2024-08-20T02:45:53.911 INFO:root:teuthology version: 1.2.0 2024-08-20T02:45:53.937 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-20T02:45:54.036 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390 branch: wip-hemanth-testing-2024-08-19-1357 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/blogbench}} email: hyelloji@redhat.com first_in_suite: false job_id: '7861390' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-08-19-1357 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: ef4275410d8483698016fe484b2a3c664080b4c1 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: ef4275410d8483698016fe484b2a3c664080b4c1 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-08-19-1357 sha1: ef4275410d8483698016fe484b2a3c664080b4c1 owner: scheduled_hyelloji@teuthology priority: 10 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7244 sha1: ef4275410d8483698016fe484b2a3c664080b4c1 sleep_before_teardown: 0 suite: fs suite_branch: wip-hemanth-testing-2024-08-19-1357 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: ef4275410d8483698016fe484b2a3c664080b4c1 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNvuXiudVw4H3/H/NQk9W+tXjD/oQP9m5qul3UpZgVm1defjGh154IzPQ4P1JgJdKsmmZRXNTUKisFysfrHYot8= smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVrwCFIf66LkdiqocXre3AjYoL95+1YBSSa9u5Y3dLpP9tsLzZkvj7nz1qBHblVpAOasHJgAah3Yu9ki8kENPI= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLkxexCI8mXgA4dPZsivoTJOUWQBBbaAz0OX5QRPSh328486mINecxy7abU0vIsrhDvZG/4IEEqOQ7FGiSbnqg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-19_19:57:23 tube: smithi user: hyelloji verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-20T02:45:54.037 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa; will attempt to use it 2024-08-20T02:45:54.037 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks 2024-08-20T02:45:54.038 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-20T02:45:54.039 INFO:teuthology.task.internal:Checking packages... 2024-08-20T02:45:54.057 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ef4275410d8483698016fe484b2a3c664080b4c1' 2024-08-20T02:45:54.057 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-20T02:45:54.057 INFO:teuthology.packaging:ref: None 2024-08-20T02:45:54.057 INFO:teuthology.packaging:tag: None 2024-08-20T02:45:54.057 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-08-19-1357 2024-08-20T02:45:54.058 INFO:teuthology.packaging:sha1: ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T02:45:54.058 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-08-19-1357 2024-08-20T02:45:54.215 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-5872.gef427541 2024-08-20T02:45:54.216 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-20T02:45:54.228 INFO:teuthology.task.internal:no buildpackages task found 2024-08-20T02:45:54.228 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-20T02:45:54.435 INFO:teuthology.task.internal:Saving configuration 2024-08-20T02:45:54.454 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-20T02:45:54.515 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-20T02:45:54.546 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-20 02:40:37.454065', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNvuXiudVw4H3/H/NQk9W+tXjD/oQP9m5qul3UpZgVm1defjGh154IzPQ4P1JgJdKsmmZRXNTUKisFysfrHYot8='} 2024-08-20T02:45:54.578 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-20 02:40:37.455396', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVrwCFIf66LkdiqocXre3AjYoL95+1YBSSa9u5Y3dLpP9tsLzZkvj7nz1qBHblVpAOasHJgAah3Yu9ki8kENPI='} 2024-08-20T02:45:54.608 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-20 02:40:37.452532', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLkxexCI8mXgA4dPZsivoTJOUWQBBbaAz0OX5QRPSh328486mINecxy7abU0vIsrhDvZG/4IEEqOQ7FGiSbnqg='} 2024-08-20T02:45:54.608 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-20T02:45:54.629 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-08-20T02:45:54.629 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-08-20T02:45:54.629 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-08-20T02:45:54.629 INFO:teuthology.run_tasks:Running task console_log... 2024-08-20T02:45:54.830 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb0304afbe0>, signals=[15]) 2024-08-20T02:45:54.830 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-20T02:45:54.846 INFO:teuthology.task.internal:Opening connections... 2024-08-20T02:45:54.846 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-08-20T02:45:54.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T02:45:54.931 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-08-20T02:45:54.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T02:45:55.013 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-08-20T02:45:55.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T02:45:55.096 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-20T02:45:55.105 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-08-20T02:45:55.123 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-08-20T02:45:55.123 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-08-20T02:45:55.181 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-08-20T02:45:55.181 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-08-20T02:45:55.181 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-08-20T02:45:55.181 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-08-20T02:45:55.181 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-08-20T02:45:55.181 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-08-20T02:45:55.181 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-20T02:45:55.181 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-08-20T02:45:55.181 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-08-20T02:45:55.181 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-20T02:45:55.182 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-08-20T02:45:55.182 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-20T02:45:55.182 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-20T02:45:55.182 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-20T02:45:55.182 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-08-20T02:45:55.214 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-08-20T02:45:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-08-20T02:45:55.233 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-08-20T02:45:55.291 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2024-08-20T02:45:55.291 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2024-08-20T02:45:55.291 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2024-08-20T02:45:55.291 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2024-08-20T02:45:55.291 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2024-08-20T02:45:55.291 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2024-08-20T02:45:55.291 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-20T02:45:55.291 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2024-08-20T02:45:55.291 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2024-08-20T02:45:55.292 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-20T02:45:55.292 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2024-08-20T02:45:55.292 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-20T02:45:55.292 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-20T02:45:55.292 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-20T02:45:55.292 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-08-20T02:45:55.322 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-08-20T02:45:55.340 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-08-20T02:45:55.340 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-20T02:45:55.399 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-20T02:45:55.400 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-20T02:45:55.400 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-08-20T02:45:55.431 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-20T02:45:55.441 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-20T02:45:55.534 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-20T02:45:55.534 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-08-20T02:45:55.537 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-08-20T02:45:55.539 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-08-20T02:45:55.556 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-20T02:45:55.606 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-20T02:45:55.606 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-08-20T02:45:55.609 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-08-20T02:45:55.611 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-08-20T02:45:55.640 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-20T02:45:55.810 INFO:teuthology.run_tasks:Running task kernel... 2024-08-20T02:45:55.832 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-08-20T02:45:55.832 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-08-20T02:45:55.832 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-08-20T02:45:55.833 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-20T02:45:55.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T02:45:55.850 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-08-20T02:45:55.908 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-496.el9.x86_64 2024-08-20T02:45:55.908 INFO:teuthology.task.kernel:Running kernel on smithi044: 5.14.0-496.el9.x86_64 2024-08-20T02:45:55.908 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum install -y kernel 2024-08-20T02:46:01.280 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 8.2 MB 00:03 2024-08-20T02:46:10.171 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-08-20T02:46:19.386 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 1.2 MB/s | 6.6 MB 00:05 2024-08-20T02:46:20.985 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 18 kB 00:00 2024-08-20T02:46:22.543 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 16 MB/s | 22 MB 00:01 2024-08-20T02:46:28.079 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-08-20T02:46:30.077 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-20T02:46:30.077 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-496.el9.x86_64 is already installed. 2024-08-20T02:46:30.168 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-08-20T02:46:30.169 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-08-20T02:46:30.169 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-08-20T02:46:30.293 DEBUG:teuthology.orchestra.run.smithi044:> echo no | sudo yum reinstall kernel || true 2024-08-20T02:46:30.854 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:02 ago on Tue 20 Aug 2024 02:46:28 AM UTC. 2024-08-20T02:46:31.001 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-08-20T02:46:31.005 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-08-20T02:46:31.005 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-08-20T02:46:31.005 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-08-20T02:46:31.005 INFO:teuthology.orchestra.run.smithi044.stdout:Reinstalling: 2024-08-20T02:46:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: kernel x86_64 5.14.0-496.el9 baseos 1.8 M 2024-08-20T02:46:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:46:31.006 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-08-20T02:46:31.006 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-08-20T02:46:31.006 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:46:31.006 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 1.8 M 2024-08-20T02:46:31.006 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 0 2024-08-20T02:46:31.006 INFO:teuthology.orchestra.run.smithi044.stderr:Operation aborted. 2024-08-20T02:46:31.068 INFO:teuthology.orchestra.run.smithi044.stdout:Is this ok [y/N]: 2024-08-20T02:46:31.069 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum reinstall -y kernel || true 2024-08-20T02:46:31.625 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:03 ago on Tue 20 Aug 2024 02:46:28 AM UTC. 2024-08-20T02:46:31.751 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-08-20T02:46:31.751 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-08-20T02:46:31.751 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-08-20T02:46:31.752 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-08-20T02:46:31.752 INFO:teuthology.orchestra.run.smithi044.stdout:Reinstalling: 2024-08-20T02:46:31.752 INFO:teuthology.orchestra.run.smithi044.stdout: kernel x86_64 5.14.0-496.el9 baseos 1.8 M 2024-08-20T02:46:31.752 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:46:31.752 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-08-20T02:46:31.752 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-08-20T02:46:31.752 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:46:31.752 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 1.8 M 2024-08-20T02:46:31.752 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 0 2024-08-20T02:46:31.752 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-08-20T02:46:31.853 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:46:31.945 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:46:32.037 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 172.65.90.23) 2024-08-20T02:46:32.169 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 172.65.90.23) 2024-08-20T02:46:32.227 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:46:32.354 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:46:32.419 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://linux-mirrors.fnal.gov/linux/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 131.225.105.75) 2024-08-20T02:46:32.529 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://linux-mirrors.fnal.gov/linux/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 131.225.105.75) 2024-08-20T02:46:32.733 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:46:32.733 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:46:32.733 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:46:32.752 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:46:32.813 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:46:32.918 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:46:32.953 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T02:46:33.112 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:46:33.444 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:46:33.606 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:46:33.938 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:46:33.990 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://ord.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:46:34.079 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://ord.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:46:34.129 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:46:34.147 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:46:35.422 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-496.el9.x86_64.rpm 506 kB/s | 1.8 MB 00:03 2024-08-20T02:46:35.425 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-08-20T02:46:35.425 INFO:teuthology.orchestra.run.smithi044.stdout:Total 499 kB/s | 1.8 MB 00:03 2024-08-20T02:46:35.456 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-08-20T02:46:35.728 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-08-20T02:46:35.728 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-08-20T02:46:35.758 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-08-20T02:46:35.758 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-08-20T02:46:35.920 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-08-20T02:46:35.991 INFO:teuthology.orchestra.run.smithi044.stdout: Reinstalling : kernel-5.14.0-496.el9.x86_64 1/2 2024-08-20T02:46:35.991 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : kernel-5.14.0-496.el9.x86_64 2/2 2024-08-20T02:46:36.865 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-5.14.0-496.el9.x86_64 2/2 2024-08-20T02:46:36.866 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-5.14.0-496.el9.x86_64 1/2 2024-08-20T02:46:37.621 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-5.14.0-496.el9.x86_64 2/2 2024-08-20T02:46:37.621 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:46:37.621 INFO:teuthology.orchestra.run.smithi044.stdout:Reinstalled: 2024-08-20T02:46:37.621 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-5.14.0-496.el9.x86_64 2024-08-20T02:46:37.621 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:46:37.621 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-08-20T02:46:37.786 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-08-20T02:46:37.845 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-496.el9.x86_64 2024-08-20T02:46:37.845 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-496.el9.x86_64 2024-08-20T02:46:37.846 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-08-20T02:46:37.846 DEBUG:teuthology.parallel:result is None 2024-08-20T02:46:37.846 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-20T02:46:38.029 INFO:teuthology.task.internal:Creating test directory... 2024-08-20T02:46:38.030 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-20T02:46:38.032 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-20T02:46:38.035 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-20T02:46:38.054 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-20T02:46:38.273 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-20T02:46:38.305 INFO:teuthology.task.internal:Creating archive directory... 2024-08-20T02:46:38.306 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-20T02:46:38.308 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-20T02:46:38.311 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-20T02:46:38.409 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-20T02:46:38.436 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-20T02:46:38.437 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-20T02:46:38.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T02:46:38.454 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-20T02:46:38.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T02:46:38.471 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-20T02:46:38.473 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-20T02:46:38.497 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-20T02:46:38.508 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T02:46:38.544 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T02:46:38.550 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T02:46:38.568 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T02:46:38.570 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T02:46:38.591 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T02:46:38.593 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-20T02:46:38.601 INFO:teuthology.task.internal:Configuring sudo... 2024-08-20T02:46:38.601 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-20T02:46:38.603 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-20T02:46:38.606 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-20T02:46:38.637 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-20T02:46:38.646 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-20T02:46:38.646 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-20T02:46:38.669 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-20T02:46:38.675 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-20T02:46:38.695 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T02:46:38.773 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T02:46:38.810 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T02:46:38.899 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T02:46:38.932 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:46:38.932 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-20T02:46:38.995 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T02:46:39.049 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T02:46:39.094 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T02:46:39.185 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T02:46:39.220 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-08-20T02:46:39.220 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-20T02:46:39.286 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T02:46:39.343 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T02:46:39.393 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T02:46:39.440 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T02:46:39.478 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-08-20T02:46:39.478 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-20T02:46:39.544 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-08-20T02:46:39.547 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-08-20T02:46:39.549 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-08-20T02:46:39.590 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T02:46:39.591 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T02:46:39.626 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T02:46:40.082 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-20T02:46:40.091 INFO:teuthology.task.internal:Starting timer... 2024-08-20T02:46:40.091 INFO:teuthology.run_tasks:Running task pcp... 2024-08-20T02:46:40.214 INFO:teuthology.run_tasks:Running task selinux... 2024-08-20T02:46:40.264 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-08-20T02:46:40.264 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-08-20T02:46:40.321 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-08-20T02:46:40.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2024-08-20T02:46:40.393 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2024-08-20T02:46:40.395 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-08-20T02:46:40.449 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-08-20T02:46:40.451 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-20T02:46:40.451 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-08-20T02:46:40.468 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-08-20T02:46:40.468 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2024-08-20T02:46:40.496 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2024-08-20T02:46:40.496 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-08-20T02:46:40.519 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-08-20T02:46:40.520 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive'} 2024-08-20T02:46:40.520 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T02:46:40.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T02:46:40.545 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T02:46:40.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T02:46:40.573 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T02:46:40.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T02:46:40.601 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-20T02:46:40.602 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-08-20T02:46:40.631 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2024-08-20T02:46:40.666 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-08-20T02:46:40.691 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-20T02:46:40.701 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-20T02:46:40.713 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-20T02:46:40.713 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi044.front.sepia.ceph.com,smithi099.front.sepia.ceph.com,smithi165.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-20T02:48:52.502 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com')] 2024-08-20T02:48:52.504 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-08-20T02:48:52.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T02:48:52.590 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-08-20T02:48:52.669 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-08-20T02:48:52.670 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-08-20T02:48:52.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T02:48:52.752 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-08-20T02:48:52.830 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-08-20T02:48:52.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-08-20T02:48:52.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T02:48:52.930 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-08-20T02:48:53.012 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-08-20T02:48:53.013 INFO:teuthology.run_tasks:Running task clock... 2024-08-20T02:48:53.023 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-20T02:48:53.024 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-20T02:48:53.024 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T02:48:53.027 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-20T02:48:53.027 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T02:48:53.030 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-20T02:48:53.030 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T02:48:53.056 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-20T02:48:53.058 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-20T02:48:53.070 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-20T02:48:53.073 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-20T02:48:53.085 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-20T02:48:53.095 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-08-20T02:48:53.097 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2024-08-20T02:48:53.101 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-20T02:48:53.107 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-08-20T02:48:53.108 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2024-08-20T02:48:53.121 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-20T02:48:53.122 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-20T02:48:53.126 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-08-20T02:48:53.136 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-20T02:48:53.136 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-20T02:48:53.138 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-08-20T02:48:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-20T02:48:53.167 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-20T02:48:53.192 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-08-20T02:48:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T02:48:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-08-20T02:48:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T02:48:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T02:48:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T02:48:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T02:48:53.199 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-08-20T02:48:53.202 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T02:48:53.202 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-08-20T02:48:53.203 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T02:48:53.203 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T02:48:53.203 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T02:48:53.203 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T02:48:53.234 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-08-20T02:48:53.238 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T02:48:53.238 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-08-20T02:48:53.238 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T02:48:53.238 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T02:48:53.238 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T02:48:53.238 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T02:48:53.239 INFO:teuthology.run_tasks:Running task install... 2024-08-20T02:48:53.250 DEBUG:teuthology.task.install:project ceph 2024-08-20T02:48:53.250 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1'}} 2024-08-20T02:48:53.250 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1'} 2024-08-20T02:48:53.250 INFO:teuthology.task.install:Using flavor: default 2024-08-20T02:48:53.255 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-20T02:48:53.255 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-08-20T02:48:53.255 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'tag': None, 'wait_for_package': False} 2024-08-20T02:48:53.256 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T02:48:53.258 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'tag': None, 'wait_for_package': False} 2024-08-20T02:48:53.258 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T02:48:53.259 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'tag': None, 'wait_for_package': False} 2024-08-20T02:48:53.259 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T02:48:53.412 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/ 2024-08-20T02:48:53.412 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5872.gef427541 2024-08-20T02:48:53.425 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/ 2024-08-20T02:48:53.425 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5872.gef427541 2024-08-20T02:48:53.437 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/ 2024-08-20T02:48:53.437 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5872.gef427541 2024-08-20T02:48:53.560 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-20T02:48:53.561 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:48:53.561 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-20T02:48:53.564 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-20T02:48:53.565 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-08-20T02:48:53.565 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-20T02:48:53.568 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-20T02:48:53.568 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-08-20T02:48:53.568 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-20T02:48:53.587 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-20T02:48:53.587 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ef4275410d8483698016fe484b2a3c664080b4c1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-20T02:48:53.594 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-20T02:48:53.594 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ef4275410d8483698016fe484b2a3c664080b4c1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-20T02:48:53.595 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-20T02:48:53.595 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ef4275410d8483698016fe484b2a3c664080b4c1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-20T02:48:53.653 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-20T02:48:53.659 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-20T02:48:53.661 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-20T02:48:53.730 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-20T02:48:53.735 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-20T02:48:53.736 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-20T02:48:53.795 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-08-20T02:48:53.796 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-08-20T02:48:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-08-20T02:48:53.800 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2024-08-20T02:48:53.801 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-08-20T02:48:53.802 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2024-08-20T02:48:54.023 INFO:teuthology.orchestra.run.smithi165.stdout:45 files removed 2024-08-20T02:48:54.027 INFO:teuthology.orchestra.run.smithi044.stdout:45 files removed 2024-08-20T02:48:54.028 INFO:teuthology.orchestra.run.smithi099.stdout:45 files removed 2024-08-20T02:48:54.050 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-20T02:48:54.054 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-20T02:48:54.055 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-20T02:48:54.604 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-08-20T02:48:54.606 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-08-20T02:48:54.610 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 304 kB/s | 84 kB 00:00 2024-08-20T02:48:54.903 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-08-20T02:48:54.905 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 93 kB/s | 24 kB 00:00 2024-08-20T02:48:54.907 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-08-20T02:48:55.172 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-08-20T02:48:55.180 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-08-20T02:48:55.182 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-08-20T02:48:59.539 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.2 MB 00:04 2024-08-20T02:48:59.648 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.2 MB 00:04 2024-08-20T02:49:00.537 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 1.5 MB/s | 8.2 MB 00:05 2024-08-20T02:49:03.463 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 20 MB 00:02 2024-08-20T02:49:03.746 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 20 MB 00:02 2024-08-20T02:49:08.147 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 20 MB 00:07 2024-08-20T02:49:11.555 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.6 MB 00:04 2024-08-20T02:49:11.753 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.6 MB 00:04 2024-08-20T02:49:13.211 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 18 kB 00:00 2024-08-20T02:49:13.516 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 18 kB 00:00 2024-08-20T02:49:14.600 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.6 MB 00:02 2024-08-20T02:49:15.426 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 11 MB/s | 22 MB 00:02 2024-08-20T02:49:15.553 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 12 MB/s | 22 MB 00:01 2024-08-20T02:49:16.326 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 18 kB 00:00 2024-08-20T02:49:17.299 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 26 MB/s | 22 MB 00:00 2024-08-20T02:49:20.779 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-08-20T02:49:21.174 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-08-20T02:49:22.744 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-08-20T02:49:22.941 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T02:49:22.942 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T02:49:22.955 INFO:teuthology.orchestra.run.smithi165.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-20T02:49:22.956 INFO:teuthology.orchestra.run.smithi165.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-20T02:49:22.957 INFO:teuthology.orchestra.run.smithi165.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-20T02:49:22.958 INFO:teuthology.orchestra.run.smithi165.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-20T02:49:22.960 INFO:teuthology.orchestra.run.smithi165.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-20T02:49:22.963 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-20T02:49:22.964 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-20T02:49:23.025 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-08-20T02:49:23.052 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-08-20T02:49:23.052 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-08-20T02:49:23.052 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-08-20T02:49:23.052 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-08-20T02:49:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-20T02:49:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:19.0.0-5872.gef427541.el9 ceph 6.6 k 2024-08-20T02:49:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T02:49:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 901 k 2024-08-20T02:49:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5872.gef427541.el9 ceph 145 k 2024-08-20T02:49:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:19.0.0-5872.gef427541.el9 ceph 1.5 M 2024-08-20T02:49:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 150 k 2024-08-20T02:49:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 14 M 2024-08-20T02:49:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 7.4 M 2024-08-20T02:49:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 49 k 2024-08-20T02:49:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 11 M 2024-08-20T02:49:23.053 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:19.0.0-5872.gef427541.el9 ceph 50 M 2024-08-20T02:49:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 285 k 2024-08-20T02:49:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 772 k 2024-08-20T02:49:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 225 k 2024-08-20T02:49:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell x86_64 2:19.0.0-5872.gef427541.el9 ceph 27 k 2024-08-20T02:49:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 18 k 2024-08-20T02:49:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-20T02:49:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-20T02:49:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-20T02:49:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 32 k 2024-08-20T02:49:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 755 k 2024-08-20T02:49:23.054 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 126 k 2024-08-20T02:49:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-20T02:49:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-20T02:49:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-20T02:49:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-20T02:49:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-20T02:49:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-20T02:49:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:19.0.0-5872.gef427541.el9 ceph 188 k 2024-08-20T02:49:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:19.0.0-5872.gef427541.el9 ceph 371 k 2024-08-20T02:49:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 355 k 2024-08-20T02:49:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 116 k 2024-08-20T02:49:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 85 k 2024-08-20T02:49:23.055 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 170 k 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.5 M 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 22 M 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 34 k 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:19.0.0-5872.gef427541.el9 ceph 2.4 M 2024-08-20T02:49:23.056 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 284 k 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:19.0.0-5872.gef427541.el9 ceph 4.8 M 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:19.0.0-5872.gef427541.el9 ceph 17 M 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 16 k 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:19.0.0-5872.gef427541.el9 ceph 25 k 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-20T02:49:23.057 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:19.0.0-5872.gef427541.el9 ceph 163 k 2024-08-20T02:49:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-20T02:49:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-20T02:49:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-20T02:49:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-20T02:49:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-20T02:49:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 508 k 2024-08-20T02:49:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-20T02:49:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T02:49:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-20T02:49:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-20T02:49:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-20T02:49:23.058 INFO:teuthology.orchestra.run.smithi165.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.26-1.el9 appstream 38 k 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp x86_64 0.3.26-1.el9 appstream 4.9 M 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-20T02:49:23.059 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:19.0.0-5872.gef427541.el9 ceph 45 k 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 138 k 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel x86_64 3.9.19-7.el9 appstream 245 k 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-20T02:49:23.060 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-20T02:49:23.061 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-20T02:49:23.062 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-20T02:49:23.063 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-20T02:49:23.064 INFO:teuthology.orchestra.run.smithi165.stdout:Installing weak dependencies: 2024-08-20T02:49:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-20T02:49:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:49:23.065 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-08-20T02:49:23.065 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-08-20T02:49:23.065 INFO:teuthology.orchestra.run.smithi165.stdout:Install 148 Packages 2024-08-20T02:49:23.065 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 2 Packages 2024-08-20T02:49:23.065 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:49:23.065 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 232 M 2024-08-20T02:49:23.066 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-08-20T02:49:23.404 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T02:49:23.405 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T02:49:23.419 INFO:teuthology.orchestra.run.smithi044.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-20T02:49:23.420 INFO:teuthology.orchestra.run.smithi044.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-20T02:49:23.421 INFO:teuthology.orchestra.run.smithi044.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-20T02:49:23.422 INFO:teuthology.orchestra.run.smithi044.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-20T02:49:23.424 INFO:teuthology.orchestra.run.smithi044.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-20T02:49:23.427 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-20T02:49:23.428 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-20T02:49:23.494 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-08-20T02:49:23.503 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-08-20T02:49:23.503 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-08-20T02:49:23.503 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-08-20T02:49:23.503 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-08-20T02:49:23.503 INFO:teuthology.orchestra.run.smithi044.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-20T02:49:23.503 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:19.0.0-5872.gef427541.el9 ceph 6.6 k 2024-08-20T02:49:23.503 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T02:49:23.503 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 901 k 2024-08-20T02:49:23.504 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5872.gef427541.el9 ceph 145 k 2024-08-20T02:49:23.504 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:19.0.0-5872.gef427541.el9 ceph 1.5 M 2024-08-20T02:49:23.504 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 150 k 2024-08-20T02:49:23.504 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 14 M 2024-08-20T02:49:23.504 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 7.4 M 2024-08-20T02:49:23.504 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 49 k 2024-08-20T02:49:23.504 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 11 M 2024-08-20T02:49:23.504 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:19.0.0-5872.gef427541.el9 ceph 50 M 2024-08-20T02:49:23.504 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 285 k 2024-08-20T02:49:23.504 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 772 k 2024-08-20T02:49:23.504 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 225 k 2024-08-20T02:49:23.504 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell x86_64 2:19.0.0-5872.gef427541.el9 ceph 27 k 2024-08-20T02:49:23.505 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 18 k 2024-08-20T02:49:23.505 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-20T02:49:23.505 INFO:teuthology.orchestra.run.smithi044.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-20T02:49:23.505 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-20T02:49:23.505 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 32 k 2024-08-20T02:49:23.505 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 755 k 2024-08-20T02:49:23.505 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 126 k 2024-08-20T02:49:23.505 INFO:teuthology.orchestra.run.smithi044.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-20T02:49:23.505 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-20T02:49:23.505 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-20T02:49:23.505 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-20T02:49:23.506 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-20T02:49:23.506 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-20T02:49:23.506 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:19.0.0-5872.gef427541.el9 ceph 188 k 2024-08-20T02:49:23.506 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:19.0.0-5872.gef427541.el9 ceph 371 k 2024-08-20T02:49:23.506 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 355 k 2024-08-20T02:49:23.506 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 116 k 2024-08-20T02:49:23.506 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 85 k 2024-08-20T02:49:23.506 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T02:49:23.506 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 170 k 2024-08-20T02:49:23.506 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-20T02:49:23.506 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-08-20T02:49:23.506 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.5 M 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 22 M 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 34 k 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:19.0.0-5872.gef427541.el9 ceph 2.4 M 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 284 k 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:19.0.0-5872.gef427541.el9 ceph 4.8 M 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:19.0.0-5872.gef427541.el9 ceph 17 M 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 16 k 2024-08-20T02:49:23.507 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:19.0.0-5872.gef427541.el9 ceph 25 k 2024-08-20T02:49:23.508 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-20T02:49:23.508 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-20T02:49:23.508 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-20T02:49:23.508 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-20T02:49:23.508 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-20T02:49:23.508 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-20T02:49:23.508 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-20T02:49:23.508 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-20T02:49:23.508 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:19.0.0-5872.gef427541.el9 ceph 163 k 2024-08-20T02:49:23.508 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-20T02:49:23.508 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 508 k 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-20T02:49:23.509 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.26-1.el9 appstream 38 k 2024-08-20T02:49:23.510 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.26-1.el9 appstream 4.9 M 2024-08-20T02:49:23.510 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-20T02:49:23.510 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-20T02:49:23.510 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-20T02:49:23.510 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-20T02:49:23.510 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-20T02:49:23.510 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-20T02:49:23.510 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-20T02:49:23.510 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-20T02:49:23.510 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-20T02:49:23.510 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:19.0.0-5872.gef427541.el9 ceph 45 k 2024-08-20T02:49:23.510 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 138 k 2024-08-20T02:49:23.511 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-20T02:49:23.511 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-20T02:49:23.511 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-20T02:49:23.511 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-20T02:49:23.511 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-20T02:49:23.511 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-20T02:49:23.511 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-20T02:49:23.511 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.19-7.el9 appstream 245 k 2024-08-20T02:49:23.511 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-20T02:49:23.511 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-20T02:49:23.511 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-20T02:49:23.511 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-20T02:49:23.512 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-20T02:49:23.513 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-20T02:49:23.514 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-20T02:49:23.515 INFO:teuthology.orchestra.run.smithi044.stdout:Installing weak dependencies: 2024-08-20T02:49:23.516 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-20T02:49:23.516 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:49:23.516 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-08-20T02:49:23.516 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-08-20T02:49:23.516 INFO:teuthology.orchestra.run.smithi044.stdout:Install 148 Packages 2024-08-20T02:49:23.516 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 2 Packages 2024-08-20T02:49:23.516 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:49:23.516 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 232 M 2024-08-20T02:49:23.517 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-08-20T02:49:23.597 INFO:teuthology.orchestra.run.smithi165.stdout:(1/150): ceph-19.0.0-5872.gef427541.el9.x86_64. 51 kB/s | 6.6 kB 00:00 2024-08-20T02:49:23.822 INFO:teuthology.orchestra.run.smithi165.stdout:(2/150): ceph-fuse-19.0.0-5872.gef427541.el9.x8 3.9 MB/s | 901 kB 00:00 2024-08-20T02:49:23.889 INFO:teuthology.orchestra.run.smithi165.stdout:(3/150): ceph-immutable-object-cache-19.0.0-587 2.1 MB/s | 145 kB 00:00 2024-08-20T02:49:23.973 INFO:teuthology.orchestra.run.smithi165.stdout:(4/150): ceph-base-19.0.0-5872.gef427541.el9.x8 11 MB/s | 5.5 MB 00:00 2024-08-20T02:49:24.098 INFO:teuthology.orchestra.run.smithi165.stdout:(5/150): ceph-mgr-19.0.0-5872.gef427541.el9.x86 12 MB/s | 1.5 MB 00:00 2024-08-20T02:49:24.157 INFO:teuthology.orchestra.run.smithi165.stdout:(6/150): ceph-mds-19.0.0-5872.gef427541.el9.x86 8.8 MB/s | 2.4 MB 00:00 2024-08-20T02:49:24.217 INFO:teuthology.orchestra.run.smithi044.stdout:(1/150): ceph-19.0.0-5872.gef427541.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-08-20T02:49:24.391 INFO:teuthology.orchestra.run.smithi165.stdout:(7/150): ceph-mon-19.0.0-5872.gef427541.el9.x86 16 MB/s | 4.8 MB 00:00 2024-08-20T02:49:24.633 INFO:teuthology.orchestra.run.smithi165.stdout:(8/150): ceph-common-19.0.0-5872.gef427541.el9. 19 MB/s | 22 MB 00:01 2024-08-20T02:49:24.659 INFO:teuthology.orchestra.run.smithi044.stdout:(2/150): ceph-fuse-19.0.0-5872.gef427541.el9.x8 2.0 MB/s | 901 kB 00:00 2024-08-20T02:49:24.666 INFO:teuthology.orchestra.run.smithi165.stdout:(9/150): ceph-selinux-19.0.0-5872.gef427541.el9 764 kB/s | 25 kB 00:00 2024-08-20T02:49:24.793 INFO:teuthology.orchestra.run.smithi044.stdout:(3/150): ceph-immutable-object-cache-19.0.0-587 1.1 MB/s | 145 kB 00:00 2024-08-20T02:49:24.887 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T02:49:24.888 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T02:49:24.899 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-20T02:49:24.899 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-20T02:49:24.900 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-20T02:49:24.901 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-20T02:49:24.902 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-20T02:49:24.905 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-20T02:49:24.905 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-20T02:49:24.965 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-08-20T02:49:24.984 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-08-20T02:49:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-08-20T02:49:24.985 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-08-20T02:49:24.985 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-08-20T02:49:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-20T02:49:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:19.0.0-5872.gef427541.el9 ceph 6.6 k 2024-08-20T02:49:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T02:49:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 901 k 2024-08-20T02:49:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5872.gef427541.el9 ceph 145 k 2024-08-20T02:49:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:19.0.0-5872.gef427541.el9 ceph 1.5 M 2024-08-20T02:49:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 150 k 2024-08-20T02:49:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 14 M 2024-08-20T02:49:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 7.4 M 2024-08-20T02:49:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 49 k 2024-08-20T02:49:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 11 M 2024-08-20T02:49:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:19.0.0-5872.gef427541.el9 ceph 50 M 2024-08-20T02:49:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 285 k 2024-08-20T02:49:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 772 k 2024-08-20T02:49:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 225 k 2024-08-20T02:49:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell x86_64 2:19.0.0-5872.gef427541.el9 ceph 27 k 2024-08-20T02:49:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 18 k 2024-08-20T02:49:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-20T02:49:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-20T02:49:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-20T02:49:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 32 k 2024-08-20T02:49:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 755 k 2024-08-20T02:49:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 126 k 2024-08-20T02:49:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-20T02:49:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-20T02:49:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-20T02:49:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-20T02:49:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-20T02:49:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-20T02:49:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:19.0.0-5872.gef427541.el9 ceph 188 k 2024-08-20T02:49:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:19.0.0-5872.gef427541.el9 ceph 371 k 2024-08-20T02:49:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 355 k 2024-08-20T02:49:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 116 k 2024-08-20T02:49:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 85 k 2024-08-20T02:49:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T02:49:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 170 k 2024-08-20T02:49:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-20T02:49:24.988 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-08-20T02:49:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.5 M 2024-08-20T02:49:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T02:49:24.989 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-08-20T02:49:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-20T02:49:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-20T02:49:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-20T02:49:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 22 M 2024-08-20T02:49:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 34 k 2024-08-20T02:49:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:19.0.0-5872.gef427541.el9 ceph 2.4 M 2024-08-20T02:49:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 284 k 2024-08-20T02:49:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:19.0.0-5872.gef427541.el9 ceph 4.8 M 2024-08-20T02:49:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:19.0.0-5872.gef427541.el9 ceph 17 M 2024-08-20T02:49:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 16 k 2024-08-20T02:49:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:19.0.0-5872.gef427541.el9 ceph 25 k 2024-08-20T02:49:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-20T02:49:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-20T02:49:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-20T02:49:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-20T02:49:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-20T02:49:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-20T02:49:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-20T02:49:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-20T02:49:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:19.0.0-5872.gef427541.el9 ceph 163 k 2024-08-20T02:49:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-20T02:49:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-20T02:49:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-20T02:49:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-20T02:49:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-20T02:49:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 508 k 2024-08-20T02:49:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-20T02:49:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T02:49:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-20T02:49:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-20T02:49:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-20T02:49:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-20T02:49:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-20T02:49:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-20T02:49:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.26-1.el9 appstream 38 k 2024-08-20T02:49:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp x86_64 0.3.26-1.el9 appstream 4.9 M 2024-08-20T02:49:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-20T02:49:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-20T02:49:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-20T02:49:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-20T02:49:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-20T02:49:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-20T02:49:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-20T02:49:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-20T02:49:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-20T02:49:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:19.0.0-5872.gef427541.el9 ceph 45 k 2024-08-20T02:49:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 138 k 2024-08-20T02:49:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-20T02:49:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-20T02:49:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-20T02:49:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-20T02:49:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-20T02:49:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-20T02:49:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-20T02:49:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel x86_64 3.9.19-7.el9 appstream 245 k 2024-08-20T02:49:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-20T02:49:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-20T02:49:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-20T02:49:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-20T02:49:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-20T02:49:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-20T02:49:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-20T02:49:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-20T02:49:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-20T02:49:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-20T02:49:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-20T02:49:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-20T02:49:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-20T02:49:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-20T02:49:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-20T02:49:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-20T02:49:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-20T02:49:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-20T02:49:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-20T02:49:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-20T02:49:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-20T02:49:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-20T02:49:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-20T02:49:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-20T02:49:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-20T02:49:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-20T02:49:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-20T02:49:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-20T02:49:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-20T02:49:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-20T02:49:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-20T02:49:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-20T02:49:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-20T02:49:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-20T02:49:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-20T02:49:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-20T02:49:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-20T02:49:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-20T02:49:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-20T02:49:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-20T02:49:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-20T02:49:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-20T02:49:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-20T02:49:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-20T02:49:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-20T02:49:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-20T02:49:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-20T02:49:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-20T02:49:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-20T02:49:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-20T02:49:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-20T02:49:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-20T02:49:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-20T02:49:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-20T02:49:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-20T02:49:25.000 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2024-08-20T02:49:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-20T02:49:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:49:25.001 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-08-20T02:49:25.001 INFO:teuthology.orchestra.run.smithi099.stdout:======================================================================================= 2024-08-20T02:49:25.001 INFO:teuthology.orchestra.run.smithi099.stdout:Install 148 Packages 2024-08-20T02:49:25.001 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 2 Packages 2024-08-20T02:49:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:49:25.002 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 232 M 2024-08-20T02:49:25.002 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-08-20T02:49:25.075 INFO:teuthology.orchestra.run.smithi165.stdout:(10/150): ceph-radosgw-19.0.0-5872.gef427541.el 16 MB/s | 11 MB 00:00 2024-08-20T02:49:25.133 INFO:teuthology.orchestra.run.smithi165.stdout:(11/150): cephfs-mirror-19.0.0-5872.gef427541.e 3.8 MB/s | 225 kB 00:00 2024-08-20T02:49:25.192 INFO:teuthology.orchestra.run.smithi165.stdout:(12/150): cephfs-shell-19.0.0-5872.gef427541.el 462 kB/s | 27 kB 00:00 2024-08-20T02:49:25.251 INFO:teuthology.orchestra.run.smithi165.stdout:(13/150): libcephfs-devel-19.0.0-5872.gef427541 545 kB/s | 32 kB 00:00 2024-08-20T02:49:25.334 INFO:teuthology.orchestra.run.smithi165.stdout:(14/150): libcephfs2-19.0.0-5872.gef427541.el9. 8.9 MB/s | 755 kB 00:00 2024-08-20T02:49:25.503 INFO:teuthology.orchestra.run.smithi099.stdout:(1/150): ceph-19.0.0-5872.gef427541.el9.x86_64. 51 kB/s | 6.6 kB 00:00 2024-08-20T02:49:25.518 INFO:teuthology.orchestra.run.smithi165.stdout:(15/150): ceph-osd-19.0.0-5872.gef427541.el9.x8 13 MB/s | 17 MB 00:01 2024-08-20T02:49:25.543 INFO:teuthology.orchestra.run.smithi165.stdout:(16/150): libcephsqlite-19.0.0-5872.gef427541.e 781 kB/s | 163 kB 00:00 2024-08-20T02:49:25.568 INFO:teuthology.orchestra.run.smithi165.stdout:(17/150): librados-devel-19.0.0-5872.gef427541. 2.5 MB/s | 126 kB 00:00 2024-08-20T02:49:25.602 INFO:teuthology.orchestra.run.smithi165.stdout:(18/150): libradosstriper1-19.0.0-5872.gef42754 8.4 MB/s | 508 kB 00:00 2024-08-20T02:49:25.669 INFO:teuthology.orchestra.run.smithi165.stdout:(19/150): python3-ceph-argparse-19.0.0-5872.gef 679 kB/s | 45 kB 00:00 2024-08-20T02:49:25.718 INFO:teuthology.orchestra.run.smithi044.stdout:(4/150): ceph-mds-19.0.0-5872.gef427541.el9.x86 2.6 MB/s | 2.4 MB 00:00 2024-08-20T02:49:25.744 INFO:teuthology.orchestra.run.smithi165.stdout:(20/150): python3-ceph-common-19.0.0-5872.gef42 1.8 MB/s | 138 kB 00:00 2024-08-20T02:49:25.820 INFO:teuthology.orchestra.run.smithi165.stdout:(21/150): python3-cephfs-19.0.0-5872.gef427541. 2.5 MB/s | 188 kB 00:00 2024-08-20T02:49:25.912 INFO:teuthology.orchestra.run.smithi165.stdout:(22/150): python3-rados-19.0.0-5872.gef427541.e 4.0 MB/s | 371 kB 00:00 2024-08-20T02:49:25.945 INFO:teuthology.orchestra.run.smithi099.stdout:(2/150): ceph-fuse-19.0.0-5872.gef427541.el9.x8 2.0 MB/s | 901 kB 00:00 2024-08-20T02:49:25.995 INFO:teuthology.orchestra.run.smithi165.stdout:(23/150): python3-rbd-19.0.0-5872.gef427541.el9 4.2 MB/s | 355 kB 00:00 2024-08-20T02:49:26.098 INFO:teuthology.orchestra.run.smithi099.stdout:(3/150): ceph-immutable-object-cache-19.0.0-587 1.2 MB/s | 145 kB 00:00 2024-08-20T02:49:26.098 INFO:teuthology.orchestra.run.smithi165.stdout:(24/150): librgw2-19.0.0-5872.gef427541.el9.x86 11 MB/s | 5.5 MB 00:00 2024-08-20T02:49:26.099 INFO:teuthology.orchestra.run.smithi165.stdout:(25/150): python3-rgw-19.0.0-5872.gef427541.el9 1.4 MB/s | 116 kB 00:00 2024-08-20T02:49:26.131 INFO:teuthology.orchestra.run.smithi165.stdout:(26/150): rbd-fuse-19.0.0-5872.gef427541.el9.x8 1.3 MB/s | 85 kB 00:00 2024-08-20T02:49:26.213 INFO:teuthology.orchestra.run.smithi165.stdout:(27/150): rbd-nbd-19.0.0-5872.gef427541.el9.x86 2.0 MB/s | 170 kB 00:00 2024-08-20T02:49:26.297 INFO:teuthology.orchestra.run.smithi165.stdout:(28/150): ceph-grafana-dashboards-19.0.0-5872.g 411 kB/s | 34 kB 00:00 2024-08-20T02:49:26.376 INFO:teuthology.orchestra.run.smithi044.stdout:(5/150): ceph-base-19.0.0-5872.gef427541.el9.x8 2.4 MB/s | 5.5 MB 00:02 2024-08-20T02:49:26.380 INFO:teuthology.orchestra.run.smithi165.stdout:(29/150): ceph-mgr-cephadm-19.0.0-5872.gef42754 1.8 MB/s | 150 kB 00:00 2024-08-20T02:49:26.439 INFO:teuthology.orchestra.run.smithi165.stdout:(30/150): rbd-mirror-19.0.0-5872.gef427541.el9. 8.8 MB/s | 3.2 MB 00:00 2024-08-20T02:49:26.501 INFO:teuthology.orchestra.run.smithi044.stdout:(6/150): ceph-mgr-19.0.0-5872.gef427541.el9.x86 1.9 MB/s | 1.5 MB 00:00 2024-08-20T02:49:26.762 INFO:teuthology.orchestra.run.smithi099.stdout:(4/150): ceph-base-19.0.0-5872.gef427541.el9.x8 4.0 MB/s | 5.5 MB 00:01 2024-08-20T02:49:26.885 INFO:teuthology.orchestra.run.smithi044.stdout:(7/150): ceph-common-19.0.0-5872.gef427541.el9. 7.6 MB/s | 22 MB 00:02 2024-08-20T02:49:27.012 INFO:teuthology.orchestra.run.smithi099.stdout:(5/150): ceph-mds-19.0.0-5872.gef427541.el9.x86 2.5 MB/s | 2.4 MB 00:00 2024-08-20T02:49:27.171 INFO:teuthology.orchestra.run.smithi099.stdout:(6/150): ceph-mgr-19.0.0-5872.gef427541.el9.x86 3.7 MB/s | 1.5 MB 00:00 2024-08-20T02:49:27.247 INFO:teuthology.orchestra.run.smithi165.stdout:(31/150): ceph-mgr-diskprediction-local-19.0.0- 9.2 MB/s | 7.4 MB 00:00 2024-08-20T02:49:27.342 INFO:teuthology.orchestra.run.smithi165.stdout:(32/150): ceph-mgr-modules-core-19.0.0-5872.gef 3.0 MB/s | 284 kB 00:00 2024-08-20T02:49:27.702 INFO:teuthology.orchestra.run.smithi044.stdout:(8/150): ceph-radosgw-19.0.0-5872.gef427541.el9 13 MB/s | 11 MB 00:00 2024-08-20T02:49:27.744 INFO:teuthology.orchestra.run.smithi044.stdout:(9/150): ceph-selinux-19.0.0-5872.gef427541.el9 607 kB/s | 25 kB 00:00 2024-08-20T02:49:27.790 INFO:teuthology.orchestra.run.smithi165.stdout:(33/150): ceph-test-19.0.0-5872.gef427541.el9.x 16 MB/s | 50 MB 00:03 2024-08-20T02:49:27.815 INFO:teuthology.orchestra.run.smithi165.stdout:(34/150): ceph-mgr-rook-19.0.0-5872.gef427541.e 104 kB/s | 49 kB 00:00 2024-08-20T02:49:27.848 INFO:teuthology.orchestra.run.smithi165.stdout:(35/150): ceph-prometheus-alerts-19.0.0-5872.ge 283 kB/s | 16 kB 00:00 2024-08-20T02:49:27.882 INFO:teuthology.orchestra.run.smithi165.stdout:(36/150): ceph-volume-19.0.0-5872.gef427541.el9 4.2 MB/s | 285 kB 00:00 2024-08-20T02:49:27.966 INFO:teuthology.orchestra.run.smithi165.stdout:(37/150): cephfs-top-19.0.0-5872.gef427541.el9. 219 kB/s | 18 kB 00:00 2024-08-20T02:49:27.991 INFO:teuthology.orchestra.run.smithi165.stdout:(38/150): cephadm-19.0.0-5872.gef427541.el9.noa 5.3 MB/s | 772 kB 00:00 2024-08-20T02:49:28.121 INFO:teuthology.orchestra.run.smithi099.stdout:(7/150): ceph-mon-19.0.0-5872.gef427541.el9.x86 4.3 MB/s | 4.8 MB 00:01 2024-08-20T02:49:28.124 INFO:teuthology.orchestra.run.smithi165.stdout:(39/150): ceph-mgr-dashboard-19.0.0-5872.gef427 8.1 MB/s | 14 MB 00:01 2024-08-20T02:49:28.154 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:28.183 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:28.219 INFO:teuthology.orchestra.run.smithi044.stdout:(10/150): ceph-mon-19.0.0-5872.gef427541.el9.x8 2.6 MB/s | 4.8 MB 00:01 2024-08-20T02:49:28.258 INFO:teuthology.orchestra.run.smithi165.stdout:(40/150): fuse-2.9.9-16.el9.x86_64.rpm 273 kB/s | 80 kB 00:00 2024-08-20T02:49:28.287 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:28.311 INFO:teuthology.orchestra.run.smithi044.stdout:(11/150): cephfs-mirror-19.0.0-5872.gef427541.e 2.4 MB/s | 225 kB 00:00 2024-08-20T02:49:28.317 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:28.367 INFO:teuthology.orchestra.run.smithi165.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-08-20T02:49:28.368 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T02:49:28.369 INFO:teuthology.orchestra.run.smithi044.stdout:(12/150): cephfs-shell-19.0.0-5872.gef427541.el 464 kB/s | 27 kB 00:00 2024-08-20T02:49:28.408 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T02:49:28.428 INFO:teuthology.orchestra.run.smithi044.stdout:(13/150): libcephfs-devel-19.0.0-5872.gef427541 544 kB/s | 32 kB 00:00 2024-08-20T02:49:28.454 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T02:49:28.457 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T02:49:28.501 INFO:teuthology.orchestra.run.smithi165.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-08-20T02:49:28.527 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ord.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:49:28.527 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ord.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:49:28.559 INFO:teuthology.orchestra.run.smithi165.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-08-20T02:49:28.619 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ord.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:49:28.628 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ord.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:49:28.650 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:28.653 INFO:teuthology.orchestra.run.smithi044.stdout:(14/150): libcephfs2-19.0.0-5872.gef427541.el9. 3.3 MB/s | 755 kB 00:00 2024-08-20T02:49:28.660 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:28.684 INFO:teuthology.orchestra.run.smithi165.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-08-20T02:49:28.744 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:28.745 INFO:teuthology.orchestra.run.smithi044.stdout:(15/150): libcephsqlite-19.0.0-5872.gef427541.e 1.7 MB/s | 163 kB 00:00 2024-08-20T02:49:28.748 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:28.784 INFO:teuthology.orchestra.run.smithi165.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 1.4 MB/s | 125 kB 00:00 2024-08-20T02:49:28.801 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T02:49:28.802 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T02:49:28.804 INFO:teuthology.orchestra.run.smithi044.stdout:(16/150): librados-devel-19.0.0-5872.gef427541. 2.1 MB/s | 126 kB 00:00 2024-08-20T02:49:28.860 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:28.860 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:28.926 INFO:teuthology.orchestra.run.smithi165.stdout:(46/150): python3-urllib3-1.26.5-6.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-08-20T02:49:28.946 INFO:teuthology.orchestra.run.smithi044.stdout:(17/150): libradosstriper1-19.0.0-5872.gef42754 3.5 MB/s | 508 kB 00:00 2024-08-20T02:49:28.954 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:28.962 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:28.997 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:29.004 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:29.029 INFO:teuthology.orchestra.run.smithi099.stdout:(8/150): ceph-osd-19.0.0-5872.gef427541.el9.x86 9.2 MB/s | 17 MB 00:01 2024-08-20T02:49:29.073 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:29.079 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:29.088 INFO:teuthology.orchestra.run.smithi099.stdout:(9/150): ceph-selinux-19.0.0-5872.gef427541.el9 434 kB/s | 25 kB 00:00 2024-08-20T02:49:29.133 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:29.139 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:29.236 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:29.250 INFO:teuthology.orchestra.run.smithi165.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:29.588 INFO:teuthology.orchestra.run.smithi099.stdout:(10/150): ceph-common-19.0.0-5872.gef427541.el9 5.1 MB/s | 22 MB 00:04 2024-08-20T02:49:29.630 INFO:teuthology.orchestra.run.smithi099.stdout:(11/150): cephfs-mirror-19.0.0-5872.gef427541.e 5.3 MB/s | 225 kB 00:00 2024-08-20T02:49:29.672 INFO:teuthology.orchestra.run.smithi099.stdout:(12/150): cephfs-shell-19.0.0-5872.gef427541.el 645 kB/s | 27 kB 00:00 2024-08-20T02:49:29.730 INFO:teuthology.orchestra.run.smithi099.stdout:(13/150): libcephfs-devel-19.0.0-5872.gef427541 544 kB/s | 32 kB 00:00 2024-08-20T02:49:29.793 INFO:teuthology.orchestra.run.smithi165.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 796 kB/s | 689 kB 00:00 2024-08-20T02:49:29.822 INFO:teuthology.orchestra.run.smithi099.stdout:(14/150): libcephfs2-19.0.0-5872.gef427541.el9. 8.1 MB/s | 755 kB 00:00 2024-08-20T02:49:29.873 INFO:teuthology.orchestra.run.smithi099.stdout:(15/150): libcephsqlite-19.0.0-5872.gef427541.e 3.2 MB/s | 163 kB 00:00 2024-08-20T02:49:29.931 INFO:teuthology.orchestra.run.smithi099.stdout:(16/150): librados-devel-19.0.0-5872.gef427541. 2.1 MB/s | 126 kB 00:00 2024-08-20T02:49:29.946 INFO:teuthology.orchestra.run.smithi044.stdout:(18/150): ceph-test-19.0.0-5872.gef427541.el9.x 23 MB/s | 50 MB 00:02 2024-08-20T02:49:29.985 INFO:teuthology.orchestra.run.smithi165.stdout:(48/150): libquadmath-11.5.0-2.el9.x86_64.rpm 103 kB/s | 191 kB 00:01 2024-08-20T02:49:29.998 INFO:teuthology.orchestra.run.smithi044.stdout:(19/150): python3-ceph-argparse-19.0.0-5872.gef 913 kB/s | 45 kB 00:00 2024-08-20T02:49:30.047 INFO:teuthology.orchestra.run.smithi044.stdout:(20/150): python3-ceph-common-19.0.0-5872.gef42 2.7 MB/s | 138 kB 00:00 2024-08-20T02:49:30.073 INFO:teuthology.orchestra.run.smithi099.stdout:(17/150): ceph-radosgw-19.0.0-5872.gef427541.el 5.6 MB/s | 11 MB 00:01 2024-08-20T02:49:30.089 INFO:teuthology.orchestra.run.smithi044.stdout:(21/150): python3-cephfs-19.0.0-5872.gef427541. 4.4 MB/s | 188 kB 00:00 2024-08-20T02:49:30.107 INFO:teuthology.orchestra.run.smithi099.stdout:(18/150): libradosstriper1-19.0.0-5872.gef42754 2.8 MB/s | 508 kB 00:00 2024-08-20T02:49:30.127 INFO:teuthology.orchestra.run.smithi165.stdout:(49/150): libgfortran-11.5.0-2.el9.x86_64.rpm 375 kB/s | 801 kB 00:02 2024-08-20T02:49:30.139 INFO:teuthology.orchestra.run.smithi044.stdout:(22/150): python3-rados-19.0.0-5872.gef427541.e 7.3 MB/s | 371 kB 00:00 2024-08-20T02:49:30.149 INFO:teuthology.orchestra.run.smithi099.stdout:(19/150): python3-ceph-argparse-19.0.0-5872.gef 1.1 MB/s | 45 kB 00:00 2024-08-20T02:49:30.199 INFO:teuthology.orchestra.run.smithi099.stdout:(20/150): python3-ceph-common-19.0.0-5872.gef42 2.7 MB/s | 138 kB 00:00 2024-08-20T02:49:30.231 INFO:teuthology.orchestra.run.smithi044.stdout:(23/150): librgw2-19.0.0-5872.gef427541.el9.x86 4.3 MB/s | 5.5 MB 00:01 2024-08-20T02:49:30.249 INFO:teuthology.orchestra.run.smithi099.stdout:(21/150): python3-cephfs-19.0.0-5872.gef427541. 3.7 MB/s | 188 kB 00:00 2024-08-20T02:49:30.256 INFO:teuthology.orchestra.run.smithi044.stdout:(24/150): python3-rbd-19.0.0-5872.gef427541.el9 3.0 MB/s | 355 kB 00:00 2024-08-20T02:49:30.282 INFO:teuthology.orchestra.run.smithi044.stdout:(25/150): python3-rgw-19.0.0-5872.gef427541.el9 2.2 MB/s | 116 kB 00:00 2024-08-20T02:49:30.307 INFO:teuthology.orchestra.run.smithi044.stdout:(26/150): rbd-fuse-19.0.0-5872.gef427541.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-20T02:49:30.316 INFO:teuthology.orchestra.run.smithi099.stdout:(22/150): python3-rados-19.0.0-5872.gef427541.e 5.5 MB/s | 371 kB 00:00 2024-08-20T02:49:30.349 INFO:teuthology.orchestra.run.smithi044.stdout:(27/150): rbd-nbd-19.0.0-5872.gef427541.el9.x86 4.0 MB/s | 170 kB 00:00 2024-08-20T02:49:30.383 INFO:teuthology.orchestra.run.smithi099.stdout:(23/150): python3-rbd-19.0.0-5872.gef427541.el9 5.2 MB/s | 355 kB 00:00 2024-08-20T02:49:30.391 INFO:teuthology.orchestra.run.smithi044.stdout:(28/150): ceph-grafana-dashboards-19.0.0-5872.g 824 kB/s | 34 kB 00:00 2024-08-20T02:49:30.434 INFO:teuthology.orchestra.run.smithi099.stdout:(24/150): python3-rgw-19.0.0-5872.gef427541.el9 2.3 MB/s | 116 kB 00:00 2024-08-20T02:49:30.441 INFO:teuthology.orchestra.run.smithi044.stdout:(29/150): ceph-mgr-cephadm-19.0.0-5872.gef42754 3.0 MB/s | 150 kB 00:00 2024-08-20T02:49:30.484 INFO:teuthology.orchestra.run.smithi099.stdout:(25/150): rbd-fuse-19.0.0-5872.gef427541.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-20T02:49:30.660 INFO:teuthology.orchestra.run.smithi165.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 158 kB/s | 107 kB 00:00 2024-08-20T02:49:30.686 INFO:teuthology.orchestra.run.smithi165.stdout:(51/150): elfutils-libelf-devel-0.191-4.el9.x86 39 kB/s | 22 kB 00:00 2024-08-20T02:49:30.719 INFO:teuthology.orchestra.run.smithi165.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-08-20T02:49:30.883 INFO:teuthology.orchestra.run.smithi044.stdout:(30/150): rbd-mirror-19.0.0-5872.gef427541.el9. 5.3 MB/s | 3.2 MB 00:00 2024-08-20T02:49:30.901 INFO:teuthology.orchestra.run.smithi099.stdout:(26/150): librgw2-19.0.0-5872.gef427541.el9.x86 6.7 MB/s | 5.5 MB 00:00 2024-08-20T02:49:30.956 INFO:teuthology.orchestra.run.smithi099.stdout:(27/150): rbd-nbd-19.0.0-5872.gef427541.el9.x86 3.0 MB/s | 170 kB 00:00 2024-08-20T02:49:31.033 INFO:teuthology.orchestra.run.smithi099.stdout:(28/150): rbd-mirror-19.0.0-5872.gef427541.el9. 5.8 MB/s | 3.2 MB 00:00 2024-08-20T02:49:31.059 INFO:teuthology.orchestra.run.smithi099.stdout:(29/150): ceph-grafana-dashboards-19.0.0-5872.g 336 kB/s | 34 kB 00:00 2024-08-20T02:49:31.084 INFO:teuthology.orchestra.run.smithi099.stdout:(30/150): ceph-mgr-cephadm-19.0.0-5872.gef42754 2.9 MB/s | 150 kB 00:00 2024-08-20T02:49:31.126 INFO:teuthology.orchestra.run.smithi044.stdout:(31/150): ceph-mgr-dashboard-19.0.0-5872.gef427 21 MB/s | 14 MB 00:00 2024-08-20T02:49:31.184 INFO:teuthology.orchestra.run.smithi044.stdout:(32/150): ceph-mgr-modules-core-19.0.0-5872.gef 4.8 MB/s | 284 kB 00:00 2024-08-20T02:49:31.241 INFO:teuthology.orchestra.run.smithi165.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-08-20T02:49:31.242 INFO:teuthology.orchestra.run.smithi044.stdout:(33/150): ceph-mgr-rook-19.0.0-5872.gef427541.e 919 kB/s | 49 kB 00:00 2024-08-20T02:49:31.281 INFO:teuthology.orchestra.run.smithi044.stdout:(34/150): ceph-prometheus-alerts-19.0.0-5872.ge 395 kB/s | 16 kB 00:00 2024-08-20T02:49:31.347 INFO:teuthology.orchestra.run.smithi044.stdout:(35/150): ceph-volume-19.0.0-5872.gef427541.el9 4.2 MB/s | 285 kB 00:00 2024-08-20T02:49:31.406 INFO:teuthology.orchestra.run.smithi044.stdout:(36/150): cephadm-19.0.0-5872.gef427541.el9.noa 13 MB/s | 772 kB 00:00 2024-08-20T02:49:31.445 INFO:teuthology.orchestra.run.smithi165.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 400 kB/s | 313 kB 00:00 2024-08-20T02:49:31.447 INFO:teuthology.orchestra.run.smithi044.stdout:(37/150): cephfs-top-19.0.0-5872.gef427541.el9. 439 kB/s | 18 kB 00:00 2024-08-20T02:49:31.523 INFO:teuthology.orchestra.run.smithi044.stdout:(38/150): fuse-2.9.9-16.el9.x86_64.rpm 1.0 MB/s | 80 kB 00:00 2024-08-20T02:49:31.706 INFO:teuthology.orchestra.run.smithi044.stdout:(39/150): ceph-osd-19.0.0-5872.gef427541.el9.x8 3.3 MB/s | 17 MB 00:05 2024-08-20T02:49:31.720 INFO:teuthology.orchestra.run.smithi165.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-08-20T02:49:31.725 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:31.743 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:31.745 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:31.776 INFO:teuthology.orchestra.run.smithi099.stdout:(31/150): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-08-20T02:49:31.779 INFO:teuthology.orchestra.run.smithi165.stdout:(56/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-08-20T02:49:31.781 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:31.802 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:31.840 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:31.843 INFO:teuthology.orchestra.run.smithi099.stdout:(32/150): ceph-mgr-modules-core-19.0.0-5872.gef 4.2 MB/s | 284 kB 00:00 2024-08-20T02:49:31.893 INFO:teuthology.orchestra.run.smithi099.stdout:(33/150): ceph-mgr-rook-19.0.0-5872.gef427541.e 986 kB/s | 49 kB 00:00 2024-08-20T02:49:31.935 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:31.943 INFO:teuthology.orchestra.run.smithi099.stdout:(34/150): ceph-prometheus-alerts-19.0.0-5872.ge 330 kB/s | 16 kB 00:00 2024-08-20T02:49:31.966 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:31.967 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:31.970 INFO:teuthology.orchestra.run.smithi165.stdout:(57/150): libaio-devel-0.3.111-13.el9.x86_64.rp 21 kB/s | 11 kB 00:00 2024-08-20T02:49:32.010 INFO:teuthology.orchestra.run.smithi099.stdout:(35/150): ceph-volume-19.0.0-5872.gef427541.el9 4.2 MB/s | 285 kB 00:00 2024-08-20T02:49:32.065 INFO:teuthology.orchestra.run.smithi044.stdout:(40/150): ceph-mgr-diskprediction-local-19.0.0- 6.3 MB/s | 7.4 MB 00:01 2024-08-20T02:49:32.080 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:32.119 INFO:teuthology.orchestra.run.smithi099.stdout:(36/150): cephadm-19.0.0-5872.gef427541.el9.noa 7.0 MB/s | 772 kB 00:00 2024-08-20T02:49:32.121 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:32.157 INFO:teuthology.orchestra.run.smithi044.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-08-20T02:49:32.169 INFO:teuthology.orchestra.run.smithi099.stdout:(37/150): cephfs-top-19.0.0-5872.gef427541.el9. 364 kB/s | 18 kB 00:00 2024-08-20T02:49:32.174 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:32.179 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:32.217 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:32.255 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:32.278 INFO:teuthology.orchestra.run.smithi099.stdout:(38/150): fuse-2.9.9-16.el9.x86_64.rpm 739 kB/s | 80 kB 00:00 2024-08-20T02:49:32.295 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T02:49:32.299 INFO:teuthology.orchestra.run.smithi044.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.2 MB 00:00 2024-08-20T02:49:32.323 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:32.331 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:32.352 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:32.354 INFO:teuthology.orchestra.run.smithi165.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 79 kB/s | 45 kB 00:00 2024-08-20T02:49:32.366 INFO:teuthology.orchestra.run.smithi044.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-08-20T02:49:32.395 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:32.429 INFO:teuthology.orchestra.run.smithi165.stdout:(59/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 226 kB/s | 160 kB 00:00 2024-08-20T02:49:32.441 INFO:teuthology.orchestra.run.smithi044.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-08-20T02:49:32.450 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:32.517 INFO:teuthology.orchestra.run.smithi044.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-08-20T02:49:32.517 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:32.517 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:32.545 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:32.578 INFO:teuthology.orchestra.run.smithi099.stdout:(39/150): ceph-mgr-dashboard-19.0.0-5872.gef427 9.3 MB/s | 14 MB 00:01 2024-08-20T02:49:32.579 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:32.592 INFO:teuthology.orchestra.run.smithi044.stdout:(46/150): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-08-20T02:49:32.596 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T02:49:32.621 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:32.624 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:32.652 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:32.659 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:32.749 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:32.783 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:32.818 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:32.867 INFO:teuthology.orchestra.run.smithi044.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-08-20T02:49:32.871 INFO:teuthology.orchestra.run.smithi165.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 736 kB/s | 662 kB 00:00 2024-08-20T02:49:32.972 INFO:teuthology.orchestra.run.smithi165.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 47 kB/s | 25 kB 00:00 2024-08-20T02:49:33.001 INFO:teuthology.orchestra.run.smithi044.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 7.4 MB/s | 943 kB 00:00 2024-08-20T02:49:33.059 INFO:teuthology.orchestra.run.smithi044.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-08-20T02:49:33.110 INFO:teuthology.orchestra.run.smithi044.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 435 kB/s | 22 kB 00:00 2024-08-20T02:49:33.139 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:33.161 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:33.212 INFO:teuthology.orchestra.run.smithi099.stdout:(40/150): ceph-test-19.0.0-5872.gef427541.el9.x 12 MB/s | 50 MB 00:04 2024-08-20T02:49:33.213 INFO:teuthology.orchestra.run.smithi165.stdout:(62/150): libtool-2.4.6-46.el9.x86_64.rpm 677 kB/s | 581 kB 00:00 2024-08-20T02:49:33.220 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:33.285 INFO:teuthology.orchestra.run.smithi044.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-08-20T02:49:33.335 INFO:teuthology.orchestra.run.smithi044.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-08-20T02:49:33.379 INFO:teuthology.orchestra.run.smithi099.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-08-20T02:49:33.455 INFO:teuthology.orchestra.run.smithi165.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-08-20T02:49:33.470 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:33.521 INFO:teuthology.orchestra.run.smithi099.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.2 MB 00:00 2024-08-20T02:49:33.551 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:33.551 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:33.581 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:33.654 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:33.704 INFO:teuthology.orchestra.run.smithi099.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 581 kB/s | 106 kB 00:00 2024-08-20T02:49:33.705 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:33.726 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:33.747 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:33.780 INFO:teuthology.orchestra.run.smithi099.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-08-20T02:49:33.802 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:33.820 INFO:teuthology.orchestra.run.smithi165.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 345 kB/s | 292 kB 00:00 2024-08-20T02:49:33.847 INFO:teuthology.orchestra.run.smithi099.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-08-20T02:49:33.847 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:33.885 INFO:teuthology.orchestra.run.smithi044.stdout:(53/150): libquadmath-11.5.0-2.el9.x86_64.rpm 88 kB/s | 191 kB 00:02 2024-08-20T02:49:33.962 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:33.970 INFO:teuthology.orchestra.run.smithi165.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 252 kB/s | 188 kB 00:00 2024-08-20T02:49:34.002 INFO:teuthology.orchestra.run.smithi044.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-08-20T02:49:34.008 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:34.036 INFO:teuthology.orchestra.run.smithi044.stdout:(55/150): libgfortran-11.5.0-2.el9.x86_64.rpm 319 kB/s | 801 kB 00:02 2024-08-20T02:49:34.037 INFO:teuthology.orchestra.run.smithi165.stdout:(66/150): openblas-0.3.26-1.el9.x86_64.rpm 67 kB/s | 38 kB 00:00 2024-08-20T02:49:34.038 INFO:teuthology.orchestra.run.smithi099.stdout:(46/150): python3-urllib3-1.26.5-6.el9.noarch.r 1.1 MB/s | 215 kB 00:00 2024-08-20T02:49:34.061 INFO:teuthology.orchestra.run.smithi044.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-08-20T02:49:34.164 INFO:teuthology.orchestra.run.smithi099.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-08-20T02:49:34.178 INFO:teuthology.orchestra.run.smithi044.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-08-20T02:49:34.203 INFO:teuthology.orchestra.run.smithi044.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 319 kB/s | 45 kB 00:00 2024-08-20T02:49:34.298 INFO:teuthology.orchestra.run.smithi099.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 6.9 MB/s | 943 kB 00:00 2024-08-20T02:49:34.298 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:34.320 INFO:teuthology.orchestra.run.smithi044.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-08-20T02:49:34.330 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:34.342 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:34.358 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:34.380 INFO:teuthology.orchestra.run.smithi044.stdout:(60/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.9 MB/s | 3.0 MB 00:01 2024-08-20T02:49:34.381 INFO:teuthology.orchestra.run.smithi099.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-08-20T02:49:34.404 INFO:teuthology.orchestra.run.smithi044.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 304 kB/s | 25 kB 00:00 2024-08-20T02:49:34.413 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:34.438 INFO:teuthology.orchestra.run.smithi044.stdout:(62/150): libtool-2.4.6-46.el9.x86_64.rpm 2.4 MB/s | 581 kB 00:00 2024-08-20T02:49:34.456 INFO:teuthology.orchestra.run.smithi099.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 323 kB/s | 22 kB 00:00 2024-08-20T02:49:34.457 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:34.463 INFO:teuthology.orchestra.run.smithi044.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 572 kB/s | 47 kB 00:00 2024-08-20T02:49:34.485 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:34.489 INFO:teuthology.orchestra.run.smithi044.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-08-20T02:49:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-08-20T02:49:34.514 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:34.519 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:34.523 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:34.557 INFO:teuthology.orchestra.run.smithi099.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-08-20T02:49:34.632 INFO:teuthology.orchestra.run.smithi099.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-08-20T02:49:34.634 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:34.642 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:34.649 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:34.662 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:34.722 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:34.836 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:34.854 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:34.891 INFO:teuthology.orchestra.run.smithi099.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-08-20T02:49:34.896 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:34.919 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:34.949 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:34.954 INFO:teuthology.orchestra.run.smithi165.stdout:(67/150): openblas-openmp-0.3.26-1.el9.x86_64.r 4.4 MB/s | 4.9 MB 00:01 2024-08-20T02:49:34.966 INFO:teuthology.orchestra.run.smithi099.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 297 kB/s | 18 kB 00:00 2024-08-20T02:49:34.996 INFO:teuthology.orchestra.run.smithi165.stdout:(68/150): postgresql-13.16-1.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-08-20T02:49:35.003 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:35.016 INFO:teuthology.orchestra.run.smithi099.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-08-20T02:49:35.035 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:35.059 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:35.075 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:35.108 INFO:teuthology.orchestra.run.smithi099.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-08-20T02:49:35.114 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:35.149 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:35.149 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T02:49:35.167 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T02:49:35.183 INFO:teuthology.orchestra.run.smithi099.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 604 kB/s | 45 kB 00:00 2024-08-20T02:49:35.213 INFO:teuthology.orchestra.run.smithi165.stdout:(69/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 3.6 MB/s | 4.4 MB 00:01 2024-08-20T02:49:35.256 INFO:teuthology.orchestra.run.smithi044.stdout:(66/150): openblas-0.3.26-1.el9.x86_64.rpm 48 kB/s | 38 kB 00:00 2024-08-20T02:49:35.300 INFO:teuthology.orchestra.run.smithi099.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 5.6 MB/s | 662 kB 00:00 2024-08-20T02:49:35.390 INFO:teuthology.orchestra.run.smithi044.stdout:(67/150): openblas-openmp-0.3.26-1.el9.x86_64.r 5.5 MB/s | 4.9 MB 00:00 2024-08-20T02:49:35.417 INFO:teuthology.orchestra.run.smithi099.stdout:(59/150): libquadmath-11.5.0-2.el9.x86_64.rpm 67 kB/s | 191 kB 00:02 2024-08-20T02:49:35.429 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:35.443 INFO:teuthology.orchestra.run.smithi099.stdout:(60/150): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-08-20T02:49:35.493 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:35.509 INFO:teuthology.orchestra.run.smithi099.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 276 kB/s | 25 kB 00:00 2024-08-20T02:49:35.518 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:35.535 INFO:teuthology.orchestra.run.smithi099.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 560 kB/s | 47 kB 00:00 2024-08-20T02:49:35.550 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:35.574 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:35.603 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:35.635 INFO:teuthology.orchestra.run.smithi099.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-08-20T02:49:35.669 INFO:teuthology.orchestra.run.smithi099.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-08-20T02:49:35.677 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:35.687 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:35.706 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:35.707 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:35.727 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:35.733 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:35.739 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:35.784 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:35.788 INFO:teuthology.orchestra.run.smithi165.stdout:(70/150): postgresql-private-libs-13.16-1.el9.x 174 kB/s | 137 kB 00:00 2024-08-20T02:49:35.794 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:35.824 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:35.836 INFO:teuthology.orchestra.run.smithi099.stdout:(65/150): libgfortran-11.5.0-2.el9.x86_64.rpm 225 kB/s | 801 kB 00:03 2024-08-20T02:49:35.843 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:35.843 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:35.878 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:35.878 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T02:49:35.894 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:35.899 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:35.910 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:35.916 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T02:49:35.931 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:35.934 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:35.939 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:35.946 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:35.959 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:35.963 INFO:teuthology.orchestra.run.smithi165.stdout:(71/150): postgresql-contrib-13.16-1.el9.x86_64 902 kB/s | 910 kB 00:01 2024-08-20T02:49:35.967 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:35.975 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:35.995 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T02:49:36.004 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:36.013 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:36.028 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:36.106 INFO:teuthology.orchestra.run.smithi044.stdout:(68/150): postgresql-contrib-13.16-1.el9.x86_64 1.2 MB/s | 910 kB 00:00 2024-08-20T02:49:36.144 INFO:teuthology.orchestra.run.smithi099.stdout:(66/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-08-20T02:49:36.145 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:36.148 INFO:teuthology.orchestra.run.smithi044.stdout:(69/150): postgresql-13.16-1.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-08-20T02:49:36.149 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:36.162 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:36.190 INFO:teuthology.orchestra.run.smithi044.stdout:(70/150): postgresql-private-libs-13.16-1.el9.x 1.7 MB/s | 137 kB 00:00 2024-08-20T02:49:36.191 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] postgresql-server-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:36.191 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:36.224 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:36.233 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:36.261 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:36.278 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T02:49:36.293 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:36.311 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:36.340 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:36.349 INFO:teuthology.orchestra.run.smithi044.stdout:(71/150): postgresql-server-13.16-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-08-20T02:49:36.370 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:36.450 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:36.475 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:36.516 INFO:teuthology.orchestra.run.smithi044.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 3.1 MB/s | 1.0 MB 00:00 2024-08-20T02:49:36.533 INFO:teuthology.orchestra.run.smithi044.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-08-20T02:49:36.539 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:36.551 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:36.581 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:36.583 INFO:teuthology.orchestra.run.smithi044.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-08-20T02:49:36.605 INFO:teuthology.orchestra.run.smithi165.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 136 kB/s | 87 kB 00:00 2024-08-20T02:49:36.612 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] python3-devel-3.9.19-7.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-devel-3.9.19-7.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:36.642 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:36.650 INFO:teuthology.orchestra.run.smithi044.stdout:(75/150): python3-devel-3.9.19-7.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-08-20T02:49:36.682 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:36.684 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:36.728 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:36.757 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:36.767 INFO:teuthology.orchestra.run.smithi044.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-08-20T02:49:36.783 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:36.801 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:36.813 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:36.843 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:36.851 INFO:teuthology.orchestra.run.smithi044.stdout:(77/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 1.9 MB/s | 4.4 MB 00:02 2024-08-20T02:49:36.860 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 172.65.90.23) 2024-08-20T02:49:36.865 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:36.867 INFO:teuthology.orchestra.run.smithi044.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-08-20T02:49:36.887 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:36.893 INFO:teuthology.orchestra.run.smithi044.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 459 kB/s | 19 kB 00:00 2024-08-20T02:49:36.918 INFO:teuthology.orchestra.run.smithi044.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-08-20T02:49:36.935 INFO:teuthology.orchestra.run.smithi044.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 828 kB/s | 35 kB 00:00 2024-08-20T02:49:36.950 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:37.069 INFO:teuthology.orchestra.run.smithi044.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-08-20T02:49:37.087 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 172.65.90.23) 2024-08-20T02:49:37.116 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.facebook.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:37.144 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:37.155 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 172.65.90.23) 2024-08-20T02:49:37.210 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:37.219 INFO:teuthology.orchestra.run.smithi044.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-08-20T02:49:37.303 INFO:teuthology.orchestra.run.smithi044.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 925 kB/s | 77 kB 00:00 2024-08-20T02:49:37.315 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.facebook.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:37.353 INFO:teuthology.orchestra.run.smithi044.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-08-20T02:49:37.377 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.facebook.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:37.452 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:37.517 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://linux-mirrors.fnal.gov/linux/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 131.225.105.75) 2024-08-20T02:49:37.522 INFO:teuthology.orchestra.run.smithi165.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 596 kB/s | 1.0 MB 00:01 2024-08-20T02:49:37.570 INFO:teuthology.orchestra.run.smithi044.stdout:(86/150): python3-babel-2.9.1-2.el9.noarch.rpm 5.7 MB/s | 6.0 MB 00:01 2024-08-20T02:49:37.596 INFO:teuthology.orchestra.run.smithi044.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 1.1 MB/s | 269 kB 00:00 2024-08-20T02:49:37.629 INFO:teuthology.orchestra.run.smithi044.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-08-20T02:49:37.630 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://linux-mirrors.fnal.gov/linux/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 131.225.105.75) 2024-08-20T02:49:37.647 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:37.679 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://linux-mirrors.fnal.gov/linux/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 131.225.105.75) 2024-08-20T02:49:37.680 INFO:teuthology.orchestra.run.smithi044.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-20T02:49:37.681 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ord.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:49:37.718 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:37.718 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://linux-mirrors.fnal.gov/linux/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 131.225.105.75) 2024-08-20T02:49:37.729 INFO:teuthology.orchestra.run.smithi044.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-20T02:49:37.750 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://linux-mirrors.fnal.gov/linux/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 131.225.105.75) 2024-08-20T02:49:37.770 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ord.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:49:37.772 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ord.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:49:37.780 INFO:teuthology.orchestra.run.smithi044.stdout:(91/150): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-08-20T02:49:37.782 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://linux-mirrors.fnal.gov/linux/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 131.225.105.75) 2024-08-20T02:49:37.834 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ord.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:49:37.838 INFO:teuthology.orchestra.run.smithi044.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 716 kB/s | 42 kB 00:00 2024-08-20T02:49:37.855 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ord.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:49:37.897 INFO:teuthology.orchestra.run.smithi044.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-08-20T02:49:37.922 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ord.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:49:38.147 INFO:teuthology.orchestra.run.smithi165.stdout:(74/150): postgresql-server-13.16-1.el9.x86_64. 2.0 MB/s | 5.8 MB 00:02 2024-08-20T02:49:38.164 INFO:teuthology.orchestra.run.smithi044.stdout:(94/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-08-20T02:49:38.174 INFO:teuthology.orchestra.run.smithi165.stdout:(75/150): python3-colorama-0.4.6-3.el9.noarch.r 94 kB/s | 61 kB 00:00 2024-08-20T02:49:38.189 INFO:teuthology.orchestra.run.smithi044.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 756 kB/s | 221 kB 00:00 2024-08-20T02:49:38.215 INFO:teuthology.orchestra.run.smithi044.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-08-20T02:49:38.248 INFO:teuthology.orchestra.run.smithi044.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-08-20T02:49:38.274 INFO:teuthology.orchestra.run.smithi044.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-08-20T02:49:38.299 INFO:teuthology.orchestra.run.smithi044.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-08-20T02:49:38.302 INFO:teuthology.orchestra.run.smithi044.stdout:[MIRROR] xfsprogs-devel-6.4.0-4.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/xfsprogs-devel-6.4.0-4.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:38.328 INFO:teuthology.orchestra.run.smithi099.stdout:(67/150): openblas-0.3.26-1.el9.x86_64.rpm 14 kB/s | 38 kB 00:02 2024-08-20T02:49:38.341 INFO:teuthology.orchestra.run.smithi044.stdout:(100/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-08-20T02:49:38.346 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:38.391 INFO:teuthology.orchestra.run.smithi044.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 692 kB/s | 64 kB 00:00 2024-08-20T02:49:38.403 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:38.416 INFO:teuthology.orchestra.run.smithi044.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-08-20T02:49:38.451 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:38.484 INFO:teuthology.orchestra.run.smithi044.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-08-20T02:49:38.525 INFO:teuthology.orchestra.run.smithi165.stdout:(76/150): python3-babel-2.9.1-2.el9.noarch.rpm 3.1 MB/s | 6.0 MB 00:01 2024-08-20T02:49:38.536 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T02:49:38.542 INFO:teuthology.orchestra.run.smithi044.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-08-20T02:49:38.567 INFO:teuthology.orchestra.run.smithi044.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-08-20T02:49:38.569 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:38.593 INFO:teuthology.orchestra.run.smithi044.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 785 kB/s | 19 kB 00:00 2024-08-20T02:49:38.603 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T02:49:38.643 INFO:teuthology.orchestra.run.smithi044.stdout:(107/150): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-08-20T02:49:38.662 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:38.693 INFO:teuthology.orchestra.run.smithi044.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-08-20T02:49:38.719 INFO:teuthology.orchestra.run.smithi044.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-20T02:49:38.734 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T02:49:38.744 INFO:teuthology.orchestra.run.smithi044.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-20T02:49:38.811 INFO:teuthology.orchestra.run.smithi044.stdout:(111/150): libarrow-9.0.0-11.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-08-20T02:49:38.878 INFO:teuthology.orchestra.run.smithi044.stdout:(112/150): python3-numpy-1.20.1-5.el9.x86_64.rp 2.6 MB/s | 5.1 MB 00:01 2024-08-20T02:49:38.894 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:38.903 INFO:teuthology.orchestra.run.smithi044.stdout:(113/150): luarocks-3.9.2-5.el9.noarch.rpm 952 kB/s | 151 kB 00:00 2024-08-20T02:49:38.936 INFO:teuthology.orchestra.run.smithi099.stdout:(68/150): postgresql-13.16-1.el9.x86_64.rpm 582 kB/s | 1.6 MB 00:02 2024-08-20T02:49:38.937 INFO:teuthology.orchestra.run.smithi044.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.4 MB/s | 43 kB 00:00 2024-08-20T02:49:38.962 INFO:teuthology.orchestra.run.smithi044.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-20T02:49:38.963 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:38.987 INFO:teuthology.orchestra.run.smithi044.stdout:(116/150): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-08-20T02:49:39.001 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:39.012 INFO:teuthology.orchestra.run.smithi044.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-08-20T02:49:39.033 INFO:teuthology.orchestra.run.smithi165.stdout:(77/150): python3-devel-3.9.19-7.el9.x86_64.rpm 276 kB/s | 245 kB 00:00 2024-08-20T02:49:39.038 INFO:teuthology.orchestra.run.smithi044.stdout:(118/150): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-08-20T02:49:39.040 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:39.067 INFO:teuthology.orchestra.run.smithi165.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 47 kB/s | 26 kB 00:00 2024-08-20T02:49:39.071 INFO:teuthology.orchestra.run.smithi044.stdout:(119/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.1 MB/s | 839 kB 00:00 2024-08-20T02:49:39.092 INFO:teuthology.orchestra.run.smithi165.stdout:(79/150): python3-jinja2-2.11.3-6.el9.noarch.rp 271 kB/s | 248 kB 00:00 2024-08-20T02:49:39.100 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:39.113 INFO:teuthology.orchestra.run.smithi044.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-08-20T02:49:39.127 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T02:49:39.139 INFO:teuthology.orchestra.run.smithi044.stdout:(121/150): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-08-20T02:49:39.164 INFO:teuthology.orchestra.run.smithi044.stdout:(122/150): python3-google-auth-2.33.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-08-20T02:49:39.189 INFO:teuthology.orchestra.run.smithi044.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-08-20T02:49:39.206 INFO:teuthology.orchestra.run.smithi044.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 256 kB/s | 11 kB 00:00 2024-08-20T02:49:39.223 INFO:teuthology.orchestra.run.smithi044.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 526 kB/s | 18 kB 00:00 2024-08-20T02:49:39.240 INFO:teuthology.orchestra.run.smithi044.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 694 kB/s | 23 kB 00:00 2024-08-20T02:49:39.257 INFO:teuthology.orchestra.run.smithi044.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 581 kB/s | 19 kB 00:00 2024-08-20T02:49:39.274 INFO:teuthology.orchestra.run.smithi044.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 584 kB/s | 20 kB 00:00 2024-08-20T02:49:39.291 INFO:teuthology.orchestra.run.smithi044.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 2.8 MB/s | 46 kB 00:00 2024-08-20T02:49:39.325 INFO:teuthology.orchestra.run.smithi044.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-08-20T02:49:39.342 INFO:teuthology.orchestra.run.smithi044.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-08-20T02:49:39.375 INFO:teuthology.orchestra.run.smithi044.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-08-20T02:49:39.401 INFO:teuthology.orchestra.run.smithi044.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-08-20T02:49:39.426 INFO:teuthology.orchestra.run.smithi044.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-08-20T02:49:39.431 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:39.451 INFO:teuthology.orchestra.run.smithi044.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-08-20T02:49:39.456 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T02:49:39.476 INFO:teuthology.orchestra.run.smithi044.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-08-20T02:49:39.477 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:39.499 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:39.501 INFO:teuthology.orchestra.run.smithi044.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-08-20T02:49:39.534 INFO:teuthology.orchestra.run.smithi165.stdout:(80/150): python3-jsonpointer-2.0-4.el9.noarch. 38 kB/s | 19 kB 00:00 2024-08-20T02:49:39.552 INFO:teuthology.orchestra.run.smithi044.stdout:(138/150): python3-grpcio-1.46.7-10.el9.x86_64. 4.6 MB/s | 2.0 MB 00:00 2024-08-20T02:49:39.579 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:39.586 INFO:teuthology.orchestra.run.smithi044.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 705 kB/s | 59 kB 00:00 2024-08-20T02:49:39.602 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T02:49:39.611 INFO:teuthology.orchestra.run.smithi044.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 607 kB/s | 36 kB 00:00 2024-08-20T02:49:39.636 INFO:teuthology.orchestra.run.smithi044.stdout:(141/150): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-08-20T02:49:39.642 INFO:teuthology.orchestra.run.smithi165.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 63 kB/s | 35 kB 00:00 2024-08-20T02:49:39.662 INFO:teuthology.orchestra.run.smithi044.stdout:(142/150): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-08-20T02:49:39.662 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 172.65.90.23) 2024-08-20T02:49:39.687 INFO:teuthology.orchestra.run.smithi044.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-08-20T02:49:39.712 INFO:teuthology.orchestra.run.smithi044.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-08-20T02:49:39.738 INFO:teuthology.orchestra.run.smithi044.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-08-20T02:49:39.739 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T02:49:39.763 INFO:teuthology.orchestra.run.smithi044.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-20T02:49:39.772 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 172.65.90.23) 2024-08-20T02:49:39.788 INFO:teuthology.orchestra.run.smithi044.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-20T02:49:39.838 INFO:teuthology.orchestra.run.smithi044.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-20T02:49:39.851 INFO:teuthology.orchestra.run.smithi165.stdout:(82/150): python3-mako-1.1.4-6.el9.noarch.rpm 220 kB/s | 172 kB 00:00 2024-08-20T02:49:39.897 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.facebook.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:39.932 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.facebook.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:40.048 INFO:teuthology.orchestra.run.smithi099.stdout:(69/150): openblas-openmp-0.3.26-1.el9.x86_64.r 1.1 MB/s | 4.9 MB 00:04 2024-08-20T02:49:40.049 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:40.087 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-server-13.16-1.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.16-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:40.104 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-server-13.16-1.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.16-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:40.114 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://linux-mirrors.fnal.gov/linux/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 131.225.105.75) 2024-08-20T02:49:40.140 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 172.65.90.21) 2024-08-20T02:49:40.147 INFO:teuthology.orchestra.run.smithi044.stdout:(149/150): librados2-19.0.0-5872.gef427541.el9. 9.0 MB/s | 3.5 MB 00:00 2024-08-20T02:49:40.170 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-server-13.16-1.el9.x86_64.rpm: Status code: 404 for http://linux-mirrors.fnal.gov/linux/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.16-1.el9.x86_64.rpm (IP: 131.225.105.75) 2024-08-20T02:49:40.222 INFO:teuthology.orchestra.run.smithi044.stdout:(150/150): librbd1-19.0.0-5872.gef427541.el9.x8 7.6 MB/s | 3.2 MB 00:00 2024-08-20T02:49:40.229 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-08-20T02:49:40.230 INFO:teuthology.orchestra.run.smithi044.stdout:Total 14 MB/s | 232 MB 00:16 2024-08-20T02:49:40.440 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:40.463 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:40.491 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ord.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:49:40.501 INFO:teuthology.orchestra.run.smithi165.stdout:(83/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 292 kB/s | 250 kB 00:00 2024-08-20T02:49:40.503 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-server-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.16-1.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T02:49:40.515 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ord.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 166.78.229.131) 2024-08-20T02:49:40.735 INFO:teuthology.orchestra.run.smithi165.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 252 kB/s | 222 kB 00:00 2024-08-20T02:49:41.160 INFO:teuthology.orchestra.run.smithi165.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 117 kB/s | 77 kB 00:00 2024-08-20T02:49:41.257 INFO:teuthology.orchestra.run.smithi099.stdout:(70/150): postgresql-private-libs-13.16-1.el9.x 59 kB/s | 137 kB 00:02 2024-08-20T02:49:41.358 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-08-20T02:49:41.368 INFO:teuthology.orchestra.run.smithi165.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 67 kB/s | 42 kB 00:00 2024-08-20T02:49:41.415 INFO:teuthology.orchestra.run.smithi099.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-08-20T02:49:41.490 INFO:teuthology.orchestra.run.smithi099.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-08-20T02:49:41.495 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-08-20T02:49:41.495 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-08-20T02:49:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-08-20T02:49:41.957 INFO:teuthology.orchestra.run.smithi099.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 823 kB/s | 61 kB 00:00 2024-08-20T02:49:42.010 INFO:teuthology.orchestra.run.smithi165.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 317 kB/s | 269 kB 00:00 2024-08-20T02:49:42.033 INFO:teuthology.orchestra.run.smithi099.stdout:(75/150): postgresql-server-13.16-1.el9.x86_64. 2.9 MB/s | 5.8 MB 00:01 2024-08-20T02:49:42.066 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] python3-devel-3.9.19-7.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-devel-3.9.19-7.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:42.069 INFO:teuthology.orchestra.run.smithi165.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 227 kB/s | 159 kB 00:00 2024-08-20T02:49:42.084 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] python3-devel-3.9.19-7.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-devel-3.9.19-7.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:42.124 INFO:teuthology.orchestra.run.smithi099.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-08-20T02:49:42.208 INFO:teuthology.orchestra.run.smithi099.stdout:(77/150): postgresql-contrib-13.16-1.el9.x86_64 234 kB/s | 910 kB 00:03 2024-08-20T02:49:42.225 INFO:teuthology.orchestra.run.smithi099.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-08-20T02:49:42.284 INFO:teuthology.orchestra.run.smithi099.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 256 kB/s | 19 kB 00:00 2024-08-20T02:49:42.334 INFO:teuthology.orchestra.run.smithi099.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-08-20T02:49:42.368 INFO:teuthology.orchestra.run.smithi099.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 414 kB/s | 35 kB 00:00 2024-08-20T02:49:42.436 INFO:teuthology.orchestra.run.smithi165.stdout:(89/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.8 MB/s | 5.1 MB 00:02 2024-08-20T02:49:42.460 INFO:teuthology.orchestra.run.smithi099.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-08-20T02:49:42.552 INFO:teuthology.orchestra.run.smithi099.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-08-20T02:49:42.635 INFO:teuthology.orchestra.run.smithi099.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 929 kB/s | 77 kB 00:00 2024-08-20T02:49:42.669 INFO:teuthology.orchestra.run.smithi165.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 85 kB/s | 51 kB 00:00 2024-08-20T02:49:42.744 INFO:teuthology.orchestra.run.smithi099.stdout:(85/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-08-20T02:49:42.769 INFO:teuthology.orchestra.run.smithi099.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 317 kB/s | 42 kB 00:00 2024-08-20T02:49:42.861 INFO:teuthology.orchestra.run.smithi099.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-08-20T02:49:42.887 INFO:teuthology.orchestra.run.smithi099.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-08-20T02:49:42.903 INFO:teuthology.orchestra.run.smithi165.stdout:(91/150): python3-pyasn1-modules-0.4.8-6.el9.no 313 kB/s | 279 kB 00:00 2024-08-20T02:49:42.962 INFO:teuthology.orchestra.run.smithi099.stdout:(89/150): python3-devel-3.9.19-7.el9.x86_64.rpm 244 kB/s | 245 kB 00:01 2024-08-20T02:49:42.987 INFO:teuthology.orchestra.run.smithi099.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 546 kB/s | 51 kB 00:00 2024-08-20T02:49:43.029 INFO:teuthology.orchestra.run.smithi099.stdout:(91/150): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-08-20T02:49:43.063 INFO:teuthology.orchestra.run.smithi099.stdout:(92/150): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-08-20T02:49:43.070 INFO:teuthology.orchestra.run.smithi165.stdout:(92/150): python3-requests-oauthlib-1.3.0-12.el 85 kB/s | 54 kB 00:00 2024-08-20T02:49:43.138 INFO:teuthology.orchestra.run.smithi099.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 386 kB/s | 42 kB 00:00 2024-08-20T02:49:43.163 INFO:teuthology.orchestra.run.smithi099.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 473 kB/s | 44 kB 00:00 2024-08-20T02:49:43.238 INFO:teuthology.orchestra.run.smithi099.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 529 kB/s | 36 kB 00:00 2024-08-20T02:49:43.264 INFO:teuthology.orchestra.run.smithi099.stdout:(96/150): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-08-20T02:49:43.339 INFO:teuthology.orchestra.run.smithi099.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-08-20T02:49:43.398 INFO:teuthology.orchestra.run.smithi099.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 2.3 MB/s | 304 kB 00:00 2024-08-20T02:49:43.423 INFO:teuthology.orchestra.run.smithi099.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 694 kB/s | 58 kB 00:00 2024-08-20T02:49:43.457 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] xfsprogs-devel-6.4.0-4.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/xfsprogs-devel-6.4.0-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:43.474 INFO:teuthology.orchestra.run.smithi099.stdout:[MIRROR] xfsprogs-devel-6.4.0-4.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/xfsprogs-devel-6.4.0-4.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T02:49:43.506 INFO:teuthology.orchestra.run.smithi099.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 836 kB/s | 64 kB 00:00 2024-08-20T02:49:43.520 INFO:teuthology.orchestra.run.smithi165.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 68 kB/s | 42 kB 00:00 2024-08-20T02:49:43.582 INFO:teuthology.orchestra.run.smithi099.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 599 kB/s | 45 kB 00:00 2024-08-20T02:49:43.637 INFO:teuthology.orchestra.run.smithi165.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 78 kB/s | 44 kB 00:00 2024-08-20T02:49:43.649 INFO:teuthology.orchestra.run.smithi099.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 335 kB/s | 22 kB 00:00 2024-08-20T02:49:43.755 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-08-20T02:49:43.755 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-08-20T02:49:43.816 INFO:teuthology.orchestra.run.smithi099.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-08-20T02:49:43.866 INFO:teuthology.orchestra.run.smithi099.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-20T02:49:43.891 INFO:teuthology.orchestra.run.smithi099.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-08-20T02:49:43.925 INFO:teuthology.orchestra.run.smithi099.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 580 kB/s | 19 kB 00:00 2024-08-20T02:49:44.092 INFO:teuthology.orchestra.run.smithi099.stdout:(107/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-08-20T02:49:44.126 INFO:teuthology.orchestra.run.smithi099.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 788 kB/s | 26 kB 00:00 2024-08-20T02:49:44.151 INFO:teuthology.orchestra.run.smithi099.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-20T02:49:44.176 INFO:teuthology.orchestra.run.smithi099.stdout:(110/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 123 kB/s | 96 kB 00:00 2024-08-20T02:49:44.196 INFO:teuthology.orchestra.run.smithi165.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 64 kB/s | 36 kB 00:00 2024-08-20T02:49:44.201 INFO:teuthology.orchestra.run.smithi099.stdout:(111/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-08-20T02:49:44.293 INFO:teuthology.orchestra.run.smithi099.stdout:(112/150): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-08-20T02:49:44.319 INFO:teuthology.orchestra.run.smithi099.stdout:(113/150): luarocks-3.9.2-5.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-08-20T02:49:44.361 INFO:teuthology.orchestra.run.smithi099.stdout:(114/150): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-08-20T02:49:44.379 INFO:teuthology.orchestra.run.smithi165.stdout:(96/150): qatlib-24.02.0-1.el9.x86_64.rpm 257 kB/s | 221 kB 00:00 2024-08-20T02:49:44.386 INFO:teuthology.orchestra.run.smithi099.stdout:(115/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 649 kB/s | 43 kB 00:00 2024-08-20T02:49:44.411 INFO:teuthology.orchestra.run.smithi099.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-08-20T02:49:44.437 INFO:teuthology.orchestra.run.smithi099.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-08-20T02:49:44.462 INFO:teuthology.orchestra.run.smithi099.stdout:(118/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.2 MB/s | 839 kB 00:00 2024-08-20T02:49:44.487 INFO:teuthology.orchestra.run.smithi099.stdout:(119/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-08-20T02:49:44.512 INFO:teuthology.orchestra.run.smithi099.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-08-20T02:49:44.563 INFO:teuthology.orchestra.run.smithi099.stdout:(121/150): python3-google-auth-2.33.0-1.el9.noa 2.8 MB/s | 220 kB 00:00 2024-08-20T02:49:44.588 INFO:teuthology.orchestra.run.smithi099.stdout:(122/150): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-08-20T02:49:44.613 INFO:teuthology.orchestra.run.smithi099.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-08-20T02:49:44.639 INFO:teuthology.orchestra.run.smithi099.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-08-20T02:49:44.664 INFO:teuthology.orchestra.run.smithi099.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-08-20T02:49:44.689 INFO:teuthology.orchestra.run.smithi099.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-08-20T02:49:44.714 INFO:teuthology.orchestra.run.smithi099.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-08-20T02:49:44.740 INFO:teuthology.orchestra.run.smithi099.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-08-20T02:49:44.765 INFO:teuthology.orchestra.run.smithi099.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-20T02:49:44.771 INFO:teuthology.orchestra.run.smithi165.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-08-20T02:49:44.790 INFO:teuthology.orchestra.run.smithi099.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-20T02:49:44.815 INFO:teuthology.orchestra.run.smithi099.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-08-20T02:49:44.849 INFO:teuthology.orchestra.run.smithi099.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-08-20T02:49:44.874 INFO:teuthology.orchestra.run.smithi099.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-08-20T02:49:44.900 INFO:teuthology.orchestra.run.smithi099.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-08-20T02:49:44.926 INFO:teuthology.orchestra.run.smithi099.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-08-20T02:49:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-08-20T02:49:44.976 INFO:teuthology.orchestra.run.smithi099.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-08-20T02:49:45.001 INFO:teuthology.orchestra.run.smithi099.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-20T02:49:45.027 INFO:teuthology.orchestra.run.smithi099.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-08-20T02:49:45.135 INFO:teuthology.orchestra.run.smithi099.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 268 kB/s | 36 kB 00:00 2024-08-20T02:49:45.219 INFO:teuthology.orchestra.run.smithi099.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 388 kB/s | 74 kB 00:00 2024-08-20T02:49:45.269 INFO:teuthology.orchestra.run.smithi099.stdout:(142/150): python3-grpcio-1.46.7-10.el9.x86_64. 2.7 MB/s | 2.0 MB 00:00 2024-08-20T02:49:45.295 INFO:teuthology.orchestra.run.smithi099.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-08-20T02:49:45.320 INFO:teuthology.orchestra.run.smithi099.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-08-20T02:49:45.337 INFO:teuthology.orchestra.run.smithi099.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 476 kB/s | 20 kB 00:00 2024-08-20T02:49:45.362 INFO:teuthology.orchestra.run.smithi099.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-08-20T02:49:45.396 INFO:teuthology.orchestra.run.smithi099.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-20T02:49:45.413 INFO:teuthology.orchestra.run.smithi165.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 90 kB/s | 58 kB 00:00 2024-08-20T02:49:45.446 INFO:teuthology.orchestra.run.smithi165.stdout:(99/150): socat-1.7.4.1-6.el9.x86_64.rpm 285 kB/s | 304 kB 00:01 2024-08-20T02:49:45.454 INFO:teuthology.orchestra.run.smithi099.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-08-20T02:49:45.722 INFO:teuthology.orchestra.run.smithi165.stdout:(100/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.3 MB/s | 16 MB 00:03 2024-08-20T02:49:45.763 INFO:teuthology.orchestra.run.smithi099.stdout:(149/150): librados2-19.0.0-5872.gef427541.el9. 8.7 MB/s | 3.5 MB 00:00 2024-08-20T02:49:45.830 INFO:teuthology.orchestra.run.smithi099.stdout:(150/150): librbd1-19.0.0-5872.gef427541.el9.x8 7.6 MB/s | 3.2 MB 00:00 2024-08-20T02:49:45.839 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-08-20T02:49:45.839 INFO:teuthology.orchestra.run.smithi099.stdout:Total 11 MB/s | 232 MB 00:20 2024-08-20T02:49:46.230 INFO:teuthology.orchestra.run.smithi165.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 81 kB/s | 64 kB 00:00 2024-08-20T02:49:46.255 INFO:teuthology.orchestra.run.smithi165.stdout:(102/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 114 kB/s | 96 kB 00:00 2024-08-20T02:49:46.339 INFO:teuthology.orchestra.run.smithi165.stdout:(103/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-08-20T02:49:46.381 INFO:teuthology.orchestra.run.smithi165.stdout:(104/150): lua-devel-5.4.4-4.el9.x86_64.rpm 150 kB/s | 22 kB 00:00 2024-08-20T02:49:46.439 INFO:teuthology.orchestra.run.smithi165.stdout:(105/150): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-08-20T02:49:46.481 INFO:teuthology.orchestra.run.smithi165.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-08-20T02:49:46.515 INFO:teuthology.orchestra.run.smithi165.stdout:(107/150): abseil-cpp-20211102.0-3.el9.x86_64.r 3.0 MB/s | 548 kB 00:00 2024-08-20T02:49:46.540 INFO:teuthology.orchestra.run.smithi165.stdout:(108/150): grpc-data-1.46.7-10.el9.noarch.rpm 197 kB/s | 19 kB 00:00 2024-08-20T02:49:46.565 INFO:teuthology.orchestra.run.smithi165.stdout:(109/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-08-20T02:49:46.591 INFO:teuthology.orchestra.run.smithi165.stdout:(110/150): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-08-20T02:49:46.616 INFO:teuthology.orchestra.run.smithi165.stdout:(111/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-08-20T02:49:46.641 INFO:teuthology.orchestra.run.smithi165.stdout:(112/150): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-08-20T02:49:46.683 INFO:teuthology.orchestra.run.smithi165.stdout:(113/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-08-20T02:49:46.709 INFO:teuthology.orchestra.run.smithi165.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-20T02:49:46.734 INFO:teuthology.orchestra.run.smithi165.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-20T02:49:46.759 INFO:teuthology.orchestra.run.smithi165.stdout:(116/150): python3-certifi-2023.05.07-4.el9.noa 570 kB/s | 14 kB 00:00 2024-08-20T02:49:46.784 INFO:teuthology.orchestra.run.smithi165.stdout:(117/150): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-08-20T02:49:46.809 INFO:teuthology.orchestra.run.smithi165.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-08-20T02:49:46.843 INFO:teuthology.orchestra.run.smithi165.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-08-20T02:49:46.920 INFO:teuthology.orchestra.run.smithi165.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-08-20T02:49:46.944 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-08-20T02:49:46.952 INFO:teuthology.orchestra.run.smithi165.stdout:(121/150): python3-google-auth-2.33.0-1.el9.noa 2.0 MB/s | 220 kB 00:00 2024-08-20T02:49:46.986 INFO:teuthology.orchestra.run.smithi165.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-08-20T02:49:47.011 INFO:teuthology.orchestra.run.smithi165.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-08-20T02:49:47.036 INFO:teuthology.orchestra.run.smithi165.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-08-20T02:49:47.062 INFO:teuthology.orchestra.run.smithi165.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 931 kB/s | 23 kB 00:00 2024-08-20T02:49:47.073 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-08-20T02:49:47.073 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-08-20T02:49:47.087 INFO:teuthology.orchestra.run.smithi165.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-08-20T02:49:47.112 INFO:teuthology.orchestra.run.smithi165.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-08-20T02:49:47.171 INFO:teuthology.orchestra.run.smithi165.stdout:(128/150): libarrow-9.0.0-11.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-08-20T02:49:47.213 INFO:teuthology.orchestra.run.smithi165.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-08-20T02:49:47.229 INFO:teuthology.orchestra.run.smithi165.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 4.7 MB/s | 79 kB 00:00 2024-08-20T02:49:47.255 INFO:teuthology.orchestra.run.smithi165.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-20T02:49:47.305 INFO:teuthology.orchestra.run.smithi165.stdout:(132/150): python3-grpcio-1.46.7-10.el9.x86_64. 5.3 MB/s | 2.0 MB 00:00 2024-08-20T02:49:47.330 INFO:teuthology.orchestra.run.smithi165.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-08-20T02:49:47.355 INFO:teuthology.orchestra.run.smithi165.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 329 kB/s | 16 kB 00:00 2024-08-20T02:49:47.381 INFO:teuthology.orchestra.run.smithi165.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-08-20T02:49:47.406 INFO:teuthology.orchestra.run.smithi165.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-08-20T02:49:47.431 INFO:teuthology.orchestra.run.smithi165.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-08-20T02:49:47.465 INFO:teuthology.orchestra.run.smithi165.stdout:(138/150): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-08-20T02:49:47.490 INFO:teuthology.orchestra.run.smithi165.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-08-20T02:49:47.516 INFO:teuthology.orchestra.run.smithi165.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 717 kB/s | 36 kB 00:00 2024-08-20T02:49:47.532 INFO:teuthology.orchestra.run.smithi165.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-08-20T02:49:47.558 INFO:teuthology.orchestra.run.smithi165.stdout:(142/150): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-08-20T02:49:47.583 INFO:teuthology.orchestra.run.smithi165.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-08-20T02:49:47.608 INFO:teuthology.orchestra.run.smithi165.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-08-20T02:49:47.625 INFO:teuthology.orchestra.run.smithi165.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-08-20T02:49:47.659 INFO:teuthology.orchestra.run.smithi165.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 5.7 MB/s | 192 kB 00:00 2024-08-20T02:49:47.684 INFO:teuthology.orchestra.run.smithi165.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-20T02:49:47.743 INFO:teuthology.orchestra.run.smithi165.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-08-20T02:49:48.034 INFO:teuthology.orchestra.run.smithi165.stdout:(149/150): librados2-19.0.0-5872.gef427541.el9. 9.2 MB/s | 3.5 MB 00:00 2024-08-20T02:49:48.109 INFO:teuthology.orchestra.run.smithi165.stdout:(150/150): librbd1-19.0.0-5872.gef427541.el9.x8 7.7 MB/s | 3.2 MB 00:00 2024-08-20T02:49:48.119 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-08-20T02:49:48.119 INFO:teuthology.orchestra.run.smithi165.stdout:Total 9.3 MB/s | 232 MB 00:25 2024-08-20T02:49:49.245 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-08-20T02:49:49.276 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-08-20T02:49:49.276 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-08-20T02:49:49.377 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-08-20T02:49:49.377 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-08-20T02:49:49.809 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-08-20T02:49:49.884 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-20T02:49:49.909 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-20T02:49:49.984 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-20T02:49:50.397 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-20T02:49:50.484 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T02:49:50.614 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T02:49:50.644 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T02:49:50.741 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T02:49:50.787 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T02:49:50.815 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-20T02:49:50.842 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-20T02:49:51.064 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-20T02:49:51.114 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-20T02:49:51.141 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T02:49:51.234 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T02:49:51.259 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T02:49:51.306 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T02:49:51.327 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T02:49:51.352 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T02:49:51.592 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-08-20T02:49:51.592 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-08-20T02:49:51.683 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T02:49:51.719 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T02:49:51.831 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-20T02:49:51.850 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-20T02:49:51.894 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-20T02:49:51.971 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-20T02:49:52.021 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-20T02:49:52.061 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-20T02:49:52.104 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-20T02:49:52.155 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-20T02:49:52.193 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-20T02:49:52.289 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-20T02:49:52.347 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-20T02:49:52.385 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-20T02:49:52.430 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-20T02:49:52.507 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-20T02:49:52.547 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 30/152 2024-08-20T02:49:52.593 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 31/152 2024-08-20T02:49:52.619 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 32/152 2024-08-20T02:49:52.693 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-20T02:49:52.734 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-20T02:49:52.803 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-20T02:49:52.952 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-20T02:49:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-20T02:49:53.587 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-20T02:49:53.678 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-20T02:49:53.711 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-20T02:49:53.755 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-20T02:49:53.793 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 42/152 2024-08-20T02:49:53.826 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-20T02:49:53.922 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-20T02:49:53.984 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-20T02:49:54.058 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-20T02:49:54.095 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-20T02:49:54.131 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-20T02:49:54.179 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-20T02:49:54.214 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-20T02:49:54.257 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-20T02:49:54.300 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-20T02:49:54.444 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-20T02:49:54.484 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-20T02:49:54.518 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-20T02:49:54.628 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-20T02:49:54.669 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-20T02:49:54.710 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-20T02:49:54.750 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-20T02:49:54.774 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-20T02:49:54.863 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-20T02:49:54.991 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-20T02:49:55.024 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-20T02:49:55.056 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-20T02:49:55.087 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-20T02:49:55.135 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-20T02:49:55.177 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-20T02:49:55.217 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-20T02:49:55.355 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-20T02:49:55.390 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-20T02:49:55.405 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-08-20T02:49:55.423 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-20T02:49:55.466 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-20T02:49:55.473 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-20T02:49:55.498 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-20T02:49:55.499 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-20T02:49:55.584 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-20T02:49:55.600 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-20T02:49:55.656 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-20T02:49:55.694 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-20T02:49:55.805 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-20T02:49:55.864 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-20T02:49:55.940 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-20T02:49:56.041 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T02:49:56.165 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T02:49:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T02:49:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T02:49:56.335 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T02:49:56.365 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-20T02:49:56.389 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-20T02:49:56.589 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-20T02:49:56.642 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-20T02:49:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-20T02:49:56.684 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-20T02:49:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T02:49:56.743 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-20T02:49:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T02:49:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T02:49:56.856 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-20T02:49:56.880 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T02:49:56.907 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.19-7.el9.x86_64 83/152 2024-08-20T02:49:56.908 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T02:49:56.934 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T02:49:56.953 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-20T02:49:57.043 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-20T02:49:57.079 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-20T02:49:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T02:49:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T02:49:57.272 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.26-1.el9.x86_64 87/152 2024-08-20T02:49:57.301 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.26-1.el9.x86_64 88/152 2024-08-20T02:49:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-20T02:49:57.373 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-20T02:49:57.382 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-20T02:49:57.408 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-20T02:49:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-20T02:49:57.523 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-08-20T02:49:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-20T02:49:57.575 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-20T02:49:57.595 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-20T02:49:57.618 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-20T02:49:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-20T02:49:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-20T02:49:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-20T02:49:57.712 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-20T02:49:57.756 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-20T02:49:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-20T02:49:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-20T02:49:57.883 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-20T02:49:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-20T02:49:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 30/152 2024-08-20T02:49:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 31/152 2024-08-20T02:49:58.066 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 32/152 2024-08-20T02:49:58.089 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-20T02:49:58.102 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-20T02:49:58.138 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-20T02:49:58.180 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T02:49:58.182 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-20T02:49:58.198 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-20T02:49:58.258 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-20T02:49:58.312 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T02:49:58.347 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T02:49:58.392 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-20T02:49:58.444 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T02:49:58.451 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-20T02:49:58.482 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T02:49:58.510 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-20T02:49:58.536 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-20T02:49:58.722 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-20T02:49:58.785 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-20T02:49:58.810 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T02:49:58.916 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T02:49:58.945 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T02:49:58.995 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-20T02:49:59.009 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T02:49:59.037 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T02:49:59.060 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T02:49:59.073 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-20T02:49:59.117 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-20T02:49:59.160 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-20T02:49:59.198 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 42/152 2024-08-20T02:49:59.231 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-20T02:49:59.325 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-20T02:49:59.382 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T02:49:59.389 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-20T02:49:59.412 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T02:49:59.462 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-20T02:49:59.500 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-20T02:49:59.519 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-20T02:49:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-20T02:49:59.545 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-20T02:49:59.564 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-20T02:49:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-20T02:49:59.603 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-20T02:49:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-20T02:49:59.634 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T02:49:59.655 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-20T02:49:59.673 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-20T02:49:59.677 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-20T02:49:59.696 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-20T02:49:59.703 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-20T02:49:59.730 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-20T02:49:59.762 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-20T02:49:59.804 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-20T02:49:59.838 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-20T02:49:59.838 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-20T02:49:59.865 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-20T02:49:59.890 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-20T02:49:59.932 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-20T02:49:59.933 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-20T02:49:59.987 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-20T02:50:00.018 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-20T02:50:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-20T02:50:00.048 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-20T02:50:00.061 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-20T02:50:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-20T02:50:00.099 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T02:50:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-20T02:50:00.138 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-20T02:50:00.155 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-20T02:50:00.172 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 30/152 2024-08-20T02:50:00.180 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-20T02:50:00.195 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T02:50:00.218 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 31/152 2024-08-20T02:50:00.221 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 98/152 2024-08-20T02:50:00.244 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 32/152 2024-08-20T02:50:00.257 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-20T02:50:00.285 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-20T02:50:00.316 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-20T02:50:00.358 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-20T02:50:00.409 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-20T02:50:00.426 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-20T02:50:00.447 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-20T02:50:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-20T02:50:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-20T02:50:00.562 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-20T02:50:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-20T02:50:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-20T02:50:00.620 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-20T02:50:00.647 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-20T02:50:00.677 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-20T02:50:00.711 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-20T02:50:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-20T02:50:00.781 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-20T02:50:00.787 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-20T02:50:00.810 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T02:50:00.821 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-20T02:50:00.875 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T02:50:00.904 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 102/152 2024-08-20T02:50:00.910 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-20T02:50:01.062 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-20T02:50:01.100 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-20T02:50:01.152 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-20T02:50:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-20T02:50:01.241 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-20T02:50:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-20T02:50:01.277 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-20T02:50:01.320 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-20T02:50:01.357 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 42/152 2024-08-20T02:50:01.390 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-20T02:50:01.484 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-20T02:50:01.547 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-20T02:50:01.620 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-20T02:50:01.651 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-20T02:50:01.687 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-20T02:50:01.735 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-20T02:50:01.769 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-20T02:50:01.812 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-20T02:50:01.846 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-20T02:50:01.994 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-20T02:50:02.000 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-20T02:50:02.048 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-20T02:50:02.048 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-20T02:50:02.082 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-20T02:50:02.089 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-20T02:50:02.176 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-20T02:50:02.194 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-20T02:50:02.224 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-20T02:50:02.229 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-devel-3.9.19-7.el9.x86_64 83/152 2024-08-20T02:50:02.256 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-20T02:50:02.275 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-20T02:50:02.282 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-20T02:50:02.304 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-20T02:50:02.364 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-20T02:50:02.391 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-20T02:50:02.409 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-20T02:50:02.525 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-20T02:50:02.562 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-20T02:50:02.585 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.26-1.el9.x86_64 87/152 2024-08-20T02:50:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-20T02:50:02.607 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-openmp-0.3.26-1.el9.x86_64 88/152 2024-08-20T02:50:02.625 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-20T02:50:02.675 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-20T02:50:02.685 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-20T02:50:02.713 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-20T02:50:02.745 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-20T02:50:02.775 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-20T02:50:02.809 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-20T02:50:02.843 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-20T02:50:02.884 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-20T02:50:02.910 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-20T02:50:02.990 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-20T02:50:03.041 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-20T02:50:03.075 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-20T02:50:03.178 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-20T02:50:03.231 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-20T02:50:03.300 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T02:50:03.334 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T02:50:03.360 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-20T02:50:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-20T02:50:03.919 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T02:50:03.954 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T02:50:04.003 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-20T02:50:04.044 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-20T02:50:04.052 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T02:50:04.053 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-20T02:50:04.053 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-20T02:50:04.053 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:04.078 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-20T02:50:04.089 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T02:50:04.181 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-20T02:50:04.242 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-devel-3.9.19-7.el9.x86_64 83/152 2024-08-20T02:50:04.296 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-20T02:50:04.375 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-20T02:50:04.414 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-20T02:50:04.602 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.26-1.el9.x86_64 87/152 2024-08-20T02:50:04.628 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-openmp-0.3.26-1.el9.x86_64 88/152 2024-08-20T02:50:04.707 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-20T02:50:04.768 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-20T02:50:04.849 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T02:50:04.880 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-20T02:50:04.909 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-20T02:50:05.233 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-20T02:50:05.281 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T02:50:05.374 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-20T02:50:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T02:50:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 98/152 2024-08-20T02:50:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-20T02:50:05.495 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-20T02:50:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-20T02:50:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T02:50:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T02:50:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 102/152 2024-08-20T02:50:06.787 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-20T02:50:06.889 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T02:50:06.921 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-20T02:50:06.950 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-20T02:50:07.274 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-20T02:50:07.314 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T02:50:07.524 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T02:50:07.634 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 98/152 2024-08-20T02:50:07.671 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-20T02:50:08.165 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-20T02:50:08.190 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T02:50:08.246 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T02:50:08.267 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 102/152 2024-08-20T02:50:08.411 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T02:50:08.440 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T02:50:08.998 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T02:50:09.036 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T02:50:09.119 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T02:50:09.119 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-20T02:50:09.119 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-20T02:50:09.119 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:09.154 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T02:50:10.562 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T02:50:10.604 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T02:50:11.156 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T02:50:11.199 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T02:50:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T02:50:11.295 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-20T02:50:11.295 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-20T02:50:11.295 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:11.334 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T02:50:16.017 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T02:50:16.017 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-08-20T02:50:16.017 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-08-20T02:50:16.017 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-08-20T02:50:16.017 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-08-20T02:50:16.017 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-08-20T02:50:16.017 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-08-20T02:50:16.017 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-08-20T02:50:16.018 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:16.244 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T02:50:16.260 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T02:50:16.260 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:16.260 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-20T02:50:16.261 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:17.822 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T02:50:17.849 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T02:50:17.849 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:17.849 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-20T02:50:17.850 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T02:50:17.850 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T02:50:17.850 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:18.081 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T02:50:18.104 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T02:50:18.104 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:18.104 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-20T02:50:18.105 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T02:50:18.105 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T02:50:18.105 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:18.581 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T02:50:18.604 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T02:50:18.605 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:18.605 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-20T02:50:18.605 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T02:50:18.605 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T02:50:18.605 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:18.721 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-20T02:50:18.779 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-20T02:50:18.842 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-20T02:50:19.036 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-20T02:50:19.092 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-20T02:50:19.173 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-20T02:50:19.262 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-20T02:50:19.405 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-20T02:50:19.454 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-20T02:50:19.490 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-20T02:50:19.517 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-20T02:50:19.897 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T02:50:19.924 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T02:50:19.957 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 122/152 2024-08-20T02:50:20.032 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 123/152 2024-08-20T02:50:20.064 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T02:50:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T02:50:20.775 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2024-08-20T02:50:20.775 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2024-08-20T02:50:20.775 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2024-08-20T02:50:20.776 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2024-08-20T02:50:20.776 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2024-08-20T02:50:20.776 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /root 2024-08-20T02:50:20.776 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2024-08-20T02:50:20.776 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:20.952 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T02:50:20.968 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T02:50:20.969 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:20.969 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-20T02:50:20.969 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:22.472 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T02:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T02:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-20T02:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T02:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T02:50:22.499 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T02:50:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T02:50:22.746 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:22.747 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-20T02:50:22.747 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T02:50:22.747 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T02:50:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:23.120 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T02:50:23.120 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-08-20T02:50:23.120 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-08-20T02:50:23.121 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-08-20T02:50:23.121 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-08-20T02:50:23.121 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-08-20T02:50:23.121 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /root 2024-08-20T02:50:23.121 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-08-20T02:50:23.121 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:23.198 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T02:50:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T02:50:23.222 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:23.222 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-20T02:50:23.222 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T02:50:23.222 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T02:50:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:23.300 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T02:50:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T02:50:23.317 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:23.317 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-20T02:50:23.317 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-20T02:50:23.377 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-20T02:50:23.413 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T02:50:23.432 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-20T02:50:23.470 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T02:50:23.617 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-20T02:50:23.657 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-20T02:50:23.727 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-20T02:50:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-20T02:50:23.972 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-20T02:50:24.013 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-20T02:50:24.048 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-20T02:50:24.076 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-20T02:50:24.485 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T02:50:24.497 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T02:50:24.512 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T02:50:24.525 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T02:50:24.557 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 122/152 2024-08-20T02:50:24.622 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 123/152 2024-08-20T02:50:24.630 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T02:50:24.657 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T02:50:24.799 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 127/152 2024-08-20T02:50:24.823 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T02:50:24.829 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T02:50:24.846 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T02:50:24.847 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:24.847 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-20T02:50:24.847 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T02:50:24.847 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T02:50:24.847 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:24.856 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T02:50:24.857 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:24.857 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-20T02:50:24.857 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T02:50:24.857 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T02:50:24.857 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:24.899 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T02:50:24.910 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T02:50:25.012 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:19.0.0-5872.gef427541.el9.x86_64 130/152 2024-08-20T02:50:25.075 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 131/152 2024-08-20T02:50:25.080 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T02:50:25.104 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T02:50:25.104 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:25.104 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-20T02:50:25.104 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T02:50:25.105 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T02:50:25.105 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:25.556 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T02:50:25.579 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T02:50:25.579 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:25.580 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-20T02:50:25.580 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T02:50:25.580 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T02:50:25.580 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:25.702 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-20T02:50:25.760 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-20T02:50:25.806 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-20T02:50:25.992 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-20T02:50:26.039 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-20T02:50:26.093 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-20T02:50:26.117 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-20T02:50:26.121 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T02:50:26.146 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T02:50:26.146 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:26.146 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-20T02:50:26.146 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T02:50:26.146 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T02:50:26.147 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:26.189 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T02:50:26.205 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-20T02:50:26.212 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T02:50:26.212 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:26.212 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-20T02:50:26.212 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:26.264 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T02:50:26.288 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T02:50:26.288 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:26.288 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-20T02:50:26.288 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T02:50:26.288 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T02:50:26.289 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:26.346 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-20T02:50:26.398 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-20T02:50:26.430 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-20T02:50:26.449 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-20T02:50:26.615 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T02:50:26.638 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T02:50:26.638 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:26.639 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-20T02:50:26.639 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T02:50:26.639 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T02:50:26.639 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:26.820 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T02:50:26.847 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T02:50:26.879 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 122/152 2024-08-20T02:50:26.945 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 123/152 2024-08-20T02:50:26.978 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T02:50:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T02:50:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T02:50:28.821 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T02:50:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T02:50:28.959 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T02:50:29.125 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 127/152 2024-08-20T02:50:29.157 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T02:50:29.181 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T02:50:29.181 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:29.181 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-20T02:50:29.181 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T02:50:29.181 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T02:50:29.181 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:29.233 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T02:50:29.244 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T02:50:29.343 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:19.0.0-5872.gef427541.el9.x86_64 130/152 2024-08-20T02:50:29.396 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 131/152 2024-08-20T02:50:30.176 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T02:50:30.242 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T02:50:30.367 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-20T02:50:30.394 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T02:50:30.417 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T02:50:30.417 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:30.418 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-20T02:50:30.418 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T02:50:30.418 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T02:50:30.418 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:30.461 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T02:50:30.484 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T02:50:30.484 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:30.484 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-20T02:50:30.484 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T02:50:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T02:50:30.560 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:30.560 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-20T02:50:30.560 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T02:50:30.560 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T02:50:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:30.871 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T02:50:30.894 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T02:50:30.894 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:30.894 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-20T02:50:30.894 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T02:50:30.894 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T02:50:30.894 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:31.209 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T02:50:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T02:50:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T02:50:31.523 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 127/152 2024-08-20T02:50:31.554 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T02:50:31.576 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T02:50:31.577 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:31.577 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-20T02:50:31.577 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T02:50:31.577 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T02:50:31.577 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:31.611 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 137/152 2024-08-20T02:50:31.628 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T02:50:31.641 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T02:50:31.653 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 138/152 2024-08-20T02:50:31.707 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 139/152 2024-08-20T02:50:31.738 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:19.0.0-5872.gef427541.el9.x86_64 130/152 2024-08-20T02:50:31.742 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-20T02:50:31.769 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-20T02:50:31.794 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 131/152 2024-08-20T02:50:31.974 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 142/152 2024-08-20T02:50:32.023 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-20T02:50:32.053 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 144/152 2024-08-20T02:50:32.691 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T02:50:32.726 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T02:50:32.751 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T02:50:32.757 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-20T02:50:32.789 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T02:50:32.813 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T02:50:32.813 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:32.813 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-20T02:50:32.813 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T02:50:32.813 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T02:50:32.813 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:32.856 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T02:50:32.880 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T02:50:32.880 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:32.880 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-20T02:50:32.880 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:32.932 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T02:50:32.955 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T02:50:32.956 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:32.956 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-20T02:50:32.956 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T02:50:32.956 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T02:50:32.956 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:33.274 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T02:50:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T02:50:33.298 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T02:50:33.298 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-20T02:50:33.298 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T02:50:33.298 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T02:50:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 146/152 2024-08-20T02:50:33.723 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-20T02:50:33.777 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-20T02:50:33.874 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-20T02:50:33.927 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-20T02:50:33.927 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T02:50:33.994 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T02:50:33.995 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T02:50:35.666 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 137/152 2024-08-20T02:50:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 138/152 2024-08-20T02:50:35.769 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 139/152 2024-08-20T02:50:35.798 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-20T02:50:35.824 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-20T02:50:36.027 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 142/152 2024-08-20T02:50:36.066 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-20T02:50:36.096 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 144/152 2024-08-20T02:50:36.685 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T02:50:36.720 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T02:50:36.740 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T02:50:37.572 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 146/152 2024-08-20T02:50:37.668 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-20T02:50:37.721 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-20T02:50:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-20T02:50:37.847 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-20T02:50:37.847 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T02:50:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T02:50:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T02:50:38.044 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 137/152 2024-08-20T02:50:38.085 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 138/152 2024-08-20T02:50:38.131 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 139/152 2024-08-20T02:50:38.158 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-20T02:50:38.185 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-20T02:50:38.377 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 142/152 2024-08-20T02:50:38.419 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-20T02:50:38.449 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 144/152 2024-08-20T02:50:39.057 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T02:50:39.097 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T02:50:39.118 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T02:50:39.970 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 146/152 2024-08-20T02:50:40.059 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-20T02:50:40.113 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-20T02:50:40.213 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-20T02:50:40.255 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-20T02:50:40.255 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T02:50:40.339 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T02:50:40.339 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T02:50:40.551 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T02:50:40.552 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:19.0.0-5872.gef427541.el9.x86_64 1/152 2024-08-20T02:50:40.552 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2/152 2024-08-20T02:50:40.552 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 3/152 2024-08-20T02:50:40.552 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 4/152 2024-08-20T02:50:40.552 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5872.gef427 5/152 2024-08-20T02:50:40.552 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T02:50:40.552 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T02:50:40.552 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 8/152 2024-08-20T02:50:40.553 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 9/152 2024-08-20T02:50:40.553 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 10/152 2024-08-20T02:50:40.553 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 11/152 2024-08-20T02:50:40.553 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T02:50:40.553 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 13/152 2024-08-20T02:50:40.553 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 14/152 2024-08-20T02:50:40.554 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 15/152 2024-08-20T02:50:40.554 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 16/152 2024-08-20T02:50:40.554 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 17/152 2024-08-20T02:50:40.554 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 18/152 2024-08-20T02:50:40.554 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 19/152 2024-08-20T02:50:40.554 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 20/152 2024-08-20T02:50:40.555 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 21/152 2024-08-20T02:50:40.555 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 22/152 2024-08-20T02:50:40.555 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 23/152 2024-08-20T02:50:40.555 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 24/152 2024-08-20T02:50:40.555 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 25/152 2024-08-20T02:50:40.555 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 26/152 2024-08-20T02:50:40.555 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 27/152 2024-08-20T02:50:40.555 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 28/152 2024-08-20T02:50:40.556 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 29/152 2024-08-20T02:50:40.556 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 30/152 2024-08-20T02:50:40.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 31/152 2024-08-20T02:50:40.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 32/152 2024-08-20T02:50:40.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 33/152 2024-08-20T02:50:40.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 34/152 2024-08-20T02:50:40.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 35/152 2024-08-20T02:50:40.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 36/152 2024-08-20T02:50:40.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 37/152 2024-08-20T02:50:40.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:19.0.0-5872.gef427541.el9.noarch 38/152 2024-08-20T02:50:40.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 39/152 2024-08-20T02:50:40.573 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-20T02:50:40.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-20T02:50:40.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-20T02:50:40.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-20T02:50:40.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-20T02:50:40.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-20T02:50:40.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-20T02:50:40.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-20T02:50:40.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-20T02:50:40.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-20T02:50:40.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-20T02:50:40.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-20T02:50:40.574 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-20T02:50:40.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-20T02:50:40.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-20T02:50:40.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-20T02:50:40.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-20T02:50:40.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-20T02:50:40.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-20T02:50:40.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-20T02:50:40.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-20T02:50:40.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-20T02:50:40.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-20T02:50:40.575 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.26-1.el9.x86_64 66/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.26-1.el9.x86_64 67/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-20T02:50:40.576 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.19-7.el9.x86_64 77/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-20T02:50:40.577 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-20T02:50:40.578 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-20T02:50:40.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-20T02:50:40.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-20T02:50:40.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-20T02:50:40.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-20T02:50:40.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-20T02:50:40.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-20T02:50:40.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-20T02:50:40.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-20T02:50:40.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-20T02:50:40.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-20T02:50:40.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-20T02:50:40.579 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-20T02:50:40.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-20T02:50:40.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-20T02:50:40.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-20T02:50:40.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-20T02:50:40.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-20T02:50:40.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-20T02:50:40.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-20T02:50:40.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-20T02:50:40.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-20T02:50:40.580 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-20T02:50:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:19.0.0-5872.gef427541.el9.x86_64 149/152 2024-08-20T02:50:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-20T02:50:40.583 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 151/152 2024-08-20T02:50:42.727 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T02:50:42.727 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:42.727 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-08-20T02:50:42.727 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.727 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.727 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-08-20T02:50:42.727 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-20T02:50:42.727 INFO:teuthology.orchestra.run.smithi044.stdout: automake-1.16.2-8.el9.noarch 2024-08-20T02:50:42.727 INFO:teuthology.orchestra.run.smithi044.stdout: bison-3.7.4-5.el9.x86_64 2024-08-20T02:50:42.727 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-20T02:50:42.728 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.728 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.728 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.728 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.728 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:42.728 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.728 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.728 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.728 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:42.728 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:42.728 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:42.728 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:42.729 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-20T02:50:42.730 INFO:teuthology.orchestra.run.smithi044.stdout: flex-2.6.4-9.el9.x86_64 2024-08-20T02:50:42.730 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-20T02:50:42.730 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-20T02:50:42.730 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-20T02:50:42.730 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-20T02:50:42.730 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-20T02:50:42.730 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-20T02:50:42.730 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-20T02:50:42.730 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-20T02:50:42.730 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-20T02:50:42.730 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.730 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-20T02:50:42.731 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: lua-5.4.4-4.el9.x86_64 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.26-1.el9.x86_64 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.26-1.el9.x86_64 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-13.16-1.el9.x86_64 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-20T02:50:42.732 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-20T02:50:42.733 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.19-7.el9.x86_64 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-20T02:50:42.734 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-20T02:50:42.735 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.736 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-20T02:50:42.737 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-3.el9.x86_64 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-20T02:50:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-20T02:50:42.739 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-20T02:50:42.739 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-20T02:50:42.739 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:50:42.739 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-08-20T02:50:43.085 DEBUG:teuthology.parallel:result is None 2024-08-20T02:50:44.509 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T02:50:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:19.0.0-5872.gef427541.el9.x86_64 1/152 2024-08-20T02:50:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2/152 2024-08-20T02:50:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 3/152 2024-08-20T02:50:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 4/152 2024-08-20T02:50:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5872.gef427 5/152 2024-08-20T02:50:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T02:50:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T02:50:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 8/152 2024-08-20T02:50:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 9/152 2024-08-20T02:50:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 10/152 2024-08-20T02:50:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 11/152 2024-08-20T02:50:44.510 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 13/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 14/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 15/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 16/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 17/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 18/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 19/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 20/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 21/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 22/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 23/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 24/152 2024-08-20T02:50:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 25/152 2024-08-20T02:50:44.512 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 26/152 2024-08-20T02:50:44.512 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 27/152 2024-08-20T02:50:44.512 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 28/152 2024-08-20T02:50:44.512 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 29/152 2024-08-20T02:50:44.512 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 30/152 2024-08-20T02:50:44.522 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 31/152 2024-08-20T02:50:44.522 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 32/152 2024-08-20T02:50:44.523 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 33/152 2024-08-20T02:50:44.523 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 34/152 2024-08-20T02:50:44.523 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 35/152 2024-08-20T02:50:44.523 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 36/152 2024-08-20T02:50:44.523 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 37/152 2024-08-20T02:50:44.523 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:19.0.0-5872.gef427541.el9.noarch 38/152 2024-08-20T02:50:44.523 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 39/152 2024-08-20T02:50:44.523 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-20T02:50:44.523 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-20T02:50:44.523 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-20T02:50:44.523 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-20T02:50:44.523 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-20T02:50:44.524 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-20T02:50:44.524 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-20T02:50:44.524 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-20T02:50:44.524 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-20T02:50:44.524 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-20T02:50:44.524 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-20T02:50:44.524 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-20T02:50:44.524 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-20T02:50:44.524 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-20T02:50:44.524 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-20T02:50:44.524 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-20T02:50:44.524 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.26-1.el9.x86_64 66/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-openmp-0.3.26-1.el9.x86_64 67/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-20T02:50:44.525 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-devel-3.9.19-7.el9.x86_64 77/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-20T02:50:44.526 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-20T02:50:44.527 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-20T02:50:44.528 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-20T02:50:44.529 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-20T02:50:44.529 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-20T02:50:44.529 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-20T02:50:44.529 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-20T02:50:44.529 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-20T02:50:44.529 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-20T02:50:44.529 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-20T02:50:44.529 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-20T02:50:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-20T02:50:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-20T02:50:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-20T02:50:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-20T02:50:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-20T02:50:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-20T02:50:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-20T02:50:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-20T02:50:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-20T02:50:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-20T02:50:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-20T02:50:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-20T02:50:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-20T02:50:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-20T02:50:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-20T02:50:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-20T02:50:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-20T02:50:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-20T02:50:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-20T02:50:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-20T02:50:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-20T02:50:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-20T02:50:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-20T02:50:44.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-20T02:50:44.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-20T02:50:44.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-20T02:50:44.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-20T02:50:44.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-20T02:50:44.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-20T02:50:44.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-20T02:50:44.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-20T02:50:44.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:19.0.0-5872.gef427541.el9.x86_64 149/152 2024-08-20T02:50:44.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-20T02:50:44.532 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 151/152 2024-08-20T02:50:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T02:50:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:46.554 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-08-20T02:50:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.554 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-08-20T02:50:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-20T02:50:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.2-8.el9.noarch 2024-08-20T02:50:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.7.4-5.el9.x86_64 2024-08-20T02:50:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-20T02:50:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-20T02:50:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.4-9.el9.x86_64 2024-08-20T02:50:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-20T02:50:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-20T02:50:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-20T02:50:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-20T02:50:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-20T02:50:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-20T02:50:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-20T02:50:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-20T02:50:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-20T02:50:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-20T02:50:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-20T02:50:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-20T02:50:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-20T02:50:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-20T02:50:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-20T02:50:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: lua-5.4.4-4.el9.x86_64 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.26-1.el9.x86_64 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp-0.3.26-1.el9.x86_64 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-13.16-1.el9.x86_64 2024-08-20T02:50:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel-3.9.19-7.el9.x86_64 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-20T02:50:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-20T02:50:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-20T02:50:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-20T02:50:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-20T02:50:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-20T02:50:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-20T02:50:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-20T02:50:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-20T02:50:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-20T02:50:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-20T02:50:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-20T02:50:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-20T02:50:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply-3.11-14.el9.noarch 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-20T02:50:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-20T02:50:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: re2-1:20211101-3.el9.x86_64 2024-08-20T02:50:46.566 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-20T02:50:46.566 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-20T02:50:46.566 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-20T02:50:46.566 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-20T02:50:46.566 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-20T02:50:46.566 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-20T02:50:46.566 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:50:46.567 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-08-20T02:50:46.745 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T02:50:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:19.0.0-5872.gef427541.el9.x86_64 1/152 2024-08-20T02:50:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2/152 2024-08-20T02:50:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 3/152 2024-08-20T02:50:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 4/152 2024-08-20T02:50:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5872.gef427 5/152 2024-08-20T02:50:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T02:50:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T02:50:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 8/152 2024-08-20T02:50:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 9/152 2024-08-20T02:50:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 10/152 2024-08-20T02:50:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 11/152 2024-08-20T02:50:46.746 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 13/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 14/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 15/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 16/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 17/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 18/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 19/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 20/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 21/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 22/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 23/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 24/152 2024-08-20T02:50:46.747 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 25/152 2024-08-20T02:50:46.748 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 26/152 2024-08-20T02:50:46.748 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 27/152 2024-08-20T02:50:46.759 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 28/152 2024-08-20T02:50:46.759 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 29/152 2024-08-20T02:50:46.759 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 30/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 31/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 32/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 33/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 34/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 35/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 36/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 37/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:19.0.0-5872.gef427541.el9.noarch 38/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 39/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-20T02:50:46.760 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-20T02:50:46.761 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-20T02:50:46.762 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-20T02:50:46.762 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-20T02:50:46.762 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-20T02:50:46.762 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-20T02:50:46.762 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-20T02:50:46.762 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-20T02:50:46.762 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-20T02:50:46.762 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-20T02:50:46.762 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.26-1.el9.x86_64 66/152 2024-08-20T02:50:46.762 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-openmp-0.3.26-1.el9.x86_64 67/152 2024-08-20T02:50:46.762 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-20T02:50:46.762 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-devel-3.9.19-7.el9.x86_64 77/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-20T02:50:46.763 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-20T02:50:46.764 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-20T02:50:46.765 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-20T02:50:46.766 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-20T02:50:46.766 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-20T02:50:46.766 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-20T02:50:46.766 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-20T02:50:46.766 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-20T02:50:46.766 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-20T02:50:46.766 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-20T02:50:46.766 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-20T02:50:46.766 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-20T02:50:46.767 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-20T02:50:46.767 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-20T02:50:46.767 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-20T02:50:46.767 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-20T02:50:46.767 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-20T02:50:46.767 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-20T02:50:46.767 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-20T02:50:46.767 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-20T02:50:46.767 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-20T02:50:46.767 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-20T02:50:46.767 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-20T02:50:46.767 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-20T02:50:46.768 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-20T02:50:46.769 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-20T02:50:46.769 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-20T02:50:46.769 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-20T02:50:46.769 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-20T02:50:46.769 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:19.0.0-5872.gef427541.el9.x86_64 149/152 2024-08-20T02:50:46.769 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-20T02:50:46.769 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 151/152 2024-08-20T02:50:46.895 DEBUG:teuthology.parallel:result is None 2024-08-20T02:50:48.799 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T02:50:48.800 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:48.800 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-08-20T02:50:48.800 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.800 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.800 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-08-20T02:50:48.800 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-20T02:50:48.800 INFO:teuthology.orchestra.run.smithi165.stdout: automake-1.16.2-8.el9.noarch 2024-08-20T02:50:48.800 INFO:teuthology.orchestra.run.smithi165.stdout: bison-3.7.4-5.el9.x86_64 2024-08-20T02:50:48.800 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-20T02:50:48.800 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.800 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.801 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.801 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.801 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:48.801 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.801 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.801 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.801 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:48.801 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:48.801 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:48.801 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:48.801 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:48.801 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.802 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.802 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:48.802 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.802 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.802 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.802 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:48.802 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:48.802 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.802 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.802 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T02:50:48.802 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-20T02:50:48.802 INFO:teuthology.orchestra.run.smithi165.stdout: flex-2.6.4-9.el9.x86_64 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.803 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-20T02:50:48.804 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-20T02:50:48.804 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-20T02:50:48.804 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-20T02:50:48.804 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-20T02:50:48.804 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.804 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.804 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-20T02:50:48.804 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.804 INFO:teuthology.orchestra.run.smithi165.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-20T02:50:48.804 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: lua-5.4.4-4.el9.x86_64 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.26-1.el9.x86_64 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp-0.3.26-1.el9.x86_64 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-13.16-1.el9.x86_64 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-20T02:50:48.805 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-20T02:50:48.806 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-20T02:50:48.806 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-20T02:50:48.806 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-20T02:50:48.806 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-20T02:50:48.806 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-20T02:50:48.806 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-20T02:50:48.806 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-20T02:50:48.806 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-20T02:50:48.806 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.806 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.806 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.806 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel-3.9.19-7.el9.x86_64 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-20T02:50:48.807 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-20T02:50:48.808 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-20T02:50:48.808 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-20T02:50:48.808 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-20T02:50:48.808 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-20T02:50:48.808 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-20T02:50:48.808 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-20T02:50:48.808 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-20T02:50:48.808 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-20T02:50:48.808 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-20T02:50:48.808 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-20T02:50:48.808 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-20T02:50:48.808 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply-3.11-14.el9.noarch 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-20T02:50:48.809 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-20T02:50:48.810 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T02:50:48.811 INFO:teuthology.orchestra.run.smithi165.stdout: re2-1:20211101-3.el9.x86_64 2024-08-20T02:50:48.812 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-20T02:50:48.812 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-20T02:50:48.812 INFO:teuthology.orchestra.run.smithi165.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-20T02:50:48.812 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-20T02:50:48.812 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-20T02:50:48.812 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-20T02:50:48.812 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:50:48.813 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-08-20T02:50:49.114 DEBUG:teuthology.parallel:result is None 2024-08-20T02:50:49.115 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T02:50:49.271 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-20T02:50:49.298 INFO:teuthology.orchestra.run.smithi044.stdout:19.0.0-5872.gef427541.el9 2024-08-20T02:50:49.299 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5872.gef427541.el9 2024-08-20T02:50:49.299 INFO:teuthology.task.install:The correct ceph version 19.0.0-5872.gef427541 is installed. 2024-08-20T02:50:49.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T02:50:49.449 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-20T02:50:49.472 INFO:teuthology.orchestra.run.smithi099.stdout:19.0.0-5872.gef427541.el9 2024-08-20T02:50:49.472 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5872.gef427541.el9 2024-08-20T02:50:49.472 INFO:teuthology.task.install:The correct ceph version 19.0.0-5872.gef427541 is installed. 2024-08-20T02:50:49.474 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T02:50:49.622 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-20T02:50:49.644 INFO:teuthology.orchestra.run.smithi165.stdout:19.0.0-5872.gef427541.el9 2024-08-20T02:50:49.644 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5872.gef427541.el9 2024-08-20T02:50:49.644 INFO:teuthology.task.install:The correct ceph version 19.0.0-5872.gef427541 is installed. 2024-08-20T02:50:49.646 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-20T02:50:49.646 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:50:49.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-20T02:50:49.678 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-08-20T02:50:49.678 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-20T02:50:49.706 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-08-20T02:50:49.706 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-20T02:50:49.733 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-20T02:50:49.733 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:50:49.733 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-08-20T02:50:49.763 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-20T02:50:49.832 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-08-20T02:50:49.832 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-08-20T02:50:49.857 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-20T02:50:49.921 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-08-20T02:50:49.921 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-08-20T02:50:49.947 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-20T02:50:50.012 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-20T02:50:50.012 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:50:50.012 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-20T02:50:50.041 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-20T02:50:50.110 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-08-20T02:50:50.110 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-20T02:50:50.135 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-20T02:50:50.200 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-08-20T02:50:50.200 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-20T02:50:50.224 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-20T02:50:50.288 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-20T02:50:50.289 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:50:50.289 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-08-20T02:50:50.318 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-20T02:50:50.388 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-08-20T02:50:50.388 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-08-20T02:50:50.413 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-20T02:50:50.478 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-08-20T02:50:50.478 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-08-20T02:50:50.504 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-20T02:50:50.571 INFO:teuthology.run_tasks:Running task cephadm... 2024-08-20T02:50:50.684 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug'}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-08-20T02:50:50.684 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T02:50:50.684 INFO:tasks.cephadm:Cluster fsid is 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:50:50.685 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-08-20T02:50:50.685 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.44', 'mon.b': '172.21.15.99', 'mon.c': '172.21.15.165'} 2024-08-20T02:50:50.685 INFO:tasks.cephadm:First mon is mon.a on smithi044 2024-08-20T02:50:50.685 INFO:tasks.cephadm:First mgr is x 2024-08-20T02:50:50.685 INFO:tasks.cephadm:Normalizing hostnames... 2024-08-20T02:50:50.685 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname $(hostname -s) 2024-08-20T02:50:50.716 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname $(hostname -s) 2024-08-20T02:50:50.741 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname $(hostname -s) 2024-08-20T02:50:50.766 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-08-20T02:50:50.767 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T02:50:50.918 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82210/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-5872-gef427541', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-5872.gef427541'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-08-19 17:15:28.176472', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-hemanth-testing-2024-08-19-1357', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-08-20T02:50:51.069 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-hemanth-testing-2024-08-19-1357, sha1 ef4275410d8483698016fe484b2a3c664080b4c1 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T02:50:51.071 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm 2024-08-20T02:50:51.071 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm 2024-08-20T02:50:51.072 DEBUG:teuthology.orchestra.run.smithi044:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-20T02:50:51.454 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r--. 1 ubuntu ubuntu 792014 Aug 20 02:50 /home/ubuntu/cephtest/cephadm 2024-08-20T02:50:51.454 DEBUG:teuthology.orchestra.run.smithi099:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-20T02:50:51.830 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r--. 1 ubuntu ubuntu 792014 Aug 20 02:50 /home/ubuntu/cephtest/cephadm 2024-08-20T02:50:51.831 DEBUG:teuthology.orchestra.run.smithi165:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-20T02:50:52.236 INFO:teuthology.orchestra.run.smithi165.stdout:-rw-r--r--. 1 ubuntu ubuntu 792014 Aug 20 02:50 /home/ubuntu/cephtest/cephadm 2024-08-20T02:50:52.236 DEBUG:teuthology.orchestra.run.smithi044:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-20T02:50:52.256 DEBUG:teuthology.orchestra.run.smithi099:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-20T02:50:52.276 DEBUG:teuthology.orchestra.run.smithi165:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-20T02:50:52.311 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 on all hosts... 2024-08-20T02:50:52.311 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 pull 2024-08-20T02:50:52.314 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 pull 2024-08-20T02:50:52.318 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 pull 2024-08-20T02:50:54.019 INFO:teuthology.orchestra.run.smithi099.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T02:50:54.044 INFO:teuthology.orchestra.run.smithi165.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T02:50:54.172 INFO:teuthology.orchestra.run.smithi044.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T02:51:34.844 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-08-20T02:51:34.844 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_version": "ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev)", 2024-08-20T02:51:34.844 INFO:teuthology.orchestra.run.smithi165.stdout: "image_id": "9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a", 2024-08-20T02:51:34.845 INFO:teuthology.orchestra.run.smithi165.stdout: "repo_digests": [ 2024-08-20T02:51:34.845 INFO:teuthology.orchestra.run.smithi165.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8" 2024-08-20T02:51:34.845 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-08-20T02:51:34.845 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-08-20T02:51:50.793 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-08-20T02:51:50.793 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev)", 2024-08-20T02:51:50.793 INFO:teuthology.orchestra.run.smithi099.stdout: "image_id": "9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a", 2024-08-20T02:51:50.793 INFO:teuthology.orchestra.run.smithi099.stdout: "repo_digests": [ 2024-08-20T02:51:50.793 INFO:teuthology.orchestra.run.smithi099.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8" 2024-08-20T02:51:50.793 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-08-20T02:51:50.794 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-08-20T02:51:52.747 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T02:51:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev)", 2024-08-20T02:51:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "image_id": "9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a", 2024-08-20T02:51:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "repo_digests": [ 2024-08-20T02:51:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8" 2024-08-20T02:51:52.747 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-08-20T02:51:52.748 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T02:51:52.771 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph 2024-08-20T02:51:52.821 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph 2024-08-20T02:51:52.854 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph 2024-08-20T02:51:52.888 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /etc/ceph 2024-08-20T02:51:52.913 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /etc/ceph 2024-08-20T02:51:52.941 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /etc/ceph 2024-08-20T02:51:52.970 INFO:tasks.cephadm:Writing seed config... 2024-08-20T02:51:52.971 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-08-20T02:51:52.971 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-08-20T02:51:52.971 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-08-20T02:51:52.971 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-08-20T02:51:52.971 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-08-20T02:51:52.971 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-08-20T02:51:52.972 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-08-20T02:51:52.973 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-08-20T02:51:52.973 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-08-20T02:51:52.973 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-08-20T02:51:52.973 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-08-20T02:51:52.973 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-08-20T02:51:52.974 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:51:52.974 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-08-20T02:51:52.989 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0284e16c-5e9f-11ef-bccf-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 2024-08-20T02:51:52.989 DEBUG:teuthology.orchestra.run.smithi044:mon.a> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.a.service 2024-08-20T02:51:53.031 DEBUG:teuthology.orchestra.run.smithi044:mgr.x> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.x.service 2024-08-20T02:51:53.075 INFO:tasks.cephadm:Bootstrapping... 2024-08-20T02:51:53.075 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 -v bootstrap --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.44 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-08-20T02:51:53.285 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-08-20T02:51:53.285 INFO:teuthology.orchestra.run.smithi044.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', '-v', 'bootstrap', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.44', '--skip-admin-label'] 2024-08-20T02:51:53.285 INFO:teuthology.orchestra.run.smithi044.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-08-20T02:51:53.285 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying podman|docker is present... 2024-08-20T02:51:53.309 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 5.2.1 2024-08-20T02:51:53.309 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying lvm2 is present... 2024-08-20T02:51:53.310 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying time synchronization is in place... 2024-08-20T02:51:53.317 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-20T02:51:53.317 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-20T02:51:53.323 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-20T02:51:53.324 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-08-20T02:51:53.330 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout enabled 2024-08-20T02:51:53.336 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout active 2024-08-20T02:51:53.336 INFO:teuthology.orchestra.run.smithi044.stdout:Unit chronyd.service is enabled and running 2024-08-20T02:51:53.336 INFO:teuthology.orchestra.run.smithi044.stdout:Repeating the final host check... 2024-08-20T02:51:53.358 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 5.2.1 2024-08-20T02:51:53.359 INFO:teuthology.orchestra.run.smithi044.stdout:podman (/bin/podman) version 5.2.1 is present 2024-08-20T02:51:53.359 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl is present 2024-08-20T02:51:53.359 INFO:teuthology.orchestra.run.smithi044.stdout:lvcreate is present 2024-08-20T02:51:53.367 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-20T02:51:53.367 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-20T02:51:53.371 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-20T02:51:53.371 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-08-20T02:51:53.378 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout enabled 2024-08-20T02:51:53.384 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout active 2024-08-20T02:51:53.384 INFO:teuthology.orchestra.run.smithi044.stdout:Unit chronyd.service is enabled and running 2024-08-20T02:51:53.384 INFO:teuthology.orchestra.run.smithi044.stdout:Host looks OK 2024-08-20T02:51:53.384 INFO:teuthology.orchestra.run.smithi044.stdout:Cluster fsid: 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:51:53.384 INFO:teuthology.orchestra.run.smithi044.stdout:Acquiring lock 139735652519792 on /run/cephadm/0284e16c-5e9f-11ef-bccf-c7b262605968.lock 2024-08-20T02:51:53.384 INFO:teuthology.orchestra.run.smithi044.stdout:Lock 139735652519792 acquired on /run/cephadm/0284e16c-5e9f-11ef-bccf-c7b262605968.lock 2024-08-20T02:51:53.385 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying IP 172.21.15.44 port 3300 ... 2024-08-20T02:51:53.385 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying IP 172.21.15.44 port 6789 ... 2024-08-20T02:51:53.385 INFO:teuthology.orchestra.run.smithi044.stdout:Base mon IP(s) is [172.21.15.44:3300, 172.21.15.44:6789], mon addrv is [v2:172.21.15.44:3300,v1:172.21.15.44:6789] 2024-08-20T02:51:53.388 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.44 metric 100 2024-08-20T02:51:53.388 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.44 metric 100 2024-08-20T02:51:53.391 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-08-20T02:51:53.391 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-08-20T02:51:53.391 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-08-20T02:51:53.393 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-08-20T02:51:53.393 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-08-20T02:51:53.393 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-20T02:51:53.393 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout 2: ens1f0: mtu 1500 state UP qlen 1000 2024-08-20T02:51:53.394 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout inet6 fe80::858d:6445:9466:81d1/64 scope link noprefixroute 2024-08-20T02:51:53.394 INFO:teuthology.orchestra.run.smithi044.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-20T02:51:53.394 INFO:teuthology.orchestra.run.smithi044.stdout:Mon IP `172.21.15.44` is in CIDR network `172.21.0.0/20` 2024-08-20T02:51:53.394 INFO:teuthology.orchestra.run.smithi044.stdout:Mon IP `172.21.15.44` is in CIDR network `172.21.0.0/20` 2024-08-20T02:51:53.394 INFO:teuthology.orchestra.run.smithi044.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-08-20T02:51:53.395 INFO:teuthology.orchestra.run.smithi044.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-08-20T02:51:53.395 INFO:teuthology.orchestra.run.smithi044.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T02:51:53.822 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stdout 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a 2024-08-20T02:51:53.823 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T02:51:53.823 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Getting image source signatures 2024-08-20T02:51:53.823 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Copying blob sha256:0beb2e0cf0f87d34a70e3776822a6ff39fafdc5a4d91bc4481cb39d566d52b0d 2024-08-20T02:51:53.823 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-08-20T02:51:53.823 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Copying config sha256:9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a 2024-08-20T02:51:53.823 INFO:teuthology.orchestra.run.smithi044.stdout:/bin/podman: stderr Writing manifest to image destination 2024-08-20T02:51:54.728 INFO:teuthology.orchestra.run.smithi044.stdout:ceph: stdout ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev) 2024-08-20T02:51:54.728 INFO:teuthology.orchestra.run.smithi044.stdout:Ceph version: ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev) 2024-08-20T02:51:54.728 INFO:teuthology.orchestra.run.smithi044.stdout:Extracting ceph user uid/gid from container image... 2024-08-20T02:51:55.848 INFO:teuthology.orchestra.run.smithi044.stdout:stat: stdout 167 167 2024-08-20T02:51:55.848 INFO:teuthology.orchestra.run.smithi044.stdout:Creating initial keys... 2024-08-20T02:52:04.656 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-authtool: stdout AQDOBMRmoYTQLhAA3wlIx5wBKI4DvLrWo6F6ew== 2024-08-20T02:52:06.792 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-authtool: stdout AQDWBMRmEvp/EBAAv7F6LO71WhTZy8QgNKwHsA== 2024-08-20T02:52:07.690 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-authtool: stdout AQDXBMRmlPNrCBAAqQc6kOubBxfkzIjrMO+Igw== 2024-08-20T02:52:07.691 INFO:teuthology.orchestra.run.smithi044.stdout:Creating initial monmap... 2024-08-20T02:52:08.638 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-20T02:52:08.638 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-08-20T02:52:08.638 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:52:08.638 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-20T02:52:08.638 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool for a [v2:172.21.15.44:3300,v1:172.21.15.44:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-20T02:52:08.639 INFO:teuthology.orchestra.run.smithi044.stdout:setting min_mon_release = quincy 2024-08-20T02:52:08.639 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: set fsid to 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:52:08.639 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-20T02:52:08.639 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:52:08.639 INFO:teuthology.orchestra.run.smithi044.stdout:Creating mon... 2024-08-20T02:52:09.561 INFO:teuthology.orchestra.run.smithi044.stdout:create mon.a on 2024-08-20T02:52:09.779 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-08-20T02:52:09.921 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-08-20T02:52:10.092 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0284e16c-5e9f-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-0284e16c-5e9f-11ef-bccf-c7b262605968.target. 2024-08-20T02:52:10.092 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0284e16c-5e9f-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-0284e16c-5e9f-11ef-bccf-c7b262605968.target. 2024-08-20T02:52:10.335 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.a 2024-08-20T02:52:10.335 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.a.service: Unit ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.a.service not loaded. 2024-08-20T02:52:10.510 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0284e16c-5e9f-11ef-bccf-c7b262605968.target.wants/ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@.service. 2024-08-20T02:52:11.022 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:10 smithi044 ceph-mon[18420]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-20T02:52:11.022 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 bash[18400]: d713942a76a31c0e6233b67acb5344ada12687d2b04749012a553e13e8fa8939 2024-08-20T02:52:11.035 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-20T02:52:11.035 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2024-08-20T02:52:11.043 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-20T02:52:11.043 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-08-20T02:52:11.043 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2024-08-20T02:52:11.043 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to enable service . firewalld.service is not available 2024-08-20T02:52:11.044 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mon to start... 2024-08-20T02:52:11.044 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mon... 2024-08-20T02:52:11.273 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 systemd[1]: Started Ceph mon.a for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:52:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 ceph-mon[18420]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-20T02:52:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 ceph-mon[18420]: monmap epoch 1 2024-08-20T02:52:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 ceph-mon[18420]: fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:52:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 ceph-mon[18420]: last_changed 2024-08-20T02:52:08.003300+0000 2024-08-20T02:52:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 ceph-mon[18420]: created 2024-08-20T02:52:08.003300+0000 2024-08-20T02:52:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 ceph-mon[18420]: min_mon_release 19 (squid) 2024-08-20T02:52:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 ceph-mon[18420]: election_strategy: 1 2024-08-20T02:52:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 ceph-mon[18420]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-08-20T02:52:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 ceph-mon[18420]: fsmap 2024-08-20T02:52:12.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 ceph-mon[18420]: osdmap e1: 0 total, 0 up, 0 in 2024-08-20T02:52:12.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 ceph-mon[18420]: mgrmap e1: no daemons active 2024-08-20T02:52:12.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:11 smithi044 ceph-mon[18420]: from='client.? 172.21.15.44:0/3678503183' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-08-20T02:52:13.007 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout cluster: 2024-08-20T02:52:13.007 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout id: 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:52:13.008 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-08-20T02:52:13.008 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-08-20T02:52:13.008 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout services: 2024-08-20T02:52:13.008 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.702488s) 2024-08-20T02:52:13.008 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-08-20T02:52:13.008 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-08-20T02:52:13.008 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-08-20T02:52:13.008 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout data: 2024-08-20T02:52:13.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-08-20T02:52:13.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-08-20T02:52:13.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-08-20T02:52:13.010 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout pgs: 2024-08-20T02:52:13.011 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-08-20T02:52:13.011 INFO:teuthology.orchestra.run.smithi044.stdout:mon is available 2024-08-20T02:52:13.011 INFO:teuthology.orchestra.run.smithi044.stdout:Assimilating anything we can from ceph.conf... 2024-08-20T02:52:13.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:13 smithi044 ceph-mon[18420]: from='client.? 172.21.15.44:0/1962734732' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-08-20T02:52:13.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:13 smithi044 ceph-mon[18420]: from='client.? 172.21.15.44:0/1962734732' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-08-20T02:52:13.994 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-08-20T02:52:13.995 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [global] 2024-08-20T02:52:13.995 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout fsid = 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:52:13.995 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-08-20T02:52:13.995 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.44:3300,v1:172.21.15.44:6789] 2024-08-20T02:52:13.995 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-20T02:52:13.995 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-20T02:52:13.995 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-20T02:52:13.995 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-20T02:52:13.995 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-08-20T02:52:13.995 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-20T02:52:13.995 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-20T02:52:13.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-08-20T02:52:13.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [osd] 2024-08-20T02:52:13.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-20T02:52:13.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-20T02:52:13.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-20T02:52:13.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-20T02:52:13.996 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-20T02:52:13.996 INFO:teuthology.orchestra.run.smithi044.stdout:Generating new minimal ceph.conf... 2024-08-20T02:52:14.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:14 smithi044 ceph-mon[18420]: from='client.? 172.21.15.44:0/2377299403' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:52:14.984 INFO:teuthology.orchestra.run.smithi044.stdout:Restarting the monitor... 2024-08-20T02:52:15.268 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:15 smithi044 systemd[1]: Stopping Ceph mon.a for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T02:52:15.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:15 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a[18416]: 2024-08-20T02:52:15.265+0000 7fa6f4a1c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T02:52:15.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:15 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a[18416]: 2024-08-20T02:52:15.265+0000 7fa6f4a1c640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-08-20T02:52:15.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:15 smithi044 podman[18666]: 2024-08-20 02:52:15.377837711 +0000 UTC m=+0.216920202 container died d713942a76a31c0e6233b67acb5344ada12687d2b04749012a553e13e8fa8939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-20T02:52:16.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:15 smithi044 podman[18666]: 2024-08-20 02:52:15.706901498 +0000 UTC m=+0.545983978 container cleanup d713942a76a31c0e6233b67acb5344ada12687d2b04749012a553e13e8fa8939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD) 2024-08-20T02:52:16.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:15 smithi044 bash[18666]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a 2024-08-20T02:52:16.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:15 smithi044 podman[18679]: 2024-08-20 02:52:15.866291103 +0000 UTC m=+0.491894288 container remove d713942a76a31c0e6233b67acb5344ada12687d2b04749012a553e13e8fa8939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-20T02:52:16.382 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 systemd[1]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-20T02:52:16.382 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 systemd[1]: Stopped Ceph mon.a for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:52:16.382 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 systemd[1]: Starting Ceph mon.a for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T02:52:16.382 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 podman[18771]: 2024-08-20 02:52:16.292325791 +0000 UTC m=+0.103499412 container create b0653d7023932256d30639a66904b211b2a5c4af49c1738c9d97586ca38d97ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-08-20T02:52:16.382 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 podman[18771]: 2024-08-20 02:52:16.204785113 +0000 UTC m=+0.015958688 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T02:52:16.559 INFO:teuthology.orchestra.run.smithi044.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-08-20T02:52:16.664 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 podman[18771]: 2024-08-20 02:52:16.478217803 +0000 UTC m=+0.289391360 container init b0653d7023932256d30639a66904b211b2a5c4af49c1738c9d97586ca38d97ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True) 2024-08-20T02:52:16.665 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 podman[18771]: 2024-08-20 02:52:16.481315282 +0000 UTC m=+0.292488840 container start b0653d7023932256d30639a66904b211b2a5c4af49c1738c9d97586ca38d97ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-20T02:52:16.665 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: set uid:gid to 167:167 (ceph:ceph) 2024-08-20T02:52:16.665 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev), process ceph-mon, pid 2 2024-08-20T02:52:16.665 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: pidfile_write: ignore empty --pid-file 2024-08-20T02:52:16.665 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: load: jerasure load: lrc 2024-08-20T02:52:16.665 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: RocksDB version: 7.9.2 2024-08-20T02:52:16.665 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Git sha 0 2024-08-20T02:52:16.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Compile date 2024-08-19 15:50:59 2024-08-20T02:52:16.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: DB SUMMARY 2024-08-20T02:52:16.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: DB Session ID: IANBCYKXG4XGRVPD1MHB 2024-08-20T02:52:16.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: CURRENT file: CURRENT 2024-08-20T02:52:16.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: IDENTITY file: IDENTITY 2024-08-20T02:52:16.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-08-20T02:52:16.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-08-20T02:52:16.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95557 ; 2024-08-20T02:52:16.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.error_if_exists: 0 2024-08-20T02:52:16.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.create_if_missing: 0 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.paranoid_checks: 1 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.env: 0x55656fc2dbe0 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.fs: PosixFileSystem 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.info_log: 0x556571663700 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_file_opening_threads: 16 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.statistics: (nil) 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.use_fsync: 0 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_log_file_size: 0 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-20T02:52:16.667 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-20T02:52:16.668 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.keep_log_file_num: 1000 2024-08-20T02:52:16.668 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.recycle_log_file_num: 0 2024-08-20T02:52:16.668 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.allow_fallocate: 1 2024-08-20T02:52:16.668 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.allow_mmap_reads: 0 2024-08-20T02:52:16.668 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.allow_mmap_writes: 0 2024-08-20T02:52:16.668 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.use_direct_reads: 0 2024-08-20T02:52:16.668 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-20T02:52:16.668 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.create_missing_column_families: 0 2024-08-20T02:52:16.668 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.db_log_dir: 2024-08-20T02:52:16.668 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.wal_dir: 2024-08-20T02:52:16.668 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-20T02:52:16.668 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.advise_random_on_open: 1 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.db_write_buffer_size: 0 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.write_buffer_manager: 0x556571666e60 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.rate_limiter: (nil) 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-20T02:52:16.669 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.wal_recovery_mode: 2 2024-08-20T02:52:16.670 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.enable_thread_tracking: 0 2024-08-20T02:52:16.670 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.enable_pipelined_write: 0 2024-08-20T02:52:16.670 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.unordered_write: 0 2024-08-20T02:52:16.670 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-20T02:52:16.670 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-20T02:52:16.670 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-20T02:52:16.670 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-20T02:52:16.670 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.row_cache: None 2024-08-20T02:52:16.670 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.wal_filter: None 2024-08-20T02:52:16.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.allow_ingest_behind: 0 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.two_write_queues: 0 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.manual_wal_flush: 0 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.wal_compression: 0 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.atomic_flush: 0 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.log_readahead_size: 0 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.best_efforts_recovery: 0 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-20T02:52:16.672 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.allow_data_in_errors: 0 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.db_host_id: __hostname__ 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_background_jobs: 2 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_background_compactions: -1 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_subcompactions: 1 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_total_wal_size: 0 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-20T02:52:16.673 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-20T02:52:16.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-20T02:52:16.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_open_files: -1 2024-08-20T02:52:16.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bytes_per_sync: 0 2024-08-20T02:52:16.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-20T02:52:16.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-20T02:52:16.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_readahead_size: 0 2024-08-20T02:52:16.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_background_flushes: -1 2024-08-20T02:52:16.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Compression algorithms supported: 2024-08-20T02:52:16.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: kZSTD supported: 0 2024-08-20T02:52:16.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: kXpressCompression supported: 0 2024-08-20T02:52:16.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: kBZip2Compression supported: 0 2024-08-20T02:52:16.674 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: kLZ4Compression supported: 1 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: kZlibCompression supported: 1 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: kLZ4HCCompression supported: 1 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: kSnappyCompression supported: 1 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.merge_operator: 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_filter: None 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_filter_factory: None 2024-08-20T02:52:16.675 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.sst_partitioner_factory: None 2024-08-20T02:52:16.676 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-20T02:52:16.676 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-20T02:52:16.676 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556571663380) 2024-08-20T02:52:16.676 INFO:journalctl@ceph.mon.a.smithi044.stdout: cache_index_and_filter_blocks: 1 2024-08-20T02:52:16.676 INFO:journalctl@ceph.mon.a.smithi044.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-20T02:52:16.676 INFO:journalctl@ceph.mon.a.smithi044.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-20T02:52:16.676 INFO:journalctl@ceph.mon.a.smithi044.stdout: pin_top_level_index_and_filter: 1 2024-08-20T02:52:16.676 INFO:journalctl@ceph.mon.a.smithi044.stdout: index_type: 0 2024-08-20T02:52:16.676 INFO:journalctl@ceph.mon.a.smithi044.stdout: data_block_index_type: 0 2024-08-20T02:52:16.676 INFO:journalctl@ceph.mon.a.smithi044.stdout: index_shortening: 1 2024-08-20T02:52:16.676 INFO:journalctl@ceph.mon.a.smithi044.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-20T02:52:16.677 INFO:journalctl@ceph.mon.a.smithi044.stdout: checksum: 4 2024-08-20T02:52:16.677 INFO:journalctl@ceph.mon.a.smithi044.stdout: no_block_cache: 0 2024-08-20T02:52:16.677 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache: 0x556571657610 2024-08-20T02:52:16.677 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache_name: BinnedLRUCache 2024-08-20T02:52:16.677 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache_options: 2024-08-20T02:52:16.677 INFO:journalctl@ceph.mon.a.smithi044.stdout: capacity : 536870912 2024-08-20T02:52:16.677 INFO:journalctl@ceph.mon.a.smithi044.stdout: num_shard_bits : 4 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: strict_capacity_limit : 0 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: high_pri_pool_ratio: 0.000 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_cache_compressed: (nil) 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: persistent_cache: (nil) 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_size: 4096 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_size_deviation: 10 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_restart_interval: 16 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: index_block_restart_interval: 1 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: metadata_block_size: 4096 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: partition_filters: 0 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: use_delta_encoding: 1 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: filter_policy: bloomfilter 2024-08-20T02:52:16.678 INFO:journalctl@ceph.mon.a.smithi044.stdout: whole_key_filtering: 1 2024-08-20T02:52:16.679 INFO:journalctl@ceph.mon.a.smithi044.stdout: verify_compression: 0 2024-08-20T02:52:16.679 INFO:journalctl@ceph.mon.a.smithi044.stdout: read_amp_bytes_per_bit: 0 2024-08-20T02:52:16.679 INFO:journalctl@ceph.mon.a.smithi044.stdout: format_version: 5 2024-08-20T02:52:16.679 INFO:journalctl@ceph.mon.a.smithi044.stdout: enable_index_compression: 1 2024-08-20T02:52:16.679 INFO:journalctl@ceph.mon.a.smithi044.stdout: block_align: 0 2024-08-20T02:52:16.679 INFO:journalctl@ceph.mon.a.smithi044.stdout: max_auto_readahead_size: 262144 2024-08-20T02:52:16.679 INFO:journalctl@ceph.mon.a.smithi044.stdout: prepopulate_block_cache: 0 2024-08-20T02:52:16.679 INFO:journalctl@ceph.mon.a.smithi044.stdout: initial_auto_readahead_size: 8192 2024-08-20T02:52:16.679 INFO:journalctl@ceph.mon.a.smithi044.stdout: num_file_reads_for_auto_readahead: 2 2024-08-20T02:52:16.679 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.write_buffer_size: 33554432 2024-08-20T02:52:16.680 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_write_buffer_number: 2 2024-08-20T02:52:16.680 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compression: NoCompression 2024-08-20T02:52:16.680 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bottommost_compression: Disabled 2024-08-20T02:52:16.680 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.prefix_extractor: nullptr 2024-08-20T02:52:16.680 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-20T02:52:16.680 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.num_levels: 7 2024-08-20T02:52:16.680 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-20T02:52:16.680 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-20T02:52:16.680 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-20T02:52:16.680 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-20T02:52:16.680 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-20T02:52:16.680 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-20T02:52:16.681 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-20T02:52:16.681 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-20T02:52:16.681 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-20T02:52:16.681 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-20T02:52:16.681 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-20T02:52:16.681 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-20T02:52:16.681 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-20T02:52:16.681 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compression_opts.level: 32767 2024-08-20T02:52:16.681 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compression_opts.strategy: 0 2024-08-20T02:52:16.681 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compression_opts.enabled: false 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.target_file_size_base: 67108864 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-20T02:52:16.682 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.arena_block_size: 1048576 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-20T02:52:16.683 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-20T02:52:16.684 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.disable_auto_compactions: 0 2024-08-20T02:52:16.684 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-20T02:52:16.684 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-20T02:52:16.684 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-20T02:52:16.684 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-20T02:52:16.684 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-20T02:52:16.685 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-20T02:52:16.685 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.inplace_update_support: 0 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.bloom_locality: 0 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.max_successive_merges: 0 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-20T02:52:16.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.paranoid_file_checks: 0 2024-08-20T02:52:16.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.force_consistency_checks: 1 2024-08-20T02:52:16.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.report_bg_io_stats: 0 2024-08-20T02:52:16.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.ttl: 2592000 2024-08-20T02:52:16.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-20T02:52:16.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-20T02:52:16.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-20T02:52:16.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.enable_blob_files: false 2024-08-20T02:52:16.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.min_blob_size: 0 2024-08-20T02:52:16.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.blob_file_size: 268435456 2024-08-20T02:52:16.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-20T02:52:16.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-20T02:52:16.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-20T02:52:16.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-20T02:52:16.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-20T02:52:16.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.blob_file_starting_level: 0 2024-08-20T02:52:16.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-20T02:52:16.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-08-20T02:52:16.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-08-20T02:52:16.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bb4eb8ab-b023-46c8-90fb-2d6b4668af6b 2024-08-20T02:52:16.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724122336508051, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-08-20T02:52:16.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-08-20T02:52:16.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724122336508929, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91664, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 284, "table_properties": {"data_size": 89711, "index_size": 273, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11760, "raw_average_key_size": 48, "raw_value_size": 83099, "raw_average_value_size": 341, "num_data_blocks": 12, "num_entries": 243, "num_filter_entries": 243, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724122336, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bb4eb8ab-b023-46c8-90fb-2d6b4668af6b", "db_session_id": "IANBCYKXG4XGRVPD1MHB", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-08-20T02:52:16.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724122336509025, "job": 1, "event": "recovery_finished"} 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556571686e00 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: DB pointer 0x556571784000 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** DB Stats ** 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T02:52:16.689 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** Compaction Stats [default] ** 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: L0 2/0 91.38 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: Sum 2/0 91.38 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** Compaction Stats [default] ** 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 155.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T02:52:16.690 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: Cumulative compaction: 0.00 GB write, 28.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: Interval compaction: 0.00 GB write, 28.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: Block cache BinnedLRUCache@0x556571657610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-08-20T02:52:16.691 INFO:journalctl@ceph.mon.a.smithi044.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: starting mon.a rank 0 at public addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] at bind addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: mon.a@-1(???) e1 preinit fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: mon.a@-1(???).mds e1 new map 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: mon.a@-1(???).mds e1 print_map 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout: e1 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout: btime 2024-08-20T02:52:10:979066+0000 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout: legacy client fscid: -1 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout: 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout: No filesystems configured 2024-08-20T02:52:16.692 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 bash[18771]: b0653d7023932256d30639a66904b211b2a5c4af49c1738c9d97586ca38d97ce 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 systemd[1]: Started Ceph mon.a for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: monmap epoch 1 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: last_changed 2024-08-20T02:52:08.003300+0000 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: created 2024-08-20T02:52:08.003300+0000 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: min_mon_release 19 (squid) 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: election_strategy: 1 2024-08-20T02:52:16.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-08-20T02:52:16.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: fsmap 2024-08-20T02:52:16.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: osdmap e1: 0 total, 0 up, 0 in 2024-08-20T02:52:16.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:16 smithi044 ceph-mon[18790]: mgrmap e1: no daemons active 2024-08-20T02:52:17.506 INFO:teuthology.orchestra.run.smithi044.stdout:Wrote config to /etc/ceph/ceph.conf 2024-08-20T02:52:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-08-20T02:52:17.528 INFO:teuthology.orchestra.run.smithi044.stdout:Creating mgr... 2024-08-20T02:52:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying port 0.0.0.0:9283 ... 2024-08-20T02:52:17.529 INFO:teuthology.orchestra.run.smithi044.stdout:Verifying port 0.0.0.0:8765 ... 2024-08-20T02:52:17.926 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.x 2024-08-20T02:52:17.926 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.x.service: Unit ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.x.service not loaded. 2024-08-20T02:52:18.069 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0284e16c-5e9f-11ef-bccf-c7b262605968.target.wants/ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@.service. 2024-08-20T02:52:18.403 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:18 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2805952067' entity='client.admin' 2024-08-20T02:52:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-20T02:52:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2024-08-20T02:52:18.600 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-20T02:52:18.600 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-08-20T02:52:18.600 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2024-08-20T02:52:18.601 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to enable service . firewalld.service is not available 2024-08-20T02:52:18.611 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-20T02:52:18.611 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2024-08-20T02:52:18.620 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-20T02:52:18.620 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-08-20T02:52:18.620 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2024-08-20T02:52:18.620 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-08-20T02:52:18.620 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr to start... 2024-08-20T02:52:18.620 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr... 2024-08-20T02:52:18.743 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:18 smithi044 systemd[1]: Started Ceph mgr.x for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:52:19.323 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:19 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1219248650' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-20T02:52:19.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-08-20T02:52:19.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-08-20T02:52:19.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T02:52:19.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2024-08-20T02:52:19.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-20T02:52:19.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-20T02:52:19.622 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-20T02:52:19.623 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:19.623 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-20T02:52:19.623 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-20T02:52:19.623 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2024-08-20T02:52:19.623 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-08-20T02:52:19.623 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-20T02:52:19.623 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2024-08-20T02:52:19.623 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-08-20T02:52:19.623 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-08-20T02:52:19.623 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-20T02:52:19.623 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T02:52:19.624 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-20T02:52:19.624 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-20T02:52:19.624 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:19.624 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-20T02:52:19.624 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T02:52:19.624 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-20T02:52:19.624 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-20T02:52:19.624 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-20T02:52:19.630 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-20T02:52:19.630 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-20T02:52:19.630 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-20T02:52:19.630 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:19.630 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-20T02:52:19.630 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-20T02:52:19.630 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-20T02:52:19.630 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-20T02:52:19.630 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-20T02:52:19.630 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-20T02:52:19.631 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-20T02:52:19.631 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-20T02:52:19.631 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-20T02:52:19.631 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:19.631 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-20T02:52:19.631 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T02:52:19.631 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "btime": "2024-08-20T02:52:10:979066+0000", 2024-08-20T02:52:19.631 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-20T02:52:19.631 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-20T02:52:19.631 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:19.631 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2024-08-20T02:52:10.979768+0000", 2024-08-20T02:52:19.632 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T02:52:19.633 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:19.633 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-20T02:52:19.633 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-08-20T02:52:19.633 INFO:teuthology.orchestra.run.smithi044.stdout:mgr not available, waiting (1/15)... 2024-08-20T02:52:22.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:22 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1723228514' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-20T02:52:23.053 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-08-20T02:52:23.054 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-08-20T02:52:23.054 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T02:52:23.054 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2024-08-20T02:52:23.054 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-20T02:52:23.054 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-20T02:52:23.054 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-20T02:52:23.054 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:23.054 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-20T02:52:23.054 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-20T02:52:23.054 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2024-08-20T02:52:23.054 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-20T02:52:23.055 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-20T02:52:23.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-20T02:52:23.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-20T02:52:23.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-20T02:52:23.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:23.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-20T02:52:23.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-20T02:52:23.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-20T02:52:23.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-20T02:52:23.059 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "btime": "2024-08-20T02:52:10:979066+0000", 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-20T02:52:23.060 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2024-08-20T02:52:10.979768+0000", 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:23.061 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-20T02:52:23.062 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-08-20T02:52:23.062 INFO:teuthology.orchestra.run.smithi044.stdout:mgr not available, waiting (2/15)... 2024-08-20T02:52:24.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: Activating manager daemon x 2024-08-20T02:52:24.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: mgrmap e2: x(active, starting, since 0.00303176s) 2024-08-20T02:52:24.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: from='mgr.14100 172.21.15.44:0/3522587560' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T02:52:24.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: from='mgr.14100 172.21.15.44:0/3522587560' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T02:52:24.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: from='mgr.14100 172.21.15.44:0/3522587560' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T02:52:24.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: from='mgr.14100 172.21.15.44:0/3522587560' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:52:24.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: from='mgr.14100 172.21.15.44:0/3522587560' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T02:52:24.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: Manager daemon x is now available 2024-08-20T02:52:24.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: from='mgr.14100 172.21.15.44:0/3522587560' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T02:52:24.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: from='mgr.14100 172.21.15.44:0/3522587560' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T02:52:24.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: from='mgr.14100 172.21.15.44:0/3522587560' entity='mgr.x' 2024-08-20T02:52:24.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: from='mgr.14100 172.21.15.44:0/3522587560' entity='mgr.x' 2024-08-20T02:52:24.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:23 smithi044 ceph-mon[18790]: from='mgr.14100 172.21.15.44:0/3522587560' entity='mgr.x' 2024-08-20T02:52:25.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:25 smithi044 ceph-mon[18790]: mgrmap e3: x(active, since 1.00726s) 2024-08-20T02:52:25.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:25 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2577608839' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-20T02:52:26.304 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-08-20T02:52:26.305 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-08-20T02:52:26.305 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T02:52:26.305 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "health": { 2024-08-20T02:52:26.305 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-20T02:52:26.305 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-20T02:52:26.305 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-20T02:52:26.305 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:26.305 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-20T02:52:26.305 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-20T02:52:26.305 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 0 2024-08-20T02:52:26.305 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "a" 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-20T02:52:26.306 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-20T02:52:26.307 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-20T02:52:26.310 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-20T02:52:26.310 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-20T02:52:26.310 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-20T02:52:26.311 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:26.311 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-20T02:52:26.311 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-20T02:52:26.311 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-20T02:52:26.311 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-20T02:52:26.311 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "btime": "2024-08-20T02:52:10:979066+0000", 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-20T02:52:26.312 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "restful" 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ], 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "modified": "2024-08-20T02:52:10.979768+0000", 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout }, 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-20T02:52:26.313 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-08-20T02:52:26.314 INFO:teuthology.orchestra.run.smithi044.stdout:mgr is available 2024-08-20T02:52:26.849 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:26 smithi044 ceph-mon[18790]: mgrmap e4: x(active, since 2s) 2024-08-20T02:52:27.476 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-08-20T02:52:27.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [global] 2024-08-20T02:52:27.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout fsid = 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:52:27.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-08-20T02:52:27.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.44:3300,v1:172.21.15.44:6789] 2024-08-20T02:52:27.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-20T02:52:27.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-20T02:52:27.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-20T02:52:27.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-20T02:52:27.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-08-20T02:52:27.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-20T02:52:27.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-20T02:52:27.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 2024-08-20T02:52:27.478 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout [osd] 2024-08-20T02:52:27.478 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-20T02:52:27.478 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-20T02:52:27.478 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-20T02:52:27.478 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-20T02:52:27.478 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-20T02:52:27.478 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling cephadm module... 2024-08-20T02:52:28.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:27 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1588498449' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-08-20T02:52:29.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:28 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1617598134' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-08-20T02:52:29.036 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:28 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: ignoring --setuser ceph since I am not root 2024-08-20T02:52:29.036 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:28 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: ignoring --setgroup ceph since I am not root 2024-08-20T02:52:30.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:29 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1617598134' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-08-20T02:52:30.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:29 smithi044 ceph-mon[18790]: mgrmap e5: x(active, since 5s) 2024-08-20T02:52:30.737 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-08-20T02:52:30.738 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-08-20T02:52:30.738 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-20T02:52:30.738 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-20T02:52:30.738 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-20T02:52:30.738 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-08-20T02:52:30.738 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for the mgr to restart... 2024-08-20T02:52:30.738 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr epoch 5... 2024-08-20T02:52:31.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:30 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/459444097' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-08-20T02:52:34.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:34 smithi044 ceph-mon[18790]: Active manager daemon x restarted 2024-08-20T02:52:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:34 smithi044 ceph-mon[18790]: Activating manager daemon x 2024-08-20T02:52:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:34 smithi044 ceph-mon[18790]: osdmap e2: 0 total, 0 up, 0 in 2024-08-20T02:52:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:34 smithi044 ceph-mon[18790]: mgrmap e6: x(active, starting, since 0.00320462s) 2024-08-20T02:52:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:34 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:52:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:34 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T02:52:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:34 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T02:52:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:34 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T02:52:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:34 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T02:52:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:34 smithi044 ceph-mon[18790]: Manager daemon x is now available 2024-08-20T02:52:35.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-08-20T02:52:35.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-08-20T02:52:35.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-20T02:52:35.535 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-08-20T02:52:35.535 INFO:teuthology.orchestra.run.smithi044.stdout:mgr epoch 5 is available 2024-08-20T02:52:35.535 INFO:teuthology.orchestra.run.smithi044.stdout:Setting orchestrator backend to cephadm... 2024-08-20T02:52:35.918 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:35 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:35.918 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:35 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:35.918 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:35 smithi044 ceph-mon[18790]: Found migration_current of "None". Setting to last migration. 2024-08-20T02:52:35.919 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:35 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:35.919 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:35 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:35.919 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:35 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:52:35.919 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:35 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:52:35.919 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:35 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T02:52:35.919 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:35 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T02:52:35.919 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:35 smithi044 ceph-mon[18790]: mgrmap e7: x(active, since 1.00538s) 2024-08-20T02:52:36.739 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T02:52:36.361+0000 7fccc1ecc640 -1 log_channel(cephadm) log [ERR] : [20/Aug/2024:02:52:36] ENGINE Error in HTTPServer.serve 2024-08-20T02:52:36.739 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: Traceback (most recent call last): 2024-08-20T02:52:36.739 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T02:52:36.739 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: self._connections.run(self.expiration_interval) 2024-08-20T02:52:36.739 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: self._run(expiration_interval) 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: s = self.context.wrap_socket( 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: return self.sslsocket_class._create( 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: self.do_handshake() 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T02:52:36.740 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: self._sslobj.do_handshake() 2024-08-20T02:52:36.741 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T02:52:36.741 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T02:52:37.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-mon[18790]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-20T02:52:37.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-mon[18790]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-20T02:52:37.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:37.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:52:37.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:36 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:52:37.928 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:37 smithi044 ceph-mon[18790]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:52:37.928 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:37 smithi044 ceph-mon[18790]: [20/Aug/2024:02:52:36] ENGINE Bus STARTING 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:37 smithi044 ceph-mon[18790]: [20/Aug/2024:02:52:36] ENGINE Serving on https://172.21.15.44:7150 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:37 smithi044 ceph-mon[18790]: [20/Aug/2024:02:52:36] ENGINE Error in HTTPServer.serve 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout: Traceback (most recent call last): 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._connections.run(self.expiration_interval) 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._run(expiration_interval) 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T02:52:37.929 INFO:journalctl@ceph.mon.a.smithi044.stdout: s = self.context.wrap_socket( 2024-08-20T02:52:37.930 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T02:52:37.930 INFO:journalctl@ceph.mon.a.smithi044.stdout: return self.sslsocket_class._create( 2024-08-20T02:52:37.930 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T02:52:37.930 INFO:journalctl@ceph.mon.a.smithi044.stdout: self.do_handshake() 2024-08-20T02:52:37.930 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T02:52:37.930 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._sslobj.do_handshake() 2024-08-20T02:52:37.930 INFO:journalctl@ceph.mon.a.smithi044.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T02:52:37.930 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:37 smithi044 ceph-mon[18790]: [20/Aug/2024:02:52:36] ENGINE Serving on http://172.21.15.44:8765 2024-08-20T02:52:37.930 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:37 smithi044 ceph-mon[18790]: [20/Aug/2024:02:52:36] ENGINE Bus STARTED 2024-08-20T02:52:37.930 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:37 smithi044 ceph-mon[18790]: mgrmap e8: x(active, since 2s) 2024-08-20T02:52:37.931 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout value unchanged 2024-08-20T02:52:37.931 INFO:teuthology.orchestra.run.smithi044.stdout:Generating ssh key... 2024-08-20T02:52:39.074 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:38 smithi044 ceph-mon[18790]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:52:39.351 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: Generating public/private rsa key pair. 2024-08-20T02:52:39.351 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: Your identification has been saved in /tmp/tmpced0d5mn/key 2024-08-20T02:52:39.351 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: Your public key has been saved in /tmp/tmpced0d5mn/key.pub 2024-08-20T02:52:39.351 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: The key fingerprint is: 2024-08-20T02:52:39.351 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: SHA256:NwdfzCFcygZ4zxsyMYRueMPtpzeQbX9i1hJA8aeyaSM ceph-0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:52:39.351 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: The key's randomart image is: 2024-08-20T02:52:39.352 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: +---[RSA 3072]----+ 2024-08-20T02:52:39.352 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: | +o.ooo | 2024-08-20T02:52:39.352 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: | o +oo* . | 2024-08-20T02:52:39.352 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: | + o.*+ = .| 2024-08-20T02:52:39.352 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: | . * ++=. o | 2024-08-20T02:52:39.352 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: | oSoo=++. | 2024-08-20T02:52:39.352 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: | .+o=+. | 2024-08-20T02:52:39.352 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: | E==. o | 2024-08-20T02:52:39.352 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: | .oo.* o| 2024-08-20T02:52:39.352 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: | . + + | 2024-08-20T02:52:39.352 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: +----[SHA256]-----+ 2024-08-20T02:52:39.904 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-mon[18790]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:52:39.904 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-mon[18790]: Generating ssh key... 2024-08-20T02:52:39.904 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:39.905 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:39 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:40.785 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvZJ0wABQHh5SYqVhzJWLARRPo8tlAh6K75a/46HhwQaHdfkhHiwX4ZZaPoORxNdZUWvQNIpSo3BxuvvLGa4NnVzC7XhOHtbozDf4C0EB+c0lKjQUDeeNTDrtEhVMgwutj6WH1V2mKPMhMk4TFLGwa08nDoz5YmzPeKopIPHyuqNyyVUZxTOi5WAFvyUC5U61E5xE5JMqtrgcVQ2YhtjTcmZFND2MU3kmRhoYt1o2WHZk57H+h/S4UJfoFomNgFn9ItfWfBy6JuT+5sYA3t9uz9kERY3q1f5aOF49OmkUOKvcaYmcnzsn5lAmyVYTTBqbZBSWGUrAxTT66t+3oHGaw4QB5MDHKxxuSgw1i4nrFzHenPCzTBd+vRmL//20tD3S1R+yLXhKrC6ekHYDym39v3opxRB6pLwsb2EO0KG5vzfo3MtDjZXJUri9X0+PJFBSR5+AlILM0AlY4gxbLglm8YyE1FLf6f1LQNhwyZX4xVKCd8go6hxQJwsY6uD5fxz8= ceph-0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:52:40.786 INFO:teuthology.orchestra.run.smithi044.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-08-20T02:52:40.786 INFO:teuthology.orchestra.run.smithi044.stdout:Adding key to root@localhost authorized_keys... 2024-08-20T02:52:40.792 INFO:teuthology.orchestra.run.smithi044.stdout:Adding host smithi044... 2024-08-20T02:52:41.034 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:40 smithi044 ceph-mon[18790]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:52:42.101 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:41 smithi044 ceph-mon[18790]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "addr": "172.21.15.44", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:52:43.005 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:42 smithi044 ceph-mon[18790]: Deploying cephadm binary to smithi044 2024-08-20T02:52:45.953 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Added host 'smithi044' with addr '172.21.15.44' 2024-08-20T02:52:45.953 INFO:teuthology.orchestra.run.smithi044.stdout:Deploying unmanaged mon service... 2024-08-20T02:52:46.473 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:46 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:46.473 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:46 smithi044 ceph-mon[18790]: Added host smithi044 2024-08-20T02:52:46.473 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:46 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:52:47.431 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-08-20T02:52:47.431 INFO:teuthology.orchestra.run.smithi044.stdout:Deploying unmanaged mgr service... 2024-08-20T02:52:48.217 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:47 smithi044 ceph-mon[18790]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:52:48.218 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:47 smithi044 ceph-mon[18790]: Saving service mon spec with placement count:5 2024-08-20T02:52:48.218 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:47 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:48.923 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-08-20T02:52:49.240 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:49 smithi044 ceph-mon[18790]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:52:49.241 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:49 smithi044 ceph-mon[18790]: Saving service mgr spec with placement count:2 2024-08-20T02:52:49.241 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:49 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:49.241 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:49 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:49.241 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:49 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:50.556 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:50 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1895982337' entity='client.admin' 2024-08-20T02:52:51.690 INFO:teuthology.orchestra.run.smithi044.stdout:Enabling the dashboard module... 2024-08-20T02:52:52.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:52 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:52.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:52 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2106109466' entity='client.admin' 2024-08-20T02:52:53.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:53 smithi044 ceph-mon[18790]: from='mgr.14118 172.21.15.44:0/3337434737' entity='mgr.x' 2024-08-20T02:52:53.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:53 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1255683119' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-08-20T02:52:53.536 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:53 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: ignoring --setuser ceph since I am not root 2024-08-20T02:52:53.536 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:52:53 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: ignoring --setgroup ceph since I am not root 2024-08-20T02:52:54.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:54 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1255683119' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-08-20T02:52:54.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:54 smithi044 ceph-mon[18790]: mgrmap e9: x(active, since 19s) 2024-08-20T02:52:55.473 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:55 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/275939206' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-08-20T02:52:55.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-08-20T02:52:55.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-08-20T02:52:55.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-20T02:52:55.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-20T02:52:55.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-20T02:52:55.477 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-08-20T02:52:55.477 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for the mgr to restart... 2024-08-20T02:52:55.477 INFO:teuthology.orchestra.run.smithi044.stdout:Waiting for mgr epoch 9... 2024-08-20T02:52:58.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:58 smithi044 ceph-mon[18790]: Active manager daemon x restarted 2024-08-20T02:52:58.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:58 smithi044 ceph-mon[18790]: Activating manager daemon x 2024-08-20T02:52:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:58 smithi044 ceph-mon[18790]: osdmap e3: 0 total, 0 up, 0 in 2024-08-20T02:52:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:58 smithi044 ceph-mon[18790]: mgrmap e10: x(active, starting, since 0.00402943s) 2024-08-20T02:52:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:58 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:52:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:58 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T02:52:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:58 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T02:52:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:58 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T02:52:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:58 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T02:52:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:58 smithi044 ceph-mon[18790]: Manager daemon x is now available 2024-08-20T02:52:59.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:59 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:52:59.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:59 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T02:52:59.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:59 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T02:52:59.772 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:52:59 smithi044 ceph-mon[18790]: mgrmap e11: x(active, since 1.0061s) 2024-08-20T02:53:00.032 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout { 2024-08-20T02:53:00.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-08-20T02:53:00.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-20T02:53:00.033 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout } 2024-08-20T02:53:00.033 INFO:teuthology.orchestra.run.smithi044.stdout:mgr epoch 9 is available 2024-08-20T02:53:00.033 INFO:teuthology.orchestra.run.smithi044.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-08-20T02:53:00.754 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:00 smithi044 ceph-mon[18790]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-20T02:53:00.755 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:00 smithi044 ceph-mon[18790]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-20T02:53:01.648 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-mon[18790]: mgrmap e12: x(active, since 2s) 2024-08-20T02:53:01.649 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-mon[18790]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:53:01.649 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:01.649 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T02:53:01.447+0000 7f8245f00640 -1 log_channel(cephadm) log [ERR] : [20/Aug/2024:02:53:01] ENGINE Error in HTTPServer.serve 2024-08-20T02:53:01.649 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: Traceback (most recent call last): 2024-08-20T02:53:01.649 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T02:53:01.649 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: self._connections.run(self.expiration_interval) 2024-08-20T02:53:01.649 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T02:53:01.650 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: self._run(expiration_interval) 2024-08-20T02:53:01.650 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T02:53:01.650 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T02:53:01.650 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T02:53:01.650 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T02:53:01.650 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T02:53:01.650 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: s = self.context.wrap_socket( 2024-08-20T02:53:01.650 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T02:53:01.650 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: return self.sslsocket_class._create( 2024-08-20T02:53:01.651 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T02:53:01.651 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: self.do_handshake() 2024-08-20T02:53:01.651 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T02:53:01.651 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: self._sslobj.do_handshake() 2024-08-20T02:53:01.651 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T02:53:01.651 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T02:53:01.909 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUFa/Kg8h0DEArV+wVKbzpW6qpSeYwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDgyMDAyNTMwMVoXDTM0\nMDgyMTAyNTMwMVowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjQ0MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAvGfh8dsxomGPH/tihSZGiQLTgvl4JwOKMByS\n0t0UmXNnYmhyK/kTqdKW5pQ+0VbteV7aveOfP54IKFW9MLodYqLzPMY7qqD2xJ0P\nKuZtKmUNNXQYBGhzNV0nXPY3PQ7bDgIfuGCZKpseK8qOLXaD1a/RUISdD9Dr1d4D\n8m32oZqYc2BUykVvf91jv/4AFbzFy4TbjCQbOEQkiLSkdq3dcFYO7BjbUxfReJJz\nmhSr96OCOS9CPVu/CkQdGOJndHFNYb+xKWIK9FHx2c8oAZJhtTTmkRJrLIVTWGIV\npwCWqSp3q/dE9IMNFcjQ/WvZQjx2e5VdUayluZGSiJrWF15mIaHxV2gQhKV76Sba\n1rCZv1pHzgw/p+19JBH1feJzRcgnxdUFGEMMR8eSrGcVRxVfgUf8/cGQR25uYler\nyjq29IFR64kjDuMCgvV1ov81NcVqlgNmpUw2rsa4A24nuxIySaTjrXdWLXu16LNw\ng1RIR7eSAn+qZm4zpHVfylIqAS9wEj83A6hwbJY1ZztmUM40Ygv+VDNVVPXbO2zD\nYTNFk73+eET6vh2s4ZPQWdkihyrKoN+gfmagEXLe2uwZ5+xnpTiCjgLH8QDPr5yR\n1XELAIWMCyErpDCwGvjuBcqVSrzHBDCoQ77zDH8aOJb93GHpi90EBKrNTybDDWHI\nQx3aaWUCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDQ0ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPLDAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQBg\nJOB4NV36DHPtSLJNXQOjH+Y2ewyxOxpSDw4RZUCCN7o0nt6BThdaPcoyW74EX8ul\nu8206lzAgNbDGdc3I/Sv4d7aHTJFvU9yDMWb130TvViJblyZfFIahhgrltA+N3yb\nvvQsPrFjtXddPM1Af7yFvPgoZy47E2YI936ln3bFj6QHpO+SlII85aHNuX2aLJDW\n6vsAosURkowo/bXGoeqcT1tMPA+CJx8fWyLfshZwz6w+bCbbFGPFWFCW3/GOFK0h\nB183KYa6pyv2CMUl9srFpQbNA4r759t8JjHtKacKqz9f/hVnWMQKk9uahwivN7Js\nzMp/ssgQjExkzeOb6J5zAwSbG1UJCbuGbgIPN6Q7/y3vU3sKPmwEYdJGao12jVaE\nL5Vi4MMpBObPt8iD0zCDznV/Elx1rm3pL5kCA+RuPuLsVuy85QGvghB+bpwFrGEF\nt3Ana37JyErAzv6+e8tTXsRkgxrYt7MLrDvPPjDJdG+tnEFp9/+SQIoLDiUCa4EQ\nRTagJGxqHV3eQRStn2mVaN5TMLVUd6VIHOJvssj1sT4Yy9Vhom1oNOwcoaRv6jFg\n7ZuuapSM0JmTW/yGpfdF69dW5ISB31gggq94b/QEg4SihcYc4cEPpc6SLcoP40LB\nfz9F+f2+ehWQVmbEp4G9lNeZglxoEOMDA31VfP2/8g==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKQIBAAKCAgEAvGfh8dsxomGPH/tihSZGiQLTgvl4JwOKMByS0t0UmXNnYmhy\nK/kTqdKW5pQ+0VbteV7aveOfP54IKFW9MLodYqLzPMY7qqD2xJ0PKuZtKmUNNXQY\nBGhzNV0nXPY3PQ7bDgIfuGCZKpseK8qOLXaD1a/RUISdD9Dr1d4D8m32oZqYc2BU\nykVvf91jv/4AFbzFy4TbjCQbOEQkiLSkdq3dcFYO7BjbUxfReJJzmhSr96OCOS9C\nPVu/CkQdGOJndHFNYb+xKWIK9FHx2c8oAZJhtTTmkRJrLIVTWGIVpwCWqSp3q/dE\n9IMNFcjQ/WvZQjx2e5VdUayluZGSiJrWF15mIaHxV2gQhKV76Sba1rCZv1pHzgw/\np+19JBH1feJzRcgnxdUFGEMMR8eSrGcVRxVfgUf8/cGQR25uYleryjq29IFR64kj\nDuMCgvV1ov81NcVqlgNmpUw2rsa4A24nuxIySaTjrXdWLXu16LNwg1RIR7eSAn+q\nZm4zpHVfylIqAS9wEj83A6hwbJY1ZztmUM40Ygv+VDNVVPXbO2zDYTNFk73+eET6\nvh2s4ZPQWdkihyrKoN+gfmagEXLe2uwZ5+xnpTiCjgLH8QDPr5yR1XELAIWMCyEr\npDCwGvjuBcqVSrzHBDCoQ77zDH8aOJb93GHpi90EBKrNTybDDWHIQx3aaWUCAwEA\nAQKCAgADBTyqE/Qya5NP1UfkZT4G6Z9adNq9mrZBxIjWDeqhSyhX0N+DtPU2aRni\nOHOJXPYRrA5LB7XV30K3jX2q+lDeSUknqvCUm7VZEZYx/zULfkiSmCSKbYzrSMMI\nyn8Ni4oXf2aNc38BH9uBTY4EoecbKzu/1AsbOyOMxN0WAI1OVJwnvV5JSokEaRg1\noRd/Qbd/ZnHenUrnwVqUlnt3K50RNU/9Q6jAfEaaGD73iWf4YGFbaVaL2pywHXIZ\n+VyPPAHG6hEns1qu8AwLNzb8+iEhi+eRC9U2sYHUjj3/9L0gQ9NcqRTXQVfvBs59\nlu0KEs4Td8w3Mv1f1uWdKwj49u/MMyT/bKGA2Z6jNFSZ70Y7PRhPlAI2vFSi4pIl\nKFnMBkB3VHCCz8TrZZ6mG1XIUzPfScwKfjbl6EteFrH1L25O4ZRNnhbvIM6weVNx\nET4lABk5hZdQvr08xi6f7fMXTQ18+3NG4SOJHZNDvFsbv8vSOiUFJd/UDmNpLnTR\n5LoubB3B8Voxbfz+d39jTmxW2jcKCL4WVmvUezRQ6wEjGOhBwxYa/OZhCB6He3Aq\ncF4leMa+MG+mb8aSf60KPUl12KT6q0WQ3pKEZvAknLGDyIziembSAWonMmxRB/gO\nYD+COU7RegrN3yblYT8gtdPvlyj8k8uEcaAH3pvTVwhxIpLdQQKCAQEA91XGfgq9\nCZorbe67mGaTdWEu1z3PcL6H0l0xf5+xfJE7jggvPISmw5GUbK4QB0sX9+4yD9a3\nV9YXqvyfaTTDhkiq759VN5cEoDgf0rNCJ0FrTJvTaX9dy3gGAtZOQBDGMdSf1ysK\n76nQBeq7Sd5IpXPLx8o9KdE3Wrqs9CYh9B35VLMJmXBBgTKxYi8l7GN7dEJMo9cL\nQbZ9sZsbgIySWHqXnamX5gsqAfgk5V3y59U5UersckvXLZFRjPlkDeC/PVmzesVA\nblG6BJodt08vSkBnpOGBgf+D7M/2kwJF/OuT54UbY3IbqTKyxmPvSi0WW4z6+1/i\n/P3B8uYftQ6YHwKCAQEAwwGZnSBiIydjVY73EFzK69U9V6gpaRSRZVqjfl7bwWW/\nqSNtgWW4cANvNeHGaKvYVmTQ/g5TNXjZHbqRI9Mrz2CDlxsk8NMkA7e7NcvND6Ys\nTVRSXA95u2V8Tf5HGbxOl2GnMHk0eCMq9pNmhVIczhP9fC+cDpXcZ81QUfoCFURG\nnXzNLpKTPx33W0BHLGabRnyUG09y33tz01jQ5TxA6CV73ENb3MYzMR4/N9k/aERG\nndRiA1orwN32nJnnatVcLHupcRgObkxdBjss04+L4fa/XgmpXUPj+QKGC+guyLMg\nKEG3eYeiLniWzN7HTYZE6MBmpT2geXBaVJ8sa7hd+wKCAQEA7QseE0qDZiKFoPR+\ngk883/BmDg/FMg7I6Uo72E7ascIXS8Z4Tw3rLd/mWkpwPi0wKYP2IsvsJ5ieUleP\nCCtzgM1AVitkZsSrwEpJ5Yu7yRefmGp3sdJJuGmtiSaPFEN/T96x2mv8LTmYjzMq\nkF8jTL6v/SjMeblzzda/T7cEL4xSm2SDvLSjcwl8i9cYB3hIp2GrRhMhTLdzu7Ym\nI4uV7op/qLETTXxBpGWIJvNli98E8oeuWNRqw9dUS0HxLIPLhSblJrLJT3GwUBzM\nghS4A/e53YHqtwaGD344mShCn2W/+NHPL/36mKxoPk8k1PhlRxTUrfp193XUXdFD\nbikwjQKCAQEAmrlHR+ts0lWrNDSGBV6KxwnfbRCJIs3l+xQqitb65/0yUCJC/ZtW\n9tFSyN7FzUEkGpBNXnbBQLkNE9B23oTrytPaNOedrNLqTWTXTnzQzGzd09Fms5K3\n1J9WPI/XnCzT8rrRq91Pa08EBLa5bmjTYBNuWb1wxgXT5ix6YUDLrFIK8WQXbK9x\n02+UVruufSVAaHHqFa05CObr5LIgtPHU0h0DpKVxWr+L2JoWMNrXu4QIv6vGxMAj\n3RiAVoqqmhKF2umXbJtYv90pJguHqqjHpVdhSVzDG7QFSwrcjs//ZtDxX9+ckvmy\nDcsyY4UTf4hcXoxiJRO7ov8pPOE6npGfRwKCAQBnHF2JMzx1Xj44JCa64d2X7wOV\n4j6i+Jh1XzCi3/ldwAHGTlukrRTJluGcmL0kwc1cw97qEvvTWFO/zo6m8rKZfgYw\nrYOmC1xHP0ODUe7vJ2vVYEu7EjAL7L3Kh96GyQmaBqPGlxyaH+uB6pLxH+r75YI+\nziivNWIRAvTkjE2a1AcNYHTgau92KNzsQ668wCc5Q0bBJpMIkcNYEb74togUTdkI\nur27Sfv2StFVS9LtwIJtHVYVIVyBClmOmfoG+Zbm8syB453c8Cwl0HYGO+zfSIT4\nPHqAAX1aks14QDi9BiDoL9eofy1Juv3N4qhpDc+qpLxIRhxYmbscFu0T1FNl\n-----END RSA PRIVATE KEY-----\n"} 2024-08-20T02:53:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:02 smithi044 ceph-mon[18790]: [20/Aug/2024:02:53:01] ENGINE Bus STARTING 2024-08-20T02:53:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:02 smithi044 ceph-mon[18790]: [20/Aug/2024:02:53:01] ENGINE Serving on https://172.21.15.44:7150 2024-08-20T02:53:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:02 smithi044 ceph-mon[18790]: [20/Aug/2024:02:53:01] ENGINE Error in HTTPServer.serve 2024-08-20T02:53:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout: Traceback (most recent call last): 2024-08-20T02:53:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T02:53:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._connections.run(self.expiration_interval) 2024-08-20T02:53:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T02:53:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._run(expiration_interval) 2024-08-20T02:53:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T02:53:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout: s = self.context.wrap_socket( 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout: return self.sslsocket_class._create( 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout: self.do_handshake() 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._sslobj.do_handshake() 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:02 smithi044 ceph-mon[18790]: [20/Aug/2024:02:53:01] ENGINE Serving on http://172.21.15.44:8765 2024-08-20T02:53:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:02 smithi044 ceph-mon[18790]: [20/Aug/2024:02:53:01] ENGINE Bus STARTED 2024-08-20T02:53:03.288 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:02 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:03.288 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:02 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:03.713 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-08-20T02:53:04.028 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:03 smithi044 ceph-mon[18790]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:53:05.046 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-08-20T02:53:05.046 INFO:teuthology.orchestra.run.smithi044.stdout:Creating initial admin user... 2024-08-20T02:53:05.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:05 smithi044 ceph-mon[18790]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:53:05.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:05 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:06.502 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$WCiYxVKT9Ba0Qa8Gjp5qAu0FOMf4YgQNLjkSCtl2dKvp76C8I5xdO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1724122385, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-08-20T02:53:06.502 INFO:teuthology.orchestra.run.smithi044.stdout:Fetching dashboard port number... 2024-08-20T02:53:07.104 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:06 smithi044 ceph-mon[18790]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:53:07.104 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:06 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:07.777 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stdout 8443 2024-08-20T02:53:07.787 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-20T02:53:07.787 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout disabled 2024-08-20T02:53:07.795 INFO:teuthology.orchestra.run.smithi044.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-20T02:53:07.795 INFO:teuthology.orchestra.run.smithi044.stdout:systemctl: stdout inactive 2024-08-20T02:53:07.795 INFO:teuthology.orchestra.run.smithi044.stdout:firewalld.service is not enabled 2024-08-20T02:53:07.795 INFO:teuthology.orchestra.run.smithi044.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-08-20T02:53:07.797 INFO:teuthology.orchestra.run.smithi044.stdout:Ceph Dashboard is now available at: 2024-08-20T02:53:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:53:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: URL: https://smithi044:8443/ 2024-08-20T02:53:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: User: admin 2024-08-20T02:53:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: Password: ul8nvj8330 2024-08-20T02:53:07.797 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:53:07.798 INFO:teuthology.orchestra.run.smithi044.stdout:Saving cluster configuration to /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config directory 2024-08-20T02:53:08.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:07 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/700225542' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-08-20T02:53:08.985 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-08-20T02:53:08.986 INFO:teuthology.orchestra.run.smithi044.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-08-20T02:53:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:53:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:53:08.986 INFO:teuthology.orchestra.run.smithi044.stdout:Or, if you are only running a single cluster on this host: 2024-08-20T02:53:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:53:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-08-20T02:53:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:53:08.986 INFO:teuthology.orchestra.run.smithi044.stdout:Please consider enabling telemetry to help improve Ceph: 2024-08-20T02:53:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:53:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: ceph telemetry on 2024-08-20T02:53:08.986 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:53:08.987 INFO:teuthology.orchestra.run.smithi044.stdout:For more information see: 2024-08-20T02:53:08.987 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:53:08.987 INFO:teuthology.orchestra.run.smithi044.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-08-20T02:53:08.987 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:53:08.987 INFO:teuthology.orchestra.run.smithi044.stdout:Bootstrap complete. 2024-08-20T02:53:09.026 INFO:tasks.cephadm:Fetching config... 2024-08-20T02:53:09.026 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:53:09.027 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-08-20T02:53:09.043 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-08-20T02:53:09.043 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:53:09.043 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-08-20T02:53:09.097 INFO:tasks.cephadm:Fetching mon keyring... 2024-08-20T02:53:09.098 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:53:09.098 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/keyring of=/dev/stdout 2024-08-20T02:53:09.162 INFO:tasks.cephadm:Fetching pub ssh key... 2024-08-20T02:53:09.162 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:53:09.162 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-08-20T02:53:09.217 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-08-20T02:53:09.217 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvZJ0wABQHh5SYqVhzJWLARRPo8tlAh6K75a/46HhwQaHdfkhHiwX4ZZaPoORxNdZUWvQNIpSo3BxuvvLGa4NnVzC7XhOHtbozDf4C0EB+c0lKjQUDeeNTDrtEhVMgwutj6WH1V2mKPMhMk4TFLGwa08nDoz5YmzPeKopIPHyuqNyyVUZxTOi5WAFvyUC5U61E5xE5JMqtrgcVQ2YhtjTcmZFND2MU3kmRhoYt1o2WHZk57H+h/S4UJfoFomNgFn9ItfWfBy6JuT+5sYA3t9uz9kERY3q1f5aOF49OmkUOKvcaYmcnzsn5lAmyVYTTBqbZBSWGUrAxTT66t+3oHGaw4QB5MDHKxxuSgw1i4nrFzHenPCzTBd+vRmL//20tD3S1R+yLXhKrC6ekHYDym39v3opxRB6pLwsb2EO0KG5vzfo3MtDjZXJUri9X0+PJFBSR5+AlILM0AlY4gxbLglm8YyE1FLf6f1LQNhwyZX4xVKCd8go6hxQJwsY6uD5fxz8= ceph-0284e16c-5e9f-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-20T02:53:09.292 INFO:teuthology.orchestra.run.smithi044.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvZJ0wABQHh5SYqVhzJWLARRPo8tlAh6K75a/46HhwQaHdfkhHiwX4ZZaPoORxNdZUWvQNIpSo3BxuvvLGa4NnVzC7XhOHtbozDf4C0EB+c0lKjQUDeeNTDrtEhVMgwutj6WH1V2mKPMhMk4TFLGwa08nDoz5YmzPeKopIPHyuqNyyVUZxTOi5WAFvyUC5U61E5xE5JMqtrgcVQ2YhtjTcmZFND2MU3kmRhoYt1o2WHZk57H+h/S4UJfoFomNgFn9ItfWfBy6JuT+5sYA3t9uz9kERY3q1f5aOF49OmkUOKvcaYmcnzsn5lAmyVYTTBqbZBSWGUrAxTT66t+3oHGaw4QB5MDHKxxuSgw1i4nrFzHenPCzTBd+vRmL//20tD3S1R+yLXhKrC6ekHYDym39v3opxRB6pLwsb2EO0KG5vzfo3MtDjZXJUri9X0+PJFBSR5+AlILM0AlY4gxbLglm8YyE1FLf6f1LQNhwyZX4xVKCd8go6hxQJwsY6uD5fxz8= ceph-0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:53:09.304 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvZJ0wABQHh5SYqVhzJWLARRPo8tlAh6K75a/46HhwQaHdfkhHiwX4ZZaPoORxNdZUWvQNIpSo3BxuvvLGa4NnVzC7XhOHtbozDf4C0EB+c0lKjQUDeeNTDrtEhVMgwutj6WH1V2mKPMhMk4TFLGwa08nDoz5YmzPeKopIPHyuqNyyVUZxTOi5WAFvyUC5U61E5xE5JMqtrgcVQ2YhtjTcmZFND2MU3kmRhoYt1o2WHZk57H+h/S4UJfoFomNgFn9ItfWfBy6JuT+5sYA3t9uz9kERY3q1f5aOF49OmkUOKvcaYmcnzsn5lAmyVYTTBqbZBSWGUrAxTT66t+3oHGaw4QB5MDHKxxuSgw1i4nrFzHenPCzTBd+vRmL//20tD3S1R+yLXhKrC6ekHYDym39v3opxRB6pLwsb2EO0KG5vzfo3MtDjZXJUri9X0+PJFBSR5+AlILM0AlY4gxbLglm8YyE1FLf6f1LQNhwyZX4xVKCd8go6hxQJwsY6uD5fxz8= ceph-0284e16c-5e9f-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-20T02:53:09.433 INFO:teuthology.orchestra.run.smithi099.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvZJ0wABQHh5SYqVhzJWLARRPo8tlAh6K75a/46HhwQaHdfkhHiwX4ZZaPoORxNdZUWvQNIpSo3BxuvvLGa4NnVzC7XhOHtbozDf4C0EB+c0lKjQUDeeNTDrtEhVMgwutj6WH1V2mKPMhMk4TFLGwa08nDoz5YmzPeKopIPHyuqNyyVUZxTOi5WAFvyUC5U61E5xE5JMqtrgcVQ2YhtjTcmZFND2MU3kmRhoYt1o2WHZk57H+h/S4UJfoFomNgFn9ItfWfBy6JuT+5sYA3t9uz9kERY3q1f5aOF49OmkUOKvcaYmcnzsn5lAmyVYTTBqbZBSWGUrAxTT66t+3oHGaw4QB5MDHKxxuSgw1i4nrFzHenPCzTBd+vRmL//20tD3S1R+yLXhKrC6ekHYDym39v3opxRB6pLwsb2EO0KG5vzfo3MtDjZXJUri9X0+PJFBSR5+AlILM0AlY4gxbLglm8YyE1FLf6f1LQNhwyZX4xVKCd8go6hxQJwsY6uD5fxz8= ceph-0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:53:09.447 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvZJ0wABQHh5SYqVhzJWLARRPo8tlAh6K75a/46HhwQaHdfkhHiwX4ZZaPoORxNdZUWvQNIpSo3BxuvvLGa4NnVzC7XhOHtbozDf4C0EB+c0lKjQUDeeNTDrtEhVMgwutj6WH1V2mKPMhMk4TFLGwa08nDoz5YmzPeKopIPHyuqNyyVUZxTOi5WAFvyUC5U61E5xE5JMqtrgcVQ2YhtjTcmZFND2MU3kmRhoYt1o2WHZk57H+h/S4UJfoFomNgFn9ItfWfBy6JuT+5sYA3t9uz9kERY3q1f5aOF49OmkUOKvcaYmcnzsn5lAmyVYTTBqbZBSWGUrAxTT66t+3oHGaw4QB5MDHKxxuSgw1i4nrFzHenPCzTBd+vRmL//20tD3S1R+yLXhKrC6ekHYDym39v3opxRB6pLwsb2EO0KG5vzfo3MtDjZXJUri9X0+PJFBSR5+AlILM0AlY4gxbLglm8YyE1FLf6f1LQNhwyZX4xVKCd8go6hxQJwsY6uD5fxz8= ceph-0284e16c-5e9f-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-20T02:53:09.502 INFO:teuthology.orchestra.run.smithi165.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvZJ0wABQHh5SYqVhzJWLARRPo8tlAh6K75a/46HhwQaHdfkhHiwX4ZZaPoORxNdZUWvQNIpSo3BxuvvLGa4NnVzC7XhOHtbozDf4C0EB+c0lKjQUDeeNTDrtEhVMgwutj6WH1V2mKPMhMk4TFLGwa08nDoz5YmzPeKopIPHyuqNyyVUZxTOi5WAFvyUC5U61E5xE5JMqtrgcVQ2YhtjTcmZFND2MU3kmRhoYt1o2WHZk57H+h/S4UJfoFomNgFn9ItfWfBy6JuT+5sYA3t9uz9kERY3q1f5aOF49OmkUOKvcaYmcnzsn5lAmyVYTTBqbZBSWGUrAxTT66t+3oHGaw4QB5MDHKxxuSgw1i4nrFzHenPCzTBd+vRmL//20tD3S1R+yLXhKrC6ekHYDym39v3opxRB6pLwsb2EO0KG5vzfo3MtDjZXJUri9X0+PJFBSR5+AlILM0AlY4gxbLglm8YyE1FLf6f1LQNhwyZX4xVKCd8go6hxQJwsY6uD5fxz8= ceph-0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:53:09.516 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-08-20T02:53:09.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:09 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/927983434' entity='client.admin' 2024-08-20T02:53:09.771 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:53:10.991 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-08-20T02:53:10.991 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-08-20T02:53:11.387 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:53:11.627 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:11 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/219837262' entity='client.admin' 2024-08-20T02:53:12.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:12 smithi044 ceph-mon[18790]: mgrmap e13: x(active, since 12s) 2024-08-20T02:53:12.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:12 smithi044 ceph-mon[18790]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:53:12.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:12 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:12.859 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi099 2024-08-20T02:53:12.859 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-08-20T02:53:12.859 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.conf 2024-08-20T02:53:12.878 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-08-20T02:53:12.878 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:12.936 INFO:tasks.cephadm:Adding host smithi099 to orchestrator... 2024-08-20T02:53:12.937 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch host add smithi099 2024-08-20T02:53:13.516 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:53:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"} : dispatch 2024-08-20T02:53:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:53:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: Updating smithi044:/etc/ceph/ceph.conf 2024-08-20T02:53:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: Updating smithi044:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: Updating smithi044:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:53:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:53:14.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:14.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:14.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:14.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:53:14.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:14.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:53:14.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:14 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:16.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:16 smithi044 ceph-mon[18790]: Deploying cephadm binary to smithi099 2024-08-20T02:53:18.152 INFO:teuthology.orchestra.run.smithi044.stdout:Added host 'smithi099' with addr '172.21.15.99' 2024-08-20T02:53:18.698 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-20T02:53:18.944 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:53:19.381 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:19 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:19.382 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:19 smithi044 ceph-mon[18790]: Added host smithi099 2024-08-20T02:53:19.382 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:19 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:53:19.382 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:19 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:19.382 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:19 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:19.571 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:53:19.571 INFO:teuthology.orchestra.run.smithi044.stdout:[{"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}] 2024-08-20T02:53:20.122 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi165 2024-08-20T02:53:20.122 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-08-20T02:53:20.122 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.conf 2024-08-20T02:53:20.141 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-08-20T02:53:20.141 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:20.200 INFO:tasks.cephadm:Adding host smithi165 to orchestrator... 2024-08-20T02:53:20.200 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch host add smithi165 2024-08-20T02:53:20.446 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:53:20.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:20 smithi044 ceph-mon[18790]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:20.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:20 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:21.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:21 smithi044 ceph-mon[18790]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T02:53:22.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:22 smithi044 ceph-mon[18790]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:22.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:22 smithi044 ceph-mon[18790]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi165", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:53:23.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:23 smithi044 ceph-mon[18790]: Deploying cephadm binary to smithi165 2024-08-20T02:53:23.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:23 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:23.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:23 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:23.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:23 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:23.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:23 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:23.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:23 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"} : dispatch 2024-08-20T02:53:23.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:23 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:23.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:23 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:53:24.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:24 smithi044 ceph-mon[18790]: Updating smithi099:/etc/ceph/ceph.conf 2024-08-20T02:53:24.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:24 smithi044 ceph-mon[18790]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:24.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:24 smithi044 ceph-mon[18790]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:24.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:24 smithi044 ceph-mon[18790]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:24.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:24 smithi044 ceph-mon[18790]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:53:24.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:24 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:24.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:24 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:24.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:24 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:24.934 INFO:teuthology.orchestra.run.smithi044.stdout:Added host 'smithi165' with addr '172.21.15.165' 2024-08-20T02:53:25.529 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-20T02:53:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:53:26.204 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:25 smithi044 ceph-mon[18790]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:26.204 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:25 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:26.204 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:25 smithi044 ceph-mon[18790]: Added host smithi165 2024-08-20T02:53:26.204 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:25 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:53:26.204 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:25 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:26.204 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:25 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:26.411 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:53:26.411 INFO:teuthology.orchestra.run.smithi044.stdout:[{"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.165", "hostname": "smithi165", "labels": [], "status": ""}] 2024-08-20T02:53:27.029 INFO:tasks.cephadm:Setting crush tunables to default 2024-08-20T02:53:27.029 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd crush tunables default 2024-08-20T02:53:27.277 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:53:27.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:27 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:28.280 INFO:teuthology.orchestra.run.smithi044.stderr:adjusted tunables profile to default 2024-08-20T02:53:28.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:28 smithi044 ceph-mon[18790]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T02:53:28.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:28 smithi044 ceph-mon[18790]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:28.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:28 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/280895863' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-20T02:53:28.921 INFO:tasks.cephadm:Adding mon.a on smithi044 2024-08-20T02:53:28.922 INFO:tasks.cephadm:Adding mon.b on smithi099 2024-08-20T02:53:28.922 INFO:tasks.cephadm:Adding mon.c on smithi165 2024-08-20T02:53:28.922 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch apply mon '3;smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi165:172.21.15.165=c' 2024-08-20T02:53:29.168 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-20T02:53:29.213 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-20T02:53:29.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:29 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/280895863' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-20T02:53:29.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:29 smithi044 ceph-mon[18790]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T02:53:29.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:29 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:29.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:29 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:29.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:29 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:29.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:29 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:29.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:29 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-08-20T02:53:29.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:29 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:29.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:29 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:53:29.929 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mon update... 2024-08-20T02:53:30.617 DEBUG:teuthology.orchestra.run.smithi099:mon.b> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.b.service 2024-08-20T02:53:30.620 DEBUG:teuthology.orchestra.run.smithi165:mon.c> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.c.service 2024-08-20T02:53:30.622 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-20T02:53:30.623 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-20T02:53:30.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:30 smithi044 ceph-mon[18790]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:30.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:30 smithi044 ceph-mon[18790]: Updating smithi165:/etc/ceph/ceph.conf 2024-08-20T02:53:30.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:30 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:30.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:30 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:30.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:30 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:30.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:30 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:30.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:30 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:53:30.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:30 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:31.640 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:31.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:31 smithi044 ceph-mon[18790]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:31.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:31 smithi044 ceph-mon[18790]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:31.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:31 smithi044 ceph-mon[18790]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:53:31.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:31 smithi044 ceph-mon[18790]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:53:31.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:31 smithi044 ceph-mon[18790]: Saving service mon spec with placement smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi165:172.21.15.165=c;count:3 2024-08-20T02:53:31.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:31 smithi044 ceph-mon[18790]: Deploying daemon mon.c on smithi165 2024-08-20T02:53:32.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:32 smithi044 ceph-mon[18790]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:33.094 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:53:33.094 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":1,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","modified":"2024-08-20T02:52:08.003300Z","created":"2024-08-20T02:52:08.003300Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-20T02:53:33.094 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 1 2024-08-20T02:53:33.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:33 smithi044 ceph-mon[18790]: from='client.? 172.21.15.165:0/2624137318' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T02:53:34.718 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: mon.c@-1(synchronizing).mds e1 print_map 2024-08-20T02:53:34.718 INFO:journalctl@ceph.mon.c.smithi165.stdout: e1 2024-08-20T02:53:34.718 INFO:journalctl@ceph.mon.c.smithi165.stdout: btime 2024-08-20T02:52:10:979066+0000 2024-08-20T02:53:34.718 INFO:journalctl@ceph.mon.c.smithi165.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-20T02:53:34.718 INFO:journalctl@ceph.mon.c.smithi165.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T02:53:34.718 INFO:journalctl@ceph.mon.c.smithi165.stdout: legacy client fscid: -1 2024-08-20T02:53:34.718 INFO:journalctl@ceph.mon.c.smithi165.stdout: 2024-08-20T02:53:34.719 INFO:journalctl@ceph.mon.c.smithi165.stdout: No filesystems configured 2024-08-20T02:53:34.719 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-20T02:53:34.719 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-20T02:53:34.719 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-20T02:53:34.719 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-20T02:53:34.719 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-20T02:53:34.719 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-20T02:53:34.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:34 smithi044 ceph-mon[18790]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:34.941 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-20T02:53:34.941 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-20T02:53:34.968 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-20T02:53:34.968 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T02:53:34.968 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T02:53:34.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T02:53:34.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: Deploying cephadm binary to smithi165 2024-08-20T02:53:34.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"} : dispatch 2024-08-20T02:53:34.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:34.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:53:34.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: Updating smithi099:/etc/ceph/ceph.conf 2024-08-20T02:53:34.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:34.969 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:34.970 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:34.970 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:53:34.970 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.970 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.970 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.970 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:34.970 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.970 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: Added host smithi165 2024-08-20T02:53:34.970 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:53:34.970 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.970 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.970 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/280895863' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/280895863' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:34.971 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: Updating smithi165:/etc/ceph/ceph.conf 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: Saving service mon spec with placement smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi165:172.21.15.165=c;count:3 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: Deploying daemon mon.c on smithi165 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: from='client.? 172.21.15.165:0/2624137318' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:34.972 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 ceph-mon[22357]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-20T02:53:34.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 bash[22338]: b9c55899951908ba9ae4fcb88b9b5bd7023b2b3aaadfc278e664f7062d9da6f2 2024-08-20T02:53:34.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:34 smithi165 systemd[1]: Started Ceph mon.c for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:53:35.188 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:53:40.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:40.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:53:40.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: mon.a calling monitor election 2024-08-20T02:53:40.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:40.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: mon.c calling monitor election 2024-08-20T02:53:40.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:40.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-20T02:53:40.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: monmap epoch 2 2024-08-20T02:53:40.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:53:40.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: last_changed 2024-08-20T02:53:34.722147+0000 2024-08-20T02:53:40.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: created 2024-08-20T02:52:08.003300+0000 2024-08-20T02:53:40.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: min_mon_release 19 (squid) 2024-08-20T02:53:40.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: election_strategy: 1 2024-08-20T02:53:40.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-08-20T02:53:40.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-08-20T02:53:40.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: fsmap 2024-08-20T02:53:40.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T02:53:40.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: mgrmap e13: x(active, since 41s) 2024-08-20T02:53:40.039 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: overall HEALTH_OK 2024-08-20T02:53:40.039 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:40.039 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:40.039 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:40.039 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:53:40.039 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:39 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:40.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:40.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:53:40.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: mon.a calling monitor election 2024-08-20T02:53:40.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:40.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: mon.c calling monitor election 2024-08-20T02:53:40.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:40.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: monmap epoch 2 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: last_changed 2024-08-20T02:53:34.722147+0000 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: created 2024-08-20T02:52:08.003300+0000 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: min_mon_release 19 (squid) 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: election_strategy: 1 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: fsmap 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: mgrmap e13: x(active, since 41s) 2024-08-20T02:53:40.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: overall HEALTH_OK 2024-08-20T02:53:40.139 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:40.139 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:40.139 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:40.139 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:53:40.139 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:39 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:40.261 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:53:40.261 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":2,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","modified":"2024-08-20T02:53:34.722147Z","created":"2024-08-20T02:52:08.003300Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-20T02:53:40.261 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 2 2024-08-20T02:53:40.812 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:40 smithi165 ceph-mon[22357]: Deploying daemon mon.b on smithi099 2024-08-20T02:53:40.812 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:40 smithi165 ceph-mon[22357]: from='client.? 172.21.15.165:0/169648680' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T02:53:40.812 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:40 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:41.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:40 smithi044 ceph-mon[18790]: Deploying daemon mon.b on smithi099 2024-08-20T02:53:41.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:40 smithi044 ceph-mon[18790]: from='client.? 172.21.15.165:0/169648680' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T02:53:41.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:40 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:41.839 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-20T02:53:41.839 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-20T02:53:42.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:41 smithi044 ceph-mon[18790]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:42.036 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:41 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T02:53:41.721+0000 7f8279927640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-20T02:53:42.093 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:53:42.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:41 smithi165 ceph-mon[22357]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:42.752 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:53:42.752 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":2,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","modified":"2024-08-20T02:53:34.722147Z","created":"2024-08-20T02:52:08.003300Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-20T02:53:42.752 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 2 2024-08-20T02:53:43.050 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:42 smithi165 ceph-mon[22357]: from='client.? 172.21.15.165:0/509082200' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T02:53:43.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:42 smithi044 ceph-mon[18790]: from='client.? 172.21.15.165:0/509082200' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T02:53:43.434 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(???) e0 preinit fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:53:43.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 bash[21965]: 79fb5c25db4d0fa4a71f58e58bd192064e4c982b8b32ba1cae33c5283aaf5bca 2024-08-20T02:53:43.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).mds e1 new map 2024-08-20T02:53:43.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).mds e1 print_map 2024-08-20T02:53:43.915 INFO:journalctl@ceph.mon.b.smithi099.stdout: e1 2024-08-20T02:53:43.915 INFO:journalctl@ceph.mon.b.smithi099.stdout: btime 2024-08-20T02:52:10:979066+0000 2024-08-20T02:53:43.915 INFO:journalctl@ceph.mon.b.smithi099.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-20T02:53:43.916 INFO:journalctl@ceph.mon.b.smithi099.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T02:53:43.916 INFO:journalctl@ceph.mon.b.smithi099.stdout: legacy client fscid: -1 2024-08-20T02:53:43.916 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-08-20T02:53:43.916 INFO:journalctl@ceph.mon.b.smithi099.stdout: No filesystems configured 2024-08-20T02:53:43.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-20T02:53:43.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-20T02:53:43.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-20T02:53:43.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-20T02:53:43.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-20T02:53:43.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-20T02:53:43.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-20T02:53:43.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T02:53:43.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T02:53:43.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T02:53:43.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Deploying cephadm binary to smithi165 2024-08-20T02:53:43.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"} : dispatch 2024-08-20T02:53:43.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:43.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:53:43.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Updating smithi099:/etc/ceph/ceph.conf 2024-08-20T02:53:43.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:43.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:43.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:43.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:53:43.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:43.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Added host smithi165 2024-08-20T02:53:43.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:53:43.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/280895863' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/280895863' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:53:43.919 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:43.920 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Updating smithi165:/etc/ceph/ceph.conf 2024-08-20T02:53:43.920 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.920 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.920 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.920 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.920 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:53:43.920 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:43.920 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:43.920 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:43.920 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:53:43.920 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi165:172.21.15.165=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:53:43.921 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Saving service mon spec with placement smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi165:172.21.15.165=c;count:3 2024-08-20T02:53:43.921 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Deploying daemon mon.c on smithi165 2024-08-20T02:53:43.921 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:43.921 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='client.? 172.21.15.165:0/2624137318' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T02:53:43.921 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:43.921 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:43.921 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:43.921 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.a calling monitor election 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.c calling monitor election 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: monmap epoch 2 2024-08-20T02:53:43.922 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: last_changed 2024-08-20T02:53:34.722147+0000 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: created 2024-08-20T02:52:08.003300+0000 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: min_mon_release 19 (squid) 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: election_strategy: 1 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: fsmap 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mgrmap e13: x(active, since 41s) 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: overall HEALTH_OK 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.923 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:43.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:53:43.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:43.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: Deploying daemon mon.b on smithi099 2024-08-20T02:53:43.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='client.? 172.21.15.165:0/169648680' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T02:53:43.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:43.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:43.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: from='client.? 172.21.15.165:0/509082200' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T02:53:43.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 ceph-mon[21984]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-20T02:53:43.924 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:43 smithi099 systemd[1]: Started Ceph mon.b for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:53:44.313 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-20T02:53:44.313 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-20T02:53:44.566 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:53:48.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:53:48.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: mon.a calling monitor election 2024-08-20T02:53:48.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: mon.c calling monitor election 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: mon.b calling monitor election 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: monmap epoch 3 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: last_changed 2024-08-20T02:53:43.440316+0000 2024-08-20T02:53:48.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: created 2024-08-20T02:52:08.003300+0000 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: min_mon_release 19 (squid) 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: election_strategy: 1 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: fsmap 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: mgrmap e13: x(active, since 50s) 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: overall HEALTH_OK 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:48.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:48 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:48.866 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:53:48.866 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":3,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","modified":"2024-08-20T02:53:43.440316Z","created":"2024-08-20T02:52:08.003300Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:3300","nonce":0},{"type":"v1","addr":"172.21.15.165:6789","nonce":0}]},"addr":"172.21.15.165:6789/0","public_addr":"172.21.15.165:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-08-20T02:53:48.866 INFO:teuthology.orchestra.run.smithi165.stderr:dumped monmap epoch 3 2024-08-20T02:53:48.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:53:48.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: mon.a calling monitor election 2024-08-20T02:53:48.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:48.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: mon.c calling monitor election 2024-08-20T02:53:48.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:48.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: mon.b calling monitor election 2024-08-20T02:53:48.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:48.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.867 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-20T02:53:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: monmap epoch 3 2024-08-20T02:53:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:53:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: last_changed 2024-08-20T02:53:43.440316+0000 2024-08-20T02:53:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: created 2024-08-20T02:52:08.003300+0000 2024-08-20T02:53:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: min_mon_release 19 (squid) 2024-08-20T02:53:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: election_strategy: 1 2024-08-20T02:53:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-08-20T02:53:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-08-20T02:53:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-08-20T02:53:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: fsmap 2024-08-20T02:53:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T02:53:48.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: mgrmap e13: x(active, since 50s) 2024-08-20T02:53:48.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: overall HEALTH_OK 2024-08-20T02:53:48.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:48.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:48.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:48.869 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:48 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:48.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:53:48.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: mon.a calling monitor election 2024-08-20T02:53:48.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: mon.c calling monitor election 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: mon.b calling monitor election 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: monmap epoch 3 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:53:48.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: last_changed 2024-08-20T02:53:43.440316+0000 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: created 2024-08-20T02:52:08.003300+0000 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: min_mon_release 19 (squid) 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: election_strategy: 1 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: 0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: 1: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.c 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: fsmap 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: mgrmap e13: x(active, since 50s) 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: overall HEALTH_OK 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:48.917 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:48.918 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:48 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:49.443 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-08-20T02:53:49.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config generate-minimal-conf 2024-08-20T02:53:49.695 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:49 smithi165 ceph-mon[22357]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:49.695 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:49 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:53:49.695 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:49 smithi165 ceph-mon[22357]: from='client.? 172.21.15.165:0/2082543058' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T02:53:49.695 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:49 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:49.695 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:49 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:49.704 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:53:49.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:49 smithi044 ceph-mon[18790]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:49.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:49 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:53:49.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:49 smithi044 ceph-mon[18790]: from='client.? 172.21.15.165:0/2082543058' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T02:53:49.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:49 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:49.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:49 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:49.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:49 smithi099 ceph-mon[21984]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:49.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:49 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:53:49.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:49 smithi099 ceph-mon[21984]: from='client.? 172.21.15.165:0/2082543058' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T02:53:49.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:49 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:49.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:49 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:50.285 INFO:teuthology.orchestra.run.smithi044.stdout:# minimal ceph.conf for 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:53:50.286 INFO:teuthology.orchestra.run.smithi044.stdout:[global] 2024-08-20T02:53:50.286 INFO:teuthology.orchestra.run.smithi044.stdout: fsid = 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T02:53:50.286 INFO:teuthology.orchestra.run.smithi044.stdout: mon_host = [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] 2024-08-20T02:53:50.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:50 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:50.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:50 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:50.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:50 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:50.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:50 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:50.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:50 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:50.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:50 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:53:50.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:50 smithi165 ceph-mon[22357]: Updating smithi044:/etc/ceph/ceph.conf 2024-08-20T02:53:50.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:50 smithi165 ceph-mon[22357]: Updating smithi099:/etc/ceph/ceph.conf 2024-08-20T02:53:50.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:50 smithi165 ceph-mon[22357]: Updating smithi165:/etc/ceph/ceph.conf 2024-08-20T02:53:50.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:50 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3133968878' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:50.826 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:50 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:50.826 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:50 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:50.826 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:50 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:50.826 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:50 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:50.826 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:50 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:50.827 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:50 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:53:50.827 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:50 smithi044 ceph-mon[18790]: Updating smithi044:/etc/ceph/ceph.conf 2024-08-20T02:53:50.827 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:50 smithi044 ceph-mon[18790]: Updating smithi099:/etc/ceph/ceph.conf 2024-08-20T02:53:50.827 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:50 smithi044 ceph-mon[18790]: Updating smithi165:/etc/ceph/ceph.conf 2024-08-20T02:53:50.827 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:50 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3133968878' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:50.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:50 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:50.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:50 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:50.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:50 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:50.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:50 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:50.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:50 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:50.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:50 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:53:50.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:50 smithi099 ceph-mon[21984]: Updating smithi044:/etc/ceph/ceph.conf 2024-08-20T02:53:50.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:50 smithi099 ceph-mon[21984]: Updating smithi099:/etc/ceph/ceph.conf 2024-08-20T02:53:50.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:50 smithi099 ceph-mon[21984]: Updating smithi165:/etc/ceph/ceph.conf 2024-08-20T02:53:50.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:50 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3133968878' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:51.004 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-08-20T02:53:51.004 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:53:51.004 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.conf 2024-08-20T02:53:51.032 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:53:51.032 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:51.098 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-08-20T02:53:51.098 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.conf 2024-08-20T02:53:51.127 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-08-20T02:53:51.128 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:51.193 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-08-20T02:53:51.193 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.conf 2024-08-20T02:53:51.220 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-08-20T02:53:51.221 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:53:51.287 INFO:tasks.cephadm:enabling module snap_schedule 2024-08-20T02:53:51.287 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-08-20T02:53:51.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: Updating smithi044:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:51.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:51.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:51.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:51.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: Reconfiguring mon.a (unknown last config time)... 2024-08-20T02:53:51.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:53:51.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T02:53:51.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:51.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:51 smithi044 ceph-mon[18790]: Reconfiguring daemon mon.a on smithi044 2024-08-20T02:53:51.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: Updating smithi044:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:51.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:51.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:51.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:51.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: Reconfiguring mon.a (unknown last config time)... 2024-08-20T02:53:51.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:53:51.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T02:53:51.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:51.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:51 smithi165 ceph-mon[22357]: Reconfiguring daemon mon.a on smithi044 2024-08-20T02:53:51.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: Updating smithi044:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:51.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:51.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:53:51.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:53:51.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:51.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: Reconfiguring mon.a (unknown last config time)... 2024-08-20T02:53:51.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:53:51.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T02:53:51.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:51.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:51 smithi099 ceph-mon[21984]: Reconfiguring daemon mon.a on smithi044 2024-08-20T02:53:52.148 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:53:53.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:53 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:53.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:53 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:53.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:53 smithi165 ceph-mon[22357]: Reconfiguring mon.b (monmap changed)... 2024-08-20T02:53:53.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:53 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:53:53.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:53 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T02:53:53.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:53 smithi165 ceph-mon[22357]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:53.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:53 smithi165 ceph-mon[22357]: Reconfiguring daemon mon.b on smithi099 2024-08-20T02:53:53.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:53 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/427082669' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-20T02:53:53.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:53 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:53.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:53 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:53.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:53 smithi099 ceph-mon[21984]: Reconfiguring mon.b (monmap changed)... 2024-08-20T02:53:53.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:53 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:53:53.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:53 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T02:53:53.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:53 smithi099 ceph-mon[21984]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:53.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:53 smithi099 ceph-mon[21984]: Reconfiguring daemon mon.b on smithi099 2024-08-20T02:53:53.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:53 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/427082669' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-20T02:53:53.442 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:53 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: ignoring --setuser ceph since I am not root 2024-08-20T02:53:53.443 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:53:53 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: ignoring --setgroup ceph since I am not root 2024-08-20T02:53:53.443 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:53 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:53.443 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:53 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' 2024-08-20T02:53:53.443 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:53 smithi044 ceph-mon[18790]: Reconfiguring mon.b (monmap changed)... 2024-08-20T02:53:53.443 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:53 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:53:53.443 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:53 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T02:53:53.443 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:53 smithi044 ceph-mon[18790]: from='mgr.14150 172.21.15.44:0/3418549239' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:53:53.443 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:53 smithi044 ceph-mon[18790]: Reconfiguring daemon mon.b on smithi099 2024-08-20T02:53:53.443 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:53 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/427082669' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-20T02:53:53.756 INFO:tasks.cephadm:Adding mgr.x on smithi044 2024-08-20T02:53:53.756 INFO:tasks.cephadm:Adding mgr.y on smithi099 2024-08-20T02:53:53.757 INFO:tasks.cephadm:Adding mgr.z on smithi165 2024-08-20T02:53:53.757 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch apply mgr '3;smithi044=x;smithi099=y;smithi165=z' 2024-08-20T02:53:54.007 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:53:54.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:54 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/427082669' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-20T02:53:54.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:54 smithi165 ceph-mon[22357]: mgrmap e14: x(active, since 54s) 2024-08-20T02:53:54.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:54 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/427082669' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-20T02:53:54.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:54 smithi099 ceph-mon[21984]: mgrmap e14: x(active, since 54s) 2024-08-20T02:53:54.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:54 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/427082669' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-20T02:53:54.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:54 smithi044 ceph-mon[18790]: mgrmap e14: x(active, since 54s) 2024-08-20T02:53:58.639 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:58 smithi165 ceph-mon[22357]: Active manager daemon x restarted 2024-08-20T02:53:58.639 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:58 smithi165 ceph-mon[22357]: Activating manager daemon x 2024-08-20T02:53:58.639 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:58 smithi165 ceph-mon[22357]: osdmap e5: 0 total, 0 up, 0 in 2024-08-20T02:53:58.639 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:58 smithi165 ceph-mon[22357]: mgrmap e15: x(active, starting, since 0.00482956s) 2024-08-20T02:53:58.639 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:53:58.639 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:58.639 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:58.639 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T02:53:58.639 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T02:53:58.639 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T02:53:58.639 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T02:53:58.640 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:58 smithi165 ceph-mon[22357]: Manager daemon x is now available 2024-08-20T02:53:58.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:58 smithi099 ceph-mon[21984]: Active manager daemon x restarted 2024-08-20T02:53:58.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:58 smithi099 ceph-mon[21984]: Activating manager daemon x 2024-08-20T02:53:58.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:58 smithi099 ceph-mon[21984]: osdmap e5: 0 total, 0 up, 0 in 2024-08-20T02:53:58.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:58 smithi099 ceph-mon[21984]: mgrmap e15: x(active, starting, since 0.00482956s) 2024-08-20T02:53:58.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:53:58.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:58.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:58.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T02:53:58.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T02:53:58.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T02:53:58.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T02:53:58.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:58 smithi099 ceph-mon[21984]: Manager daemon x is now available 2024-08-20T02:53:58.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:58 smithi044 ceph-mon[18790]: Active manager daemon x restarted 2024-08-20T02:53:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:58 smithi044 ceph-mon[18790]: Activating manager daemon x 2024-08-20T02:53:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:58 smithi044 ceph-mon[18790]: osdmap e5: 0 total, 0 up, 0 in 2024-08-20T02:53:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:58 smithi044 ceph-mon[18790]: mgrmap e15: x(active, starting, since 0.00482956s) 2024-08-20T02:53:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:53:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:53:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:53:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T02:53:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T02:53:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T02:53:58.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T02:53:58.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:58 smithi044 ceph-mon[18790]: Manager daemon x is now available 2024-08-20T02:53:59.359 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mgr update... 2024-08-20T02:53:59.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:53:59.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T02:53:59.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T02:53:59.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:59 smithi165 ceph-mon[22357]: mgrmap e16: x(active, since 1.00871s) 2024-08-20T02:53:59.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:53:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:53:59.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:53:59.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T02:53:59.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T02:53:59.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:59 smithi099 ceph-mon[21984]: mgrmap e16: x(active, since 1.00871s) 2024-08-20T02:53:59.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:53:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:53:59.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:53:59.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T02:53:59.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T02:53:59.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:59 smithi044 ceph-mon[18790]: mgrmap e16: x(active, since 1.00871s) 2024-08-20T02:53:59.697 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:53:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:53:59.972 DEBUG:teuthology.orchestra.run.smithi099:mgr.y> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.y.service 2024-08-20T02:53:59.974 DEBUG:teuthology.orchestra.run.smithi165:mgr.z> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.z.service 2024-08-20T02:53:59.977 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-08-20T02:53:59.977 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-08-20T02:53:59.980 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-08-20T02:53:59.980 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set client debug_client 20 2024-08-20T02:54:00.023 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-08-20T02:54:00.023 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set client debug_ms 1 2024-08-20T02:54:00.065 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-08-20T02:54:00.066 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-08-20T02:54:00.109 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-08-20T02:54:00.109 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-08-20T02:54:00.152 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-08-20T02:54:00.152 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds debug_mds 20 2024-08-20T02:54:00.196 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-08-20T02:54:00.196 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-08-20T02:54:00.240 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-08-20T02:54:00.240 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds debug_ms 2 2024-08-20T02:54:00.285 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-08-20T02:54:00.286 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-08-20T02:54:00.337 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-08-20T02:54:00.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-08-20T02:54:00.338 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:00.381 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-08-20T02:54:00.381 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-08-20T02:54:00.426 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-08-20T02:54:00.426 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-08-20T02:54:00.477 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-08-20T02:54:00.478 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-08-20T02:54:00.527 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-08-20T02:54:00.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-08-20T02:54:00.536 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T02:54:00.480+0000 7efc67d30640 -1 log_channel(cephadm) log [ERR] : [20/Aug/2024:02:54:00] ENGINE Error in HTTPServer.serve 2024-08-20T02:54:00.536 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: Traceback (most recent call last): 2024-08-20T02:54:00.536 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T02:54:00.536 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: self._connections.run(self.expiration_interval) 2024-08-20T02:54:00.536 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T02:54:00.536 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: self._run(expiration_interval) 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: s = self.context.wrap_socket( 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: return self.sslsocket_class._create( 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: self.do_handshake() 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: self._sslobj.do_handshake() 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T02:54:00.537 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T02:54:00.538 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-08-20T02:54:00.538 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-08-20T02:54:00.590 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-08-20T02:54:00.590 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-08-20T02:54:00.649 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-08-20T02:54:00.649 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-08-20T02:54:00.703 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-08-20T02:54:00.703 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-08-20T02:54:00.757 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-08-20T02:54:00.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-08-20T02:54:00.806 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-08-20T02:54:00.806 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mgr debug_cephsqlite 20 2024-08-20T02:54:00.864 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-08-20T02:54:00.865 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-08-20T02:54:00.911 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-08-20T02:54:00.912 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-08-20T02:54:00.961 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-08-20T02:54:00.961 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-08-20T02:54:01.019 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-08-20T02:54:01.019 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-08-20T02:54:01.064 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-08-20T02:54:01.064 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-08-20T02:54:01.087 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-mon[18790]: Saving service mgr spec with placement smithi044=x;smithi099=y;smithi165=z;count:3 2024-08-20T02:54:01.088 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.088 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.088 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.088 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.088 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.088 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.088 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"} : dispatch 2024-08-20T02:54:01.088 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.088 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.089 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-08-20T02:54:01.089 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-08-20T02:54:01.152 DEBUG:tasks.cephadm:set 26 configs 2024-08-20T02:54:01.152 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi044') 2024-08-20T02:54:01.157 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:01.360 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:01.364 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:01.369 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:01.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:00 smithi165 ceph-mon[22357]: Saving service mgr spec with placement smithi044=x;smithi099=y;smithi165=z;count:3 2024-08-20T02:54:01.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"} : dispatch 2024-08-20T02:54:01.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:00 smithi099 ceph-mon[21984]: Saving service mgr spec with placement smithi044=x;smithi099=y;smithi165=z;count:3 2024-08-20T02:54:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"} : dispatch 2024-08-20T02:54:01.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.997 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:01 smithi044 ceph-mon[18790]: [20/Aug/2024:02:54:00] ENGINE Bus STARTING 2024-08-20T02:54:01.997 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:01 smithi044 ceph-mon[18790]: [20/Aug/2024:02:54:00] ENGINE Serving on http://172.21.15.44:8765 2024-08-20T02:54:01.997 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:01 smithi044 ceph-mon[18790]: [20/Aug/2024:02:54:00] ENGINE Error in HTTPServer.serve 2024-08-20T02:54:01.997 INFO:journalctl@ceph.mon.a.smithi044.stdout: Traceback (most recent call last): 2024-08-20T02:54:01.997 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T02:54:01.997 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._connections.run(self.expiration_interval) 2024-08-20T02:54:01.997 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T02:54:01.997 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._run(expiration_interval) 2024-08-20T02:54:01.997 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T02:54:01.997 INFO:journalctl@ceph.mon.a.smithi044.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T02:54:01.997 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout: s = self.context.wrap_socket( 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout: return self.sslsocket_class._create( 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout: self.do_handshake() 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout: self._sslobj.do_handshake() 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:01 smithi044 ceph-mon[18790]: [20/Aug/2024:02:54:00] ENGINE Serving on https://172.21.15.44:7150 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:01 smithi044 ceph-mon[18790]: [20/Aug/2024:02:54:00] ENGINE Bus STARTED 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:01 smithi044 ceph-mon[18790]: mgrmap e17: x(active, since 2s) 2024-08-20T02:54:01.998 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:01 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3294779169' entity='client.admin' 2024-08-20T02:54:01.999 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:01 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.999 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:01 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:01.999 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:01 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-08-20T02:54:02.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:01 smithi165 ceph-mon[22357]: [20/Aug/2024:02:54:00] ENGINE Bus STARTING 2024-08-20T02:54:02.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:01 smithi165 ceph-mon[22357]: [20/Aug/2024:02:54:00] ENGINE Serving on http://172.21.15.44:8765 2024-08-20T02:54:02.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:01 smithi165 ceph-mon[22357]: [20/Aug/2024:02:54:00] ENGINE Error in HTTPServer.serve 2024-08-20T02:54:02.386 INFO:journalctl@ceph.mon.c.smithi165.stdout: Traceback (most recent call last): 2024-08-20T02:54:02.386 INFO:journalctl@ceph.mon.c.smithi165.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T02:54:02.387 INFO:journalctl@ceph.mon.c.smithi165.stdout: self._connections.run(self.expiration_interval) 2024-08-20T02:54:02.387 INFO:journalctl@ceph.mon.c.smithi165.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T02:54:02.387 INFO:journalctl@ceph.mon.c.smithi165.stdout: self._run(expiration_interval) 2024-08-20T02:54:02.387 INFO:journalctl@ceph.mon.c.smithi165.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T02:54:02.387 INFO:journalctl@ceph.mon.c.smithi165.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T02:54:02.387 INFO:journalctl@ceph.mon.c.smithi165.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T02:54:02.387 INFO:journalctl@ceph.mon.c.smithi165.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T02:54:02.387 INFO:journalctl@ceph.mon.c.smithi165.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T02:54:02.387 INFO:journalctl@ceph.mon.c.smithi165.stdout: s = self.context.wrap_socket( 2024-08-20T02:54:02.387 INFO:journalctl@ceph.mon.c.smithi165.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T02:54:02.387 INFO:journalctl@ceph.mon.c.smithi165.stdout: return self.sslsocket_class._create( 2024-08-20T02:54:02.388 INFO:journalctl@ceph.mon.c.smithi165.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T02:54:02.388 INFO:journalctl@ceph.mon.c.smithi165.stdout: self.do_handshake() 2024-08-20T02:54:02.388 INFO:journalctl@ceph.mon.c.smithi165.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T02:54:02.388 INFO:journalctl@ceph.mon.c.smithi165.stdout: self._sslobj.do_handshake() 2024-08-20T02:54:02.388 INFO:journalctl@ceph.mon.c.smithi165.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T02:54:02.388 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:01 smithi165 ceph-mon[22357]: [20/Aug/2024:02:54:00] ENGINE Serving on https://172.21.15.44:7150 2024-08-20T02:54:02.388 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:01 smithi165 ceph-mon[22357]: [20/Aug/2024:02:54:00] ENGINE Bus STARTED 2024-08-20T02:54:02.388 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:01 smithi165 ceph-mon[22357]: mgrmap e17: x(active, since 2s) 2024-08-20T02:54:02.388 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:01 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3294779169' entity='client.admin' 2024-08-20T02:54:02.388 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:01 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:02.388 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:01 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:02.389 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:01 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-08-20T02:54:02.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:01 smithi099 ceph-mon[21984]: [20/Aug/2024:02:54:00] ENGINE Bus STARTING 2024-08-20T02:54:02.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:01 smithi099 ceph-mon[21984]: [20/Aug/2024:02:54:00] ENGINE Serving on http://172.21.15.44:8765 2024-08-20T02:54:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:01 smithi099 ceph-mon[21984]: [20/Aug/2024:02:54:00] ENGINE Error in HTTPServer.serve 2024-08-20T02:54:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: Traceback (most recent call last): 2024-08-20T02:54:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T02:54:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: self._connections.run(self.expiration_interval) 2024-08-20T02:54:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T02:54:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: self._run(expiration_interval) 2024-08-20T02:54:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T02:54:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T02:54:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T02:54:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T02:54:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T02:54:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout: s = self.context.wrap_socket( 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout: return self.sslsocket_class._create( 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout: self.do_handshake() 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout: self._sslobj.do_handshake() 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:01 smithi099 ceph-mon[21984]: [20/Aug/2024:02:54:00] ENGINE Serving on https://172.21.15.44:7150 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:01 smithi099 ceph-mon[21984]: [20/Aug/2024:02:54:00] ENGINE Bus STARTED 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:01 smithi099 ceph-mon[21984]: mgrmap e17: x(active, since 2s) 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:01 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3294779169' entity='client.admin' 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:01 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:02.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:01 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:02.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:01 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi165", "name": "osd_memory_target"} : dispatch 2024-08-20T02:54:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"} : dispatch 2024-08-20T02:54:03.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: Updating smithi044:/etc/ceph/ceph.conf 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: Updating smithi099:/etc/ceph/ceph.conf 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: Updating smithi165:/etc/ceph/ceph.conf 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1502991386' entity='client.admin' 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T02:54:03.288 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T02:54:03.288 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T02:54:03.288 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:03.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"} : dispatch 2024-08-20T02:54:03.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: Updating smithi044:/etc/ceph/ceph.conf 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: Updating smithi099:/etc/ceph/ceph.conf 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: Updating smithi165:/etc/ceph/ceph.conf 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1502991386' entity='client.admin' 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T02:54:03.388 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T02:54:03.388 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T02:54:03.388 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"} : dispatch 2024-08-20T02:54:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: Updating smithi044:/etc/ceph/ceph.conf 2024-08-20T02:54:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: Updating smithi099:/etc/ceph/ceph.conf 2024-08-20T02:54:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: Updating smithi165:/etc/ceph/ceph.conf 2024-08-20T02:54:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1502991386' entity='client.admin' 2024-08-20T02:54:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T02:54:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T02:54:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T02:54:03.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:04.209+0000 7f8d2f73b640 1 -- 172.21.15.44:0/938959212 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8d1003edb0 msgr2=0x7f8d10041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:04.209+0000 7f8d2f73b640 1 --2- 172.21.15.44:0/938959212 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8d1003edb0 0x7f8d10041270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d18000f80 tx=0x7f8d18031530 comp rx=0 tx=0).stop 2024-08-20T02:54:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:04.209+0000 7f8d2f73b640 1 -- 172.21.15.44:0/938959212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d28075b90 msgr2=0x7f8d2819f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:04.209+0000 7f8d2f73b640 1 --2- 172.21.15.44:0/938959212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d28075b90 0x7f8d2819f900 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00b560 tx=0x7f8d1c00ba20 comp rx=0 tx=0).stop 2024-08-20T02:54:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:04.209+0000 7f8d2f73b640 1 -- 172.21.15.44:0/938959212 shutdown_connections 2024-08-20T02:54:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:04.209+0000 7f8d2f73b640 1 --2- 172.21.15.44:0/938959212 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8d1003edb0 0x7f8d10041270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:04.209+0000 7f8d2f73b640 1 --2- 172.21.15.44:0/938959212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d28113830 0x7f8d281a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:04.209+0000 7f8d2f73b640 1 --2- 172.21.15.44:0/938959212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d28075b90 0x7f8d2819f900 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:04.209+0000 7f8d2f73b640 1 --2- 172.21.15.44:0/938959212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d28075270 0x7f8d2819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:04.209+0000 7f8d2f73b640 1 -- 172.21.15.44:0/938959212 >> 172.21.15.44:0/938959212 conn(0x7f8d280fe540 msgr2=0x7f8d280ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:04.209+0000 7f8d2f73b640 1 -- 172.21.15.44:0/938959212 shutdown_connections 2024-08-20T02:54:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:04.209+0000 7f8d2f73b640 1 -- 172.21.15.44:0/938959212 wait complete. 2024-08-20T02:54:04.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:04 smithi165 ceph-mon[22357]: Updating smithi044:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:54:04.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:04 smithi165 ceph-mon[22357]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:54:04.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:04 smithi165 ceph-mon[22357]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:54:04.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:04 smithi165 ceph-mon[22357]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:54:04.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:04 smithi165 ceph-mon[22357]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:54:04.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:04 smithi165 ceph-mon[22357]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:54:04.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:04 smithi165 ceph-mon[22357]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:54:04.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:04 smithi165 ceph-mon[22357]: Updating smithi044:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:54:04.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:04 smithi165 ceph-mon[22357]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:54:04.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:04 smithi165 ceph-mon[22357]: Deploying daemon mgr.z on smithi165 2024-08-20T02:54:04.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:04 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1343783604' entity='client.admin' 2024-08-20T02:54:04.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:04 smithi099 ceph-mon[21984]: Updating smithi044:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:54:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:04 smithi099 ceph-mon[21984]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:54:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:04 smithi099 ceph-mon[21984]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:54:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:04 smithi099 ceph-mon[21984]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:54:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:04 smithi099 ceph-mon[21984]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:54:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:04 smithi099 ceph-mon[21984]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:54:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:04 smithi099 ceph-mon[21984]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:54:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:04 smithi099 ceph-mon[21984]: Updating smithi044:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:54:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:04 smithi099 ceph-mon[21984]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:54:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:04 smithi099 ceph-mon[21984]: Deploying daemon mgr.z on smithi165 2024-08-20T02:54:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:04 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1343783604' entity='client.admin' 2024-08-20T02:54:04.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:04 smithi044 ceph-mon[18790]: Updating smithi044:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:54:04.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:04 smithi044 ceph-mon[18790]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:54:04.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:04 smithi044 ceph-mon[18790]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T02:54:04.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:04 smithi044 ceph-mon[18790]: Updating smithi165:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:54:04.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:04 smithi044 ceph-mon[18790]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:54:04.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:04 smithi044 ceph-mon[18790]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-08-20T02:54:04.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:04 smithi044 ceph-mon[18790]: Updating smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:54:04.508 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:04 smithi044 ceph-mon[18790]: Updating smithi044:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:54:04.509 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:04 smithi044 ceph-mon[18790]: Updating smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T02:54:04.509 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:04 smithi044 ceph-mon[18790]: Deploying daemon mgr.z on smithi165 2024-08-20T02:54:04.509 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:04 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1343783604' entity='client.admin' 2024-08-20T02:54:04.829 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Aug 20 02:54:04 smithi165 systemd[1]: Starting Ceph mgr.z for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T02:54:05.080 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Aug 20 02:54:04 smithi165 podman[24979]: 2024-08-20 02:54:04.729713742 +0000 UTC m=+0.015970925 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T02:54:05.080 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Aug 20 02:54:04 smithi165 podman[24979]: 2024-08-20 02:54:04.839791921 +0000 UTC m=+0.126049156 container create 1a4c2ed740a443d477e7e04c8d3086601abf45f6ef1ef5a3e1afebd25bc9db39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T02:54:05.080 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Aug 20 02:54:05 smithi165 podman[24979]: 2024-08-20 02:54:05.050555755 +0000 UTC m=+0.336812927 container init 1a4c2ed740a443d477e7e04c8d3086601abf45f6ef1ef5a3e1afebd25bc9db39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD) 2024-08-20T02:54:05.080 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Aug 20 02:54:05 smithi165 podman[24979]: 2024-08-20 02:54:05.053247478 +0000 UTC m=+0.339504650 container start 1a4c2ed740a443d477e7e04c8d3086601abf45f6ef1ef5a3e1afebd25bc9db39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-z, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-08-20T02:54:05.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:05 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/938959212' entity='client.admin' 2024-08-20T02:54:05.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:05 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:05.386 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Aug 20 02:54:05 smithi165 ceph-mgr[24997]: -- 172.21.15.165:0/2181917857 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604e56990e0 con 0x5604e5676800 2024-08-20T02:54:05.386 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Aug 20 02:54:05 smithi165 bash[24979]: 1a4c2ed740a443d477e7e04c8d3086601abf45f6ef1ef5a3e1afebd25bc9db39 2024-08-20T02:54:05.386 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Aug 20 02:54:05 smithi165 systemd[1]: Started Ceph mgr.z for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:54:05.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:05 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/938959212' entity='client.admin' 2024-08-20T02:54:05.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:05 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:05.642 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:05 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/938959212' entity='client.admin' 2024-08-20T02:54:05.642 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:05 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:05.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.718+0000 7fae764f1640 1 -- 172.21.15.44:0/2849606444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae70107d20 msgr2=0x7fae7010e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:05.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.718+0000 7fae764f1640 1 --2- 172.21.15.44:0/2849606444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae70107d20 0x7fae7010e520 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fae68009ca0 tx=0x7fae6802a8c0 comp rx=0 tx=0).stop 2024-08-20T02:54:05.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.718+0000 7fae764f1640 1 -- 172.21.15.44:0/2849606444 shutdown_connections 2024-08-20T02:54:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.718+0000 7fae764f1640 1 --2- 172.21.15.44:0/2849606444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae70107d20 0x7fae7010e520 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.718+0000 7fae764f1640 1 --2- 172.21.15.44:0/2849606444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae701073a0 0x7fae701077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.718+0000 7fae764f1640 1 --2- 172.21.15.44:0/2849606444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae70101990 0x7fae70101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.718+0000 7fae764f1640 1 -- 172.21.15.44:0/2849606444 >> 172.21.15.44:0/2849606444 conn(0x7fae700fd400 msgr2=0x7fae700ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.718+0000 7fae764f1640 1 -- 172.21.15.44:0/2849606444 shutdown_connections 2024-08-20T02:54:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.718+0000 7fae764f1640 1 -- 172.21.15.44:0/2849606444 wait complete. 2024-08-20T02:54:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.718+0000 7fae764f1640 1 Processor -- start 2024-08-20T02:54:05.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.718+0000 7fae764f1640 1 -- start start 2024-08-20T02:54:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae764f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae70101990 0x7fae70198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae764f1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae701073a0 0x7fae70199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae764f1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae70107d20 0x7fae7019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae764f1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae70111720 con 0x7fae70107d20 2024-08-20T02:54:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae764f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fae701115a0 con 0x7fae70101990 2024-08-20T02:54:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae764f1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fae701118a0 con 0x7fae701073a0 2024-08-20T02:54:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae74a8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae70107d20 0x7fae7019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae74a8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae70107d20 0x7fae7019e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38846/0 (socket says 172.21.15.44:38846) 2024-08-20T02:54:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae74a8e640 1 -- 172.21.15.44:0/1437071156 learned_addr learned my addr 172.21.15.44:0/1437071156 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae6ffff640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae70101990 0x7fae70198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:05.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae6f7fe640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae701073a0 0x7fae70199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae74a8e640 1 -- 172.21.15.44:0/1437071156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae701073a0 msgr2=0x7fae70199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae74a8e640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae701073a0 0x7fae70199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae74a8e640 1 -- 172.21.15.44:0/1437071156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae70101990 msgr2=0x7fae70198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae74a8e640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae70101990 0x7fae70198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae74a8e640 1 -- 172.21.15.44:0/1437071156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7019edb0 con 0x7fae70107d20 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.719+0000 7fae74a8e640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae70107d20 0x7fae7019e5e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae68003ba0 tx=0x7fae68003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.720+0000 7fae6d7fa640 1 -- 172.21.15.44:0/1437071156 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68004240 con 0x7fae70107d20 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.720+0000 7fae6d7fa640 1 -- 172.21.15.44:0/1437071156 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(23 keys) ==== 1014+0+0 (secure 0 0 0) 0x7fae68040070 con 0x7fae70107d20 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.720+0000 7fae764f1640 1 -- 172.21.15.44:0/1437071156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7019f040 con 0x7fae70107d20 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.720+0000 7fae6d7fa640 1 -- 172.21.15.44:0/1437071156 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68030c00 con 0x7fae70107d20 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.720+0000 7fae764f1640 1 -- 172.21.15.44:0/1437071156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae701a5760 con 0x7fae70107d20 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.721+0000 7fae6d7fa640 1 -- 172.21.15.44:0/1437071156 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 50764+0+0 (secure 0 0 0) 0x7fae6803b430 con 0x7fae70107d20 2024-08-20T02:54:05.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.721+0000 7fae764f1640 1 -- 172.21.15.44:0/1437071156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae70101d70 con 0x7fae70107d20 2024-08-20T02:54:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.721+0000 7fae6d7fa640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fae5803edb0 0x7fae58041270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.721+0000 7fae6d7fa640 1 -- 172.21.15.44:0/1437071156 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fae680748a0 con 0x7fae70107d20 2024-08-20T02:54:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.721+0000 7fae6ffff640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fae5803edb0 0x7fae58041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:05.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.722+0000 7fae6ffff640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fae5803edb0 0x7fae58041270 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fae60009bd0 tx=0x7fae60005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:05.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.726+0000 7fae6d7fa640 1 -- 172.21.15.44:0/1437071156 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fae6803a9f0 con 0x7fae70107d20 2024-08-20T02:54:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:05.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.929+0000 7fae764f1640 1 -- 172.21.15.44:0/1437071156 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fae701a5a50 con 0x7fae70107d20 2024-08-20T02:54:05.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.932+0000 7fae6d7fa640 1 -- 172.21.15.44:0/1437071156 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v14)=0 v14) ==== 114+0+0 (secure 0 0 0) 0x7fae68039a60 con 0x7fae70107d20 2024-08-20T02:54:05.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.933+0000 7fae6d7fa640 1 -- 172.21.15.44:0/1437071156 <== mon.0 v2:172.21.15.44:3300/0 8 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fae680305c0 con 0x7fae70107d20 2024-08-20T02:54:05.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.935+0000 7fae764f1640 1 -- 172.21.15.44:0/1437071156 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fae5803edb0 msgr2=0x7fae58041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:05.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.935+0000 7fae764f1640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fae5803edb0 0x7fae58041270 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fae60009bd0 tx=0x7fae60005f90 comp rx=0 tx=0).stop 2024-08-20T02:54:05.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.935+0000 7fae764f1640 1 -- 172.21.15.44:0/1437071156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae70107d20 msgr2=0x7fae7019e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:05.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.935+0000 7fae764f1640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae70107d20 0x7fae7019e5e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fae68003ba0 tx=0x7fae68003bd0 comp rx=0 tx=0).stop 2024-08-20T02:54:05.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.935+0000 7fae764f1640 1 -- 172.21.15.44:0/1437071156 shutdown_connections 2024-08-20T02:54:05.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.935+0000 7fae764f1640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fae5803edb0 0x7fae58041270 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:05.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.935+0000 7fae764f1640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae70107d20 0x7fae7019e5e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:05.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.935+0000 7fae764f1640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae701073a0 0x7fae70199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:05.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.935+0000 7fae764f1640 1 --2- 172.21.15.44:0/1437071156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae70101990 0x7fae70198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:05.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.935+0000 7fae764f1640 1 -- 172.21.15.44:0/1437071156 >> 172.21.15.44:0/1437071156 conn(0x7fae700fd400 msgr2=0x7fae7006a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:05.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.935+0000 7fae764f1640 1 -- 172.21.15.44:0/1437071156 shutdown_connections 2024-08-20T02:54:05.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:05.935+0000 7fae764f1640 1 -- 172.21.15.44:0/1437071156 wait complete. 2024-08-20T02:54:06.235 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:06.235 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:06.244 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:06.301 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi044') 2024-08-20T02:54:06.449 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:06.477 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:06 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:06.477 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:06 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:06.478 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:06 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T02:54:06.478 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:06 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T02:54:06.478 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:06 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T02:54:06.478 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:06 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:06.478 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:06 smithi099 ceph-mon[21984]: Deploying daemon mgr.y on smithi099 2024-08-20T02:54:06.478 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:06 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1437071156' entity='client.admin' 2024-08-20T02:54:06.479 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:06 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:06.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:06 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:06.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:06 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T02:54:06.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:06 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T02:54:06.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:06 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T02:54:06.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:06 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:06.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:06 smithi165 ceph-mon[22357]: Deploying daemon mgr.y on smithi099 2024-08-20T02:54:06.480 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:06 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1437071156' entity='client.admin' 2024-08-20T02:54:06.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:06 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:06.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:06 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:06.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:06 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T02:54:06.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:06 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T02:54:06.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:06 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T02:54:06.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:06 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:06.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:06 smithi044 ceph-mon[18790]: Deploying daemon mgr.y on smithi099 2024-08-20T02:54:06.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:06 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1437071156' entity='client.admin' 2024-08-20T02:54:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.257+0000 7fa6410a2640 1 -- 172.21.15.44:0/3063777564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c108e40 msgr2=0x7fa63c10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.257+0000 7fa6410a2640 1 --2- 172.21.15.44:0/3063777564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c108e40 0x7fa63c10f640 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa634009ca0 tx=0x7fa63402a8c0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.257+0000 7fa6410a2640 1 -- 172.21.15.44:0/3063777564 shutdown_connections 2024-08-20T02:54:07.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.257+0000 7fa6410a2640 1 --2- 172.21.15.44:0/3063777564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c108e40 0x7fa63c10f640 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.257+0000 7fa6410a2640 1 --2- 172.21.15.44:0/3063777564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa63c1084c0 0x7fa63c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.257+0000 7fa6410a2640 1 --2- 172.21.15.44:0/3063777564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63c102ab0 0x7fa63c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.257+0000 7fa6410a2640 1 -- 172.21.15.44:0/3063777564 >> 172.21.15.44:0/3063777564 conn(0x7fa63c0fe500 msgr2=0x7fa63c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.257+0000 7fa6410a2640 1 -- 172.21.15.44:0/3063777564 shutdown_connections 2024-08-20T02:54:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.257+0000 7fa6410a2640 1 -- 172.21.15.44:0/3063777564 wait complete. 2024-08-20T02:54:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.258+0000 7fa6410a2640 1 Processor -- start 2024-08-20T02:54:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.258+0000 7fa6410a2640 1 -- start start 2024-08-20T02:54:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.258+0000 7fa6410a2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c102ab0 0x7fa63c198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.258+0000 7fa6410a2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa63c1084c0 0x7fa63c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.258+0000 7fa6410a2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63c108e40 0x7fa63c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.258+0000 7fa6410a2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c112870 con 0x7fa63c102ab0 2024-08-20T02:54:07.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.258+0000 7fa6410a2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c1126f0 con 0x7fa63c108e40 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.258+0000 7fa6410a2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c1129f0 con 0x7fa63c1084c0 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.258+0000 7fa63ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c102ab0 0x7fa63c198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.258+0000 7fa63ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c102ab0 0x7fa63c198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38862/0 (socket says 172.21.15.44:38862) 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.258+0000 7fa63ad9d640 1 -- 172.21.15.44:0/1769700081 learned_addr learned my addr 172.21.15.44:0/1769700081 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.258+0000 7fa63a59c640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa63c1084c0 0x7fa63c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.259+0000 7fa63b59e640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63c108e40 0x7fa63c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.259+0000 7fa63ad9d640 1 -- 172.21.15.44:0/1769700081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa63c1084c0 msgr2=0x7fa63c199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.259+0000 7fa63ad9d640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa63c1084c0 0x7fa63c199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.259+0000 7fa63ad9d640 1 -- 172.21.15.44:0/1769700081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63c108e40 msgr2=0x7fa63c19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.259+0000 7fa63ad9d640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63c108e40 0x7fa63c19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.259+0000 7fa63ad9d640 1 -- 172.21.15.44:0/1769700081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63c19eed0 con 0x7fa63c102ab0 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.259+0000 7fa63ad9d640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c102ab0 0x7fa63c198e20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00b4a0 tx=0x7fa62c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.259+0000 7fa623fff640 1 -- 172.21.15.44:0/1769700081 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c014070 con 0x7fa63c102ab0 2024-08-20T02:54:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.259+0000 7fa623fff640 1 -- 172.21.15.44:0/1769700081 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fa62c010a80 con 0x7fa63c102ab0 2024-08-20T02:54:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.259+0000 7fa6410a2640 1 -- 172.21.15.44:0/1769700081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63c19f1c0 con 0x7fa63c102ab0 2024-08-20T02:54:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.259+0000 7fa623fff640 1 -- 172.21.15.44:0/1769700081 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c017ad0 con 0x7fa63c102ab0 2024-08-20T02:54:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.259+0000 7fa6410a2640 1 -- 172.21.15.44:0/1769700081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa63c1a5900 con 0x7fa63c102ab0 2024-08-20T02:54:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.260+0000 7fa623fff640 1 -- 172.21.15.44:0/1769700081 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 50764+0+0 (secure 0 0 0) 0x7fa62c017070 con 0x7fa63c102ab0 2024-08-20T02:54:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.260+0000 7fa6410a2640 1 -- 172.21.15.44:0/1769700081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa63c102e90 con 0x7fa63c102ab0 2024-08-20T02:54:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.260+0000 7fa623fff640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa62403edb0 0x7fa624041270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.260+0000 7fa623fff640 1 -- 172.21.15.44:0/1769700081 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa62c052c20 con 0x7fa63c102ab0 2024-08-20T02:54:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.260+0000 7fa63a59c640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa62403edb0 0x7fa624041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.261+0000 7fa63a59c640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa62403edb0 0x7fa624041270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa63c104fb0 tx=0x7fa63000b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.265+0000 7fa623fff640 1 -- 172.21.15.44:0/1769700081 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa62c017c70 con 0x7fa63c102ab0 2024-08-20T02:54:07.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.456+0000 7fa6410a2640 1 -- 172.21.15.44:0/1769700081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fa63c10ac20 con 0x7fa63c102ab0 2024-08-20T02:54:07.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.459+0000 7fa623fff640 1 -- 172.21.15.44:0/1769700081 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v15)=0 v15) ==== 127+0+0 (secure 0 0 0) 0x7fa62c018aa0 con 0x7fa63c102ab0 2024-08-20T02:54:07.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.462+0000 7fa6410a2640 1 -- 172.21.15.44:0/1769700081 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa62403edb0 msgr2=0x7fa624041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:07.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.462+0000 7fa6410a2640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa62403edb0 0x7fa624041270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa63c104fb0 tx=0x7fa63000b390 comp rx=0 tx=0).stop 2024-08-20T02:54:07.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.462+0000 7fa6410a2640 1 -- 172.21.15.44:0/1769700081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c102ab0 msgr2=0x7fa63c198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:07.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.462+0000 7fa6410a2640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c102ab0 0x7fa63c198e20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00b4a0 tx=0x7fa62c00b960 comp rx=0 tx=0).stop 2024-08-20T02:54:07.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.462+0000 7fa6410a2640 1 -- 172.21.15.44:0/1769700081 shutdown_connections 2024-08-20T02:54:07.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.462+0000 7fa6410a2640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa62403edb0 0x7fa624041270 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.462+0000 7fa6410a2640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa63c108e40 0x7fa63c19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.462+0000 7fa6410a2640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa63c1084c0 0x7fa63c199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.462+0000 7fa6410a2640 1 --2- 172.21.15.44:0/1769700081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa63c102ab0 0x7fa63c198e20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.462+0000 7fa6410a2640 1 -- 172.21.15.44:0/1769700081 >> 172.21.15.44:0/1769700081 conn(0x7fa63c0fe500 msgr2=0x7fa63c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:07.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.462+0000 7fa6410a2640 1 -- 172.21.15.44:0/1769700081 shutdown_connections 2024-08-20T02:54:07.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.462+0000 7fa6410a2640 1 -- 172.21.15.44:0/1769700081 wait complete. 2024-08-20T02:54:07.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.654+0000 7f3ad63d5640 1 -- 172.21.15.44:0/2551297126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ad01073c0 msgr2=0x7f3ad0107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:07.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.654+0000 7f3ad63d5640 1 --2- 172.21.15.44:0/2551297126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ad01073c0 0x7f3ad0107800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009900 tx=0x7f3ac402a410 comp rx=0 tx=0).stop 2024-08-20T02:54:07.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.654+0000 7f3ad63d5640 1 -- 172.21.15.44:0/2551297126 shutdown_connections 2024-08-20T02:54:07.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.654+0000 7f3ad63d5640 1 --2- 172.21.15.44:0/2551297126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ad0107d40 0x7f3ad010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.654+0000 7f3ad63d5640 1 --2- 172.21.15.44:0/2551297126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ad01073c0 0x7f3ad0107800 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.654+0000 7f3ad63d5640 1 --2- 172.21.15.44:0/2551297126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ad01019b0 0x7f3ad0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.654+0000 7f3ad63d5640 1 -- 172.21.15.44:0/2551297126 >> 172.21.15.44:0/2551297126 conn(0x7f3ad00fd400 msgr2=0x7f3ad00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:07.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.654+0000 7f3ad63d5640 1 -- 172.21.15.44:0/2551297126 shutdown_connections 2024-08-20T02:54:07.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.654+0000 7f3ad63d5640 1 -- 172.21.15.44:0/2551297126 wait complete. 2024-08-20T02:54:07.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.654+0000 7f3ad63d5640 1 Processor -- start 2024-08-20T02:54:07.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.654+0000 7f3ad63d5640 1 -- start start 2024-08-20T02:54:07.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3ad63d5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ad01019b0 0x7f3ad00fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:07.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3ad63d5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ad01073c0 0x7f3ad00fde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:07.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3ad63d5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ad0107d40 0x7f3ad00fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:07.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3ad63d5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01117a0 con 0x7f3ad0107d40 2024-08-20T02:54:07.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3ad63d5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0111620 con 0x7f3ad01019b0 2024-08-20T02:54:07.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3ad63d5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0111920 con 0x7f3ad01073c0 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3acffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ad01019b0 0x7f3ad00fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3acffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ad01019b0 0x7f3ad00fd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55278/0 (socket says 172.21.15.44:55278) 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3acffff640 1 -- 172.21.15.44:0/1385381321 learned_addr learned my addr 172.21.15.44:0/1385381321 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3ad4972640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ad0107d40 0x7f3ad00fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3acf7fe640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ad01073c0 0x7f3ad00fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3acffff640 1 -- 172.21.15.44:0/1385381321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ad01073c0 msgr2=0x7f3ad00fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3acffff640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ad01073c0 0x7f3ad00fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3acffff640 1 -- 172.21.15.44:0/1385381321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ad0107d40 msgr2=0x7f3ad00fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3acffff640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ad0107d40 0x7f3ad00fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.655+0000 7f3acffff640 1 -- 172.21.15.44:0/1385381321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad01a72c0 con 0x7f3ad01019b0 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.656+0000 7f3acffff640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ad01019b0 0x7f3ad00fd900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000ea60 tx=0x7f3ac000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.656+0000 7f3acd7fa640 1 -- 172.21.15.44:0/1385381321 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac000cde0 con 0x7f3ad01019b0 2024-08-20T02:54:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.656+0000 7f3ad63d5640 1 -- 172.21.15.44:0/1385381321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad01a75b0 con 0x7f3ad01019b0 2024-08-20T02:54:07.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.656+0000 7f3ad63d5640 1 -- 172.21.15.44:0/1385381321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad01a7b10 con 0x7f3ad01019b0 2024-08-20T02:54:07.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.656+0000 7f3acd7fa640 1 -- 172.21.15.44:0/1385381321 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f3ac0004490 con 0x7f3ad01019b0 2024-08-20T02:54:07.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.656+0000 7f3acd7fa640 1 -- 172.21.15.44:0/1385381321 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0018430 con 0x7f3ad01019b0 2024-08-20T02:54:07.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.657+0000 7f3acd7fa640 1 -- 172.21.15.44:0/1385381321 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 50764+0+0 (secure 0 0 0) 0x7f3ac00185d0 con 0x7f3ad01019b0 2024-08-20T02:54:07.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.657+0000 7f3ad63d5640 1 -- 172.21.15.44:0/1385381321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad0101d90 con 0x7f3ad01019b0 2024-08-20T02:54:07.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.657+0000 7f3acd7fa640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3ab803edb0 0x7f3ab8041270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:07.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.657+0000 7f3acd7fa640 1 -- 172.21.15.44:0/1385381321 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3ac0053770 con 0x7f3ad01019b0 2024-08-20T02:54:07.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.657+0000 7f3acf7fe640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3ab803edb0 0x7f3ab8041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:07.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.657+0000 7f3acf7fe640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3ab803edb0 0x7f3ab8041270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4005ab0 tx=0x7f3ac4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:07.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.662+0000 7f3acd7fa640 1 -- 172.21.15.44:0/1385381321 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f3ac001fb10 con 0x7f3ad01019b0 2024-08-20T02:54:07.664 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Aug 20 02:54:07 smithi099 systemd[1]: Started Ceph mgr.y for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:54:07.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.854+0000 7f3ad63d5640 1 -- 172.21.15.44:0/1385381321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f3ad01a7e20 con 0x7f3ad01019b0 2024-08-20T02:54:07.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.862+0000 7f3acd7fa640 1 -- 172.21.15.44:0/1385381321 <== mon.2 v2:172.21.15.99:3300/0 7 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3ac001e2e0 con 0x7f3ad01019b0 2024-08-20T02:54:07.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.862+0000 7f3acd7fa640 1 -- 172.21.15.44:0/1385381321 <== mon.2 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7f3ac0007d30 con 0x7f3ad01019b0 2024-08-20T02:54:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.865+0000 7f3ad63d5640 1 -- 172.21.15.44:0/1385381321 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3ab803edb0 msgr2=0x7f3ab8041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.865+0000 7f3ad63d5640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3ab803edb0 0x7f3ab8041270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4005ab0 tx=0x7f3ac4005a20 comp rx=0 tx=0).stop 2024-08-20T02:54:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.865+0000 7f3ad63d5640 1 -- 172.21.15.44:0/1385381321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ad01019b0 msgr2=0x7f3ad00fd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.865+0000 7f3ad63d5640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ad01019b0 0x7f3ad00fd900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000ea60 tx=0x7f3ac000ef20 comp rx=0 tx=0).stop 2024-08-20T02:54:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.865+0000 7f3ad63d5640 1 -- 172.21.15.44:0/1385381321 shutdown_connections 2024-08-20T02:54:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.865+0000 7f3ad63d5640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3ab803edb0 0x7f3ab8041270 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.865+0000 7f3ad63d5640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3ad0107d40 0x7f3ad00fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.865+0000 7f3ad63d5640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ad01073c0 0x7f3ad00fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.865+0000 7f3ad63d5640 1 --2- 172.21.15.44:0/1385381321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ad01019b0 0x7f3ad00fd900 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:07.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.865+0000 7f3ad63d5640 1 -- 172.21.15.44:0/1385381321 >> 172.21.15.44:0/1385381321 conn(0x7f3ad00fd400 msgr2=0x7f3ad006b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:07.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.865+0000 7f3ad63d5640 1 -- 172.21.15.44:0/1385381321 shutdown_connections 2024-08-20T02:54:07.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:07.865+0000 7f3ad63d5640 1 -- 172.21.15.44:0/1385381321 wait complete. 2024-08-20T02:54:08.519 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:08 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:08.519 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:08 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:08.519 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:08 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:08.519 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:08 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:08.519 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:08 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1769700081' entity='client.admin' 2024-08-20T02:54:08.519 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:08 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:08.519 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:08 smithi044 ceph-mon[18790]: from='client.? ' entity='client.admin' 2024-08-20T02:54:08.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:08 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:08.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:08 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:08.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:08 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:08.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:08 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:08.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:08 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1769700081' entity='client.admin' 2024-08-20T02:54:08.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:08 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:08.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:08 smithi165 ceph-mon[22357]: from='client.? ' entity='client.admin' 2024-08-20T02:54:08.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:08 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:08.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:08 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:08.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:08 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:08.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:08 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:08.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:08 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1769700081' entity='client.admin' 2024-08-20T02:54:08.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:08 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:08.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:08 smithi099 ceph-mon[21984]: from='client.? ' entity='client.admin' 2024-08-20T02:54:08.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.923+0000 7f0ede9fb640 1 -- 172.21.15.44:0/1881783808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed8107d20 msgr2=0x7f0ed810e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:08.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.923+0000 7f0ede9fb640 1 --2- 172.21.15.44:0/1881783808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed8107d20 0x7f0ed810e520 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009ca0 tx=0x7f0ed002a8c0 comp rx=0 tx=0).stop 2024-08-20T02:54:08.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.923+0000 7f0ede9fb640 1 -- 172.21.15.44:0/1881783808 shutdown_connections 2024-08-20T02:54:08.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.923+0000 7f0ede9fb640 1 --2- 172.21.15.44:0/1881783808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed8107d20 0x7f0ed810e520 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:08.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.923+0000 7f0ede9fb640 1 --2- 172.21.15.44:0/1881783808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ed81073a0 0x7f0ed81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:08.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.923+0000 7f0ede9fb640 1 --2- 172.21.15.44:0/1881783808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed8101990 0x7f0ed8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:08.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.923+0000 7f0ede9fb640 1 -- 172.21.15.44:0/1881783808 >> 172.21.15.44:0/1881783808 conn(0x7f0ed80fd400 msgr2=0x7f0ed80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.923+0000 7f0ede9fb640 1 -- 172.21.15.44:0/1881783808 shutdown_connections 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.923+0000 7f0ede9fb640 1 -- 172.21.15.44:0/1881783808 wait complete. 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0ede9fb640 1 Processor -- start 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0ede9fb640 1 -- start start 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0ede9fb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ed8101990 0x7f0ed8198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0ede9fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed81073a0 0x7f0ed81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0ede9fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed8107d20 0x7f0ed819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0ede9fb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed81129e0 con 0x7f0ed8107d20 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0ede9fb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed8112860 con 0x7f0ed81073a0 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0ede9fb640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed8112b60 con 0x7f0ed8101990 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0ed77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed81073a0 0x7f0ed81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0edcf98640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed8107d20 0x7f0ed819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0ed77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed81073a0 0x7f0ed81993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55296/0 (socket says 172.21.15.44:55296) 2024-08-20T02:54:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0edcf98640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed8107d20 0x7f0ed819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34460/0 (socket says 172.21.15.44:34460) 2024-08-20T02:54:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0ed7fff640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ed8101990 0x7f0ed8198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.924+0000 7f0ed77fe640 1 -- 172.21.15.44:0/2007193386 learned_addr learned my addr 172.21.15.44:0/2007193386 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.925+0000 7f0edcf98640 1 -- 172.21.15.44:0/2007193386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ed8101990 msgr2=0x7f0ed8198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.925+0000 7f0edcf98640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ed8101990 0x7f0ed8198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.925+0000 7f0edcf98640 1 -- 172.21.15.44:0/2007193386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed81073a0 msgr2=0x7f0ed81993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.925+0000 7f0edcf98640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed81073a0 0x7f0ed81993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.925+0000 7f0edcf98640 1 -- 172.21.15.44:0/2007193386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed819ef60 con 0x7f0ed8107d20 2024-08-20T02:54:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.925+0000 7f0ed77fe640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed81073a0 0x7f0ed81993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T02:54:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.925+0000 7f0edcf98640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed8107d20 0x7f0ed819e790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0005e30 tx=0x7f0ed0009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.925+0000 7f0ed57fa640 1 -- 172.21.15.44:0/2007193386 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0003c30 con 0x7f0ed8107d20 2024-08-20T02:54:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.925+0000 7f0ed57fa640 1 -- 172.21.15.44:0/2007193386 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0ed0003dd0 con 0x7f0ed8107d20 2024-08-20T02:54:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.925+0000 7f0ed57fa640 1 -- 172.21.15.44:0/2007193386 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed00045c0 con 0x7f0ed8107d20 2024-08-20T02:54:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.925+0000 7f0ede9fb640 1 -- 172.21.15.44:0/2007193386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed819f1f0 con 0x7f0ed8107d20 2024-08-20T02:54:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.925+0000 7f0ede9fb640 1 -- 172.21.15.44:0/2007193386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed81a5910 con 0x7f0ed8107d20 2024-08-20T02:54:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.926+0000 7f0ed57fa640 1 -- 172.21.15.44:0/2007193386 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 50764+0+0 (secure 0 0 0) 0x7f0ed004d440 con 0x7f0ed8107d20 2024-08-20T02:54:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.926+0000 7f0ede9fb640 1 -- 172.21.15.44:0/2007193386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed8101d70 con 0x7f0ed8107d20 2024-08-20T02:54:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.926+0000 7f0ed57fa640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0ec003edb0 0x7f0ec0041270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.926+0000 7f0ed57fa640 1 -- 172.21.15.44:0/2007193386 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0ed0074320 con 0x7f0ed8107d20 2024-08-20T02:54:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.926+0000 7f0ed7fff640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0ec003edb0 0x7f0ec0041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.927+0000 7f0ed7fff640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0ec003edb0 0x7f0ec0041270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009bd0 tx=0x7f0ec8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:08.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:08.931+0000 7f0ed57fa640 1 -- 172.21.15.44:0/2007193386 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0ed003fe10 con 0x7f0ed8107d20 2024-08-20T02:54:09.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.120+0000 7f0ede9fb640 1 -- 172.21.15.44:0/2007193386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f0ed8109b00 con 0x7f0ed8107d20 2024-08-20T02:54:09.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.124+0000 7f0ed57fa640 1 -- 172.21.15.44:0/2007193386 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v17)=0 v17) ==== 117+0+0 (secure 0 0 0) 0x7f0ed0040d40 con 0x7f0ed8107d20 2024-08-20T02:54:09.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.126+0000 7f0ede9fb640 1 -- 172.21.15.44:0/2007193386 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0ec003edb0 msgr2=0x7f0ec0041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:09.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.126+0000 7f0ede9fb640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0ec003edb0 0x7f0ec0041270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009bd0 tx=0x7f0ec8005f90 comp rx=0 tx=0).stop 2024-08-20T02:54:09.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.126+0000 7f0ede9fb640 1 -- 172.21.15.44:0/2007193386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed8107d20 msgr2=0x7f0ed819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:09.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.126+0000 7f0ede9fb640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed8107d20 0x7f0ed819e790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0005e30 tx=0x7f0ed0009890 comp rx=0 tx=0).stop 2024-08-20T02:54:09.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.126+0000 7f0ede9fb640 1 -- 172.21.15.44:0/2007193386 shutdown_connections 2024-08-20T02:54:09.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.126+0000 7f0ede9fb640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0ec003edb0 0x7f0ec0041270 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.126+0000 7f0ede9fb640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ed8107d20 0x7f0ed819e790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.126+0000 7f0ede9fb640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ed81073a0 0x7f0ed81993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.126+0000 7f0ede9fb640 1 --2- 172.21.15.44:0/2007193386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ed8101990 0x7f0ed8198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.126+0000 7f0ede9fb640 1 -- 172.21.15.44:0/2007193386 >> 172.21.15.44:0/2007193386 conn(0x7f0ed80fd400 msgr2=0x7f0ed80ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:09.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.126+0000 7f0ede9fb640 1 -- 172.21.15.44:0/2007193386 shutdown_connections 2024-08-20T02:54:09.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.126+0000 7f0ede9fb640 1 -- 172.21.15.44:0/2007193386 wait complete. 2024-08-20T02:54:09.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.618+0000 7f4b54fa5640 1 -- 172.21.15.44:0/3947044413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5010cf60 msgr2=0x7f4b5010f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.618+0000 7f4b54fa5640 1 --2- 172.21.15.44:0/3947044413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5010cf60 0x7f4b5010f350 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009ca0 tx=0x7f4b4802a8c0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.618+0000 7f4b54fa5640 1 -- 172.21.15.44:0/3947044413 shutdown_connections 2024-08-20T02:54:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.618+0000 7f4b54fa5640 1 --2- 172.21.15.44:0/3947044413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5010cf60 0x7f4b5010f350 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.618+0000 7f4b54fa5640 1 --2- 172.21.15.44:0/3947044413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b5010a630 0x7f4b5010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.618+0000 7f4b54fa5640 1 --2- 172.21.15.44:0/3947044413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b50101520 0x7f4b50101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.618+0000 7f4b54fa5640 1 -- 172.21.15.44:0/3947044413 >> 172.21.15.44:0/3947044413 conn(0x7f4b50076c80 msgr2=0x7f4b50077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:09.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.618+0000 7f4b54fa5640 1 -- 172.21.15.44:0/3947044413 shutdown_connections 2024-08-20T02:54:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.618+0000 7f4b54fa5640 1 -- 172.21.15.44:0/3947044413 wait complete. 2024-08-20T02:54:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b54fa5640 1 Processor -- start 2024-08-20T02:54:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b54fa5640 1 -- start start 2024-08-20T02:54:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b54fa5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b50101520 0x7f4b501024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b54fa5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b5010a630 0x7f4b50102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b54fa5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5010cf60 0x7f4b50102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b54fa5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50112980 con 0x7f4b5010cf60 2024-08-20T02:54:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b54fa5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50112800 con 0x7f4b50101520 2024-08-20T02:54:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b54fa5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50112b00 con 0x7f4b5010a630 2024-08-20T02:54:09.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b4ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5010cf60 0x7f4b50102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b4ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5010cf60 0x7f4b50102f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34466/0 (socket says 172.21.15.44:34466) 2024-08-20T02:54:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b4e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b50101520 0x7f4b501024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b4ed9d640 1 -- 172.21.15.44:0/769074498 learned_addr learned my addr 172.21.15.44:0/769074498 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b4dd9b640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b5010a630 0x7f4b50102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b4e59c640 1 -- 172.21.15.44:0/769074498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b5010a630 msgr2=0x7f4b50102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b4e59c640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b5010a630 0x7f4b50102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b4e59c640 1 -- 172.21.15.44:0/769074498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5010cf60 msgr2=0x7f4b50102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b4e59c640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5010cf60 0x7f4b50102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b4e59c640 1 -- 172.21.15.44:0/769074498 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b501a7250 con 0x7f4b50101520 2024-08-20T02:54:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.619+0000 7f4b4ed9d640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5010cf60 0x7f4b50102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T02:54:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.620+0000 7f4b4e59c640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b50101520 0x7f4b501024f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4b40005fd0 tx=0x7f4b4000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.620+0000 7f4b3f7fe640 1 -- 172.21.15.44:0/769074498 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40009c30 con 0x7f4b50101520 2024-08-20T02:54:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.620+0000 7f4b54fa5640 1 -- 172.21.15.44:0/769074498 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b501a7540 con 0x7f4b50101520 2024-08-20T02:54:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.620+0000 7f4b3f7fe640 1 -- 172.21.15.44:0/769074498 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4b400044c0 con 0x7f4b50101520 2024-08-20T02:54:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.620+0000 7f4b3f7fe640 1 -- 172.21.15.44:0/769074498 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40003d80 con 0x7f4b50101520 2024-08-20T02:54:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.620+0000 7f4b54fa5640 1 -- 172.21.15.44:0/769074498 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b501a7ad0 con 0x7f4b50101520 2024-08-20T02:54:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.621+0000 7f4b54fa5640 1 -- 172.21.15.44:0/769074498 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b50101900 con 0x7f4b50101520 2024-08-20T02:54:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.621+0000 7f4b3f7fe640 1 -- 172.21.15.44:0/769074498 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 17) ==== 50764+0+0 (secure 0 0 0) 0x7f4b40004030 con 0x7f4b50101520 2024-08-20T02:54:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.621+0000 7f4b3f7fe640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4b3803edb0 0x7f4b38041270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:09.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.621+0000 7f4b3f7fe640 1 -- 172.21.15.44:0/769074498 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4b40053050 con 0x7f4b50101520 2024-08-20T02:54:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.621+0000 7f4b4dd9b640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4b3803edb0 0x7f4b38041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:09.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.622+0000 7f4b4dd9b640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4b3803edb0 0x7f4b38041270 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4b50103fe0 tx=0x7f4b4400b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:09.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.630+0000 7f4b3f7fe640 1 -- 172.21.15.44:0/769074498 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4b40017020 con 0x7f4b50101520 2024-08-20T02:54:09.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:09.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:09.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: Reconfiguring mgr.x (unknown last config time)... 2024-08-20T02:54:09.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T02:54:09.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T02:54:09.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:09.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: Reconfiguring daemon mgr.x on smithi044 2024-08-20T02:54:09.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:09 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2007193386' entity='client.admin' 2024-08-20T02:54:09.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.820+0000 7f4b54fa5640 1 -- 172.21.15.44:0/769074498 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f4b501a7d30 con 0x7f4b50101520 2024-08-20T02:54:09.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.821+0000 7f77d6e3e640 1 -- 172.21.15.44:0/3903186436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77d01073a0 msgr2=0x7f77d01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.821+0000 7f77d6e3e640 1 --2- 172.21.15.44:0/3903186436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77d01073a0 0x7f77d01077e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009900 tx=0x7f77bc02a410 comp rx=0 tx=0).stop 2024-08-20T02:54:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.821+0000 7f77d6e3e640 1 -- 172.21.15.44:0/3903186436 shutdown_connections 2024-08-20T02:54:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.821+0000 7f77d6e3e640 1 --2- 172.21.15.44:0/3903186436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77d0107d20 0x7f77d010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.821+0000 7f77d6e3e640 1 --2- 172.21.15.44:0/3903186436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77d01073a0 0x7f77d01077e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.821+0000 7f77d6e3e640 1 --2- 172.21.15.44:0/3903186436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d0101990 0x7f77d0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.821+0000 7f77d6e3e640 1 -- 172.21.15.44:0/3903186436 >> 172.21.15.44:0/3903186436 conn(0x7f77d00fd400 msgr2=0x7f77d00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:09.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.821+0000 7f77d6e3e640 1 -- 172.21.15.44:0/3903186436 shutdown_connections 2024-08-20T02:54:09.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.821+0000 7f77d6e3e640 1 -- 172.21.15.44:0/3903186436 wait complete. 2024-08-20T02:54:09.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.822+0000 7f77d6e3e640 1 Processor -- start 2024-08-20T02:54:09.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.822+0000 7f77d6e3e640 1 -- start start 2024-08-20T02:54:09.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.822+0000 7f77d6e3e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77d0101990 0x7f77d00fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:09.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.822+0000 7f77d6e3e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77d01073a0 0x7f77d00fde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:09.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.822+0000 7f77d6e3e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d0107d20 0x7f77d00fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:09.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.822+0000 7f77d6e3e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01118c0 con 0x7f77d01073a0 2024-08-20T02:54:09.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.822+0000 7f77d6e3e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0111740 con 0x7f77d0101990 2024-08-20T02:54:09.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.822+0000 7f77d6e3e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0111a40 con 0x7f77d0107d20 2024-08-20T02:54:09.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.823+0000 7f77cffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77d01073a0 0x7f77d00fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:09.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.823+0000 7f77cffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77d01073a0 0x7f77d00fde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34476/0 (socket says 172.21.15.44:34476) 2024-08-20T02:54:09.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.823+0000 7f77d4bda640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77d0101990 0x7f77d00fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:09.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.823+0000 7f77cffff640 1 -- 172.21.15.44:0/631891649 learned_addr learned my addr 172.21.15.44:0/631891649 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:09.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.823+0000 7f77d53db640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d0107d20 0x7f77d00fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.823+0000 7f77cffff640 1 -- 172.21.15.44:0/631891649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d0107d20 msgr2=0x7f77d00fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.823+0000 7f77cffff640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d0107d20 0x7f77d00fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.823+0000 7f77cffff640 1 -- 172.21.15.44:0/631891649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77d0101990 msgr2=0x7f77d00fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.823+0000 7f77cffff640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77d0101990 0x7f77d00fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.823+0000 7f77cffff640 1 -- 172.21.15.44:0/631891649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d01a7380 con 0x7f77d01073a0 2024-08-20T02:54:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.823+0000 7f77d4bda640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77d0101990 0x7f77d00fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T02:54:09.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.823+0000 7f77cffff640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77d01073a0 0x7f77d00fde40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f77bc006010 tx=0x7f77bc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:09.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.825+0000 7f4b3f7fe640 1 -- 172.21.15.44:0/769074498 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v18)=0 v18) ==== 108+0+0 (secure 0 0 0) 0x7f4b40023dd0 con 0x7f4b50101520 2024-08-20T02:54:09.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.826+0000 7f77cdffb640 1 -- 172.21.15.44:0/631891649 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc03f030 con 0x7f77d01073a0 2024-08-20T02:54:09.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.827+0000 7f77cdffb640 1 -- 172.21.15.44:0/631891649 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f77bc0044c0 con 0x7f77d01073a0 2024-08-20T02:54:09.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.827+0000 7f77cdffb640 1 -- 172.21.15.44:0/631891649 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc035070 con 0x7f77d01073a0 2024-08-20T02:54:09.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.827+0000 7f77d6e3e640 1 -- 172.21.15.44:0/631891649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d01a7610 con 0x7f77d01073a0 2024-08-20T02:54:09.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.827+0000 7f77d6e3e640 1 -- 172.21.15.44:0/631891649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d01a7b40 con 0x7f77d01073a0 2024-08-20T02:54:09.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f4b54fa5640 1 -- 172.21.15.44:0/769074498 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4b3803edb0 msgr2=0x7f4b38041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:09.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f4b54fa5640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4b3803edb0 0x7f4b38041270 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4b50103fe0 tx=0x7f4b4400b390 comp rx=0 tx=0).stop 2024-08-20T02:54:09.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f4b54fa5640 1 -- 172.21.15.44:0/769074498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b50101520 msgr2=0x7f4b501024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:09.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f4b54fa5640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b50101520 0x7f4b501024f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4b40005fd0 tx=0x7f4b4000dde0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f4b54fa5640 1 -- 172.21.15.44:0/769074498 shutdown_connections 2024-08-20T02:54:09.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f4b54fa5640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4b3803edb0 0x7f4b38041270 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f4b54fa5640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b5010cf60 0x7f4b50102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f4b54fa5640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b5010a630 0x7f4b50102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f4b54fa5640 1 --2- 172.21.15.44:0/769074498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b50101520 0x7f4b501024f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:09.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f4b54fa5640 1 -- 172.21.15.44:0/769074498 >> 172.21.15.44:0/769074498 conn(0x7f4b50076c80 msgr2=0x7f4b501002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:09.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f77d6e3e640 1 -- 172.21.15.44:0/631891649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d0101d70 con 0x7f77d01073a0 2024-08-20T02:54:09.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f77cdffb640 1 -- 172.21.15.44:0/631891649 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 50764+0+0 (secure 0 0 0) 0x7f77bc042030 con 0x7f77d01073a0 2024-08-20T02:54:09.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f4b54fa5640 1 -- 172.21.15.44:0/769074498 shutdown_connections 2024-08-20T02:54:09.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.828+0000 7f4b54fa5640 1 -- 172.21.15.44:0/769074498 wait complete. 2024-08-20T02:54:09.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.829+0000 7f77cdffb640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77c003edb0 0x7f77c0041270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:09.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.829+0000 7f77cdffb640 1 -- 172.21.15.44:0/631891649 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f77bc073690 con 0x7f77d01073a0 2024-08-20T02:54:09.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.829+0000 7f77d4bda640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77c003edb0 0x7f77c0041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:09.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.829+0000 7f77d4bda640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77c003edb0 0x7f77c0041270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009fc0 tx=0x7f77c4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:09.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:09.833+0000 7f77cdffb640 1 -- 172.21.15.44:0/631891649 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f77bc04c350 con 0x7f77d01073a0 2024-08-20T02:54:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:09.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:09.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: Reconfiguring mgr.x (unknown last config time)... 2024-08-20T02:54:09.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T02:54:09.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T02:54:09.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:09.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: Reconfiguring daemon mgr.x on smithi044 2024-08-20T02:54:09.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:09 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2007193386' entity='client.admin' 2024-08-20T02:54:09.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: Reconfiguring mgr.x (unknown last config time)... 2024-08-20T02:54:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T02:54:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T02:54:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: Reconfiguring daemon mgr.x on smithi044 2024-08-20T02:54:09.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:09 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2007193386' entity='client.admin' 2024-08-20T02:54:10.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.024+0000 7f77d6e3e640 1 -- 172.21.15.44:0/631891649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f77d010ab10 con 0x7f77d01073a0 2024-08-20T02:54:10.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.028+0000 7f77cdffb640 1 -- 172.21.15.44:0/631891649 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v19)=0 v19) ==== 116+0+0 (secure 0 0 0) 0x7f77bc04a260 con 0x7f77d01073a0 2024-08-20T02:54:10.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.030+0000 7f77d6e3e640 1 -- 172.21.15.44:0/631891649 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77c003edb0 msgr2=0x7f77c0041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.030+0000 7f77d6e3e640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77c003edb0 0x7f77c0041270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009fc0 tx=0x7f77c4005f90 comp rx=0 tx=0).stop 2024-08-20T02:54:10.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.030+0000 7f77d6e3e640 1 -- 172.21.15.44:0/631891649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77d01073a0 msgr2=0x7f77d00fde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.030+0000 7f77d6e3e640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77d01073a0 0x7f77d00fde40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f77bc006010 tx=0x7f77bc0040c0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.030+0000 7f77d6e3e640 1 -- 172.21.15.44:0/631891649 shutdown_connections 2024-08-20T02:54:10.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.030+0000 7f77d6e3e640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77c003edb0 0x7f77c0041270 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.030+0000 7f77d6e3e640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d0107d20 0x7f77d00fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.030+0000 7f77d6e3e640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77d01073a0 0x7f77d00fde40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.030+0000 7f77d6e3e640 1 --2- 172.21.15.44:0/631891649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77d0101990 0x7f77d00fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.030+0000 7f77d6e3e640 1 -- 172.21.15.44:0/631891649 >> 172.21.15.44:0/631891649 conn(0x7f77d00fd400 msgr2=0x7f77d006aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:10.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.030+0000 7f77d6e3e640 1 -- 172.21.15.44:0/631891649 shutdown_connections 2024-08-20T02:54:10.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.030+0000 7f77d6e3e640 1 -- 172.21.15.44:0/631891649 wait complete. 2024-08-20T02:54:10.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.180+0000 7f392bbdd640 1 -- 172.21.15.44:0/2788296278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f392410cde0 msgr2=0x7f392410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.180+0000 7f392bbdd640 1 --2- 172.21.15.44:0/2788296278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f392410cde0 0x7f392410f250 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f391c009900 tx=0x7f391c02a410 comp rx=0 tx=0).stop 2024-08-20T02:54:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.180+0000 7f392bbdd640 1 -- 172.21.15.44:0/2788296278 shutdown_connections 2024-08-20T02:54:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.180+0000 7f392bbdd640 1 --2- 172.21.15.44:0/2788296278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f392410cde0 0x7f392410f250 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.180+0000 7f392bbdd640 1 --2- 172.21.15.44:0/2788296278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3924104230 0x7f392410c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.180+0000 7f392bbdd640 1 --2- 172.21.15.44:0/2788296278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3924103830 0x7f3924103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.180+0000 7f392bbdd640 1 -- 172.21.15.44:0/2788296278 >> 172.21.15.44:0/2788296278 conn(0x7f39240fd400 msgr2=0x7f39240ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.180+0000 7f392bbdd640 1 -- 172.21.15.44:0/2788296278 shutdown_connections 2024-08-20T02:54:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.180+0000 7f392bbdd640 1 -- 172.21.15.44:0/2788296278 wait complete. 2024-08-20T02:54:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f392bbdd640 1 Processor -- start 2024-08-20T02:54:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f392bbdd640 1 -- start start 2024-08-20T02:54:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f392bbdd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3924103830 0x7f3924198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:10.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f392bbdd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3924104230 0x7f3924199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f392bbdd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f392410cde0 0x7f392419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f392bbdd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3924112990 con 0x7f3924104230 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f392bbdd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3924112810 con 0x7f392410cde0 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f392bbdd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3924112b10 con 0x7f3924103830 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f3929178640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3924104230 0x7f3924199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f3929178640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3924104230 0x7f3924199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34502/0 (socket says 172.21.15.44:34502) 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f3929178640 1 -- 172.21.15.44:0/2532574936 learned_addr learned my addr 172.21.15.44:0/2532574936 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f392a17a640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f392410cde0 0x7f392419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.181+0000 7f3929979640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3924103830 0x7f3924198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.182+0000 7f3929178640 1 -- 172.21.15.44:0/2532574936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3924103830 msgr2=0x7f3924198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.182+0000 7f3929178640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3924103830 0x7f3924198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.182+0000 7f3929178640 1 -- 172.21.15.44:0/2532574936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f392410cde0 msgr2=0x7f392419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.182+0000 7f3929178640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f392410cde0 0x7f392419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.182+0000 7f3929178640 1 -- 172.21.15.44:0/2532574936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392419eed0 con 0x7f3924104230 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.182+0000 7f3929178640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3924104230 0x7f3924199360 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f391800d950 tx=0x7f391800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.182+0000 7f3922ffd640 1 -- 172.21.15.44:0/2532574936 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918017070 con 0x7f3924104230 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.182+0000 7f3922ffd640 1 -- 172.21.15.44:0/2532574936 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3918009c30 con 0x7f3924104230 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.182+0000 7f392bbdd640 1 -- 172.21.15.44:0/2532574936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392419f1c0 con 0x7f3924104230 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.182+0000 7f3922ffd640 1 -- 172.21.15.44:0/2532574936 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391800f620 con 0x7f3924104230 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.182+0000 7f392bbdd640 1 -- 172.21.15.44:0/2532574936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39241a5a50 con 0x7f3924104230 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.183+0000 7f3922ffd640 1 -- 172.21.15.44:0/2532574936 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 50764+0+0 (secure 0 0 0) 0x7f3918010460 con 0x7f3924104230 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.183+0000 7f392bbdd640 1 -- 172.21.15.44:0/2532574936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f392406ae50 con 0x7f3924104230 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.183+0000 7f3922ffd640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f390c03edb0 0x7f390c041270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.183+0000 7f3922ffd640 1 -- 172.21.15.44:0/2532574936 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3918052f30 con 0x7f3924104230 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.183+0000 7f3929979640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f390c03edb0 0x7f390c041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:10.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.184+0000 7f3929979640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f390c03edb0 0x7f390c041270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3914005fd0 tx=0x7f3914009680 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:10.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.188+0000 7f3922ffd640 1 -- 172.21.15.44:0/2532574936 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f3918009de0 con 0x7f3924104230 2024-08-20T02:54:10.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.364+0000 7f92eee9b640 1 -- 172.21.15.44:0/1476340823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e8101990 msgr2=0x7f92e8101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.364+0000 7f92eee9b640 1 --2- 172.21.15.44:0/1476340823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e8101990 0x7f92e8101d70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009900 tx=0x7f92dc02a3f0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.364+0000 7f92eee9b640 1 -- 172.21.15.44:0/1476340823 shutdown_connections 2024-08-20T02:54:10.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.364+0000 7f92eee9b640 1 --2- 172.21.15.44:0/1476340823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e8107d20 0x7f92e810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.364+0000 7f92eee9b640 1 --2- 172.21.15.44:0/1476340823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e81073a0 0x7f92e81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.364+0000 7f92eee9b640 1 --2- 172.21.15.44:0/1476340823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e8101990 0x7f92e8101d70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.364+0000 7f92eee9b640 1 -- 172.21.15.44:0/1476340823 >> 172.21.15.44:0/1476340823 conn(0x7f92e80fd400 msgr2=0x7f92e80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:10.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.364+0000 7f92eee9b640 1 -- 172.21.15.44:0/1476340823 shutdown_connections 2024-08-20T02:54:10.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.364+0000 7f92eee9b640 1 -- 172.21.15.44:0/1476340823 wait complete. 2024-08-20T02:54:10.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.364+0000 7f92eee9b640 1 Processor -- start 2024-08-20T02:54:10.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.364+0000 7f92eee9b640 1 -- start start 2024-08-20T02:54:10.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92eee9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e8101990 0x7f92e8198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:10.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92eee9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e81073a0 0x7f92e8199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:10.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92eee9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e8107d20 0x7f92e819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:10.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92eee9b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8111880 con 0x7f92e8101990 2024-08-20T02:54:10.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92eee9b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8111700 con 0x7f92e81073a0 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92eee9b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8111a00 con 0x7f92e8107d20 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92ecc37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e8101990 0x7f92e8198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92ecc37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e8101990 0x7f92e8198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34516/0 (socket says 172.21.15.44:34516) 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92ecc37640 1 -- 172.21.15.44:0/2846971386 learned_addr learned my addr 172.21.15.44:0/2846971386 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92ed438640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e8107d20 0x7f92e819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92e7fff640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e81073a0 0x7f92e8199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92ecc37640 1 -- 172.21.15.44:0/2846971386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e8107d20 msgr2=0x7f92e819e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92ecc37640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e8107d20 0x7f92e819e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92ecc37640 1 -- 172.21.15.44:0/2846971386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e81073a0 msgr2=0x7f92e8199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92ecc37640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e81073a0 0x7f92e8199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.365+0000 7f92ecc37640 1 -- 172.21.15.44:0/2846971386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e819efa0 con 0x7f92e8101990 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.366+0000 7f92ecc37640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e8101990 0x7f92e8198ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f92dc006010 tx=0x7f92dc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:10.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.366+0000 7f92e5ffb640 1 -- 172.21.15.44:0/2846971386 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc03f030 con 0x7f92e8101990 2024-08-20T02:54:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.366+0000 7f92e5ffb640 1 -- 172.21.15.44:0/2846971386 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f92dc0044c0 con 0x7f92e8101990 2024-08-20T02:54:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.366+0000 7f92eee9b640 1 -- 172.21.15.44:0/2846971386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e819f230 con 0x7f92e8101990 2024-08-20T02:54:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.366+0000 7f92eee9b640 1 -- 172.21.15.44:0/2846971386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e80fd7e0 con 0x7f92e8101990 2024-08-20T02:54:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.367+0000 7f92e5ffb640 1 -- 172.21.15.44:0/2846971386 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc049060 con 0x7f92e8101990 2024-08-20T02:54:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.367+0000 7f92e5ffb640 1 -- 172.21.15.44:0/2846971386 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 50764+0+0 (secure 0 0 0) 0x7f92dc042030 con 0x7f92e8101990 2024-08-20T02:54:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.367+0000 7f92eee9b640 1 -- 172.21.15.44:0/2846971386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e8101d70 con 0x7f92e8101990 2024-08-20T02:54:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.368+0000 7f92e5ffb640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92d803edb0 0x7f92d8041270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.368+0000 7f92e5ffb640 1 -- 172.21.15.44:0/2846971386 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f92dc0739f0 con 0x7f92e8101990 2024-08-20T02:54:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.368+0000 7f92e7fff640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92d803edb0 0x7f92d8041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.369+0000 7f92e7fff640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92d803edb0 0x7f92d8041270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f92d40095f0 tx=0x7f92d4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.373+0000 7f92e5ffb640 1 -- 172.21.15.44:0/2846971386 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f92dc042de0 con 0x7f92e8101990 2024-08-20T02:54:10.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.377+0000 7f392bbdd640 1 -- 172.21.15.44:0/2532574936 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f3924199f50 con 0x7f3924104230 2024-08-20T02:54:10.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.380+0000 7f3922ffd640 1 -- 172.21.15.44:0/2532574936 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f3918018270 con 0x7f3924104230 2024-08-20T02:54:10.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.382+0000 7f392bbdd640 1 -- 172.21.15.44:0/2532574936 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f390c03edb0 msgr2=0x7f390c041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.382+0000 7f392bbdd640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f390c03edb0 0x7f390c041270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3914005fd0 tx=0x7f3914009680 comp rx=0 tx=0).stop 2024-08-20T02:54:10.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.382+0000 7f392bbdd640 1 -- 172.21.15.44:0/2532574936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3924104230 msgr2=0x7f3924199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.382+0000 7f392bbdd640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3924104230 0x7f3924199360 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f391800d950 tx=0x7f391800de10 comp rx=0 tx=0).stop 2024-08-20T02:54:10.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.383+0000 7f392bbdd640 1 -- 172.21.15.44:0/2532574936 shutdown_connections 2024-08-20T02:54:10.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.383+0000 7f392bbdd640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f390c03edb0 0x7f390c041270 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.383+0000 7f392bbdd640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f392410cde0 0x7f392419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.383+0000 7f392bbdd640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3924104230 0x7f3924199360 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.383+0000 7f392bbdd640 1 --2- 172.21.15.44:0/2532574936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3924103830 0x7f3924198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.383+0000 7f392bbdd640 1 -- 172.21.15.44:0/2532574936 >> 172.21.15.44:0/2532574936 conn(0x7f39240fd400 msgr2=0x7f39240fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:10.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.383+0000 7f392bbdd640 1 -- 172.21.15.44:0/2532574936 shutdown_connections 2024-08-20T02:54:10.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.383+0000 7f392bbdd640 1 -- 172.21.15.44:0/2532574936 wait complete. 2024-08-20T02:54:10.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.563+0000 7f92eee9b640 1 -- 172.21.15.44:0/2846971386 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f92e810ab10 con 0x7f92e8101990 2024-08-20T02:54:10.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.566+0000 7f92e5ffb640 1 -- 172.21.15.44:0/2846971386 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v21)=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f92dc0319e0 con 0x7f92e8101990 2024-08-20T02:54:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.569+0000 7f92eee9b640 1 -- 172.21.15.44:0/2846971386 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92d803edb0 msgr2=0x7f92d8041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.569+0000 7f92eee9b640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92d803edb0 0x7f92d8041270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f92d40095f0 tx=0x7f92d4005f90 comp rx=0 tx=0).stop 2024-08-20T02:54:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.569+0000 7f92eee9b640 1 -- 172.21.15.44:0/2846971386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e8101990 msgr2=0x7f92e8198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.569+0000 7f92eee9b640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e8101990 0x7f92e8198ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f92dc006010 tx=0x7f92dc0040c0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.569+0000 7f92eee9b640 1 -- 172.21.15.44:0/2846971386 shutdown_connections 2024-08-20T02:54:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.569+0000 7f92eee9b640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92d803edb0 0x7f92d8041270 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.569+0000 7f92eee9b640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e8107d20 0x7f92e819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.569+0000 7f92eee9b640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e81073a0 0x7f92e8199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.569+0000 7f92eee9b640 1 --2- 172.21.15.44:0/2846971386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e8101990 0x7f92e8198ef0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.569+0000 7f92eee9b640 1 -- 172.21.15.44:0/2846971386 >> 172.21.15.44:0/2846971386 conn(0x7f92e80fd400 msgr2=0x7f92e806ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.569+0000 7f92eee9b640 1 -- 172.21.15.44:0/2846971386 shutdown_connections 2024-08-20T02:54:10.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.569+0000 7f92eee9b640 1 -- 172.21.15.44:0/2846971386 wait complete. 2024-08-20T02:54:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.774+0000 7fe92fb23640 1 -- 172.21.15.44:0/2748169759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe928108e40 msgr2=0x7fe92810f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.774+0000 7fe92fb23640 1 --2- 172.21.15.44:0/2748169759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe928108e40 0x7fe92810f640 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe920009ca0 tx=0x7fe92002a8c0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.774+0000 7fe92fb23640 1 -- 172.21.15.44:0/2748169759 shutdown_connections 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.774+0000 7fe92fb23640 1 --2- 172.21.15.44:0/2748169759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe928108e40 0x7fe92810f640 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.774+0000 7fe92fb23640 1 --2- 172.21.15.44:0/2748169759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9281084c0 0x7fe928108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.774+0000 7fe92fb23640 1 --2- 172.21.15.44:0/2748169759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe928102ab0 0x7fe928102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.774+0000 7fe92fb23640 1 -- 172.21.15.44:0/2748169759 >> 172.21.15.44:0/2748169759 conn(0x7fe9280fe500 msgr2=0x7fe928100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.774+0000 7fe92fb23640 1 -- 172.21.15.44:0/2748169759 shutdown_connections 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.774+0000 7fe92fb23640 1 -- 172.21.15.44:0/2748169759 wait complete. 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.775+0000 7fe92fb23640 1 Processor -- start 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.775+0000 7fe92fb23640 1 -- start start 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.775+0000 7fe92fb23640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe928102ab0 0x7fe9280751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.775+0000 7fe92fb23640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9281084c0 0x7fe928075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.775+0000 7fe92fb23640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe928108e40 0x7fe928075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:10.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.775+0000 7fe92fb23640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281129a0 con 0x7fe9281084c0 2024-08-20T02:54:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.775+0000 7fe92fb23640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe928112820 con 0x7fe928108e40 2024-08-20T02:54:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.775+0000 7fe92fb23640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe928112b20 con 0x7fe928102ab0 2024-08-20T02:54:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.775+0000 7fe92d0be640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9281084c0 0x7fe928075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.775+0000 7fe92d0be640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9281084c0 0x7fe928075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34530/0 (socket says 172.21.15.44:34530) 2024-08-20T02:54:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.775+0000 7fe92d0be640 1 -- 172.21.15.44:0/4094402127 learned_addr learned my addr 172.21.15.44:0/4094402127 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.775+0000 7fe92d8bf640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe928102ab0 0x7fe9280751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.776+0000 7fe92e0c0640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe928108e40 0x7fe928075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.776+0000 7fe92d0be640 1 -- 172.21.15.44:0/4094402127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe928102ab0 msgr2=0x7fe9280751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.776+0000 7fe92d0be640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe928102ab0 0x7fe9280751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.776+0000 7fe92d0be640 1 -- 172.21.15.44:0/4094402127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe928108e40 msgr2=0x7fe928075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.776+0000 7fe92d0be640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe928108e40 0x7fe928075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.776+0000 7fe92d0be640 1 -- 172.21.15.44:0/4094402127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9281a7270 con 0x7fe9281084c0 2024-08-20T02:54:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.776+0000 7fe92d0be640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9281084c0 0x7fe928075710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe91800e9f0 tx=0x7fe91800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.776+0000 7fe91effd640 1 -- 172.21.15.44:0/4094402127 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91800cd80 con 0x7fe9281084c0 2024-08-20T02:54:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.776+0000 7fe91effd640 1 -- 172.21.15.44:0/4094402127 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe9180108c0 con 0x7fe9281084c0 2024-08-20T02:54:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.776+0000 7fe92fb23640 1 -- 172.21.15.44:0/4094402127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9281a7560 con 0x7fe9281084c0 2024-08-20T02:54:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.776+0000 7fe91effd640 1 -- 172.21.15.44:0/4094402127 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9180187d0 con 0x7fe9281084c0 2024-08-20T02:54:10.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.776+0000 7fe92fb23640 1 -- 172.21.15.44:0/4094402127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9281a7af0 con 0x7fe9281084c0 2024-08-20T02:54:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.777+0000 7fe91effd640 1 -- 172.21.15.44:0/4094402127 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) ==== 50764+0+0 (secure 0 0 0) 0x7fe918018a70 con 0x7fe9281084c0 2024-08-20T02:54:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.777+0000 7fe92fb23640 1 -- 172.21.15.44:0/4094402127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe928102e90 con 0x7fe9281084c0 2024-08-20T02:54:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.778+0000 7fe91effd640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe91403edb0 0x7fe914041270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.778+0000 7fe91effd640 1 -- 172.21.15.44:0/4094402127 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe9180538c0 con 0x7fe9281084c0 2024-08-20T02:54:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.778+0000 7fe92d8bf640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe91403edb0 0x7fe914041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:10.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.778+0000 7fe92d8bf640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe91403edb0 0x7fe914041270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe924005d00 tx=0x7fe92400b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:10.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.782+0000 7fe91effd640 1 -- 172.21.15.44:0/4094402127 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe918010d50 con 0x7fe9281084c0 2024-08-20T02:54:10.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.971+0000 7fe92fb23640 1 -- 172.21.15.44:0/4094402127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fe92810ac20 con 0x7fe9281084c0 2024-08-20T02:54:10.973 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:10 smithi044 ceph-mon[18790]: from='client.? ' entity='client.admin' 2024-08-20T02:54:10.973 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:10 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/631891649' entity='client.admin' 2024-08-20T02:54:10.973 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:10 smithi044 ceph-mon[18790]: Standby manager daemon z started 2024-08-20T02:54:10.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:10 smithi044 ceph-mon[18790]: from='mgr.? 172.21.15.165:0/3786243303' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-20T02:54:10.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:10 smithi044 ceph-mon[18790]: from='mgr.? 172.21.15.165:0/3786243303' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T02:54:10.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:10 smithi044 ceph-mon[18790]: from='mgr.? 172.21.15.165:0/3786243303' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-20T02:54:10.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:10 smithi044 ceph-mon[18790]: from='mgr.? 172.21.15.165:0/3786243303' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T02:54:10.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:10 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2532574936' entity='client.admin' 2024-08-20T02:54:10.974 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:10 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2846971386' entity='client.admin' 2024-08-20T02:54:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.974+0000 7fe91effd640 1 -- 172.21.15.44:0/4094402127 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v22)=0 v22) ==== 123+0+0 (secure 0 0 0) 0x7fe918018d60 con 0x7fe9281084c0 2024-08-20T02:54:10.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.974+0000 7fe91effd640 1 -- 172.21.15.44:0/4094402127 <== mon.0 v2:172.21.15.44:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe918010ae0 con 0x7fe9281084c0 2024-08-20T02:54:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.976+0000 7fe92fb23640 1 -- 172.21.15.44:0/4094402127 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe91403edb0 msgr2=0x7fe914041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.976+0000 7fe92fb23640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe91403edb0 0x7fe914041270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe924005d00 tx=0x7fe92400b410 comp rx=0 tx=0).stop 2024-08-20T02:54:10.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.976+0000 7fe92fb23640 1 -- 172.21.15.44:0/4094402127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9281084c0 msgr2=0x7fe928075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.976+0000 7fe92fb23640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9281084c0 0x7fe928075710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe91800e9f0 tx=0x7fe91800eeb0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.976+0000 7fe92fb23640 1 -- 172.21.15.44:0/4094402127 shutdown_connections 2024-08-20T02:54:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.976+0000 7fe92fb23640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe91403edb0 0x7fe914041270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.976+0000 7fe92fb23640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe928108e40 0x7fe928075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.976+0000 7fe92fb23640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9281084c0 0x7fe928075710 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.976+0000 7fe92fb23640 1 --2- 172.21.15.44:0/4094402127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe928102ab0 0x7fe9280751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.976+0000 7fe92fb23640 1 -- 172.21.15.44:0/4094402127 >> 172.21.15.44:0/4094402127 conn(0x7fe9280fe500 msgr2=0x7fe9280fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.976+0000 7fe92fb23640 1 -- 172.21.15.44:0/4094402127 shutdown_connections 2024-08-20T02:54:10.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:10.976+0000 7fe92fb23640 1 -- 172.21.15.44:0/4094402127 wait complete. 2024-08-20T02:54:11.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:10 smithi165 ceph-mon[22357]: from='client.? ' entity='client.admin' 2024-08-20T02:54:11.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:10 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/631891649' entity='client.admin' 2024-08-20T02:54:11.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:10 smithi165 ceph-mon[22357]: Standby manager daemon z started 2024-08-20T02:54:11.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:10 smithi165 ceph-mon[22357]: from='mgr.? 172.21.15.165:0/3786243303' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-20T02:54:11.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:10 smithi165 ceph-mon[22357]: from='mgr.? 172.21.15.165:0/3786243303' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T02:54:11.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:10 smithi165 ceph-mon[22357]: from='mgr.? 172.21.15.165:0/3786243303' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-20T02:54:11.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:10 smithi165 ceph-mon[22357]: from='mgr.? 172.21.15.165:0/3786243303' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T02:54:11.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:10 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2532574936' entity='client.admin' 2024-08-20T02:54:11.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:10 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2846971386' entity='client.admin' 2024-08-20T02:54:11.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:10 smithi099 ceph-mon[21984]: from='client.? ' entity='client.admin' 2024-08-20T02:54:11.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:10 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/631891649' entity='client.admin' 2024-08-20T02:54:11.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:10 smithi099 ceph-mon[21984]: Standby manager daemon z started 2024-08-20T02:54:11.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:10 smithi099 ceph-mon[21984]: from='mgr.? 172.21.15.165:0/3786243303' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-20T02:54:11.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:10 smithi099 ceph-mon[21984]: from='mgr.? 172.21.15.165:0/3786243303' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T02:54:11.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:10 smithi099 ceph-mon[21984]: from='mgr.? 172.21.15.165:0/3786243303' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-20T02:54:11.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:10 smithi099 ceph-mon[21984]: from='mgr.? 172.21.15.165:0/3786243303' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T02:54:11.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:10 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2532574936' entity='client.admin' 2024-08-20T02:54:11.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:10 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2846971386' entity='client.admin' 2024-08-20T02:54:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:11 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/4094402127' entity='client.admin' 2024-08-20T02:54:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:11 smithi044 ceph-mon[18790]: mgrmap e18: x(active, since 12s), standbys: z 2024-08-20T02:54:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-20T02:54:12.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:11 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/4094402127' entity='client.admin' 2024-08-20T02:54:12.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:11 smithi099 ceph-mon[21984]: mgrmap e18: x(active, since 12s), standbys: z 2024-08-20T02:54:12.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-20T02:54:12.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:11 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/4094402127' entity='client.admin' 2024-08-20T02:54:12.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:11 smithi165 ceph-mon[22357]: mgrmap e18: x(active, since 12s), standbys: z 2024-08-20T02:54:12.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-20T02:54:12.873 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi044') 2024-08-20T02:54:12.874 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi044') 2024-08-20T02:54:13.210 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:12 smithi044 ceph-mon[18790]: Standby manager daemon y started 2024-08-20T02:54:13.210 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:12 smithi044 ceph-mon[18790]: from='mgr.? 172.21.15.99:0/3683581866' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-20T02:54:13.210 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:12 smithi044 ceph-mon[18790]: from='mgr.? 172.21.15.99:0/3683581866' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T02:54:13.210 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:12 smithi044 ceph-mon[18790]: from='mgr.? 172.21.15.99:0/3683581866' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-20T02:54:13.210 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:12 smithi044 ceph-mon[18790]: from='mgr.? 172.21.15.99:0/3683581866' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T02:54:13.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:12 smithi165 ceph-mon[22357]: Standby manager daemon y started 2024-08-20T02:54:13.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:12 smithi165 ceph-mon[22357]: from='mgr.? 172.21.15.99:0/3683581866' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-20T02:54:13.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:12 smithi165 ceph-mon[22357]: from='mgr.? 172.21.15.99:0/3683581866' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T02:54:13.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:12 smithi165 ceph-mon[22357]: from='mgr.? 172.21.15.99:0/3683581866' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-20T02:54:13.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:12 smithi165 ceph-mon[22357]: from='mgr.? 172.21.15.99:0/3683581866' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T02:54:13.404 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:13.404 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:13.404 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:13.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:12 smithi099 ceph-mon[21984]: Standby manager daemon y started 2024-08-20T02:54:13.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:12 smithi099 ceph-mon[21984]: from='mgr.? 172.21.15.99:0/3683581866' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-20T02:54:13.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:12 smithi099 ceph-mon[21984]: from='mgr.? 172.21.15.99:0/3683581866' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T02:54:13.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:12 smithi099 ceph-mon[21984]: from='mgr.? 172.21.15.99:0/3683581866' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-20T02:54:13.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:12 smithi099 ceph-mon[21984]: from='mgr.? 172.21.15.99:0/3683581866' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T02:54:14.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:13 smithi044 ceph-mon[18790]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-08-20T02:54:14.106 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-20T02:54:14.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:13 smithi165 ceph-mon[22357]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-08-20T02:54:14.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-20T02:54:14.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:13 smithi099 ceph-mon[21984]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-08-20T02:54:14.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-20T02:54:15.457 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi044') 2024-08-20T02:54:15.457 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi044') 2024-08-20T02:54:15.457 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi044') 2024-08-20T02:54:15.457 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi044') 2024-08-20T02:54:15.457 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi044') 2024-08-20T02:54:15.457 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi044') 2024-08-20T02:54:15.458 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi044') 2024-08-20T02:54:15.458 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi044') 2024-08-20T02:54:15.465 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:15.465 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:15.467 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:15.468 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:15.468 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:15.468 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:15.470 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:15.470 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:15.474 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:16.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.147+0000 7fc1ea27c640 1 -- 172.21.15.44:0/912014480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e4107d40 msgr2=0x7fc1e410e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:16.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.147+0000 7fc1ea27c640 1 --2- 172.21.15.44:0/912014480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e4107d40 0x7fc1e410e540 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009ca0 tx=0x7fc1dc02a8c0 comp rx=0 tx=0).stop 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.147+0000 7fc1ea27c640 1 -- 172.21.15.44:0/912014480 shutdown_connections 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.147+0000 7fc1ea27c640 1 --2- 172.21.15.44:0/912014480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e4107d40 0x7fc1e410e540 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.147+0000 7fc1ea27c640 1 --2- 172.21.15.44:0/912014480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e41073c0 0x7fc1e4107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.147+0000 7fc1ea27c640 1 --2- 172.21.15.44:0/912014480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1e41019b0 0x7fc1e4101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.147+0000 7fc1ea27c640 1 -- 172.21.15.44:0/912014480 >> 172.21.15.44:0/912014480 conn(0x7fc1e40fd460 msgr2=0x7fc1e40ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.147+0000 7fc1ea27c640 1 -- 172.21.15.44:0/912014480 shutdown_connections 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.147+0000 7fc1ea27c640 1 -- 172.21.15.44:0/912014480 wait complete. 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.148+0000 7fc1ea27c640 1 Processor -- start 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.148+0000 7fc1ea27c640 1 -- start start 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.148+0000 7fc1ea27c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1e41019b0 0x7fc1e4199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.148+0000 7fc1ea27c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e41073c0 0x7fc1e419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.148+0000 7fc1ea27c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e4107d40 0x7fc1e419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.148+0000 7fc1ea27c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4111a00 con 0x7fc1e4107d40 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.148+0000 7fc1ea27c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4111880 con 0x7fc1e41073c0 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.148+0000 7fc1ea27c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e4111b80 con 0x7fc1e41019b0 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e8819640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e4107d40 0x7fc1e419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e8819640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e4107d40 0x7fc1e419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34544/0 (socket says 172.21.15.44:34544) 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e8819640 1 -- 172.21.15.44:0/3850207613 learned_addr learned my addr 172.21.15.44:0/3850207613 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e37fe640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e41073c0 0x7fc1e419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e3fff640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1e41019b0 0x7fc1e4199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e8819640 1 -- 172.21.15.44:0/3850207613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1e41019b0 msgr2=0x7fc1e4199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e8819640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1e41019b0 0x7fc1e4199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e8819640 1 -- 172.21.15.44:0/3850207613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e41073c0 msgr2=0x7fc1e419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e8819640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e41073c0 0x7fc1e419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e8819640 1 -- 172.21.15.44:0/3850207613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e41a5090 con 0x7fc1e4107d40 2024-08-20T02:54:16.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e37fe640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e41073c0 0x7fc1e419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:54:16.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e8819640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e4107d40 0x7fc1e419a750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc005cc0 tx=0x7fc1dc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:16.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e17fa640 1 -- 172.21.15.44:0/3850207613 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc003bf0 con 0x7fc1e4107d40 2024-08-20T02:54:16.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e17fa640 1 -- 172.21.15.44:0/3850207613 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1dc003d90 con 0x7fc1e4107d40 2024-08-20T02:54:16.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1ea27c640 1 -- 172.21.15.44:0/3850207613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e41a5320 con 0x7fc1e4107d40 2024-08-20T02:54:16.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1e17fa640 1 -- 172.21.15.44:0/3850207613 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc02fbc0 con 0x7fc1e4107d40 2024-08-20T02:54:16.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.149+0000 7fc1ea27c640 1 -- 172.21.15.44:0/3850207613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e41a58b0 con 0x7fc1e4107d40 2024-08-20T02:54:16.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.150+0000 7fc1ea27c640 1 -- 172.21.15.44:0/3850207613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e4101d90 con 0x7fc1e4107d40 2024-08-20T02:54:16.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.152+0000 7fc1e17fa640 1 -- 172.21.15.44:0/3850207613 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fc1dc02fd60 con 0x7fc1e4107d40 2024-08-20T02:54:16.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.152+0000 7fc1e17fa640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc1cc0b3c10 0x7fc1cc0b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:16.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.153+0000 7fc1e3fff640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc1cc0b3c10 0x7fc1cc0b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:16.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.153+0000 7fc1e17fa640 1 -- 172.21.15.44:0/3850207613 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc1dc103900 con 0x7fc1e4107d40 2024-08-20T02:54:16.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.153+0000 7fc1e3fff640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc1cc0b3c10 0x7fc1cc0b60d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4005fd0 tx=0x7fc1d4005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:16.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.155+0000 7fc1e17fa640 1 -- 172.21.15.44:0/3850207613 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fc1dc0ccb50 con 0x7fc1e4107d40 2024-08-20T02:54:16.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.341+0000 7fc1ea27c640 1 -- 172.21.15.44:0/3850207613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fc1e406b3e0 con 0x7fc1e4107d40 2024-08-20T02:54:16.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.345+0000 7fc1e17fa640 1 -- 172.21.15.44:0/3850207613 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v23)=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7fc1dc0cc450 con 0x7fc1e4107d40 2024-08-20T02:54:16.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.347+0000 7fc1ea27c640 1 -- 172.21.15.44:0/3850207613 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc1cc0b3c10 msgr2=0x7fc1cc0b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.347+0000 7fc1ea27c640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc1cc0b3c10 0x7fc1cc0b60d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4005fd0 tx=0x7fc1d4005850 comp rx=0 tx=0).stop 2024-08-20T02:54:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.347+0000 7fc1ea27c640 1 -- 172.21.15.44:0/3850207613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e4107d40 msgr2=0x7fc1e419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.347+0000 7fc1ea27c640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e4107d40 0x7fc1e419a750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc005cc0 tx=0x7fc1dc02f380 comp rx=0 tx=0).stop 2024-08-20T02:54:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.347+0000 7fc1ea27c640 1 -- 172.21.15.44:0/3850207613 shutdown_connections 2024-08-20T02:54:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.347+0000 7fc1ea27c640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc1cc0b3c10 0x7fc1cc0b60d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.347+0000 7fc1ea27c640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e4107d40 0x7fc1e419a750 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.347+0000 7fc1ea27c640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1e41073c0 0x7fc1e419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.347+0000 7fc1ea27c640 1 --2- 172.21.15.44:0/3850207613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1e41019b0 0x7fc1e4199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.347+0000 7fc1ea27c640 1 -- 172.21.15.44:0/3850207613 >> 172.21.15.44:0/3850207613 conn(0x7fc1e40fd460 msgr2=0x7fc1e41009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.347+0000 7fc1ea27c640 1 -- 172.21.15.44:0/3850207613 shutdown_connections 2024-08-20T02:54:16.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:16.347+0000 7fc1ea27c640 1 -- 172.21.15.44:0/3850207613 wait complete. 2024-08-20T02:54:16.803 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:17.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.351+0000 7fd4db899640 1 -- 172.21.15.44:0/3093592468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d41039e0 msgr2=0x7fd4d4103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.351+0000 7fd4db899640 1 --2- 172.21.15.44:0/3093592468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d41039e0 0x7fd4d4103dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009900 tx=0x7fd4d002a410 comp rx=0 tx=0).stop 2024-08-20T02:54:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.351+0000 7fd4db899640 1 -- 172.21.15.44:0/3093592468 shutdown_connections 2024-08-20T02:54:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.351+0000 7fd4db899640 1 --2- 172.21.15.44:0/3093592468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d410ce40 0x7fd4d410f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.351+0000 7fd4db899640 1 --2- 172.21.15.44:0/3093592468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4d4104300 0x7fd4d410c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.351+0000 7fd4db899640 1 --2- 172.21.15.44:0/3093592468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d41039e0 0x7fd4d4103dc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.351+0000 7fd4db899640 1 -- 172.21.15.44:0/3093592468 >> 172.21.15.44:0/3093592468 conn(0x7fd4d40fd420 msgr2=0x7fd4d40ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.351+0000 7fd4db899640 1 -- 172.21.15.44:0/3093592468 shutdown_connections 2024-08-20T02:54:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.351+0000 7fd4db899640 1 -- 172.21.15.44:0/3093592468 wait complete. 2024-08-20T02:54:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.352+0000 7fd4db899640 1 Processor -- start 2024-08-20T02:54:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.352+0000 7fd4db899640 1 -- start start 2024-08-20T02:54:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.352+0000 7fd4db899640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4d41039e0 0x7fd4d4101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.352+0000 7fd4db899640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d4104300 0x7fd4d4101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.352+0000 7fd4db899640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d410ce40 0x7fd4d4102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.352+0000 7fd4db899640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4112b00 con 0x7fd4d410ce40 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.352+0000 7fd4db899640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4112980 con 0x7fd4d41039e0 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.352+0000 7fd4db899640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4112c80 con 0x7fd4d4104300 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.352+0000 7fd4d9e36640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d410ce40 0x7fd4d4102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.352+0000 7fd4d9e36640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d410ce40 0x7fd4d4102110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34560/0 (socket says 172.21.15.44:34560) 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.352+0000 7fd4d9e36640 1 -- 172.21.15.44:0/2125227312 learned_addr learned my addr 172.21.15.44:0/2125227312 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4d9635640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4d41039e0 0x7fd4d4101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4d8e34640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d4104300 0x7fd4d4101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4d9e36640 1 -- 172.21.15.44:0/2125227312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d4104300 msgr2=0x7fd4d4101bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4d9e36640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d4104300 0x7fd4d4101bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4d9e36640 1 -- 172.21.15.44:0/2125227312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4d41039e0 msgr2=0x7fd4d4101690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4d9e36640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4d41039e0 0x7fd4d4101690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4d9e36640 1 -- 172.21.15.44:0/2125227312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d41a73b0 con 0x7fd4d410ce40 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4d8e34640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d4104300 0x7fd4d4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4d9e36640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d410ce40 0x7fd4d4102110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00eaa0 tx=0x7fd4cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4ca7fc640 1 -- 172.21.15.44:0/2125227312 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00cdb0 con 0x7fd4d410ce40 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4ca7fc640 1 -- 172.21.15.44:0/2125227312 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4cc005020 con 0x7fd4d410ce40 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4db899640 1 -- 172.21.15.44:0/2125227312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d41a76a0 con 0x7fd4d410ce40 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4ca7fc640 1 -- 172.21.15.44:0/2125227312 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0116d0 con 0x7fd4d410ce40 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.353+0000 7fd4db899640 1 -- 172.21.15.44:0/2125227312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d41a7c30 con 0x7fd4d410ce40 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.354+0000 7fd4db899640 1 -- 172.21.15.44:0/2125227312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d4103dc0 con 0x7fd4d410ce40 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.355+0000 7fd4ca7fc640 1 -- 172.21.15.44:0/2125227312 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fd4cc0051d0 con 0x7fd4d410ce40 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.356+0000 7fd4ca7fc640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4c00b3c10 0x7fd4c00b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.356+0000 7fd4d9635640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4c00b3c10 0x7fd4c00b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:17.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.356+0000 7fd4ca7fc640 1 -- 172.21.15.44:0/2125227312 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd4cc0e2500 con 0x7fd4d410ce40 2024-08-20T02:54:17.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.357+0000 7fd4d9635640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4c00b3c10 0x7fd4c00b60d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005ab0 tx=0x7fd4d0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:17.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.359+0000 7fd4ca7fc640 1 -- 172.21.15.44:0/2125227312 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd4cc016030 con 0x7fd4d410ce40 2024-08-20T02:54:17.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:17 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3850207613' entity='client.admin' 2024-08-20T02:54:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:17 smithi044 ceph-mon[18790]: Reconfiguring mon.b (monmap changed)... 2024-08-20T02:54:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:54:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T02:54:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:17 smithi044 ceph-mon[18790]: Reconfiguring daemon mon.b on smithi099 2024-08-20T02:54:17.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.549+0000 7fd4db899640 1 -- 172.21.15.44:0/2125227312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fd4d4102990 con 0x7fd4d410ce40 2024-08-20T02:54:17.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.552+0000 7fd4ca7fc640 1 -- 172.21.15.44:0/2125227312 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v24)=0 v24) ==== 120+0+0 (secure 0 0 0) 0x7fd4d4102990 con 0x7fd4d410ce40 2024-08-20T02:54:17.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.555+0000 7fd4db899640 1 -- 172.21.15.44:0/2125227312 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4c00b3c10 msgr2=0x7fd4c00b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:17.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.555+0000 7fd4db899640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4c00b3c10 0x7fd4c00b60d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005ab0 tx=0x7fd4d0005a20 comp rx=0 tx=0).stop 2024-08-20T02:54:17.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.555+0000 7fd4db899640 1 -- 172.21.15.44:0/2125227312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d410ce40 msgr2=0x7fd4d4102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:17.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.555+0000 7fd4db899640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d410ce40 0x7fd4d4102110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00eaa0 tx=0x7fd4cc00ef60 comp rx=0 tx=0).stop 2024-08-20T02:54:17.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.555+0000 7fd4db899640 1 -- 172.21.15.44:0/2125227312 shutdown_connections 2024-08-20T02:54:17.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.555+0000 7fd4db899640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4c00b3c10 0x7fd4c00b60d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:17.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.555+0000 7fd4db899640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4d410ce40 0x7fd4d4102110 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:17.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.555+0000 7fd4db899640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d4104300 0x7fd4d4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:17.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.555+0000 7fd4db899640 1 --2- 172.21.15.44:0/2125227312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4d41039e0 0x7fd4d4101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:17.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.555+0000 7fd4db899640 1 -- 172.21.15.44:0/2125227312 >> 172.21.15.44:0/2125227312 conn(0x7fd4d40fd420 msgr2=0x7fd4d410f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:17.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.555+0000 7fd4db899640 1 -- 172.21.15.44:0/2125227312 shutdown_connections 2024-08-20T02:54:17.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:17.555+0000 7fd4db899640 1 -- 172.21.15.44:0/2125227312 wait complete. 2024-08-20T02:54:17.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:17 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3850207613' entity='client.admin' 2024-08-20T02:54:17.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:17.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:17.572 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:17 smithi099 ceph-mon[21984]: Reconfiguring mon.b (monmap changed)... 2024-08-20T02:54:17.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:54:17.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T02:54:17.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:17.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:17 smithi099 ceph-mon[21984]: Reconfiguring daemon mon.b on smithi099 2024-08-20T02:54:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:17 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3850207613' entity='client.admin' 2024-08-20T02:54:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:17 smithi165 ceph-mon[22357]: Reconfiguring mon.b (monmap changed)... 2024-08-20T02:54:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:54:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T02:54:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:17.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:17 smithi165 ceph-mon[22357]: Reconfiguring daemon mon.b on smithi099 2024-08-20T02:54:18.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:18 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2125227312' entity='client.admin' 2024-08-20T02:54:18.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:18 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:18.750 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:18 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:18.751 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:18 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:54:18.751 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:18 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T02:54:18.751 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:18 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:18.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:18 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2125227312' entity='client.admin' 2024-08-20T02:54:18.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:18 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:18.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:18 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:18.808 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:18 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:54:18.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:18 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T02:54:18.809 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:18 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:18.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.832+0000 7f346e44c640 1 -- 172.21.15.44:0/2278528099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34681084e0 msgr2=0x7f3468108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:18.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.832+0000 7f346e44c640 1 --2- 172.21.15.44:0/2278528099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34681084e0 0x7f3468108920 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f345c009900 tx=0x7f345c02a410 comp rx=0 tx=0).stop 2024-08-20T02:54:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.832+0000 7f346e44c640 1 -- 172.21.15.44:0/2278528099 shutdown_connections 2024-08-20T02:54:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.832+0000 7f346e44c640 1 --2- 172.21.15.44:0/2278528099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3468108e60 0x7f346810f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.832+0000 7f346e44c640 1 --2- 172.21.15.44:0/2278528099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34681084e0 0x7f3468108920 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.832+0000 7f346e44c640 1 --2- 172.21.15.44:0/2278528099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3468102ad0 0x7f3468102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.832+0000 7f346e44c640 1 -- 172.21.15.44:0/2278528099 >> 172.21.15.44:0/2278528099 conn(0x7f34680fe540 msgr2=0x7f3468100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.832+0000 7f346e44c640 1 -- 172.21.15.44:0/2278528099 shutdown_connections 2024-08-20T02:54:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.832+0000 7f346e44c640 1 -- 172.21.15.44:0/2278528099 wait complete. 2024-08-20T02:54:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f346e44c640 1 Processor -- start 2024-08-20T02:54:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f346e44c640 1 -- start start 2024-08-20T02:54:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f346e44c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3468102ad0 0x7f346819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:18.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f346e44c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34681084e0 0x7f346819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f346e44c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3468108e60 0x7f34681a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f346e44c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3468112b20 con 0x7f34681084e0 2024-08-20T02:54:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f346e44c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f34681129a0 con 0x7f3468102ad0 2024-08-20T02:54:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f346e44c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3468112ca0 con 0x7f3468108e60 2024-08-20T02:54:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f34677fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34681084e0 0x7f346819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f34677fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34681084e0 0x7f346819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59792/0 (socket says 172.21.15.44:59792) 2024-08-20T02:54:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f34677fe640 1 -- 172.21.15.44:0/2332369778 learned_addr learned my addr 172.21.15.44:0/2332369778 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:18.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f346c9e9640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3468108e60 0x7f34681a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.833+0000 7f3467fff640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3468102ad0 0x7f346819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.834+0000 7f34677fe640 1 -- 172.21.15.44:0/2332369778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3468108e60 msgr2=0x7f34681a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.834+0000 7f34677fe640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3468108e60 0x7f34681a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.834+0000 7f34677fe640 1 -- 172.21.15.44:0/2332369778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3468102ad0 msgr2=0x7f346819b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.834+0000 7f34677fe640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3468102ad0 0x7f346819b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.834+0000 7f34677fe640 1 -- 172.21.15.44:0/2332369778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34681a12e0 con 0x7f34681084e0 2024-08-20T02:54:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.834+0000 7f34677fe640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34681084e0 0x7f346819b770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f345c005940 tx=0x7f345c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.834+0000 7f34657fa640 1 -- 172.21.15.44:0/2332369778 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c03f030 con 0x7f34681084e0 2024-08-20T02:54:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.834+0000 7f34657fa640 1 -- 172.21.15.44:0/2332369778 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f345c003cb0 con 0x7f34681084e0 2024-08-20T02:54:18.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.834+0000 7f346e44c640 1 -- 172.21.15.44:0/2332369778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34681a1570 con 0x7f34681084e0 2024-08-20T02:54:18.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.834+0000 7f34657fa640 1 -- 172.21.15.44:0/2332369778 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c035070 con 0x7f34681084e0 2024-08-20T02:54:18.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.834+0000 7f346e44c640 1 -- 172.21.15.44:0/2332369778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34681a7c90 con 0x7f34681084e0 2024-08-20T02:54:18.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.835+0000 7f346e44c640 1 -- 172.21.15.44:0/2332369778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3468102eb0 con 0x7f34681084e0 2024-08-20T02:54:18.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.836+0000 7f34657fa640 1 -- 172.21.15.44:0/2332369778 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f345c042030 con 0x7f34681084e0 2024-08-20T02:54:18.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.837+0000 7f34657fa640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f34500b3b90 0x7f34500b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:18.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.838+0000 7f3467fff640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f34500b3b90 0x7f34500b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:18.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.838+0000 7f34657fa640 1 -- 172.21.15.44:0/2332369778 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f345c102bf0 con 0x7f34681084e0 2024-08-20T02:54:18.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.838+0000 7f3467fff640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f34500b3b90 0x7f34500b6050 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3458005fd0 tx=0x7f345800b590 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:18.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.841+0000 7f34657fa640 1 -- 172.21.15.44:0/2332369778 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f345c0cbda0 con 0x7f34681084e0 2024-08-20T02:54:18.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:18 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2125227312' entity='client.admin' 2024-08-20T02:54:18.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:18 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:18.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:18 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:18.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:18 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T02:54:18.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:18 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T02:54:18.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:18 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:18.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.991+0000 7fdb735ac640 1 -- 172.21.15.44:0/559600496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb6c102ab0 msgr2=0x7fdb6c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:18.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.991+0000 7fdb735ac640 1 --2- 172.21.15.44:0/559600496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb6c102ab0 0x7fdb6c102e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009900 tx=0x7fdb6802a410 comp rx=0 tx=0).stop 2024-08-20T02:54:18.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.991+0000 7fdb735ac640 1 -- 172.21.15.44:0/559600496 shutdown_connections 2024-08-20T02:54:18.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.991+0000 7fdb735ac640 1 --2- 172.21.15.44:0/559600496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb6c108e40 0x7fdb6c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:18.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.991+0000 7fdb735ac640 1 --2- 172.21.15.44:0/559600496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb6c1084c0 0x7fdb6c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:18.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.991+0000 7fdb735ac640 1 --2- 172.21.15.44:0/559600496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb6c102ab0 0x7fdb6c102e90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:18.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.991+0000 7fdb735ac640 1 -- 172.21.15.44:0/559600496 >> 172.21.15.44:0/559600496 conn(0x7fdb6c0fe500 msgr2=0x7fdb6c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:18.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.991+0000 7fdb735ac640 1 -- 172.21.15.44:0/559600496 shutdown_connections 2024-08-20T02:54:18.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.991+0000 7fdb735ac640 1 -- 172.21.15.44:0/559600496 wait complete. 2024-08-20T02:54:18.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb735ac640 1 Processor -- start 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb735ac640 1 -- start start 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb735ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb6c102ab0 0x7fdb6c0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb735ac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb6c1084c0 0x7fdb6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb735ac640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb6c108e40 0x7fdb6c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb735ac640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c112b00 con 0x7fdb6c102ab0 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb735ac640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c112980 con 0x7fdb6c1084c0 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb735ac640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c112c80 con 0x7fdb6c108e40 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb71348640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb6c102ab0 0x7fdb6c0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb71348640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb6c102ab0 0x7fdb6c0751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59814/0 (socket says 172.21.15.44:59814) 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb71348640 1 -- 172.21.15.44:0/1174562905 learned_addr learned my addr 172.21.15.44:0/1174562905 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb71b49640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb6c108e40 0x7fdb6c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb70b47640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb6c1084c0 0x7fdb6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:18.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb71348640 1 -- 172.21.15.44:0/1174562905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb6c108e40 msgr2=0x7fdb6c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb71348640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb6c108e40 0x7fdb6c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb71348640 1 -- 172.21.15.44:0/1174562905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb6c1084c0 msgr2=0x7fdb6c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb71348640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb6c1084c0 0x7fdb6c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb71348640 1 -- 172.21.15.44:0/1174562905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6c1a73f0 con 0x7fdb6c102ab0 2024-08-20T02:54:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.992+0000 7fdb70b47640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb6c1084c0 0x7fdb6c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T02:54:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.993+0000 7fdb71348640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb6c102ab0 0x7fdb6c0751d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb68005940 tx=0x7fdb68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.993+0000 7fdb627fc640 1 -- 172.21.15.44:0/1174562905 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6803f030 con 0x7fdb6c102ab0 2024-08-20T02:54:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.993+0000 7fdb627fc640 1 -- 172.21.15.44:0/1174562905 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb68003aa0 con 0x7fdb6c102ab0 2024-08-20T02:54:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.993+0000 7fdb735ac640 1 -- 172.21.15.44:0/1174562905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6c1a7680 con 0x7fdb6c102ab0 2024-08-20T02:54:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.993+0000 7fdb627fc640 1 -- 172.21.15.44:0/1174562905 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68035070 con 0x7fdb6c102ab0 2024-08-20T02:54:18.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.993+0000 7fdb735ac640 1 -- 172.21.15.44:0/1174562905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb6c1a7be0 con 0x7fdb6c102ab0 2024-08-20T02:54:18.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.994+0000 7fdb735ac640 1 -- 172.21.15.44:0/1174562905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb6c102e90 con 0x7fdb6c102ab0 2024-08-20T02:54:18.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.996+0000 7fdb627fc640 1 -- 172.21.15.44:0/1174562905 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fdb68042030 con 0x7fdb6c102ab0 2024-08-20T02:54:18.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.997+0000 7fdb627fc640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdb580b3b90 0x7fdb580b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:18.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.997+0000 7fdb70b47640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdb580b3b90 0x7fdb580b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:19.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.998+0000 7fdb627fc640 1 -- 172.21.15.44:0/1174562905 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdb68102a10 con 0x7fdb6c102ab0 2024-08-20T02:54:19.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:18.998+0000 7fdb70b47640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdb580b3b90 0x7fdb580b6050 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c006fd0 tx=0x7fdb5c008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:19.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.000+0000 7fdb627fc640 1 -- 172.21.15.44:0/1174562905 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdb680cbc60 con 0x7fdb6c102ab0 2024-08-20T02:54:19.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.028+0000 7f346e44c640 1 -- 172.21.15.44:0/2332369778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f346810f050 con 0x7f34681084e0 2024-08-20T02:54:19.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.032+0000 7f34657fa640 1 -- 172.21.15.44:0/2332369778 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v25)=0 v25) ==== 120+0+0 (secure 0 0 0) 0x7f345c0d4070 con 0x7f34681084e0 2024-08-20T02:54:19.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.035+0000 7f346e44c640 1 -- 172.21.15.44:0/2332369778 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f34500b3b90 msgr2=0x7f34500b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:19.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.035+0000 7f346e44c640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f34500b3b90 0x7f34500b6050 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3458005fd0 tx=0x7f345800b590 comp rx=0 tx=0).stop 2024-08-20T02:54:19.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.035+0000 7f346e44c640 1 -- 172.21.15.44:0/2332369778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34681084e0 msgr2=0x7f346819b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:19.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.035+0000 7f346e44c640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34681084e0 0x7f346819b770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f345c005940 tx=0x7f345c0042f0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.035+0000 7f346e44c640 1 -- 172.21.15.44:0/2332369778 shutdown_connections 2024-08-20T02:54:19.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.035+0000 7f346e44c640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f34500b3b90 0x7f34500b6050 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.035+0000 7f346e44c640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3468108e60 0x7f34681a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.035+0000 7f346e44c640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34681084e0 0x7f346819b770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.035+0000 7f346e44c640 1 --2- 172.21.15.44:0/2332369778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3468102ad0 0x7f346819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.035+0000 7f346e44c640 1 -- 172.21.15.44:0/2332369778 >> 172.21.15.44:0/2332369778 conn(0x7f34680fe540 msgr2=0x7f3468104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:19.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.035+0000 7f346e44c640 1 -- 172.21.15.44:0/2332369778 shutdown_connections 2024-08-20T02:54:19.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.035+0000 7f346e44c640 1 -- 172.21.15.44:0/2332369778 wait complete. 2024-08-20T02:54:19.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.200+0000 7fdb735ac640 1 -- 172.21.15.44:0/1174562905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fdb6c0764d0 con 0x7fdb6c102ab0 2024-08-20T02:54:19.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.204+0000 7fdb627fc640 1 -- 172.21.15.44:0/1174562905 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v26)=0 v26) ==== 129+0+0 (secure 0 0 0) 0x7fdb680cb560 con 0x7fdb6c102ab0 2024-08-20T02:54:19.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.206+0000 7fdb735ac640 1 -- 172.21.15.44:0/1174562905 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdb580b3b90 msgr2=0x7fdb580b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:19.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.206+0000 7fdb735ac640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdb580b3b90 0x7fdb580b6050 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c006fd0 tx=0x7fdb5c008040 comp rx=0 tx=0).stop 2024-08-20T02:54:19.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.206+0000 7fdb735ac640 1 -- 172.21.15.44:0/1174562905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb6c102ab0 msgr2=0x7fdb6c0751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:19.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.206+0000 7fdb735ac640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb6c102ab0 0x7fdb6c0751d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb68005940 tx=0x7fdb68004270 comp rx=0 tx=0).stop 2024-08-20T02:54:19.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.206+0000 7fdb735ac640 1 -- 172.21.15.44:0/1174562905 shutdown_connections 2024-08-20T02:54:19.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.206+0000 7fdb735ac640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdb580b3b90 0x7fdb580b6050 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.206+0000 7fdb735ac640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdb6c108e40 0x7fdb6c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.206+0000 7fdb735ac640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb6c1084c0 0x7fdb6c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.206+0000 7fdb735ac640 1 --2- 172.21.15.44:0/1174562905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb6c102ab0 0x7fdb6c0751d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.206+0000 7fdb735ac640 1 -- 172.21.15.44:0/1174562905 >> 172.21.15.44:0/1174562905 conn(0x7fdb6c0fe500 msgr2=0x7fdb6c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:19.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.206+0000 7fdb735ac640 1 -- 172.21.15.44:0/1174562905 shutdown_connections 2024-08-20T02:54:19.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.206+0000 7fdb735ac640 1 -- 172.21.15.44:0/1174562905 wait complete. 2024-08-20T02:54:19.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.498+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/4264721237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1f410ce20 msgr2=0x7fa1f410f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.498+0000 7fa1fc6e5640 1 --2- 172.21.15.44:0/4264721237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1f410ce20 0x7fa1f410f270 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009ca0 tx=0x7fa1ec02a8c0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.498+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/4264721237 shutdown_connections 2024-08-20T02:54:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.498+0000 7fa1fc6e5640 1 --2- 172.21.15.44:0/4264721237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1f410ce20 0x7fa1f410f270 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.498+0000 7fa1fc6e5640 1 --2- 172.21.15.44:0/4264721237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1f4104270 0x7fa1f410c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.498+0000 7fa1fc6e5640 1 --2- 172.21.15.44:0/4264721237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f4103870 0x7fa1f4103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.498+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/4264721237 >> 172.21.15.44:0/4264721237 conn(0x7fa1f40fd400 msgr2=0x7fa1f40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.498+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/4264721237 shutdown_connections 2024-08-20T02:54:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.498+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/4264721237 wait complete. 2024-08-20T02:54:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.499+0000 7fa1fc6e5640 1 Processor -- start 2024-08-20T02:54:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.499+0000 7fa1fc6e5640 1 -- start start 2024-08-20T02:54:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.499+0000 7fa1fc6e5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1f4103870 0x7fa1f4198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:19.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.499+0000 7fa1fc6e5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1f4104270 0x7fa1f41994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.499+0000 7fa1fc6e5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f410ce20 0x7fa1f419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.499+0000 7fa1fc6e5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f4112a80 con 0x7fa1f4104270 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.499+0000 7fa1fc6e5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f4112900 con 0x7fa1f410ce20 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.499+0000 7fa1fc6e5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f4112c00 con 0x7fa1f4103870 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.499+0000 7fa1f9c80640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1f4104270 0x7fa1f41994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1f9c80640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1f4104270 0x7fa1f41994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59842/0 (socket says 172.21.15.44:59842) 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1f9c80640 1 -- 172.21.15.44:0/190201970 learned_addr learned my addr 172.21.15.44:0/190201970 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1fac82640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f410ce20 0x7fa1f419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1fa481640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1f4103870 0x7fa1f4198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1f9c80640 1 -- 172.21.15.44:0/190201970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1f4103870 msgr2=0x7fa1f4198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1f9c80640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1f4103870 0x7fa1f4198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1f9c80640 1 -- 172.21.15.44:0/190201970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f410ce20 msgr2=0x7fa1f419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:19.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1f9c80640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f410ce20 0x7fa1f419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1f9c80640 1 -- 172.21.15.44:0/190201970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f419f020 con 0x7fa1f4104270 2024-08-20T02:54:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1fac82640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f410ce20 0x7fa1f419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:54:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1f9c80640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1f4104270 0x7fa1f41994b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400d770 tx=0x7fa1e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1eb7fe640 1 -- 172.21.15.44:0/190201970 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4017070 con 0x7fa1f4104270 2024-08-20T02:54:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1eb7fe640 1 -- 172.21.15.44:0/190201970 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1e40054d0 con 0x7fa1f4104270 2024-08-20T02:54:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/190201970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f419f310 con 0x7fa1f4104270 2024-08-20T02:54:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1eb7fe640 1 -- 172.21.15.44:0/190201970 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4004c30 con 0x7fa1f4104270 2024-08-20T02:54:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.500+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/190201970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f41a59d0 con 0x7fa1f4104270 2024-08-20T02:54:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.501+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/190201970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f4103c50 con 0x7fa1f4104270 2024-08-20T02:54:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.503+0000 7fa1eb7fe640 1 -- 172.21.15.44:0/190201970 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fa1e4004dd0 con 0x7fa1f4104270 2024-08-20T02:54:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.504+0000 7fa1eb7fe640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa1e00b3c10 0x7fa1e00b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:19.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.504+0000 7fa1fa481640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa1e00b3c10 0x7fa1e00b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.504+0000 7fa1eb7fe640 1 -- 172.21.15.44:0/190201970 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa1e40e1f40 con 0x7fa1f4104270 2024-08-20T02:54:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.504+0000 7fa1fa481640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa1e00b3c10 0x7fa1e00b60d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009b60 tx=0x7fa1f0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.506+0000 7fa1eb7fe640 1 -- 172.21.15.44:0/190201970 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa1e40ab210 con 0x7fa1f4104270 2024-08-20T02:54:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.699+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/190201970 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fa1f41a5d40 con 0x7fa1f4104270 2024-08-20T02:54:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.703+0000 7fa1eb7fe640 1 -- 172.21.15.44:0/190201970 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v27)=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7fa1e40aab10 con 0x7fa1f4104270 2024-08-20T02:54:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.705+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/190201970 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa1e00b3c10 msgr2=0x7fa1e00b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.705+0000 7fa1fc6e5640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa1e00b3c10 0x7fa1e00b60d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009b60 tx=0x7fa1f0005f90 comp rx=0 tx=0).stop 2024-08-20T02:54:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.705+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/190201970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1f4104270 msgr2=0x7fa1f41994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.705+0000 7fa1fc6e5640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1f4104270 0x7fa1f41994b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400d770 tx=0x7fa1e400dc30 comp rx=0 tx=0).stop 2024-08-20T02:54:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.705+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/190201970 shutdown_connections 2024-08-20T02:54:19.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.705+0000 7fa1fc6e5640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa1e00b3c10 0x7fa1e00b60d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.705+0000 7fa1fc6e5640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f410ce20 0x7fa1f419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.705+0000 7fa1fc6e5640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa1f4104270 0x7fa1f41994b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.705+0000 7fa1fc6e5640 1 --2- 172.21.15.44:0/190201970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1f4103870 0x7fa1f4198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:19.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.705+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/190201970 >> 172.21.15.44:0/190201970 conn(0x7fa1f40fd400 msgr2=0x7fa1f40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:19.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.705+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/190201970 shutdown_connections 2024-08-20T02:54:19.855 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.705+0000 7fa1fc6e5640 1 -- 172.21.15.44:0/190201970 wait complete. 2024-08-20T02:54:20.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: Reconfiguring mon.c (monmap changed)... 2024-08-20T02:54:20.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: Reconfiguring daemon mon.c on smithi165 2024-08-20T02:54:20.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:20.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2332369778' entity='client.admin' 2024-08-20T02:54:20.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1174562905' entity='client.admin' 2024-08-20T02:54:20.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:20.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:20.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:20.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:20.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:20.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:20.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/190201970' entity='client.admin' 2024-08-20T02:54:20.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:20 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3966418947' entity='client.admin' 2024-08-20T02:54:20.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: Reconfiguring mon.c (monmap changed)... 2024-08-20T02:54:20.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: Reconfiguring daemon mon.c on smithi165 2024-08-20T02:54:20.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:20.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2332369778' entity='client.admin' 2024-08-20T02:54:20.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1174562905' entity='client.admin' 2024-08-20T02:54:20.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:20.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:20.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:20.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:20.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:20.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:20.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/190201970' entity='client.admin' 2024-08-20T02:54:20.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:20 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3966418947' entity='client.admin' 2024-08-20T02:54:20.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: Reconfiguring mon.c (monmap changed)... 2024-08-20T02:54:20.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: Reconfiguring daemon mon.c on smithi165 2024-08-20T02:54:20.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:20.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2332369778' entity='client.admin' 2024-08-20T02:54:20.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1174562905' entity='client.admin' 2024-08-20T02:54:20.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:20.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:20.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:20.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:20.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:20.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:20.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/190201970' entity='client.admin' 2024-08-20T02:54:20.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:20 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3966418947' entity='client.admin' 2024-08-20T02:54:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.622+0000 7fe93718c640 1 -- 172.21.15.44:0/341516650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe930101990 msgr2=0x7fe930101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.622+0000 7fe93718c640 1 --2- 172.21.15.44:0/341516650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe930101990 0x7fe930101d70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe924009900 tx=0x7fe92402a410 comp rx=0 tx=0).stop 2024-08-20T02:54:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.622+0000 7fe93718c640 1 -- 172.21.15.44:0/341516650 shutdown_connections 2024-08-20T02:54:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.622+0000 7fe93718c640 1 --2- 172.21.15.44:0/341516650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe930107d20 0x7fe93010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:20.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.622+0000 7fe93718c640 1 --2- 172.21.15.44:0/341516650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9301073a0 0x7fe9301077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.622+0000 7fe93718c640 1 --2- 172.21.15.44:0/341516650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe930101990 0x7fe930101d70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.622+0000 7fe93718c640 1 -- 172.21.15.44:0/341516650 >> 172.21.15.44:0/341516650 conn(0x7fe9300fd400 msgr2=0x7fe9300ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.622+0000 7fe93718c640 1 -- 172.21.15.44:0/341516650 shutdown_connections 2024-08-20T02:54:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.622+0000 7fe93718c640 1 -- 172.21.15.44:0/341516650 wait complete. 2024-08-20T02:54:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe93718c640 1 Processor -- start 2024-08-20T02:54:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe93718c640 1 -- start start 2024-08-20T02:54:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe93718c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe930101990 0x7fe9300fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe93718c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9301073a0 0x7fe9300fde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe93718c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe930107d20 0x7fe9300fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe93718c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe930111a20 con 0x7fe930101990 2024-08-20T02:54:20.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe93718c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9301118a0 con 0x7fe9301073a0 2024-08-20T02:54:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe93718c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe930111ba0 con 0x7fe930107d20 2024-08-20T02:54:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe934f28640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe930101990 0x7fe9300fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe934f28640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe930101990 0x7fe9300fd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59854/0 (socket says 172.21.15.44:59854) 2024-08-20T02:54:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe934f28640 1 -- 172.21.15.44:0/3966418947 learned_addr learned my addr 172.21.15.44:0/3966418947 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe92ffff640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9301073a0 0x7fe9300fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe935729640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe930107d20 0x7fe9300fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe934f28640 1 -- 172.21.15.44:0/3966418947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe930107d20 msgr2=0x7fe9300fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe934f28640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe930107d20 0x7fe9300fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe934f28640 1 -- 172.21.15.44:0/3966418947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9301073a0 msgr2=0x7fe9300fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe934f28640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9301073a0 0x7fe9300fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:20.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe934f28640 1 -- 172.21.15.44:0/3966418947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9301a7580 con 0x7fe930101990 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.623+0000 7fe934f28640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe930101990 0x7fe9300fd900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe924006010 tx=0x7fe9240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.624+0000 7fe92dffb640 1 -- 172.21.15.44:0/3966418947 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92403f030 con 0x7fe930101990 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.624+0000 7fe93718c640 1 -- 172.21.15.44:0/3966418947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9301a7810 con 0x7fe930101990 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.624+0000 7fe92dffb640 1 -- 172.21.15.44:0/3966418947 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe924003800 con 0x7fe930101990 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.624+0000 7fe92dffb640 1 -- 172.21.15.44:0/3966418947 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924035070 con 0x7fe930101990 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.624+0000 7fe93718c640 1 -- 172.21.15.44:0/3966418947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9301a7d40 con 0x7fe930101990 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.625+0000 7fe93718c640 1 -- 172.21.15.44:0/3966418947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe930101df0 con 0x7fe930101990 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.627+0000 7fe92dffb640 1 -- 172.21.15.44:0/3966418947 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fe924003cb0 con 0x7fe930101990 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.628+0000 7fe92dffb640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe9200b3c10 0x7fe9200b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.628+0000 7fe92ffff640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe9200b3c10 0x7fe9200b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.628+0000 7fe92dffb640 1 -- 172.21.15.44:0/3966418947 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe924102d00 con 0x7fe930101990 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.629+0000 7fe92ffff640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe9200b3c10 0x7fe9200b60d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe9300ff3f0 tx=0x7fe91c008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.631+0000 7fe92dffb640 1 -- 172.21.15.44:0/3966418947 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe9240cbeb0 con 0x7fe930101990 2024-08-20T02:54:20.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.824+0000 7fe93718c640 1 -- 172.21.15.44:0/3966418947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fe93010ab10 con 0x7fe930101990 2024-08-20T02:54:20.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.827+0000 7fe92dffb640 1 -- 172.21.15.44:0/3966418947 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v28)=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7fe9240d4180 con 0x7fe930101990 2024-08-20T02:54:20.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.830+0000 7fe93718c640 1 -- 172.21.15.44:0/3966418947 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe9200b3c10 msgr2=0x7fe9200b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:20.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.830+0000 7fe93718c640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe9200b3c10 0x7fe9200b60d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe9300ff3f0 tx=0x7fe91c008040 comp rx=0 tx=0).stop 2024-08-20T02:54:20.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.830+0000 7fe93718c640 1 -- 172.21.15.44:0/3966418947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe930101990 msgr2=0x7fe9300fd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:20.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.830+0000 7fe93718c640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe930101990 0x7fe9300fd900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe924006010 tx=0x7fe9240042f0 comp rx=0 tx=0).stop 2024-08-20T02:54:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.830+0000 7fe93718c640 1 -- 172.21.15.44:0/3966418947 shutdown_connections 2024-08-20T02:54:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.830+0000 7fe93718c640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe9200b3c10 0x7fe9200b60d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.830+0000 7fe93718c640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe930107d20 0x7fe9300fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.830+0000 7fe93718c640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9301073a0 0x7fe9300fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.830+0000 7fe93718c640 1 --2- 172.21.15.44:0/3966418947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe930101990 0x7fe9300fd900 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.830+0000 7fe93718c640 1 -- 172.21.15.44:0/3966418947 >> 172.21.15.44:0/3966418947 conn(0x7fe9300fd400 msgr2=0x7fe93006ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.830+0000 7fe93718c640 1 -- 172.21.15.44:0/3966418947 shutdown_connections 2024-08-20T02:54:20.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:19.830+0000 7fe93718c640 1 -- 172.21.15.44:0/3966418947 wait complete. 2024-08-20T02:54:21.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.117+0000 7f2e146be640 1 -- 172.21.15.44:0/791651213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c1039c0 msgr2=0x7f2e0c103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:21.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.117+0000 7f2e146be640 1 --2- 172.21.15.44:0/791651213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c1039c0 0x7f2e0c103da0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009900 tx=0x7f2e0802a410 comp rx=0 tx=0).stop 2024-08-20T02:54:21.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.117+0000 7f2e146be640 1 -- 172.21.15.44:0/791651213 shutdown_connections 2024-08-20T02:54:21.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.117+0000 7f2e146be640 1 --2- 172.21.15.44:0/791651213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c10ce20 0x7f2e0c10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.117+0000 7f2e146be640 1 --2- 172.21.15.44:0/791651213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0c1042e0 0x7f2e0c10c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.117+0000 7f2e146be640 1 --2- 172.21.15.44:0/791651213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c1039c0 0x7f2e0c103da0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.117+0000 7f2e146be640 1 -- 172.21.15.44:0/791651213 >> 172.21.15.44:0/791651213 conn(0x7f2e0c0fd400 msgr2=0x7f2e0c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:21.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.117+0000 7f2e146be640 1 -- 172.21.15.44:0/791651213 shutdown_connections 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.117+0000 7f2e146be640 1 -- 172.21.15.44:0/791651213 wait complete. 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.118+0000 7f2e146be640 1 Processor -- start 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.118+0000 7f2e146be640 1 -- start start 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.118+0000 7f2e146be640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0c1039c0 0x7f2e0c198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.118+0000 7f2e146be640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c1042e0 0x7f2e0c199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.118+0000 7f2e146be640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c10ce20 0x7f2e0c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.118+0000 7f2e146be640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c112a90 con 0x7f2e0c10ce20 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.118+0000 7f2e146be640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c112910 con 0x7f2e0c1042e0 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.118+0000 7f2e146be640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c112c10 con 0x7f2e0c1039c0 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e12c5b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c10ce20 0x7f2e0c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e12c5b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c10ce20 0x7f2e0c19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59886/0 (socket says 172.21.15.44:59886) 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e12c5b640 1 -- 172.21.15.44:0/2334158543 learned_addr learned my addr 172.21.15.44:0/2334158543 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:21.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e11c59640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c1042e0 0x7f2e0c199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e1245a640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0c1039c0 0x7f2e0c198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e12c5b640 1 -- 172.21.15.44:0/2334158543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0c1039c0 msgr2=0x7f2e0c198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e12c5b640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0c1039c0 0x7f2e0c198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e12c5b640 1 -- 172.21.15.44:0/2334158543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c1042e0 msgr2=0x7f2e0c199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e12c5b640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c1042e0 0x7f2e0c199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e12c5b640 1 -- 172.21.15.44:0/2334158543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c19f090 con 0x7f2e0c10ce20 2024-08-20T02:54:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e11c59640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c1042e0 0x7f2e0c199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:54:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e12c5b640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c10ce20 0x7f2e0c19e8c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009fd0 tx=0x7f2e0400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e037fe640 1 -- 172.21.15.44:0/2334158543 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04017070 con 0x7f2e0c10ce20 2024-08-20T02:54:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e037fe640 1 -- 172.21.15.44:0/2334158543 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e04003e20 con 0x7f2e0c10ce20 2024-08-20T02:54:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e146be640 1 -- 172.21.15.44:0/2334158543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0c19f380 con 0x7f2e0c10ce20 2024-08-20T02:54:21.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e037fe640 1 -- 172.21.15.44:0/2334158543 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04010460 con 0x7f2e0c10ce20 2024-08-20T02:54:21.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.119+0000 7f2e146be640 1 -- 172.21.15.44:0/2334158543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0c1a5a40 con 0x7f2e0c10ce20 2024-08-20T02:54:21.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.120+0000 7f2e146be640 1 -- 172.21.15.44:0/2334158543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e0c103da0 con 0x7f2e0c10ce20 2024-08-20T02:54:21.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.122+0000 7f2e037fe640 1 -- 172.21.15.44:0/2334158543 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f2e04003970 con 0x7f2e0c10ce20 2024-08-20T02:54:21.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.122+0000 7f2e037fe640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2df80b3c10 0x7f2df80b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:21.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.123+0000 7f2e1245a640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2df80b3c10 0x7f2df80b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:21.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.123+0000 7f2e037fe640 1 -- 172.21.15.44:0/2334158543 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f2e040e1e30 con 0x7f2e0c10ce20 2024-08-20T02:54:21.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.123+0000 7f2e1245a640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2df80b3c10 0x7f2df80b60d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2e08031440 tx=0x7f2e080095f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:21.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.127+0000 7f2e037fe640 1 -- 172.21.15.44:0/2334158543 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2e040ab080 con 0x7f2e0c10ce20 2024-08-20T02:54:21.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.290+0000 7f9260e53640 1 -- 172.21.15.44:0/3866877016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f925c108e20 msgr2=0x7f925c10f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:21.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.290+0000 7f9260e53640 1 --2- 172.21.15.44:0/3866877016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f925c108e20 0x7f925c10f620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9254009ca0 tx=0x7f925402a8c0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.290+0000 7f9260e53640 1 -- 172.21.15.44:0/3866877016 shutdown_connections 2024-08-20T02:54:21.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.290+0000 7f9260e53640 1 --2- 172.21.15.44:0/3866877016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f925c108e20 0x7f925c10f620 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.290+0000 7f9260e53640 1 --2- 172.21.15.44:0/3866877016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f925c1084a0 0x7f925c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.290+0000 7f9260e53640 1 --2- 172.21.15.44:0/3866877016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c102a90 0x7f925c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.290+0000 7f9260e53640 1 -- 172.21.15.44:0/3866877016 >> 172.21.15.44:0/3866877016 conn(0x7f925c0fe540 msgr2=0x7f925c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:21.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.290+0000 7f9260e53640 1 -- 172.21.15.44:0/3866877016 shutdown_connections 2024-08-20T02:54:21.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.290+0000 7f9260e53640 1 -- 172.21.15.44:0/3866877016 wait complete. 2024-08-20T02:54:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f9260e53640 1 Processor -- start 2024-08-20T02:54:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f9260e53640 1 -- start start 2024-08-20T02:54:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f9260e53640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f925c102a90 0x7f925c0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f9260e53640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c1084a0 0x7f925c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f9260e53640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f925c108e20 0x7f925c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f9260e53640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f925c112b30 con 0x7f925c108e20 2024-08-20T02:54:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f9260e53640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f925c1129b0 con 0x7f925c102a90 2024-08-20T02:54:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f925ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f925c108e20 0x7f925c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f925ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f925c108e20 0x7f925c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59902/0 (socket says 172.21.15.44:59902) 2024-08-20T02:54:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f925ad9d640 1 -- 172.21.15.44:0/2639619960 learned_addr learned my addr 172.21.15.44:0/2639619960 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f9260e53640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f925c112cb0 con 0x7f925c1084a0 2024-08-20T02:54:21.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f925a59c640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f925c102a90 0x7f925c0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.291+0000 7f9259d9b640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c1084a0 0x7f925c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.292+0000 7f925ad9d640 1 -- 172.21.15.44:0/2639619960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c1084a0 msgr2=0x7f925c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.292+0000 7f925ad9d640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c1084a0 0x7f925c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.292+0000 7f925ad9d640 1 -- 172.21.15.44:0/2639619960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f925c102a90 msgr2=0x7f925c0751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.292+0000 7f925ad9d640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f925c102a90 0x7f925c0751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.292+0000 7f925ad9d640 1 -- 172.21.15.44:0/2639619960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c1a7380 con 0x7f925c108e20 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.292+0000 7f925a59c640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f925c102a90 0x7f925c0751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.292+0000 7f925ad9d640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f925c108e20 0x7f925c075c50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9254031a20 tx=0x7f9254031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.292+0000 7f924b7fe640 1 -- 172.21.15.44:0/2639619960 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254003d30 con 0x7f925c108e20 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.292+0000 7f9260e53640 1 -- 172.21.15.44:0/2639619960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925c1a7610 con 0x7f925c108e20 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.292+0000 7f9260e53640 1 -- 172.21.15.44:0/2639619960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925c1a7c20 con 0x7f925c108e20 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.292+0000 7f924b7fe640 1 -- 172.21.15.44:0/2639619960 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92540044f0 con 0x7f925c108e20 2024-08-20T02:54:21.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.293+0000 7f924b7fe640 1 -- 172.21.15.44:0/2639619960 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925402fb20 con 0x7f925c108e20 2024-08-20T02:54:21.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.294+0000 7f9260e53640 1 -- 172.21.15.44:0/2639619960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923c0052c0 con 0x7f925c108e20 2024-08-20T02:54:21.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.295+0000 7f924b7fe640 1 -- 172.21.15.44:0/2639619960 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f92540046a0 con 0x7f925c108e20 2024-08-20T02:54:21.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.296+0000 7f924b7fe640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92440b3c10 0x7f92440b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:21.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.296+0000 7f925a59c640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92440b3c10 0x7f92440b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:21.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.297+0000 7f924b7fe640 1 -- 172.21.15.44:0/2639619960 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f92541036f0 con 0x7f925c108e20 2024-08-20T02:54:21.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.297+0000 7f925a59c640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92440b3c10 0x7f92440b60d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f924c009930 tx=0x7f924c008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:21.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.301+0000 7f924b7fe640 1 -- 172.21.15.44:0/2639619960 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f92540cc9d0 con 0x7f925c108e20 2024-08-20T02:54:21.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.319+0000 7f2e146be640 1 -- 172.21.15.44:0/2334158543 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7f2e0c107900 con 0x7f2e0c10ce20 2024-08-20T02:54:21.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.323+0000 7f2e037fe640 1 -- 172.21.15.44:0/2334158543 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v29)=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7f2e040aa980 con 0x7f2e0c10ce20 2024-08-20T02:54:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.326+0000 7f2e146be640 1 -- 172.21.15.44:0/2334158543 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2df80b3c10 msgr2=0x7f2df80b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.326+0000 7f2e146be640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2df80b3c10 0x7f2df80b60d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2e08031440 tx=0x7f2e080095f0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.326+0000 7f2e146be640 1 -- 172.21.15.44:0/2334158543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c10ce20 msgr2=0x7f2e0c19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.326+0000 7f2e146be640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c10ce20 0x7f2e0c19e8c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009fd0 tx=0x7f2e0400b260 comp rx=0 tx=0).stop 2024-08-20T02:54:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.326+0000 7f2e146be640 1 -- 172.21.15.44:0/2334158543 shutdown_connections 2024-08-20T02:54:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.326+0000 7f2e146be640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2df80b3c10 0x7f2df80b60d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.326+0000 7f2e146be640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2e0c10ce20 0x7f2e0c19e8c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.326+0000 7f2e146be640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e0c1042e0 0x7f2e0c199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.326+0000 7f2e146be640 1 --2- 172.21.15.44:0/2334158543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0c1039c0 0x7f2e0c198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.326+0000 7f2e146be640 1 -- 172.21.15.44:0/2334158543 >> 172.21.15.44:0/2334158543 conn(0x7f2e0c0fd400 msgr2=0x7f2e0c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.326+0000 7f2e146be640 1 -- 172.21.15.44:0/2334158543 shutdown_connections 2024-08-20T02:54:21.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.326+0000 7f2e146be640 1 -- 172.21.15.44:0/2334158543 wait complete. 2024-08-20T02:54:21.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.501+0000 7f9260e53640 1 -- 172.21.15.44:0/2639619960 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f923c0055b0 con 0x7f925c108e20 2024-08-20T02:54:21.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.505+0000 7f924b7fe640 1 -- 172.21.15.44:0/2639619960 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v30)=0 v30) ==== 119+0+0 (secure 0 0 0) 0x7f92540cc2d0 con 0x7f925c108e20 2024-08-20T02:54:21.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.507+0000 7f9260e53640 1 -- 172.21.15.44:0/2639619960 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92440b3c10 msgr2=0x7f92440b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.507+0000 7f9260e53640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92440b3c10 0x7f92440b60d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f924c009930 tx=0x7f924c008040 comp rx=0 tx=0).stop 2024-08-20T02:54:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.507+0000 7f9260e53640 1 -- 172.21.15.44:0/2639619960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f925c108e20 msgr2=0x7f925c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.507+0000 7f9260e53640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f925c108e20 0x7f925c075c50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9254031a20 tx=0x7f9254031a50 comp rx=0 tx=0).stop 2024-08-20T02:54:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.507+0000 7f9260e53640 1 -- 172.21.15.44:0/2639619960 shutdown_connections 2024-08-20T02:54:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.507+0000 7f9260e53640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92440b3c10 0x7f92440b60d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.507+0000 7f9260e53640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f925c108e20 0x7f925c075c50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.507+0000 7f9260e53640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c1084a0 0x7f925c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.507+0000 7f9260e53640 1 --2- 172.21.15.44:0/2639619960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f925c102a90 0x7f925c0751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.507+0000 7f9260e53640 1 -- 172.21.15.44:0/2639619960 >> 172.21.15.44:0/2639619960 conn(0x7f925c0fe540 msgr2=0x7f925c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.507+0000 7f9260e53640 1 -- 172.21.15.44:0/2639619960 shutdown_connections 2024-08-20T02:54:21.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:21.507+0000 7f9260e53640 1 -- 172.21.15.44:0/2639619960 wait complete. 2024-08-20T02:54:22.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:22 smithi099 ceph-mon[21984]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:22.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:22 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2334158543' entity='client.admin' 2024-08-20T02:54:22.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:22 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:22.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:22 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:22.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:22 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:22.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:22 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:22.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:22 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2639619960' entity='client.admin' 2024-08-20T02:54:22.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:22 smithi044 ceph-mon[18790]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:22.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:22 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2334158543' entity='client.admin' 2024-08-20T02:54:22.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:22 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:22.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:22 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:22.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:22 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:22.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:22 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:22.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:22 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2639619960' entity='client.admin' 2024-08-20T02:54:22.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:22 smithi165 ceph-mon[22357]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:22.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:22 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2334158543' entity='client.admin' 2024-08-20T02:54:22.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:22 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:22.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:22 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:22.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:22 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:22.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:22 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:22.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:22 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2639619960' entity='client.admin' 2024-08-20T02:54:23.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.224+0000 7efe41b3b640 1 -- 172.21.15.44:0/2635989563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe3c101f40 msgr2=0x7efe3c10c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:23.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.224+0000 7efe41b3b640 1 --2- 172.21.15.44:0/2635989563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe3c101f40 0x7efe3c10c6b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efe24009900 tx=0x7efe2402a410 comp rx=0 tx=0).stop 2024-08-20T02:54:23.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.224+0000 7efe41b3b640 1 -- 172.21.15.44:0/2635989563 shutdown_connections 2024-08-20T02:54:23.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.225+0000 7efe41b3b640 1 --2- 172.21.15.44:0/2635989563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe3c10cdf0 0x7efe3c10f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:23.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.225+0000 7efe41b3b640 1 --2- 172.21.15.44:0/2635989563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe3c101f40 0x7efe3c10c6b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:23.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.225+0000 7efe41b3b640 1 --2- 172.21.15.44:0/2635989563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe3c101590 0x7efe3c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:23.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.225+0000 7efe41b3b640 1 -- 172.21.15.44:0/2635989563 >> 172.21.15.44:0/2635989563 conn(0x7efe3c076bd0 msgr2=0x7efe3c076fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:23.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.225+0000 7efe41b3b640 1 -- 172.21.15.44:0/2635989563 shutdown_connections 2024-08-20T02:54:23.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.225+0000 7efe41b3b640 1 -- 172.21.15.44:0/2635989563 wait complete. 2024-08-20T02:54:23.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.225+0000 7efe41b3b640 1 Processor -- start 2024-08-20T02:54:23.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.225+0000 7efe41b3b640 1 -- start start 2024-08-20T02:54:23.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe41b3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe3c101590 0x7efe3c102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:23.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe41b3b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe3c101f40 0x7efe3c102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:23.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe41b3b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe3c10cdf0 0x7efe3c102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:23.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe41b3b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c112ae0 con 0x7efe3c10cdf0 2024-08-20T02:54:23.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe41b3b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c112960 con 0x7efe3c101590 2024-08-20T02:54:23.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe41b3b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c112c60 con 0x7efe3c101f40 2024-08-20T02:54:23.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe3c10cdf0 0x7efe3c102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:23.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe3bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe3c10cdf0 0x7efe3c102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59932/0 (socket says 172.21.15.44:59932) 2024-08-20T02:54:23.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe3bfff640 1 -- 172.21.15.44:0/3692125701 learned_addr learned my addr 172.21.15.44:0/3692125701 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:23.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe3b7fe640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe3c101590 0x7efe3c102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:23.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe3affd640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe3c101f40 0x7efe3c102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:23.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe3bfff640 1 -- 172.21.15.44:0/3692125701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe3c101f40 msgr2=0x7efe3c102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:23.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe3bfff640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe3c101f40 0x7efe3c102aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:23.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe3bfff640 1 -- 172.21.15.44:0/3692125701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe3c101590 msgr2=0x7efe3c102560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:23.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe3bfff640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe3c101590 0x7efe3c102560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:23.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe3bfff640 1 -- 172.21.15.44:0/3692125701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c1a74b0 con 0x7efe3c10cdf0 2024-08-20T02:54:23.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe3bfff640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe3c10cdf0 0x7efe3c102fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe3400c970 tx=0x7efe3400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:23.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe38ff9640 1 -- 172.21.15.44:0/3692125701 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34007bb0 con 0x7efe3c10cdf0 2024-08-20T02:54:23.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe38ff9640 1 -- 172.21.15.44:0/3692125701 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe34007d50 con 0x7efe3c10cdf0 2024-08-20T02:54:23.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe41b3b640 1 -- 172.21.15.44:0/3692125701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3c1a77a0 con 0x7efe3c10cdf0 2024-08-20T02:54:23.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.226+0000 7efe38ff9640 1 -- 172.21.15.44:0/3692125701 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34004e60 con 0x7efe3c10cdf0 2024-08-20T02:54:23.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.227+0000 7efe41b3b640 1 -- 172.21.15.44:0/3692125701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3c1a7cb0 con 0x7efe3c10cdf0 2024-08-20T02:54:23.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.227+0000 7efe41b3b640 1 -- 172.21.15.44:0/3692125701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3c101970 con 0x7efe3c10cdf0 2024-08-20T02:54:23.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.229+0000 7efe38ff9640 1 -- 172.21.15.44:0/3692125701 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7efe34005000 con 0x7efe3c10cdf0 2024-08-20T02:54:23.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.229+0000 7efe38ff9640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efe280b3c10 0x7efe280b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:23.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.230+0000 7efe3b7fe640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efe280b3c10 0x7efe280b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:23.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.230+0000 7efe38ff9640 1 -- 172.21.15.44:0/3692125701 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7efe340e20c0 con 0x7efe3c10cdf0 2024-08-20T02:54:23.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.230+0000 7efe3b7fe640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efe280b3c10 0x7efe280b60d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00a400 tx=0x7efe2c00d040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:23.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.232+0000 7efe38ff9640 1 -- 172.21.15.44:0/3692125701 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7efe34014030 con 0x7efe3c10cdf0 2024-08-20T02:54:23.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.418+0000 7efe41b3b640 1 -- 172.21.15.44:0/3692125701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7efe3c103860 con 0x7efe3c10cdf0 2024-08-20T02:54:23.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.421+0000 7efe38ff9640 1 -- 172.21.15.44:0/3692125701 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7efe3c103860 con 0x7efe3c10cdf0 2024-08-20T02:54:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.424+0000 7efe41b3b640 1 -- 172.21.15.44:0/3692125701 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efe280b3c10 msgr2=0x7efe280b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.424+0000 7efe41b3b640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efe280b3c10 0x7efe280b60d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00a400 tx=0x7efe2c00d040 comp rx=0 tx=0).stop 2024-08-20T02:54:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.424+0000 7efe41b3b640 1 -- 172.21.15.44:0/3692125701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe3c10cdf0 msgr2=0x7efe3c102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.424+0000 7efe41b3b640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe3c10cdf0 0x7efe3c102fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe3400c970 tx=0x7efe3400ce30 comp rx=0 tx=0).stop 2024-08-20T02:54:23.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.424+0000 7efe41b3b640 1 -- 172.21.15.44:0/3692125701 shutdown_connections 2024-08-20T02:54:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.424+0000 7efe41b3b640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efe280b3c10 0x7efe280b60d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.424+0000 7efe41b3b640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe3c10cdf0 0x7efe3c102fe0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.424+0000 7efe41b3b640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe3c101f40 0x7efe3c102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.424+0000 7efe41b3b640 1 --2- 172.21.15.44:0/3692125701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe3c101590 0x7efe3c102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.424+0000 7efe41b3b640 1 -- 172.21.15.44:0/3692125701 >> 172.21.15.44:0/3692125701 conn(0x7efe3c076bd0 msgr2=0x7efe3c0ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.424+0000 7efe41b3b640 1 -- 172.21.15.44:0/3692125701 shutdown_connections 2024-08-20T02:54:23.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:23.424+0000 7efe41b3b640 1 -- 172.21.15.44:0/3692125701 wait complete. 2024-08-20T02:54:23.563 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi044') 2024-08-20T02:54:23.563 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi044') 2024-08-20T02:54:23.563 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi044') 2024-08-20T02:54:24.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.234+0000 7f286b12b640 1 -- 172.21.15.44:0/822894220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28641042c0 msgr2=0x7f286410c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:24.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.234+0000 7f286b12b640 1 --2- 172.21.15.44:0/822894220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28641042c0 0x7f286410c7b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2858009900 tx=0x7f285802a410 comp rx=0 tx=0).stop 2024-08-20T02:54:24.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.234+0000 7f286b12b640 1 -- 172.21.15.44:0/822894220 shutdown_connections 2024-08-20T02:54:24.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.234+0000 7f286b12b640 1 --2- 172.21.15.44:0/822894220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f286410ce00 0x7f286410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:24.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.234+0000 7f286b12b640 1 --2- 172.21.15.44:0/822894220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28641042c0 0x7f286410c7b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:24.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.235+0000 7f286b12b640 1 --2- 172.21.15.44:0/822894220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28641039a0 0x7f2864103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:24.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.235+0000 7f286b12b640 1 -- 172.21.15.44:0/822894220 >> 172.21.15.44:0/822894220 conn(0x7f28640fd400 msgr2=0x7f28640ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:24.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.235+0000 7f286b12b640 1 -- 172.21.15.44:0/822894220 shutdown_connections 2024-08-20T02:54:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.235+0000 7f286b12b640 1 -- 172.21.15.44:0/822894220 wait complete. 2024-08-20T02:54:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.235+0000 7f286b12b640 1 Processor -- start 2024-08-20T02:54:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.235+0000 7f286b12b640 1 -- start start 2024-08-20T02:54:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f286b12b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28641039a0 0x7f2864198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f286b12b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28641042c0 0x7f28641994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f286b12b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410ce00 0x7f286419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f286b12b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2864112a70 con 0x7f286410ce00 2024-08-20T02:54:24.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f286b12b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f28641128f0 con 0x7f28641039a0 2024-08-20T02:54:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f286b12b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2864112bf0 con 0x7f28641042c0 2024-08-20T02:54:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f28696c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410ce00 0x7f286419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f28696c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410ce00 0x7f286419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59944/0 (socket says 172.21.15.44:59944) 2024-08-20T02:54:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f28696c8640 1 -- 172.21.15.44:0/3606650120 learned_addr learned my addr 172.21.15.44:0/3606650120 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f2863fff640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28641042c0 0x7f28641994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f2868ec7640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28641039a0 0x7f2864198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f28696c8640 1 -- 172.21.15.44:0/3606650120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28641042c0 msgr2=0x7f28641994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f28696c8640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28641042c0 0x7f28641994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f28696c8640 1 -- 172.21.15.44:0/3606650120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28641039a0 msgr2=0x7f2864198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f28696c8640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28641039a0 0x7f2864198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f28696c8640 1 -- 172.21.15.44:0/3606650120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286419f030 con 0x7f286410ce00 2024-08-20T02:54:24.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f2863fff640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28641042c0 0x7f28641994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:54:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f2868ec7640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28641039a0 0x7f2864198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:54:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.236+0000 7f28696c8640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410ce00 0x7f286419e860 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f285c00cad0 tx=0x7f285c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.237+0000 7f2861ffb640 1 -- 172.21.15.44:0/3606650120 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c00ae20 con 0x7f286410ce00 2024-08-20T02:54:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.237+0000 7f2861ffb640 1 -- 172.21.15.44:0/3606650120 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f285c005020 con 0x7f286410ce00 2024-08-20T02:54:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.237+0000 7f286b12b640 1 -- 172.21.15.44:0/3606650120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286419f320 con 0x7f286410ce00 2024-08-20T02:54:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.237+0000 7f2861ffb640 1 -- 172.21.15.44:0/3606650120 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c00f6b0 con 0x7f286410ce00 2024-08-20T02:54:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.237+0000 7f286b12b640 1 -- 172.21.15.44:0/3606650120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28641a59e0 con 0x7f286410ce00 2024-08-20T02:54:24.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.238+0000 7f286b12b640 1 -- 172.21.15.44:0/3606650120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2864103d80 con 0x7f286410ce00 2024-08-20T02:54:24.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.239+0000 7f2861ffb640 1 -- 172.21.15.44:0/3606650120 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f285c0051d0 con 0x7f286410ce00 2024-08-20T02:54:24.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.240+0000 7f2861ffb640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f284c0b3b90 0x7f284c0b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:24.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.240+0000 7f2868ec7640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f284c0b3b90 0x7f284c0b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:24.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.240+0000 7f2861ffb640 1 -- 172.21.15.44:0/3606650120 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f285c0e2200 con 0x7f286410ce00 2024-08-20T02:54:24.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.240+0000 7f2868ec7640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f284c0b3b90 0x7f284c0b6050 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f285400abe0 tx=0x7f285400a5c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:24.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.244+0000 7f2861ffb640 1 -- 172.21.15.44:0/3606650120 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f285c014030 con 0x7f286410ce00 2024-08-20T02:54:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:24 smithi165 ceph-mon[22357]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:24 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3692125701' entity='client.admin' 2024-08-20T02:54:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:24 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:24 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:24 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:24 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:24.393 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:24 smithi044 ceph-mon[18790]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:24.394 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:24 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3692125701' entity='client.admin' 2024-08-20T02:54:24.394 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:24 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:24.394 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:24 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:24.394 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:24 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:24.394 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:24 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:24.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:24 smithi099 ceph-mon[21984]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:24.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:24 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3692125701' entity='client.admin' 2024-08-20T02:54:24.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:24 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:24.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:24 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:24.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:24 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:24.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:24 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:24.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.433+0000 7f286b12b640 1 -- 172.21.15.44:0/3606650120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f286402e340 con 0x7f286410ce00 2024-08-20T02:54:24.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.437+0000 7f2861ffb640 1 -- 172.21.15.44:0/3606650120 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v32)=0 v32) ==== 121+0+0 (secure 0 0 0) 0x7f285c0cd070 con 0x7f286410ce00 2024-08-20T02:54:24.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.439+0000 7f286b12b640 1 -- 172.21.15.44:0/3606650120 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f284c0b3b90 msgr2=0x7f284c0b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:24.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.439+0000 7f286b12b640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f284c0b3b90 0x7f284c0b6050 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f285400abe0 tx=0x7f285400a5c0 comp rx=0 tx=0).stop 2024-08-20T02:54:24.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.440+0000 7f286b12b640 1 -- 172.21.15.44:0/3606650120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410ce00 msgr2=0x7f286419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:24.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.440+0000 7f286b12b640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410ce00 0x7f286419e860 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f285c00cad0 tx=0x7f285c00cf90 comp rx=0 tx=0).stop 2024-08-20T02:54:24.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.440+0000 7f286b12b640 1 -- 172.21.15.44:0/3606650120 shutdown_connections 2024-08-20T02:54:24.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.440+0000 7f286b12b640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f284c0b3b90 0x7f284c0b6050 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:24.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.440+0000 7f286b12b640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f286410ce00 0x7f286419e860 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:24.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.440+0000 7f286b12b640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28641042c0 0x7f28641994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:24.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.440+0000 7f286b12b640 1 --2- 172.21.15.44:0/3606650120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28641039a0 0x7f2864198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:24.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.440+0000 7f286b12b640 1 -- 172.21.15.44:0/3606650120 >> 172.21.15.44:0/3606650120 conn(0x7f28640fd400 msgr2=0x7f28640fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:24.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.440+0000 7f286b12b640 1 -- 172.21.15.44:0/3606650120 shutdown_connections 2024-08-20T02:54:24.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:24.440+0000 7f286b12b640 1 -- 172.21.15.44:0/3606650120 wait complete. 2024-08-20T02:54:25.584 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:25 smithi044 ceph-mon[18790]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:25.584 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:25 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3606650120' entity='client.admin' 2024-08-20T02:54:25.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.582+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/386029715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b641073a0 msgr2=0x7f3b641077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.582+0000 7f3b6c1c0640 1 --2- 172.21.15.44:0/386029715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b641073a0 0x7f3b641077e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009900 tx=0x7f3b5802a410 comp rx=0 tx=0).stop 2024-08-20T02:54:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.582+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/386029715 shutdown_connections 2024-08-20T02:54:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.582+0000 7f3b6c1c0640 1 --2- 172.21.15.44:0/386029715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b64107d20 0x7f3b6410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.582+0000 7f3b6c1c0640 1 --2- 172.21.15.44:0/386029715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b641073a0 0x7f3b641077e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.582+0000 7f3b6c1c0640 1 --2- 172.21.15.44:0/386029715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b64101990 0x7f3b64101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.582+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/386029715 >> 172.21.15.44:0/386029715 conn(0x7f3b640fd400 msgr2=0x7f3b640ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.582+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/386029715 shutdown_connections 2024-08-20T02:54:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.582+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/386029715 wait complete. 2024-08-20T02:54:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.582+0000 7f3b6c1c0640 1 Processor -- start 2024-08-20T02:54:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b6c1c0640 1 -- start start 2024-08-20T02:54:25.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b6c1c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b64101990 0x7f3b64199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b6c1c0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b641073a0 0x7f3b64199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b6c1c0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b64107d20 0x7f3b6419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b6c1c0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3b641119e0 con 0x7f3b64101990 2024-08-20T02:54:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b6c1c0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3b64111860 con 0x7f3b64107d20 2024-08-20T02:54:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b6c1c0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3b64111b60 con 0x7f3b641073a0 2024-08-20T02:54:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b69f5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b64101990 0x7f3b64199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b6a75d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b64107d20 0x7f3b6419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b69f5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b64101990 0x7f3b64199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59984/0 (socket says 172.21.15.44:59984) 2024-08-20T02:54:25.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b69f5c640 1 -- 172.21.15.44:0/765712501 learned_addr learned my addr 172.21.15.44:0/765712501 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b6975b640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b641073a0 0x7f3b64199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b69f5c640 1 -- 172.21.15.44:0/765712501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b641073a0 msgr2=0x7f3b64199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b69f5c640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b641073a0 0x7f3b64199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b69f5c640 1 -- 172.21.15.44:0/765712501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b64107d20 msgr2=0x7f3b6419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b69f5c640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b64107d20 0x7f3b6419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b69f5c640 1 -- 172.21.15.44:0/765712501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6419f100 con 0x7f3b64101990 2024-08-20T02:54:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.583+0000 7f3b6a75d640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b64107d20 0x7f3b6419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T02:54:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.584+0000 7f3b69f5c640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b64101990 0x7f3b64199050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400edf0 tx=0x7f3b5400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.584+0000 7f3b62ffd640 1 -- 172.21.15.44:0/765712501 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54004b20 con 0x7f3b64101990 2024-08-20T02:54:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.584+0000 7f3b62ffd640 1 -- 172.21.15.44:0/765712501 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b54004cc0 con 0x7f3b64101990 2024-08-20T02:54:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.584+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/765712501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6419f3f0 con 0x7f3b64101990 2024-08-20T02:54:25.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.585+0000 7f3b62ffd640 1 -- 172.21.15.44:0/765712501 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b540107f0 con 0x7f3b64101990 2024-08-20T02:54:25.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.585+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/765712501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b641a5ab0 con 0x7f3b64101990 2024-08-20T02:54:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.586+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/765712501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b64101d70 con 0x7f3b64101990 2024-08-20T02:54:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.587+0000 7f3b62ffd640 1 -- 172.21.15.44:0/765712501 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f3b54005020 con 0x7f3b64101990 2024-08-20T02:54:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.588+0000 7f3b62ffd640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3b4c0b3b90 0x7f3b4c0b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.588+0000 7f3b6975b640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3b4c0b3b90 0x7f3b4c0b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.588+0000 7f3b62ffd640 1 -- 172.21.15.44:0/765712501 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3b540e2680 con 0x7f3b64101990 2024-08-20T02:54:25.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.589+0000 7f3b6975b640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3b4c0b3b90 0x7f3b4c0b6050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b58031440 tx=0x7f3b58005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:25.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.592+0000 7f3b62ffd640 1 -- 172.21.15.44:0/765712501 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f3b540ab8d0 con 0x7f3b64101990 2024-08-20T02:54:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.768+0000 7f5793662640 1 -- 172.21.15.44:0/3220033099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c113840 msgr2=0x7f578c11a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.768+0000 7f5793662640 1 --2- 172.21.15.44:0/3220033099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c113840 0x7f578c11a040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5784009ca0 tx=0x7f578402a8c0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.768+0000 7f5793662640 1 -- 172.21.15.44:0/3220033099 shutdown_connections 2024-08-20T02:54:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.768+0000 7f5793662640 1 --2- 172.21.15.44:0/3220033099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c113840 0x7f578c11a040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.768+0000 7f5793662640 1 --2- 172.21.15.44:0/3220033099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f578c112ec0 0x7f578c113300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.768+0000 7f5793662640 1 --2- 172.21.15.44:0/3220033099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f578c10d4b0 0x7f578c10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.768+0000 7f5793662640 1 -- 172.21.15.44:0/3220033099 >> 172.21.15.44:0/3220033099 conn(0x7f578c06d9b0 msgr2=0x7f578c06ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f5793662640 1 -- 172.21.15.44:0/3220033099 shutdown_connections 2024-08-20T02:54:25.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f5793662640 1 -- 172.21.15.44:0/3220033099 wait complete. 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f5793662640 1 Processor -- start 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f5793662640 1 -- start start 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f5793662640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c10d4b0 0x7f578c1a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f5793662640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f578c112ec0 0x7f578c1a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f5793662640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f578c113840 0x7f578c1a9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f5793662640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f578c11d470 con 0x7f578c10d4b0 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f5793662640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f578c11d2f0 con 0x7f578c112ec0 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f5793662640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f578c11d5f0 con 0x7f578c113840 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f57913fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c10d4b0 0x7f578c1a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f57913fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c10d4b0 0x7f578c1a3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60002/0 (socket says 172.21.15.44:60002) 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.769+0000 7f57913fe640 1 -- 172.21.15.44:0/2536827719 learned_addr learned my addr 172.21.15.44:0/2536827719 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.770+0000 7f57913fe640 1 -- 172.21.15.44:0/2536827719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f578c113840 msgr2=0x7f578c1a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:25.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.770+0000 7f57913fe640 1 --2- 172.21.15.44:0/2536827719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f578c113840 0x7f578c1a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.770+0000 7f57913fe640 1 -- 172.21.15.44:0/2536827719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f578c112ec0 msgr2=0x7f578c1a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.770+0000 7f57913fe640 1 --2- 172.21.15.44:0/2536827719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f578c112ec0 0x7f578c1a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.770+0000 7f57913fe640 1 -- 172.21.15.44:0/2536827719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578c1a9a30 con 0x7f578c10d4b0 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.770+0000 7f57913fe640 1 --2- 172.21.15.44:0/2536827719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c10d4b0 0x7f578c1a3980 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f578800e9f0 tx=0x7f578800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.770+0000 7f57827fc640 1 -- 172.21.15.44:0/2536827719 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578800cd80 con 0x7f578c10d4b0 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.770+0000 7f57827fc640 1 -- 172.21.15.44:0/2536827719 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57880108e0 con 0x7f578c10d4b0 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.770+0000 7f57827fc640 1 -- 172.21.15.44:0/2536827719 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788018980 con 0x7f578c10d4b0 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.770+0000 7f5793662640 1 -- 172.21.15.44:0/2536827719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578c1a9d20 con 0x7f578c10d4b0 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.770+0000 7f5793662640 1 -- 172.21.15.44:0/2536827719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578c1b03e0 con 0x7f578c10d4b0 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.771+0000 7f5793662640 1 -- 172.21.15.44:0/2536827719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f578c10d890 con 0x7f578c10d4b0 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.773+0000 7f57827fc640 1 -- 172.21.15.44:0/2536827719 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f5788010430 con 0x7f578c10d4b0 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.773+0000 7f57827fc640 1 --2- 172.21.15.44:0/2536827719 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f57780b3b90 0x7f57780b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:25.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.773+0000 7f5790bfd640 1 --2- 172.21.15.44:0/2536827719 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f57780b3b90 0x7f57780b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:25.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.773+0000 7f57827fc640 1 -- 172.21.15.44:0/2536827719 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f57880e2950 con 0x7f578c10d4b0 2024-08-20T02:54:25.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.774+0000 7f5790bfd640 1 --2- 172.21.15.44:0/2536827719 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f57780b3b90 0x7f57780b6050 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f577c005fd0 tx=0x7f577c005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:25.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.776+0000 7f57827fc640 1 -- 172.21.15.44:0/2536827719 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f57880abba0 con 0x7f578c10d4b0 2024-08-20T02:54:25.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.787+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/765712501 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f3b6410ab10 con 0x7f3b64101990 2024-08-20T02:54:25.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.790+0000 7f3b62ffd640 1 -- 172.21.15.44:0/765712501 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v34)=0 v34) ==== 121+0+0 (secure 0 0 0) 0x7f3b540ab1d0 con 0x7f3b64101990 2024-08-20T02:54:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.793+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/765712501 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3b4c0b3b90 msgr2=0x7f3b4c0b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.793+0000 7f3b6c1c0640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3b4c0b3b90 0x7f3b4c0b6050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b58031440 tx=0x7f3b58005b50 comp rx=0 tx=0).stop 2024-08-20T02:54:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.793+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/765712501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b64101990 msgr2=0x7f3b64199050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.793+0000 7f3b6c1c0640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b64101990 0x7f3b64199050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400edf0 tx=0x7f3b5400c6a0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.793+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/765712501 shutdown_connections 2024-08-20T02:54:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.793+0000 7f3b6c1c0640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3b4c0b3b90 0x7f3b4c0b6050 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.793+0000 7f3b6c1c0640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b64107d20 0x7f3b6419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.793+0000 7f3b6c1c0640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b641073a0 0x7f3b64199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.793+0000 7f3b6c1c0640 1 --2- 172.21.15.44:0/765712501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b64101990 0x7f3b64199050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.793+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/765712501 >> 172.21.15.44:0/765712501 conn(0x7f3b640fd400 msgr2=0x7f3b6406aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:25.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.793+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/765712501 shutdown_connections 2024-08-20T02:54:25.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.793+0000 7f3b6c1c0640 1 -- 172.21.15.44:0/765712501 wait complete. 2024-08-20T02:54:25.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:25 smithi165 ceph-mon[22357]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:25.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:25 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3606650120' entity='client.admin' 2024-08-20T02:54:25.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:25 smithi099 ceph-mon[21984]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:25.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:25 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3606650120' entity='client.admin' 2024-08-20T02:54:25.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.967+0000 7f5793662640 1 -- 172.21.15.44:0/2536827719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f578c1b0860 con 0x7f578c10d4b0 2024-08-20T02:54:25.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.971+0000 7f57827fc640 1 -- 172.21.15.44:0/2536827719 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) ==== 138+0+0 (secure 0 0 0) 0x7f57880b2e20 con 0x7f578c10d4b0 2024-08-20T02:54:25.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.973+0000 7f5793662640 1 -- 172.21.15.44:0/2536827719 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f57780b3b90 msgr2=0x7f57780b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:25.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.973+0000 7f5793662640 1 --2- 172.21.15.44:0/2536827719 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f57780b3b90 0x7f57780b6050 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f577c005fd0 tx=0x7f577c005850 comp rx=0 tx=0).stop 2024-08-20T02:54:25.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.973+0000 7f5793662640 1 -- 172.21.15.44:0/2536827719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c10d4b0 msgr2=0x7f578c1a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:25.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.973+0000 7f5793662640 1 --2- 172.21.15.44:0/2536827719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c10d4b0 0x7f578c1a3980 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f578800e9f0 tx=0x7f578800eeb0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.973+0000 7f5793662640 1 -- 172.21.15.44:0/2536827719 shutdown_connections 2024-08-20T02:54:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.973+0000 7f5793662640 1 --2- 172.21.15.44:0/2536827719 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f57780b3b90 0x7f57780b6050 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.973+0000 7f5793662640 1 --2- 172.21.15.44:0/2536827719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f578c113840 0x7f578c1a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.973+0000 7f5793662640 1 --2- 172.21.15.44:0/2536827719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f578c112ec0 0x7f578c1a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.973+0000 7f5793662640 1 --2- 172.21.15.44:0/2536827719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f578c10d4b0 0x7f578c1a3980 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.973+0000 7f5793662640 1 -- 172.21.15.44:0/2536827719 >> 172.21.15.44:0/2536827719 conn(0x7f578c06d9b0 msgr2=0x7f578c110e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.973+0000 7f5793662640 1 -- 172.21.15.44:0/2536827719 shutdown_connections 2024-08-20T02:54:25.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.973+0000 7f5793662640 1 -- 172.21.15.44:0/2536827719 wait complete. 2024-08-20T02:54:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.473+0000 7fd7728d3640 1 -- 172.21.15.44:0/1127391143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd76c10ce40 msgr2=0x7fd76c10f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.473+0000 7fd7728d3640 1 --2- 172.21.15.44:0/1127391143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd76c10ce40 0x7fd76c10f2a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd764009ca0 tx=0x7fd76402a8c0 comp rx=0 tx=0).stop 2024-08-20T02:54:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.473+0000 7fd7728d3640 1 -- 172.21.15.44:0/1127391143 shutdown_connections 2024-08-20T02:54:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.473+0000 7fd7728d3640 1 --2- 172.21.15.44:0/1127391143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd76c10ce40 0x7fd76c10f2a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.473+0000 7fd7728d3640 1 --2- 172.21.15.44:0/1127391143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd76c104300 0x7fd76c10c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:26.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.473+0000 7fd7728d3640 1 --2- 172.21.15.44:0/1127391143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c1039e0 0x7fd76c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.473+0000 7fd7728d3640 1 -- 172.21.15.44:0/1127391143 >> 172.21.15.44:0/1127391143 conn(0x7fd76c0fd440 msgr2=0x7fd76c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.473+0000 7fd7728d3640 1 -- 172.21.15.44:0/1127391143 shutdown_connections 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.473+0000 7fd7728d3640 1 -- 172.21.15.44:0/1127391143 wait complete. 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.474+0000 7fd7728d3640 1 Processor -- start 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.474+0000 7fd7728d3640 1 -- start start 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.474+0000 7fd7728d3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd76c1039e0 0x7fd76c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.474+0000 7fd7728d3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c104300 0x7fd76c199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.474+0000 7fd7728d3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd76c10ce40 0x7fd76c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.474+0000 7fd7728d3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c112b10 con 0x7fd76c10ce40 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.474+0000 7fd7728d3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c112990 con 0x7fd76c104300 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.474+0000 7fd7728d3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c112c90 con 0x7fd76c1039e0 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd770e70640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd76c10ce40 0x7fd76c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:26.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd770e70640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd76c10ce40 0x7fd76c19e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59952/0 (socket says 172.21.15.44:59952) 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd770e70640 1 -- 172.21.15.44:0/719688633 learned_addr learned my addr 172.21.15.44:0/719688633 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd76b7fe640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c104300 0x7fd76c199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd76bfff640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd76c1039e0 0x7fd76c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd770e70640 1 -- 172.21.15.44:0/719688633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd76c1039e0 msgr2=0x7fd76c1990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd770e70640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd76c1039e0 0x7fd76c1990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd770e70640 1 -- 172.21.15.44:0/719688633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c104300 msgr2=0x7fd76c199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd770e70640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c104300 0x7fd76c199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd770e70640 1 -- 172.21.15.44:0/719688633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c19f170 con 0x7fd76c10ce40 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd770e70640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd76c10ce40 0x7fd76c19e9a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd764005cc0 tx=0x7fd76402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd7697fa640 1 -- 172.21.15.44:0/719688633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764003bf0 con 0x7fd76c10ce40 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd7728d3640 1 -- 172.21.15.44:0/719688633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c19f400 con 0x7fd76c10ce40 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd7697fa640 1 -- 172.21.15.44:0/719688633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd764003d90 con 0x7fd76c10ce40 2024-08-20T02:54:26.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd7697fa640 1 -- 172.21.15.44:0/719688633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76402fbc0 con 0x7fd76c10ce40 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.475+0000 7fd7728d3640 1 -- 172.21.15.44:0/719688633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c1a5b20 con 0x7fd76c10ce40 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.476+0000 7fd7728d3640 1 -- 172.21.15.44:0/719688633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76c103dc0 con 0x7fd76c10ce40 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.478+0000 7fd7697fa640 1 -- 172.21.15.44:0/719688633 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fd76402fd60 con 0x7fd76c10ce40 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.478+0000 7fd7697fa640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd7540b3b90 0x7fd7540b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.478+0000 7fd76bfff640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd7540b3b90 0x7fd7540b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.478+0000 7fd7697fa640 1 -- 172.21.15.44:0/719688633 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd7641038d0 con 0x7fd76c10ce40 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.479+0000 7fd76bfff640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd7540b3b90 0x7fd7540b6050 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd75c005fd0 tx=0x7fd75c005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.481+0000 7fd7697fa640 1 -- 172.21.15.44:0/719688633 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd7640ccb00 con 0x7fd76c10ce40 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.673+0000 7fd7728d3640 1 -- 172.21.15.44:0/719688633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fd76c19a1f0 con 0x7fd76c10ce40 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.676+0000 7fd7697fa640 1 -- 172.21.15.44:0/719688633 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v33)=0 v33) ==== 125+0+0 (secure 0 0 0) 0x7fd76c19a1f0 con 0x7fd76c10ce40 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.678+0000 7fd7728d3640 1 -- 172.21.15.44:0/719688633 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd7540b3b90 msgr2=0x7fd7540b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.678+0000 7fd7728d3640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd7540b3b90 0x7fd7540b6050 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd75c005fd0 tx=0x7fd75c005850 comp rx=0 tx=0).stop 2024-08-20T02:54:26.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.678+0000 7fd7728d3640 1 -- 172.21.15.44:0/719688633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd76c10ce40 msgr2=0x7fd76c19e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.679+0000 7fd7728d3640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd76c10ce40 0x7fd76c19e9a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd764005cc0 tx=0x7fd76402f380 comp rx=0 tx=0).stop 2024-08-20T02:54:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.679+0000 7fd7728d3640 1 -- 172.21.15.44:0/719688633 shutdown_connections 2024-08-20T02:54:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.679+0000 7fd7728d3640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd7540b3b90 0x7fd7540b6050 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.679+0000 7fd7728d3640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd76c10ce40 0x7fd76c19e9a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.679+0000 7fd7728d3640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c104300 0x7fd76c199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.679+0000 7fd7728d3640 1 --2- 172.21.15.44:0/719688633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd76c1039e0 0x7fd76c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.679+0000 7fd7728d3640 1 -- 172.21.15.44:0/719688633 >> 172.21.15.44:0/719688633 conn(0x7fd76c0fd440 msgr2=0x7fd76c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.679+0000 7fd7728d3640 1 -- 172.21.15.44:0/719688633 shutdown_connections 2024-08-20T02:54:26.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:25.679+0000 7fd7728d3640 1 -- 172.21.15.44:0/719688633 wait complete. 2024-08-20T02:54:26.733 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:26 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/719688633' entity='client.admin' 2024-08-20T02:54:26.733 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:26 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/765712501' entity='client.admin' 2024-08-20T02:54:26.734 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:26 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2536827719' entity='client.admin' 2024-08-20T02:54:26.734 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:26 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:26.734 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:26 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:26.734 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:26 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:26.734 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:26 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:27.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:26 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/719688633' entity='client.admin' 2024-08-20T02:54:27.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:26 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/765712501' entity='client.admin' 2024-08-20T02:54:27.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:26 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2536827719' entity='client.admin' 2024-08-20T02:54:27.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:26 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:27.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:26 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:27.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:26 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:27.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:26 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:27.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:26 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/719688633' entity='client.admin' 2024-08-20T02:54:27.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:26 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/765712501' entity='client.admin' 2024-08-20T02:54:27.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:26 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2536827719' entity='client.admin' 2024-08-20T02:54:27.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:26 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:27.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:26 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:27.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:26 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:27.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:26 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:27.884 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi044') 2024-08-20T02:54:27.884 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi044') 2024-08-20T02:54:27.884 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi044') 2024-08-20T02:54:27.884 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi044') 2024-08-20T02:54:27.885 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi044') 2024-08-20T02:54:27.885 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi044') 2024-08-20T02:54:28.144 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:27 smithi044 ceph-mon[18790]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:28.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:27 smithi165 ceph-mon[22357]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:28.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:27 smithi099 ceph-mon[21984]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:28.782 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi044') 2024-08-20T02:54:28.782 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi044') 2024-08-20T02:54:28.782 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi044') 2024-08-20T02:54:28.783 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi044') 2024-08-20T02:54:28.783 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '0284e16c-5e9f-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi044') 2024-08-20T02:54:28.783 INFO:tasks.cephadm:Deploying OSDs... 2024-08-20T02:54:28.783 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T02:54:28.783 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-08-20T02:54:28.801 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-20T02:54:28.801 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-08-20T02:54:28.858 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-20T02:54:28.858 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T02:54:28.858 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-08-20T02:54:28.858 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T02:54:28.858 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T02:54:28.858 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-08-20 02:53:12.551258296 +0000 2024-08-20T02:54:28.858 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-08-20 02:48:05.814151861 +0000 2024-08-20T02:54:28.859 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-08-20 02:48:05.814151861 +0000 2024-08-20T02:54:28.859 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-08-20 02:48:05.814151861 +0000 2024-08-20T02:54:28.859 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-20T02:54:28.926 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-08-20T02:54:28.926 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-08-20T02:54:28.927 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000235276 s, 2.2 MB/s 2024-08-20T02:54:28.928 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-20T02:54:28.985 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-08-20T02:54:29.043 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-20T02:54:29.043 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T02:54:29.043 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-08-20T02:54:29.043 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T02:54:29.043 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T02:54:29.043 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-08-20 02:53:12.577257896 +0000 2024-08-20T02:54:29.044 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-08-20 02:48:06.142148668 +0000 2024-08-20T02:54:29.044 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-08-20 02:48:06.142148668 +0000 2024-08-20T02:54:29.044 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-08-20 02:48:06.142148668 +0000 2024-08-20T02:54:29.044 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-20T02:54:29.107 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-08-20T02:54:29.107 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-08-20T02:54:29.107 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000208347 s, 2.5 MB/s 2024-08-20T02:54:29.109 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-20T02:54:29.166 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-08-20T02:54:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-20T02:54:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T02:54:29.221 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-08-20T02:54:29.222 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T02:54:29.222 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T02:54:29.222 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-08-20 02:53:12.604257481 +0000 2024-08-20T02:54:29.222 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-08-20 02:48:06.463145543 +0000 2024-08-20T02:54:29.222 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-08-20 02:48:06.463145543 +0000 2024-08-20T02:54:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-08-20 02:48:06.463145543 +0000 2024-08-20T02:54:29.222 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-20T02:54:29.285 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-08-20T02:54:29.286 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-08-20T02:54:29.286 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000194043 s, 2.6 MB/s 2024-08-20T02:54:29.287 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-20T02:54:29.343 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-08-20T02:54:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-20T02:54:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T02:54:29.398 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-08-20T02:54:29.398 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T02:54:29.398 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T02:54:29.398 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-08-20 02:53:12.630257081 +0000 2024-08-20T02:54:29.399 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-08-20 02:48:06.797142292 +0000 2024-08-20T02:54:29.399 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-08-20 02:48:06.797142292 +0000 2024-08-20T02:54:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-08-20 02:48:06.797142292 +0000 2024-08-20T02:54:29.399 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-20T02:54:29.461 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-08-20T02:54:29.461 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-08-20T02:54:29.461 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000183522 s, 2.8 MB/s 2024-08-20T02:54:29.462 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-20T02:54:29.519 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-08-20T02:54:29.519 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-08-20T02:54:29.539 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-20T02:54:29.539 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-08-20T02:54:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-20T02:54:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T02:54:29.597 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-08-20T02:54:29.597 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T02:54:29.597 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T02:54:29.597 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-08-20 02:54:06.753019245 +0000 2024-08-20T02:54:29.597 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-08-20 02:48:34.276652276 +0000 2024-08-20T02:54:29.597 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-08-20 02:48:34.276652276 +0000 2024-08-20T02:54:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-08-20 02:48:34.276652276 +0000 2024-08-20T02:54:29.597 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-20T02:54:29.666 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-08-20T02:54:29.666 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-08-20T02:54:29.666 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000289457 s, 1.8 MB/s 2024-08-20T02:54:29.668 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-20T02:54:29.726 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-08-20T02:54:29.784 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-20T02:54:29.784 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T02:54:29.784 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-08-20T02:54:29.785 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T02:54:29.785 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T02:54:29.785 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-08-20 02:54:06.753019245 +0000 2024-08-20T02:54:29.785 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-08-20 02:48:34.601645085 +0000 2024-08-20T02:54:29.785 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-08-20 02:48:34.601645085 +0000 2024-08-20T02:54:29.785 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-08-20 02:48:34.601645085 +0000 2024-08-20T02:54:29.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-20T02:54:29.849 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-08-20T02:54:29.850 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-08-20T02:54:29.850 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000230731 s, 2.2 MB/s 2024-08-20T02:54:29.851 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-20T02:54:29.908 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-08-20T02:54:29.965 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-20T02:54:29.965 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T02:54:29.965 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-08-20T02:54:29.965 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T02:54:29.965 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T02:54:29.965 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-08-20 02:54:06.754019222 +0000 2024-08-20T02:54:29.966 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-08-20 02:48:34.940637584 +0000 2024-08-20T02:54:29.966 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-08-20 02:48:34.940637584 +0000 2024-08-20T02:54:29.966 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-08-20 02:48:34.940637584 +0000 2024-08-20T02:54:29.966 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-20T02:54:30.029 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-08-20T02:54:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-08-20T02:54:30.030 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000175496 s, 2.9 MB/s 2024-08-20T02:54:30.031 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-20T02:54:30.088 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-08-20T02:54:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-20T02:54:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T02:54:30.143 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-08-20T02:54:30.143 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T02:54:30.143 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T02:54:30.143 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-08-20 02:54:06.754019222 +0000 2024-08-20T02:54:30.144 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-08-20 02:48:35.276630149 +0000 2024-08-20T02:54:30.144 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-08-20 02:48:35.276630149 +0000 2024-08-20T02:54:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-08-20 02:48:35.276630149 +0000 2024-08-20T02:54:30.144 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-20T02:54:30.206 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-08-20T02:54:30.206 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-08-20T02:54:30.206 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000197959 s, 2.6 MB/s 2024-08-20T02:54:30.207 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-20T02:54:30.264 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-08-20T02:54:30.264 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-08-20T02:54:30.284 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-20T02:54:30.285 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-08-20T02:54:30.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:29 smithi044 ceph-mon[18790]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-20T02:54:30.344 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T02:54:30.344 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-08-20T02:54:30.344 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T02:54:30.344 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T02:54:30.345 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-08-20 02:54:04.556057615 +0000 2024-08-20T02:54:30.345 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-08-20 02:48:34.244374282 +0000 2024-08-20T02:54:30.345 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-08-20 02:48:34.244374282 +0000 2024-08-20T02:54:30.345 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-08-20 02:48:34.244374282 +0000 2024-08-20T02:54:30.345 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-20T02:54:30.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:29 smithi165 ceph-mon[22357]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:29 smithi099 ceph-mon[21984]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-08-20T02:54:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-08-20T02:54:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000288826 s, 1.8 MB/s 2024-08-20T02:54:30.416 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-20T02:54:30.475 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-08-20T02:54:30.532 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-20T02:54:30.532 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T02:54:30.532 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-08-20T02:54:30.533 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T02:54:30.533 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T02:54:30.533 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-08-20 02:54:04.557057617 +0000 2024-08-20T02:54:30.533 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-08-20 02:48:34.605375087 +0000 2024-08-20T02:54:30.533 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-08-20 02:48:34.605375087 +0000 2024-08-20T02:54:30.533 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-08-20 02:48:34.605375087 +0000 2024-08-20T02:54:30.533 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-20T02:54:30.598 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-08-20T02:54:30.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-08-20T02:54:30.599 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000204094 s, 2.5 MB/s 2024-08-20T02:54:30.600 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-20T02:54:30.658 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-08-20T02:54:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-20T02:54:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T02:54:30.715 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-08-20T02:54:30.715 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T02:54:30.715 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T02:54:30.715 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-08-20 02:54:04.557057617 +0000 2024-08-20T02:54:30.715 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-08-20 02:48:34.944375844 +0000 2024-08-20T02:54:30.715 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-08-20 02:48:34.944375844 +0000 2024-08-20T02:54:30.715 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-08-20 02:48:34.944375844 +0000 2024-08-20T02:54:30.716 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-20T02:54:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-08-20T02:54:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-08-20T02:54:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000209276 s, 2.4 MB/s 2024-08-20T02:54:30.781 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-20T02:54:30.839 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-08-20T02:54:30.895 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-20T02:54:30.895 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T02:54:30.896 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-08-20T02:54:30.896 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T02:54:30.896 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T02:54:30.896 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-08-20 02:54:04.557057617 +0000 2024-08-20T02:54:30.896 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-08-20 02:48:35.266376563 +0000 2024-08-20T02:54:30.896 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-08-20 02:48:35.266376563 +0000 2024-08-20T02:54:30.896 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-08-20 02:48:35.266376563 +0000 2024-08-20T02:54:30.896 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-20T02:54:30.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-08-20T02:54:30.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-08-20T02:54:30.961 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000204527 s, 2.5 MB/s 2024-08-20T02:54:30.962 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-20T02:54:31.019 INFO:tasks.cephadm:Deploying osd.0 on smithi044 with /dev/vg_nvme/lv_4... 2024-08-20T02:54:31.019 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-20T02:54:31.278 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:32.271 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:31 smithi044 ceph-mon[18790]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:31 smithi165 ceph-mon[22357]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:31 smithi099 ceph-mon[21984]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:33.704 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:54:33.731 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_4 2024-08-20T02:54:33.979 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:54:34.281 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:33 smithi044 ceph-mon[18790]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:33 smithi165 ceph-mon[22357]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:34.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:33 smithi099 ceph-mon[21984]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:34.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.483+0000 7fdf631cb640 1 -- 172.21.15.44:0/4151216264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c1039e0 msgr2=0x7fdf5c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:34.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.483+0000 7fdf631cb640 1 --2- 172.21.15.44:0/4151216264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c1039e0 0x7fdf5c103dc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009ca0 tx=0x7fdf4c02a8c0 comp rx=0 tx=0).stop 2024-08-20T02:54:34.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.483+0000 7fdf631cb640 1 -- 172.21.15.44:0/4151216264 shutdown_connections 2024-08-20T02:54:34.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.483+0000 7fdf631cb640 1 --2- 172.21.15.44:0/4151216264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf5c10ce40 0x7fdf5c10f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:34.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.483+0000 7fdf631cb640 1 --2- 172.21.15.44:0/4151216264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf5c104300 0x7fdf5c10c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:34.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.483+0000 7fdf631cb640 1 --2- 172.21.15.44:0/4151216264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c1039e0 0x7fdf5c103dc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:34.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.483+0000 7fdf631cb640 1 -- 172.21.15.44:0/4151216264 >> 172.21.15.44:0/4151216264 conn(0x7fdf5c0fd440 msgr2=0x7fdf5c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:34.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.483+0000 7fdf631cb640 1 -- 172.21.15.44:0/4151216264 shutdown_connections 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.483+0000 7fdf631cb640 1 -- 172.21.15.44:0/4151216264 wait complete. 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.483+0000 7fdf631cb640 1 Processor -- start 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.483+0000 7fdf631cb640 1 -- start start 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf631cb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c1039e0 0x7fdf5c19b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf631cb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf5c104300 0x7fdf5c19b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf631cb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf5c10ce40 0x7fdf5c1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf631cb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c112af0 con 0x7fdf5c1039e0 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf631cb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c112970 con 0x7fdf5c104300 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf631cb640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c112c70 con 0x7fdf5c10ce40 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf60f67640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c1039e0 0x7fdf5c19b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf60f67640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c1039e0 0x7fdf5c19b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60622/0 (socket says 172.21.15.44:60622) 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf60f67640 1 -- 172.21.15.44:0/1323149581 learned_addr learned my addr 172.21.15.44:0/1323149581 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf61768640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf5c10ce40 0x7fdf5c1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:34.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf5bfff640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf5c104300 0x7fdf5c19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:34.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf60f67640 1 -- 172.21.15.44:0/1323149581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf5c10ce40 msgr2=0x7fdf5c1a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:34.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf60f67640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf5c10ce40 0x7fdf5c1a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:34.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf60f67640 1 -- 172.21.15.44:0/1323149581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf5c104300 msgr2=0x7fdf5c19b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:34.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf60f67640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf5c104300 0x7fdf5c19b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:34.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf60f67640 1 -- 172.21.15.44:0/1323149581 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c1a12a0 con 0x7fdf5c1039e0 2024-08-20T02:54:34.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf5bfff640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf5c104300 0x7fdf5c19b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T02:54:34.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf61768640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf5c10ce40 0x7fdf5c1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:54:34.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf60f67640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c1039e0 0x7fdf5c19b1f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c006010 tx=0x7fdf4c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:34.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf59ffb640 1 -- 172.21.15.44:0/1323149581 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c003bf0 con 0x7fdf5c1039e0 2024-08-20T02:54:34.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf631cb640 1 -- 172.21.15.44:0/1323149581 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c1a1530 con 0x7fdf5c1039e0 2024-08-20T02:54:34.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf59ffb640 1 -- 172.21.15.44:0/1323149581 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf4c003d90 con 0x7fdf5c1039e0 2024-08-20T02:54:34.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.484+0000 7fdf59ffb640 1 -- 172.21.15.44:0/1323149581 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c02fc20 con 0x7fdf5c1039e0 2024-08-20T02:54:34.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.485+0000 7fdf631cb640 1 -- 172.21.15.44:0/1323149581 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c1a7cd0 con 0x7fdf5c1039e0 2024-08-20T02:54:34.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.486+0000 7fdf631cb640 1 -- 172.21.15.44:0/1323149581 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c103dc0 con 0x7fdf5c1039e0 2024-08-20T02:54:34.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.487+0000 7fdf59ffb640 1 -- 172.21.15.44:0/1323149581 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fdf4c0046a0 con 0x7fdf5c1039e0 2024-08-20T02:54:34.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.487+0000 7fdf59ffb640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdf480b3c10 0x7fdf480b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:34.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.488+0000 7fdf5bfff640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdf480b3c10 0x7fdf480b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:34.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.488+0000 7fdf59ffb640 1 -- 172.21.15.44:0/1323149581 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdf4c1036b0 con 0x7fdf5c1039e0 2024-08-20T02:54:34.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.488+0000 7fdf5bfff640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdf480b3c10 0x7fdf480b60d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0ffb00 tx=0x7fdf440092a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:34.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.492+0000 7fdf59ffb640 1 -- 172.21.15.44:0/1323149581 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdf4c02fdc0 con 0x7fdf5c1039e0 2024-08-20T02:54:34.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:34.685+0000 7fdf631cb640 1 -- 172.21.15.44:0/1323149581 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fdf5c0ff050 con 0x7fdf480b3c10 2024-08-20T02:54:35.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:54:35.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:54:35.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:35.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:54:35.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:54:35.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:35.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:54:35.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:54:35.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:36.008 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:36 smithi044 ceph-mon[18790]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:36.008 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:36 smithi044 ceph-mon[18790]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:54:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:36 smithi165 ceph-mon[22357]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:36 smithi165 ceph-mon[22357]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:54:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:36 smithi099 ceph-mon[21984]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:36 smithi099 ceph-mon[21984]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:54:38.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:38 smithi044 ceph-mon[18790]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:38.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:38 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2606004843' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7708f8c-a799-4b8f-ab1e-78482d73ca1e"} : dispatch 2024-08-20T02:54:38.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:38 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2606004843' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7708f8c-a799-4b8f-ab1e-78482d73ca1e"}]': finished 2024-08-20T02:54:38.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:38 smithi044 ceph-mon[18790]: osdmap e6: 1 total, 0 up, 1 in 2024-08-20T02:54:38.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:38 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:38.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:38 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3429698241' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:54:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:38 smithi165 ceph-mon[22357]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:38 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2606004843' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7708f8c-a799-4b8f-ab1e-78482d73ca1e"} : dispatch 2024-08-20T02:54:38.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:38 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2606004843' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7708f8c-a799-4b8f-ab1e-78482d73ca1e"}]': finished 2024-08-20T02:54:38.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:38 smithi165 ceph-mon[22357]: osdmap e6: 1 total, 0 up, 1 in 2024-08-20T02:54:38.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:38 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:38.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:38 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3429698241' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:54:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:38 smithi099 ceph-mon[21984]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:38 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2606004843' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a7708f8c-a799-4b8f-ab1e-78482d73ca1e"} : dispatch 2024-08-20T02:54:38.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:38 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2606004843' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7708f8c-a799-4b8f-ab1e-78482d73ca1e"}]': finished 2024-08-20T02:54:38.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:38 smithi099 ceph-mon[21984]: osdmap e6: 1 total, 0 up, 1 in 2024-08-20T02:54:38.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:38 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:38.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:38 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3429698241' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:54:40.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:40 smithi044 ceph-mon[18790]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:40 smithi165 ceph-mon[22357]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:40.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:40 smithi099 ceph-mon[21984]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:42.272 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:42 smithi044 ceph-mon[18790]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:42 smithi165 ceph-mon[22357]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:42.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:42 smithi099 ceph-mon[21984]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:44.232 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:44 smithi044 ceph-mon[18790]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:44.372 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:44 smithi165 ceph-mon[22357]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:44.399 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:44 smithi099 ceph-mon[21984]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:45.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:45 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-20T02:54:45.405 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:45 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:45.406 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:45 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-20T02:54:45.406 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:45 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:45.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:45 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-20T02:54:45.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:45 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:46.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:46 smithi044 ceph-mon[18790]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:46.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:46 smithi044 ceph-mon[18790]: Deploying daemon osd.0 on smithi044 2024-08-20T02:54:46.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:46 smithi165 ceph-mon[22357]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:46.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:46 smithi165 ceph-mon[22357]: Deploying daemon osd.0 on smithi044 2024-08-20T02:54:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:46 smithi099 ceph-mon[21984]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:46 smithi099 ceph-mon[21984]: Deploying daemon osd.0 on smithi044 2024-08-20T02:54:48.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:48 smithi044 ceph-mon[18790]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:48.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:48 smithi165 ceph-mon[22357]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:48.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:48 smithi099 ceph-mon[21984]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:50 smithi044 ceph-mon[18790]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:50 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:50 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:50 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:50 smithi165 ceph-mon[22357]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:50 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:50 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:50 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:50.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:50 smithi099 ceph-mon[21984]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:50.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:50 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:50.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:50 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:50.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:50 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:52.226 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:52 smithi044 ceph-mon[18790]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:52.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:52 smithi165 ceph-mon[22357]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:52.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:52 smithi099 ceph-mon[21984]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:53.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:53 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:53.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:53 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:53.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:53 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:53.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:53 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:53.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:53 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:53.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:53 smithi044 ceph-mon[18790]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:53.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:53 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:53.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:53 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:53.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:53 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:53.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:53 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:53.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:53 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:53.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:53 smithi165 ceph-mon[22357]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:53.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:53 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:53.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:53 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:53.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:53 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:53.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:53 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:53.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:53 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:53.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:53 smithi099 ceph-mon[21984]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:53.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.985+0000 7fdf59ffb640 1 -- 172.21.15.44:0/1323149581 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdf5c0ff050 con 0x7fdf480b3c10 2024-08-20T02:54:53.988 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 0 on host 'smithi044' 2024-08-20T02:54:53.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.987+0000 7fdf631cb640 1 -- 172.21.15.44:0/1323149581 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdf480b3c10 msgr2=0x7fdf480b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:53.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.987+0000 7fdf631cb640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdf480b3c10 0x7fdf480b60d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0ffb00 tx=0x7fdf440092a0 comp rx=0 tx=0).stop 2024-08-20T02:54:53.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.987+0000 7fdf631cb640 1 -- 172.21.15.44:0/1323149581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c1039e0 msgr2=0x7fdf5c19b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.987+0000 7fdf631cb640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c1039e0 0x7fdf5c19b1f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c006010 tx=0x7fdf4c0319f0 comp rx=0 tx=0).stop 2024-08-20T02:54:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.987+0000 7fdf631cb640 1 -- 172.21.15.44:0/1323149581 shutdown_connections 2024-08-20T02:54:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.987+0000 7fdf631cb640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdf480b3c10 0x7fdf480b60d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.987+0000 7fdf631cb640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf5c10ce40 0x7fdf5c1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.987+0000 7fdf631cb640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf5c104300 0x7fdf5c19b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.987+0000 7fdf631cb640 1 --2- 172.21.15.44:0/1323149581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf5c1039e0 0x7fdf5c19b1f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.987+0000 7fdf631cb640 1 -- 172.21.15.44:0/1323149581 >> 172.21.15.44:0/1323149581 conn(0x7fdf5c0fd440 msgr2=0x7fdf5c10f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.987+0000 7fdf631cb640 1 -- 172.21.15.44:0/1323149581 shutdown_connections 2024-08-20T02:54:53.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:54:53.987+0000 7fdf631cb640 1 -- 172.21.15.44:0/1323149581 wait complete. 2024-08-20T02:54:54.512 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:54 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:54.512 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:54 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:54.512 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:54 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:54.538 DEBUG:teuthology.orchestra.run.smithi044:osd.0> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.0.service 2024-08-20T02:54:54.541 INFO:tasks.cephadm:Deploying osd.1 on smithi099 with /dev/vg_nvme/lv_4... 2024-08-20T02:54:54.541 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-20T02:54:54.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:54 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:54.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:54 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:54.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:54 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:54.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:54 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:54:54.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:54 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:54.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:54 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:54.784 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T02:54:55.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:55 smithi099 ceph-mon[21984]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:55.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:55 smithi044 ceph-mon[18790]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:55.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:55 smithi165 ceph-mon[22357]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:55.936 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 02:54:55 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0[37020]: 2024-08-20T02:54:55.741+0000 7f1b05b14640 -1 osd.0 0 log_to_monitors true 2024-08-20T02:54:56.364 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:56 smithi099 ceph-mon[21984]: from='osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-20T02:54:56.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:56 smithi044 ceph-mon[18790]: from='osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-20T02:54:56.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:56 smithi165 ceph-mon[22357]: from='osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-20T02:54:57.053 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:54:57.081 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_4 2024-08-20T02:54:57.379 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T02:54:57.517 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:57 smithi099 ceph-mon[21984]: from='osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-20T02:54:57.517 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:57 smithi099 ceph-mon[21984]: osdmap e7: 1 total, 0 up, 1 in 2024-08-20T02:54:57.517 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:57 smithi099 ceph-mon[21984]: from='osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]} : dispatch 2024-08-20T02:54:57.518 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:57.518 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:57 smithi099 ceph-mon[21984]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:57.535 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 02:54:57 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0[37020]: 2024-08-20T02:54:57.252+0000 7f1b01a97640 -1 osd.0 0 waiting for initial osdmap 2024-08-20T02:54:57.535 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 02:54:57 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0[37020]: 2024-08-20T02:54:57.257+0000 7f1afd8be640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T02:54:57.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:57 smithi044 ceph-mon[18790]: from='osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-20T02:54:57.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:57 smithi044 ceph-mon[18790]: osdmap e7: 1 total, 0 up, 1 in 2024-08-20T02:54:57.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:57 smithi044 ceph-mon[18790]: from='osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]} : dispatch 2024-08-20T02:54:57.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:57.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:57 smithi044 ceph-mon[18790]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:57.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:57 smithi165 ceph-mon[22357]: from='osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-20T02:54:57.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:57 smithi165 ceph-mon[22357]: osdmap e7: 1 total, 0 up, 1 in 2024-08-20T02:54:57.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:57 smithi165 ceph-mon[22357]: from='osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]} : dispatch 2024-08-20T02:54:57.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:57.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:57 smithi165 ceph-mon[22357]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:57.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.836+0000 7fcff5f18640 1 -- 172.21.15.99:0/1602046099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcff0101990 msgr2=0x7fcff0101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:57.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.836+0000 7fcff5f18640 1 --2- 172.21.15.99:0/1602046099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcff0101990 0x7fcff0101d70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0009900 tx=0x7fcfe002a3f0 comp rx=0 tx=0).stop 2024-08-20T02:54:57.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.836+0000 7fcff5f18640 1 -- 172.21.15.99:0/1602046099 shutdown_connections 2024-08-20T02:54:57.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.836+0000 7fcff5f18640 1 --2- 172.21.15.99:0/1602046099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff0107d20 0x7fcff010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:57.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.836+0000 7fcff5f18640 1 --2- 172.21.15.99:0/1602046099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff01073a0 0x7fcff01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:57.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.836+0000 7fcff5f18640 1 --2- 172.21.15.99:0/1602046099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcff0101990 0x7fcff0101d70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:57.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.836+0000 7fcff5f18640 1 -- 172.21.15.99:0/1602046099 >> 172.21.15.99:0/1602046099 conn(0x7fcff00fd400 msgr2=0x7fcff00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:54:57.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.836+0000 7fcff5f18640 1 -- 172.21.15.99:0/1602046099 shutdown_connections 2024-08-20T02:54:57.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.836+0000 7fcff5f18640 1 -- 172.21.15.99:0/1602046099 wait complete. 2024-08-20T02:54:57.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.837+0000 7fcff5f18640 1 Processor -- start 2024-08-20T02:54:57.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.837+0000 7fcff5f18640 1 -- start start 2024-08-20T02:54:57.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.837+0000 7fcff5f18640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff0101990 0x7fcff0198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:57.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.837+0000 7fcff5f18640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff01073a0 0x7fcff0199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.837+0000 7fcff5f18640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcff0107d20 0x7fcff019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.837+0000 7fcff5f18640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcff01119e0 con 0x7fcff0101990 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.837+0000 7fcff5f18640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0111860 con 0x7fcff0107d20 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.837+0000 7fcff5f18640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fcff0111b60 con 0x7fcff01073a0 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfeffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcff0107d20 0x7fcff019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfeffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcff0107d20 0x7fcff019e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38474/0 (socket says 172.21.15.99:38474) 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfeffff640 1 -- 172.21.15.99:0/2937497510 learned_addr learned my addr 172.21.15.99:0/2937497510 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfef7fe640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff0101990 0x7fcff0198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfeeffd640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff01073a0 0x7fcff0199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfeffff640 1 -- 172.21.15.99:0/2937497510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff01073a0 msgr2=0x7fcff0199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfeffff640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff01073a0 0x7fcff0199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfeffff640 1 -- 172.21.15.99:0/2937497510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff0101990 msgr2=0x7fcff0198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:54:57.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfeffff640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff0101990 0x7fcff0198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:54:57.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfeffff640 1 -- 172.21.15.99:0/2937497510 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff019f090 con 0x7fcff0107d20 2024-08-20T02:54:57.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfeeffd640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff01073a0 0x7fcff0199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:54:57.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfeffff640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcff0107d20 0x7fcff019e8c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800cce0 tx=0x7fcfe800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:57.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfecff9640 1 -- 172.21.15.99:0/2937497510 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800f450 con 0x7fcff0107d20 2024-08-20T02:54:57.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfecff9640 1 -- 172.21.15.99:0/2937497510 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfe800faa0 con 0x7fcff0107d20 2024-08-20T02:54:57.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcff5f18640 1 -- 172.21.15.99:0/2937497510 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff019f380 con 0x7fcff0107d20 2024-08-20T02:54:57.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcfecff9640 1 -- 172.21.15.99:0/2937497510 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8018b70 con 0x7fcff0107d20 2024-08-20T02:54:57.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.838+0000 7fcff5f18640 1 -- 172.21.15.99:0/2937497510 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff00fd7e0 con 0x7fcff0107d20 2024-08-20T02:54:57.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.839+0000 7fcff5f18640 1 -- 172.21.15.99:0/2937497510 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff0101d70 con 0x7fcff0107d20 2024-08-20T02:54:57.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.841+0000 7fcfecff9640 1 -- 172.21.15.99:0/2937497510 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fcfe8018d10 con 0x7fcff0107d20 2024-08-20T02:54:57.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.842+0000 7fcfecff9640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fcfd80b3b90 0x7fcfd80b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:54:57.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.842+0000 7fcfecff9640 1 -- 172.21.15.99:0/2937497510 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1702+0+0 (secure 0 0 0) 0x7fcfe80e2700 con 0x7fcff0107d20 2024-08-20T02:54:57.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.843+0000 7fcfef7fe640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fcfd80b3b90 0x7fcfd80b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:54:57.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.843+0000 7fcfef7fe640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fcfd80b3b90 0x7fcfd80b6050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0031440 tx=0x7fcfe0005e80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:54:57.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:57.845+0000 7fcfecff9640 1 -- 172.21.15.99:0/2937497510 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fcfe8014030 con 0x7fcff0107d20 2024-08-20T02:54:58.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:54:58.035+0000 7fcff5f18640 1 -- 172.21.15.99:0/2937497510 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fcff00632c0 con 0x7fcfd80b3b90 2024-08-20T02:54:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-08-20T02:54:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: osdmap e8: 1 total, 0 up, 1 in 2024-08-20T02:54:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: Detected new or changed devices on smithi044 2024-08-20T02:54:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T02:54:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi044 to 17092M 2024-08-20T02:54:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:58.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:58.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:54:58.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:54:58.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:54:58.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:58.587 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-08-20T02:54:58.587 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: osdmap e8: 1 total, 0 up, 1 in 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: Detected new or changed devices on smithi044 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi044 to 17092M 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:54:58.588 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:54:58.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:58.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-08-20T02:54:58.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: osdmap e8: 1 total, 0 up, 1 in 2024-08-20T02:54:58.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:58.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:58.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: Detected new or changed devices on smithi044 2024-08-20T02:54:58.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:58.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:58.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T02:54:58.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi044 to 17092M 2024-08-20T02:54:58.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:58.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:58.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:54:58.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:54:58.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:54:58.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:54:58.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:54:58.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:54:59.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:59 smithi099 ceph-mon[21984]: purged_snaps scrub starts 2024-08-20T02:54:59.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:59 smithi099 ceph-mon[21984]: purged_snaps scrub ok 2024-08-20T02:54:59.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:59 smithi099 ceph-mon[21984]: osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316] boot 2024-08-20T02:54:59.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:59 smithi099 ceph-mon[21984]: osdmap e9: 1 total, 1 up, 1 in 2024-08-20T02:54:59.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:59.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:54:59 smithi099 ceph-mon[21984]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:59.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:59 smithi044 ceph-mon[18790]: purged_snaps scrub starts 2024-08-20T02:54:59.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:59 smithi044 ceph-mon[18790]: purged_snaps scrub ok 2024-08-20T02:54:59.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:59 smithi044 ceph-mon[18790]: osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316] boot 2024-08-20T02:54:59.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:59 smithi044 ceph-mon[18790]: osdmap e9: 1 total, 1 up, 1 in 2024-08-20T02:54:59.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:59.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:54:59 smithi044 ceph-mon[18790]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:54:59.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:59 smithi165 ceph-mon[22357]: purged_snaps scrub starts 2024-08-20T02:54:59.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:59 smithi165 ceph-mon[22357]: purged_snaps scrub ok 2024-08-20T02:54:59.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:59 smithi165 ceph-mon[22357]: osd.0 [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316] boot 2024-08-20T02:54:59.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:59 smithi165 ceph-mon[22357]: osdmap e9: 1 total, 1 up, 1 in 2024-08-20T02:54:59.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T02:54:59.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:54:59 smithi165 ceph-mon[22357]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T02:55:00.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:00 smithi044 ceph-mon[18790]: osdmap e10: 1 total, 1 up, 1 in 2024-08-20T02:55:00.548 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:00 smithi099 ceph-mon[21984]: osdmap e10: 1 total, 1 up, 1 in 2024-08-20T02:55:00.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:00 smithi165 ceph-mon[22357]: osdmap e10: 1 total, 1 up, 1 in 2024-08-20T02:55:01.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:01 smithi044 ceph-mon[18790]: from='client.? 172.21.15.99:0/2964877377' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "12f07986-60bf-498f-a012-cd7ea607b3d3"} : dispatch 2024-08-20T02:55:01.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:01 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "12f07986-60bf-498f-a012-cd7ea607b3d3"} : dispatch 2024-08-20T02:55:01.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:01 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12f07986-60bf-498f-a012-cd7ea607b3d3"}]': finished 2024-08-20T02:55:01.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:01 smithi044 ceph-mon[18790]: osdmap e11: 2 total, 1 up, 2 in 2024-08-20T02:55:01.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:01 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:01.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:01 smithi044 ceph-mon[18790]: pgmap v30: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:01.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:01 smithi044 ceph-mon[18790]: from='client.? 172.21.15.99:0/1594405743' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:55:01.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:01 smithi165 ceph-mon[22357]: from='client.? 172.21.15.99:0/2964877377' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "12f07986-60bf-498f-a012-cd7ea607b3d3"} : dispatch 2024-08-20T02:55:01.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:01 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "12f07986-60bf-498f-a012-cd7ea607b3d3"} : dispatch 2024-08-20T02:55:01.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:01 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12f07986-60bf-498f-a012-cd7ea607b3d3"}]': finished 2024-08-20T02:55:01.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:01 smithi165 ceph-mon[22357]: osdmap e11: 2 total, 1 up, 2 in 2024-08-20T02:55:01.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:01 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:01.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:01 smithi165 ceph-mon[22357]: pgmap v30: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:01.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:01 smithi165 ceph-mon[22357]: from='client.? 172.21.15.99:0/1594405743' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:55:01.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:01 smithi099 ceph-mon[21984]: from='client.? 172.21.15.99:0/2964877377' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "12f07986-60bf-498f-a012-cd7ea607b3d3"} : dispatch 2024-08-20T02:55:01.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:01 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "12f07986-60bf-498f-a012-cd7ea607b3d3"} : dispatch 2024-08-20T02:55:01.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:01 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12f07986-60bf-498f-a012-cd7ea607b3d3"}]': finished 2024-08-20T02:55:01.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:01 smithi099 ceph-mon[21984]: osdmap e11: 2 total, 1 up, 2 in 2024-08-20T02:55:01.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:01 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:01.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:01 smithi099 ceph-mon[21984]: pgmap v30: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:01.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:01 smithi099 ceph-mon[21984]: from='client.? 172.21.15.99:0/1594405743' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:55:03.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:03 smithi165 ceph-mon[22357]: pgmap v31: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:03.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:03 smithi099 ceph-mon[21984]: pgmap v31: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:03.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:03 smithi044 ceph-mon[18790]: pgmap v31: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:05.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:05 smithi165 ceph-mon[22357]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:05.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:05 smithi099 ceph-mon[21984]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:05.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:05 smithi044 ceph-mon[18790]: pgmap v32: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:07.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:07 smithi099 ceph-mon[21984]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:07.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:07 smithi044 ceph-mon[18790]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:07.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:07 smithi165 ceph-mon[22357]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:09.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:09 smithi099 ceph-mon[21984]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:09.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-20T02:55:09.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:09.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:09 smithi099 ceph-mon[21984]: Deploying daemon osd.1 on smithi099 2024-08-20T02:55:09.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:09 smithi044 ceph-mon[18790]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:09.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-20T02:55:09.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:09.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:09 smithi044 ceph-mon[18790]: Deploying daemon osd.1 on smithi099 2024-08-20T02:55:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:09 smithi165 ceph-mon[22357]: pgmap v34: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-20T02:55:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:09 smithi165 ceph-mon[22357]: Deploying daemon osd.1 on smithi099 2024-08-20T02:55:11.633 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:11 smithi099 ceph-mon[21984]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:11.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:11 smithi044 ceph-mon[18790]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:11.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:11 smithi165 ceph-mon[22357]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:13.654 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:13 smithi099 ceph-mon[21984]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:13.688 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:13 smithi165 ceph-mon[22357]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:13.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:13 smithi044 ceph-mon[18790]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:14.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:14 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:55:14.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:14 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:14.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:14 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:14.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:14 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:55:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:14 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:14 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:14.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:14 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:55:14.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:14 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:14.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:14 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:16.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:15 smithi044 ceph-mon[18790]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:16.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:15 smithi165 ceph-mon[22357]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:16.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:15 smithi099 ceph-mon[21984]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:17.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.065+0000 7fcfecff9640 1 -- 172.21.15.99:0/2937497510 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcff00632c0 con 0x7fcfd80b3b90 2024-08-20T02:55:17.068 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 1 on host 'smithi099' 2024-08-20T02:55:17.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.068+0000 7fcff5f18640 1 -- 172.21.15.99:0/2937497510 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fcfd80b3b90 msgr2=0x7fcfd80b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:55:17.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.068+0000 7fcff5f18640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fcfd80b3b90 0x7fcfd80b6050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0031440 tx=0x7fcfe0005e80 comp rx=0 tx=0).stop 2024-08-20T02:55:17.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.068+0000 7fcff5f18640 1 -- 172.21.15.99:0/2937497510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcff0107d20 msgr2=0x7fcff019e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:55:17.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.068+0000 7fcff5f18640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcff0107d20 0x7fcff019e8c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800cce0 tx=0x7fcfe800a620 comp rx=0 tx=0).stop 2024-08-20T02:55:17.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.068+0000 7fcff5f18640 1 -- 172.21.15.99:0/2937497510 shutdown_connections 2024-08-20T02:55:17.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.068+0000 7fcff5f18640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fcfd80b3b90 0x7fcfd80b6050 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:17.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.068+0000 7fcff5f18640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcff0107d20 0x7fcff019e8c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:17.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.068+0000 7fcff5f18640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcff01073a0 0x7fcff0199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:17.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.068+0000 7fcff5f18640 1 --2- 172.21.15.99:0/2937497510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcff0101990 0x7fcff0198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:17.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.068+0000 7fcff5f18640 1 -- 172.21.15.99:0/2937497510 >> 172.21.15.99:0/2937497510 conn(0x7fcff00fd400 msgr2=0x7fcff006aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:55:17.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.068+0000 7fcff5f18640 1 -- 172.21.15.99:0/2937497510 shutdown_connections 2024-08-20T02:55:17.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:55:17.068+0000 7fcff5f18640 1 -- 172.21.15.99:0/2937497510 wait complete. 2024-08-20T02:55:17.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:17.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:17.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:17 smithi099 ceph-mon[21984]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:17.363 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:17.364 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:17.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:17 smithi044 ceph-mon[18790]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:17 smithi165 ceph-mon[22357]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:17.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:17.657 DEBUG:teuthology.orchestra.run.smithi099:osd.1> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.1.service 2024-08-20T02:55:17.659 INFO:tasks.cephadm:Deploying osd.2 on smithi165 with /dev/vg_nvme/lv_4... 2024-08-20T02:55:17.659 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-20T02:55:17.928 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:55:19.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:19 smithi099 ceph-mon[21984]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:19.561 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:19 smithi165 ceph-mon[22357]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:19.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:19 smithi044 ceph-mon[18790]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:19.915 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 02:55:19 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1[27731]: 2024-08-20T02:55:19.523+0000 7f4a0d327640 -1 osd.1 0 log_to_monitors true 2024-08-20T02:55:20.214 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:55:20.239 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_4 2024-08-20T02:55:20.494 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:55:20.559 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:20 smithi099 ceph-mon[21984]: from='osd.1 [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T02:55:20.559 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:20 smithi099 ceph-mon[21984]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T02:55:20.687 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:20 smithi165 ceph-mon[22357]: from='osd.1 [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T02:55:20.687 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:20 smithi165 ceph-mon[22357]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T02:55:20.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:20 smithi044 ceph-mon[18790]: from='osd.1 [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T02:55:20.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:20 smithi044 ceph-mon[18790]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T02:55:20.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.932+0000 7fd392839640 1 -- 172.21.15.165:0/1874357231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd38c1042c0 msgr2=0x7fd38c10c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:55:20.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.932+0000 7fd392839640 1 --2- 172.21.15.165:0/1874357231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd38c1042c0 0x7fd38c10c7b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd380009900 tx=0x7fd38002a410 comp rx=0 tx=0).stop 2024-08-20T02:55:20.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.932+0000 7fd392839640 1 -- 172.21.15.165:0/1874357231 shutdown_connections 2024-08-20T02:55:20.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.932+0000 7fd392839640 1 --2- 172.21.15.165:0/1874357231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c10ce00 0x7fd38c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:20.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.932+0000 7fd392839640 1 --2- 172.21.15.165:0/1874357231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd38c1042c0 0x7fd38c10c7b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:20.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.932+0000 7fd392839640 1 --2- 172.21.15.165:0/1874357231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38c1039a0 0x7fd38c103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:20.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.932+0000 7fd392839640 1 -- 172.21.15.165:0/1874357231 >> 172.21.15.165:0/1874357231 conn(0x7fd38c0fd400 msgr2=0x7fd38c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:55:20.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.932+0000 7fd392839640 1 -- 172.21.15.165:0/1874357231 shutdown_connections 2024-08-20T02:55:20.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.932+0000 7fd392839640 1 -- 172.21.15.165:0/1874357231 wait complete. 2024-08-20T02:55:20.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.933+0000 7fd392839640 1 Processor -- start 2024-08-20T02:55:20.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.933+0000 7fd392839640 1 -- start start 2024-08-20T02:55:20.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.933+0000 7fd392839640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd38c1039a0 0x7fd38c199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:55:20.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.933+0000 7fd392839640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38c1042c0 0x7fd38c199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:55:20.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.933+0000 7fd392839640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c10ce00 0x7fd38c19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:55:20.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.933+0000 7fd392839640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c112a70 con 0x7fd38c10ce00 2024-08-20T02:55:20.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.933+0000 7fd392839640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1128f0 con 0x7fd38c1042c0 2024-08-20T02:55:20.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.933+0000 7fd392839640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c112bf0 con 0x7fd38c1039a0 2024-08-20T02:55:20.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd38c1039a0 0x7fd38c199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:55:20.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd38c1039a0 0x7fd38c199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41762/0 (socket says 172.21.15.165:41762) 2024-08-20T02:55:20.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd38bfff640 1 -- 172.21.15.165:0/368356495 learned_addr learned my addr 172.21.15.165:0/368356495 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-08-20T02:55:20.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd390dd6640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c10ce00 0x7fd38c19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd38b7fe640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38c1042c0 0x7fd38c199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd38bfff640 1 -- 172.21.15.165:0/368356495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38c1042c0 msgr2=0x7fd38c199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd38bfff640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38c1042c0 0x7fd38c199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd38bfff640 1 -- 172.21.15.165:0/368356495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c10ce00 msgr2=0x7fd38c19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd38bfff640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c10ce00 0x7fd38c19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd38bfff640 1 -- 172.21.15.165:0/368356495 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c19f0b0 con 0x7fd38c1039a0 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd38bfff640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd38c1039a0 0x7fd38c199000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00e9f0 tx=0x7fd37c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd3897fa640 1 -- 172.21.15.165:0/368356495 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c00cd80 con 0x7fd38c1039a0 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.934+0000 7fd3897fa640 1 -- 172.21.15.165:0/368356495 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd37c0054d0 con 0x7fd38c1039a0 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.935+0000 7fd3897fa640 1 -- 172.21.15.165:0/368356495 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c010690 con 0x7fd38c1039a0 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.935+0000 7fd392839640 1 -- 172.21.15.165:0/368356495 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38c19f3a0 con 0x7fd38c1039a0 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.935+0000 7fd392839640 1 -- 172.21.15.165:0/368356495 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38c1a5a60 con 0x7fd38c1039a0 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.936+0000 7fd392839640 1 -- 172.21.15.165:0/368356495 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38c103d80 con 0x7fd38c1039a0 2024-08-20T02:55:20.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.939+0000 7fd3897fa640 1 -- 172.21.15.165:0/368356495 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7fd37c004e30 con 0x7fd38c1039a0 2024-08-20T02:55:20.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.939+0000 7fd3897fa640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd3740b3c10 0x7fd3740b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:55:20.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.939+0000 7fd3897fa640 1 -- 172.21.15.165:0/368356495 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2118+0+0 (secure 0 0 0) 0x7fd37c0e2890 con 0x7fd38c1039a0 2024-08-20T02:55:20.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.940+0000 7fd38b7fe640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd3740b3c10 0x7fd3740b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:55:20.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.940+0000 7fd38b7fe640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd3740b3c10 0x7fd3740b60d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd380000c00 tx=0x7fd380031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:55:20.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:20.942+0000 7fd3897fa640 1 -- 172.21.15.165:0/368356495 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd37c055e20 con 0x7fd38c1039a0 2024-08-20T02:55:21.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:21.134+0000 7fd392839640 1 -- 172.21.15.165:0/368356495 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd38c0632c0 con 0x7fd3740b3c10 2024-08-20T02:55:21.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:21.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-20T02:55:21.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='osd.1 [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:55:21.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: osdmap e12: 2 total, 1 up, 2 in 2024-08-20T02:55:21.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:21.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:55:21.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: Detected new or changed devices on smithi099 2024-08-20T02:55:21.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T02:55:21.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi099 to 17088M 2024-08-20T02:55:21.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:21.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:55:21.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:55:21.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:21.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:55:21.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:55:21.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:55:21.667 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:55:21.667 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:21.672 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:21.672 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-20T02:55:21.673 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='osd.1 [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:55:21.673 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: osdmap e12: 2 total, 1 up, 2 in 2024-08-20T02:55:21.673 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:21.673 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:55:21.673 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: Detected new or changed devices on smithi099 2024-08-20T02:55:21.673 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.673 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.673 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T02:55:21.673 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi099 to 17088M 2024-08-20T02:55:21.673 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.673 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:21.673 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:55:21.674 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.674 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:55:21.674 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:21.674 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:55:21.674 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.674 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:55:21.674 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:55:21.674 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:55:21.674 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:21.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:21.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-20T02:55:21.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='osd.1 [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:55:21.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: osdmap e12: 2 total, 1 up, 2 in 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: Detected new or changed devices on smithi099 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi099 to 17088M 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:55:21.688 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:21.689 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:55:21.689 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:21.689 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:55:21.689 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:55:21.689 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:55:21.689 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:22.164 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1[27731]: 2024-08-20T02:55:21.771+0000 7f4a092aa640 -1 osd.1 0 waiting for initial osdmap 2024-08-20T02:55:22.165 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 02:55:21 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1[27731]: 2024-08-20T02:55:21.774+0000 7f4a010c9640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T02:55:22.632 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:22 smithi165 ceph-mon[22357]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-08-20T02:55:22.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:22 smithi165 ceph-mon[22357]: osdmap e13: 2 total, 1 up, 2 in 2024-08-20T02:55:22.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:22 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:22.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:22 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:22.633 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:22 smithi165 ceph-mon[22357]: from='osd.1 ' entity='osd.1' 2024-08-20T02:55:22.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:22 smithi099 ceph-mon[21984]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-08-20T02:55:22.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:22 smithi099 ceph-mon[21984]: osdmap e13: 2 total, 1 up, 2 in 2024-08-20T02:55:22.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:22 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:22.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:22 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:22.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:22 smithi099 ceph-mon[21984]: from='osd.1 ' entity='osd.1' 2024-08-20T02:55:22.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:22 smithi044 ceph-mon[18790]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-08-20T02:55:22.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:22 smithi044 ceph-mon[18790]: osdmap e13: 2 total, 1 up, 2 in 2024-08-20T02:55:22.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:22 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:22.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:22 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:22.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:22 smithi044 ceph-mon[18790]: from='osd.1 ' entity='osd.1' 2024-08-20T02:55:23.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:23 smithi099 ceph-mon[21984]: purged_snaps scrub starts 2024-08-20T02:55:23.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:23 smithi099 ceph-mon[21984]: purged_snaps scrub ok 2024-08-20T02:55:23.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:23 smithi099 ceph-mon[21984]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:23.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:23 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:23.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:23 smithi099 ceph-mon[21984]: osd.1 [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663] boot 2024-08-20T02:55:23.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:23 smithi099 ceph-mon[21984]: osdmap e14: 2 total, 2 up, 2 in 2024-08-20T02:55:23.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:23 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:23.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:23 smithi099 ceph-mon[21984]: from='client.? 172.21.15.165:0/4188722464' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f189000-ba7f-4412-b0d3-75e8265f6685"} : dispatch 2024-08-20T02:55:23.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:23 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f189000-ba7f-4412-b0d3-75e8265f6685"} : dispatch 2024-08-20T02:55:23.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:23 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f189000-ba7f-4412-b0d3-75e8265f6685"}]': finished 2024-08-20T02:55:23.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:23 smithi099 ceph-mon[21984]: osdmap e15: 3 total, 2 up, 3 in 2024-08-20T02:55:23.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:23 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:23.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:23 smithi044 ceph-mon[18790]: purged_snaps scrub starts 2024-08-20T02:55:23.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:23 smithi044 ceph-mon[18790]: purged_snaps scrub ok 2024-08-20T02:55:23.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:23 smithi044 ceph-mon[18790]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:23.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:23 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:23.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:23 smithi044 ceph-mon[18790]: osd.1 [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663] boot 2024-08-20T02:55:23.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:23 smithi044 ceph-mon[18790]: osdmap e14: 2 total, 2 up, 2 in 2024-08-20T02:55:23.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:23 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:23.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:23 smithi044 ceph-mon[18790]: from='client.? 172.21.15.165:0/4188722464' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f189000-ba7f-4412-b0d3-75e8265f6685"} : dispatch 2024-08-20T02:55:23.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:23 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f189000-ba7f-4412-b0d3-75e8265f6685"} : dispatch 2024-08-20T02:55:23.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:23 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f189000-ba7f-4412-b0d3-75e8265f6685"}]': finished 2024-08-20T02:55:23.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:23 smithi044 ceph-mon[18790]: osdmap e15: 3 total, 2 up, 3 in 2024-08-20T02:55:23.787 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:23 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:23.865 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:23 smithi165 ceph-mon[22357]: purged_snaps scrub starts 2024-08-20T02:55:23.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:23 smithi165 ceph-mon[22357]: purged_snaps scrub ok 2024-08-20T02:55:23.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:23 smithi165 ceph-mon[22357]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T02:55:23.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:23 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:23.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:23 smithi165 ceph-mon[22357]: osd.1 [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663] boot 2024-08-20T02:55:23.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:23 smithi165 ceph-mon[22357]: osdmap e14: 2 total, 2 up, 2 in 2024-08-20T02:55:23.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:23 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T02:55:23.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:23 smithi165 ceph-mon[22357]: from='client.? 172.21.15.165:0/4188722464' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f189000-ba7f-4412-b0d3-75e8265f6685"} : dispatch 2024-08-20T02:55:23.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:23 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0f189000-ba7f-4412-b0d3-75e8265f6685"} : dispatch 2024-08-20T02:55:23.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:23 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f189000-ba7f-4412-b0d3-75e8265f6685"}]': finished 2024-08-20T02:55:23.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:23 smithi165 ceph-mon[22357]: osdmap e15: 3 total, 2 up, 3 in 2024-08-20T02:55:23.866 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:23 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:24.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:24 smithi099 ceph-mon[21984]: from='client.? 172.21.15.165:0/1398203156' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:55:24.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:24 smithi044 ceph-mon[18790]: from='client.? 172.21.15.165:0/1398203156' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:55:24.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:24 smithi165 ceph-mon[22357]: from='client.? 172.21.15.165:0/1398203156' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:55:25.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:25 smithi099 ceph-mon[21984]: pgmap v46: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:25.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:25 smithi099 ceph-mon[21984]: osdmap e16: 3 total, 2 up, 3 in 2024-08-20T02:55:25.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:25.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:25 smithi044 ceph-mon[18790]: pgmap v46: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:25.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:25 smithi044 ceph-mon[18790]: osdmap e16: 3 total, 2 up, 3 in 2024-08-20T02:55:25.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:25.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:25 smithi165 ceph-mon[22357]: pgmap v46: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:25.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:25 smithi165 ceph-mon[22357]: osdmap e16: 3 total, 2 up, 3 in 2024-08-20T02:55:25.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:27.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:27 smithi099 ceph-mon[21984]: pgmap v48: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:27.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:27 smithi044 ceph-mon[18790]: pgmap v48: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:27.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:27 smithi165 ceph-mon[22357]: pgmap v48: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:29.699 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:29 smithi165 ceph-mon[22357]: pgmap v49: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:29.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:29 smithi044 ceph-mon[18790]: pgmap v49: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:29.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:29 smithi099 ceph-mon[21984]: pgmap v49: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:31.434 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:31 smithi165 ceph-mon[22357]: pgmap v50: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:31.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:31 smithi044 ceph-mon[18790]: pgmap v50: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:31.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:31 smithi099 ceph-mon[21984]: pgmap v50: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:32.740 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:32 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-20T02:55:32.740 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:32 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:32.740 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:32 smithi165 ceph-mon[22357]: Deploying daemon osd.2 on smithi165 2024-08-20T02:55:32.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:32 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-20T02:55:32.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:32 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:32.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:32 smithi044 ceph-mon[18790]: Deploying daemon osd.2 on smithi165 2024-08-20T02:55:32.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:32 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-20T02:55:32.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:32 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:32.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:32 smithi099 ceph-mon[21984]: Deploying daemon osd.2 on smithi165 2024-08-20T02:55:33.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:33 smithi165 ceph-mon[22357]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:33.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:33 smithi044 ceph-mon[18790]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:33.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:33 smithi099 ceph-mon[21984]: pgmap v51: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:35.546 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:35 smithi165 ceph-mon[22357]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:35.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:35 smithi044 ceph-mon[18790]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:35.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:35 smithi099 ceph-mon[21984]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:55:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:36.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:55:36.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:36.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:36.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:55:36.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:36.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:37.732 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:37 smithi165 ceph-mon[22357]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:37.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:37 smithi044 ceph-mon[18790]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:37.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:37 smithi099 ceph-mon[21984]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:40.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:39 smithi165 ceph-mon[22357]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:40.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:39 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:40.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:39 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:40.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:39 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:40.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:39 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:55:40.025 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:39 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:40.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:39 smithi099 ceph-mon[21984]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:40.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:39 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:40.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:39 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:40.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:39 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:40.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:39 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:55:40.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:39 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:40.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:39 smithi044 ceph-mon[18790]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:40.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:39 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:40.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:39 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:40.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:39 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:40.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:39 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:55:40.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:39 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:40.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.322+0000 7fd3897fa640 1 -- 172.21.15.165:0/368356495 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd38c0632c0 con 0x7fd3740b3c10 2024-08-20T02:55:40.326 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 2 on host 'smithi165' 2024-08-20T02:55:40.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.325+0000 7fd392839640 1 -- 172.21.15.165:0/368356495 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd3740b3c10 msgr2=0x7fd3740b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:55:40.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.325+0000 7fd392839640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd3740b3c10 0x7fd3740b60d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd380000c00 tx=0x7fd380031400 comp rx=0 tx=0).stop 2024-08-20T02:55:40.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.325+0000 7fd392839640 1 -- 172.21.15.165:0/368356495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd38c1039a0 msgr2=0x7fd38c199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:55:40.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.325+0000 7fd392839640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd38c1039a0 0x7fd38c199000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00e9f0 tx=0x7fd37c00edf0 comp rx=0 tx=0).stop 2024-08-20T02:55:40.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.325+0000 7fd392839640 1 -- 172.21.15.165:0/368356495 shutdown_connections 2024-08-20T02:55:40.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.325+0000 7fd392839640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd3740b3c10 0x7fd3740b60d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:40.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.325+0000 7fd392839640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd38c10ce00 0x7fd38c19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:40.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.325+0000 7fd392839640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38c1042c0 0x7fd38c199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:40.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.325+0000 7fd392839640 1 --2- 172.21.15.165:0/368356495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd38c1039a0 0x7fd38c199000 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:40.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.325+0000 7fd392839640 1 -- 172.21.15.165:0/368356495 >> 172.21.15.165:0/368356495 conn(0x7fd38c0fd400 msgr2=0x7fd38c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:55:40.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.325+0000 7fd392839640 1 -- 172.21.15.165:0/368356495 shutdown_connections 2024-08-20T02:55:40.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:55:40.325+0000 7fd392839640 1 -- 172.21.15.165:0/368356495 wait complete. 2024-08-20T02:55:40.863 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:55:40.863 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:40.863 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:40.890 DEBUG:teuthology.orchestra.run.smithi165:osd.2> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.2.service 2024-08-20T02:55:40.892 INFO:tasks.cephadm:Deploying osd.3 on smithi044 with /dev/vg_nvme/lv_3... 2024-08-20T02:55:40.892 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-20T02:55:41.154 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:55:41.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:55:41.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:41.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:41.275 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:55:41.275 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:41.275 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:42.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:41 smithi044 ceph-mon[18790]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:42.065 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:41 smithi099 ceph-mon[21984]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:42.066 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:41 smithi165 ceph-mon[22357]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:42.636 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 02:55:42 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2[28587]: 2024-08-20T02:55:42.393+0000 7f84323e7640 -1 osd.2 0 log_to_monitors true 2024-08-20T02:55:43.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:42 smithi165 ceph-mon[22357]: from='osd.2 [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T02:55:43.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:42 smithi165 ceph-mon[22357]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T02:55:43.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:42 smithi099 ceph-mon[21984]: from='osd.2 [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T02:55:43.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:42 smithi099 ceph-mon[21984]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T02:55:43.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:42 smithi044 ceph-mon[18790]: from='osd.2 [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T02:55:43.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:42 smithi044 ceph-mon[18790]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T02:55:43.619 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:55:43.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_3 2024-08-20T02:55:43.901 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:55:44.136 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2[28587]: 2024-08-20T02:55:43.813+0000 7f842e36a640 -1 osd.2 0 waiting for initial osdmap 2024-08-20T02:55:44.136 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2[28587]: 2024-08-20T02:55:43.817+0000 7f842a191640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T02:55:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-20T02:55:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: osdmap e17: 3 total, 2 up, 3 in 2024-08-20T02:55:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: from='osd.2 [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:55:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:55:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T02:55:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:55:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:44.144 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:44.144 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-20T02:55:44.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: osdmap e17: 3 total, 2 up, 3 in 2024-08-20T02:55:44.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:44.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: from='osd.2 [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:55:44.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:55:44.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:44.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:44.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T02:55:44.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:44.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:44.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:55:44.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:44.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:44.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-20T02:55:44.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: osdmap e17: 3 total, 2 up, 3 in 2024-08-20T02:55:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: from='osd.2 [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:55:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:55:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T02:55:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:55:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.352+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/2299294597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98108e20 msgr2=0x7f8b9810f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:55:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.352+0000 7f8b9f1d6640 1 --2- 172.21.15.44:0/2299294597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98108e20 0x7f8b9810f620 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009ca0 tx=0x7f8b9002a8c0 comp rx=0 tx=0).stop 2024-08-20T02:55:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.352+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/2299294597 shutdown_connections 2024-08-20T02:55:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.352+0000 7f8b9f1d6640 1 --2- 172.21.15.44:0/2299294597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98108e20 0x7f8b9810f620 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.352+0000 7f8b9f1d6640 1 --2- 172.21.15.44:0/2299294597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b981084a0 0x7f8b981088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.352+0000 7f8b9f1d6640 1 --2- 172.21.15.44:0/2299294597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b98102a90 0x7f8b98102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.352+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/2299294597 >> 172.21.15.44:0/2299294597 conn(0x7f8b980fe540 msgr2=0x7f8b98100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:55:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.352+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/2299294597 shutdown_connections 2024-08-20T02:55:44.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.352+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/2299294597 wait complete. 2024-08-20T02:55:44.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.352+0000 7f8b9f1d6640 1 Processor -- start 2024-08-20T02:55:44.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.352+0000 7f8b9f1d6640 1 -- start start 2024-08-20T02:55:44.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9f1d6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98102a90 0x7f8b98198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:55:44.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9f1d6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b981084a0 0x7f8b98199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:55:44.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9f1d6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b98108e20 0x7f8b9819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:55:44.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9f1d6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98112aa0 con 0x7f8b98102a90 2024-08-20T02:55:44.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9f1d6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98112920 con 0x7f8b98108e20 2024-08-20T02:55:44.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9f1d6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98112c20 con 0x7f8b981084a0 2024-08-20T02:55:44.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9cf72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98102a90 0x7f8b98198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:55:44.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9cf72640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98102a90 0x7f8b98198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57494/0 (socket says 172.21.15.44:57494) 2024-08-20T02:55:44.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9cf72640 1 -- 172.21.15.44:0/3955912358 learned_addr learned my addr 172.21.15.44:0/3955912358 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:55:44.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9d773640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b98108e20 0x7f8b9819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:55:44.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b97fff640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b981084a0 0x7f8b98199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:55:44.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9cf72640 1 -- 172.21.15.44:0/3955912358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b981084a0 msgr2=0x7f8b98199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:55:44.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9cf72640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b981084a0 0x7f8b98199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:44.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9cf72640 1 -- 172.21.15.44:0/3955912358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b98108e20 msgr2=0x7f8b9819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:55:44.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9cf72640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b98108e20 0x7f8b9819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:55:44.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9cf72640 1 -- 172.21.15.44:0/3955912358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9819eee0 con 0x7f8b98102a90 2024-08-20T02:55:44.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.353+0000 7f8b9cf72640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98102a90 0x7f8b98198e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00c920 tx=0x7f8b8c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:55:44.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.354+0000 7f8b95ffb640 1 -- 172.21.15.44:0/3955912358 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c007bb0 con 0x7f8b98102a90 2024-08-20T02:55:44.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.354+0000 7f8b95ffb640 1 -- 172.21.15.44:0/3955912358 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b8c007d50 con 0x7f8b98102a90 2024-08-20T02:55:44.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.354+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/3955912358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9819f1d0 con 0x7f8b98102a90 2024-08-20T02:55:44.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.354+0000 7f8b95ffb640 1 -- 172.21.15.44:0/3955912358 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c018730 con 0x7f8b98102a90 2024-08-20T02:55:44.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.354+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/3955912358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b981a5890 con 0x7f8b98102a90 2024-08-20T02:55:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.355+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/3955912358 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b98102e70 con 0x7f8b98102a90 2024-08-20T02:55:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.356+0000 7f8b95ffb640 1 -- 172.21.15.44:0/3955912358 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) ==== 151020+0+0 (secure 0 0 0) 0x7f8b8c00f450 con 0x7f8b98102a90 2024-08-20T02:55:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.357+0000 7f8b95ffb640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8b880b3b90 0x7f8b880b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:55:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.358+0000 7f8b97fff640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8b880b3b90 0x7f8b880b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:55:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.358+0000 7f8b95ffb640 1 -- 172.21.15.44:0/3955912358 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2786+0+0 (secure 0 0 0) 0x7f8b8c0e26e0 con 0x7f8b98102a90 2024-08-20T02:55:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.358+0000 7f8b97fff640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8b880b3b90 0x7f8b880b6050 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b98100e20 tx=0x7f8b8400a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:55:44.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.361+0000 7f8b95ffb640 1 -- 172.21.15.44:0/3955912358 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8b8c014030 con 0x7f8b98102a90 2024-08-20T02:55:44.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:44.555+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/3955912358 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8b980ff2e0 con 0x7f8b880b3b90 2024-08-20T02:55:44.826 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:44 smithi044 ceph-mon[18790]: Detected new or changed devices on smithi165 2024-08-20T02:55:44.826 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:44 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi165 to 17088M 2024-08-20T02:55:44.826 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:44 smithi044 ceph-mon[18790]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-08-20T02:55:44.826 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:44 smithi044 ceph-mon[18790]: osdmap e18: 3 total, 2 up, 3 in 2024-08-20T02:55:44.826 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:44 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:44.826 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:44 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:44.826 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:44 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:55:44.827 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:44 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:55:44.827 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:44 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:44.827 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:44 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:44.932 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:44 smithi165 ceph-mon[22357]: Detected new or changed devices on smithi165 2024-08-20T02:55:44.932 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:44 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi165 to 17088M 2024-08-20T02:55:44.932 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:44 smithi165 ceph-mon[22357]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-08-20T02:55:44.932 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:44 smithi165 ceph-mon[22357]: osdmap e18: 3 total, 2 up, 3 in 2024-08-20T02:55:44.933 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:44 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:44.933 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:44 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:44.933 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:44 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:55:44.933 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:44 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:55:44.933 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:44 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:44.933 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:44 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:45.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:44 smithi099 ceph-mon[21984]: Detected new or changed devices on smithi165 2024-08-20T02:55:45.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:44 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi165 to 17088M 2024-08-20T02:55:45.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:44 smithi099 ceph-mon[21984]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-08-20T02:55:45.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:44 smithi099 ceph-mon[21984]: osdmap e18: 3 total, 2 up, 3 in 2024-08-20T02:55:45.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:44 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:45.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:44 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:45.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:44 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:55:45.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:44 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:55:45.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:44 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:45.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:44 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:46.027 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:45 smithi044 ceph-mon[18790]: purged_snaps scrub starts 2024-08-20T02:55:46.027 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:45 smithi044 ceph-mon[18790]: purged_snaps scrub ok 2024-08-20T02:55:46.027 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:45 smithi044 ceph-mon[18790]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:46.027 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:45 smithi044 ceph-mon[18790]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:55:46.027 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:45 smithi044 ceph-mon[18790]: osd.2 [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465] boot 2024-08-20T02:55:46.027 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:45 smithi044 ceph-mon[18790]: osdmap e19: 3 total, 3 up, 3 in 2024-08-20T02:55:46.028 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:45 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:46.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:45 smithi165 ceph-mon[22357]: purged_snaps scrub starts 2024-08-20T02:55:46.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:45 smithi165 ceph-mon[22357]: purged_snaps scrub ok 2024-08-20T02:55:46.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:45 smithi165 ceph-mon[22357]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:46.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:45 smithi165 ceph-mon[22357]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:55:46.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:45 smithi165 ceph-mon[22357]: osd.2 [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465] boot 2024-08-20T02:55:46.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:45 smithi165 ceph-mon[22357]: osdmap e19: 3 total, 3 up, 3 in 2024-08-20T02:55:46.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:45 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:46.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:45 smithi099 ceph-mon[21984]: purged_snaps scrub starts 2024-08-20T02:55:46.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:45 smithi099 ceph-mon[21984]: purged_snaps scrub ok 2024-08-20T02:55:46.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:45 smithi099 ceph-mon[21984]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T02:55:46.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:45 smithi099 ceph-mon[21984]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:55:46.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:45 smithi099 ceph-mon[21984]: osd.2 [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465] boot 2024-08-20T02:55:46.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:45 smithi099 ceph-mon[21984]: osdmap e19: 3 total, 3 up, 3 in 2024-08-20T02:55:46.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:45 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T02:55:47.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:46 smithi044 ceph-mon[18790]: osdmap e20: 3 total, 3 up, 3 in 2024-08-20T02:55:47.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:46 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-20T02:55:47.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:46 smithi165 ceph-mon[22357]: osdmap e20: 3 total, 3 up, 3 in 2024-08-20T02:55:47.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:46 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-20T02:55:47.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:46 smithi099 ceph-mon[21984]: osdmap e20: 3 total, 3 up, 3 in 2024-08-20T02:55:47.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:46 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-20T02:55:47.375 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41462]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T02:55:47.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41462]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T02:55:47.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41462]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T02:55:47.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41462]: pam_unix(sudo:session): session closed for user root 2024-08-20T02:55:47.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41470]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T02:55:47.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41470]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T02:55:47.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41470]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T02:55:47.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41470]: pam_unix(sudo:session): session closed for user root 2024-08-20T02:55:47.376 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41453]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T02:55:47.376 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41453]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T02:55:47.376 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41453]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T02:55:47.377 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41453]: pam_unix(sudo:session): session closed for user root 2024-08-20T02:55:47.377 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41458]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T02:55:47.377 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41458]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T02:55:47.377 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41458]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T02:55:47.377 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 02:55:47 smithi044 sudo[41458]: pam_unix(sudo:session): session closed for user root 2024-08-20T02:55:47.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32214]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T02:55:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32214]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T02:55:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32214]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T02:55:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32214]: pam_unix(sudo:session): session closed for user root 2024-08-20T02:55:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32217]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T02:55:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32217]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T02:55:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32217]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T02:55:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32217]: pam_unix(sudo:session): session closed for user root 2024-08-20T02:55:47.637 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32208]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T02:55:47.637 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32208]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T02:55:47.637 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32208]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T02:55:47.638 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32208]: pam_unix(sudo:session): session closed for user root 2024-08-20T02:55:47.638 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32211]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T02:55:47.638 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32211]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T02:55:47.638 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32211]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T02:55:47.638 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 02:55:47 smithi165 sudo[32211]: pam_unix(sudo:session): session closed for user root 2024-08-20T02:55:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31987]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T02:55:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31987]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T02:55:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31987]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T02:55:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31987]: pam_unix(sudo:session): session closed for user root 2024-08-20T02:55:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31990]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T02:55:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31990]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T02:55:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31990]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T02:55:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31990]: pam_unix(sudo:session): session closed for user root 2024-08-20T02:55:47.665 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31981]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T02:55:47.666 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31981]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T02:55:47.666 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31981]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T02:55:47.666 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31981]: pam_unix(sudo:session): session closed for user root 2024-08-20T02:55:47.666 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31984]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T02:55:47.666 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31984]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T02:55:47.666 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31984]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T02:55:47.666 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 02:55:47 smithi099 sudo[31984]: pam_unix(sudo:session): session closed for user root 2024-08-20T02:55:48.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: pgmap v62: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:48.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-20T02:55:48.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: osdmap e21: 3 total, 3 up, 3 in 2024-08-20T02:55:48.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-20T02:55:48.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1831475974' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8f9b042c-d6b5-4eee-aa29-ffc41a48c20a"} : dispatch 2024-08-20T02:55:48.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-20T02:55:48.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1831475974' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f9b042c-d6b5-4eee-aa29-ffc41a48c20a"}]': finished 2024-08-20T02:55:48.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: osdmap e22: 4 total, 3 up, 4 in 2024-08-20T02:55:48.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:55:48.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:48.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:48.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:48.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T02:55:48.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T02:55:48.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:55:48.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:55:48.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:55:48.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T02:55:48.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:55:48.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:55:48.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:55:48.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T02:55:48.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T02:55:48.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:55:48.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:55:48.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:55:48.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T02:55:48.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:47 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2626071318' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:55:48.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: pgmap v62: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:48.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-20T02:55:48.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: osdmap e21: 3 total, 3 up, 3 in 2024-08-20T02:55:48.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-20T02:55:48.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1831475974' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8f9b042c-d6b5-4eee-aa29-ffc41a48c20a"} : dispatch 2024-08-20T02:55:48.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-20T02:55:48.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1831475974' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f9b042c-d6b5-4eee-aa29-ffc41a48c20a"}]': finished 2024-08-20T02:55:48.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: osdmap e22: 4 total, 3 up, 4 in 2024-08-20T02:55:48.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:55:48.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:48.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:48.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:55:48.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:55:48.139 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:55:48.139 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T02:55:48.139 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:47 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2626071318' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:55:48.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: pgmap v62: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:48.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-20T02:55:48.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: osdmap e21: 3 total, 3 up, 3 in 2024-08-20T02:55:48.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-20T02:55:48.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1831475974' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8f9b042c-d6b5-4eee-aa29-ffc41a48c20a"} : dispatch 2024-08-20T02:55:48.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-20T02:55:48.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1831475974' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f9b042c-d6b5-4eee-aa29-ffc41a48c20a"}]': finished 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: osdmap e22: 4 total, 3 up, 4 in 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:55:48.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:55:48.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T02:55:48.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T02:55:48.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T02:55:48.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T02:55:48.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T02:55:48.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T02:55:48.167 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:47 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2626071318' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:55:49.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:55:49.058+0000 7f8b95ffb640 1 -- 172.21.15.44:0/3955912358 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f8b8c0a6e90 con 0x7f8b98102a90 2024-08-20T02:55:49.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:49 smithi165 ceph-mon[22357]: osdmap e23: 4 total, 3 up, 4 in 2024-08-20T02:55:49.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:49 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:55:49.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:49 smithi099 ceph-mon[21984]: osdmap e23: 4 total, 3 up, 4 in 2024-08-20T02:55:49.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:49 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:55:49.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:49 smithi044 ceph-mon[18790]: osdmap e23: 4 total, 3 up, 4 in 2024-08-20T02:55:49.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:49 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:55:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:50 smithi165 ceph-mon[22357]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:50 smithi165 ceph-mon[22357]: mgrmap e20: x(active, since 110s), standbys: z, y 2024-08-20T02:55:50.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:50 smithi099 ceph-mon[21984]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:50.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:50 smithi099 ceph-mon[21984]: mgrmap e20: x(active, since 110s), standbys: z, y 2024-08-20T02:55:50.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:50 smithi044 ceph-mon[18790]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:50.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:50 smithi044 ceph-mon[18790]: mgrmap e20: x(active, since 110s), standbys: z, y 2024-08-20T02:55:52.223 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:52 smithi044 ceph-mon[18790]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:52.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:52 smithi165 ceph-mon[22357]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:52.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:52 smithi099 ceph-mon[21984]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:54.327 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:54 smithi044 ceph-mon[18790]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:54.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:54 smithi165 ceph-mon[22357]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:54.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:54 smithi099 ceph-mon[21984]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:56.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:56 smithi165 ceph-mon[22357]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:56.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:56 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-20T02:55:56.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:56 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:56.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:56 smithi044 ceph-mon[18790]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:56.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:56 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-20T02:55:56.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:56 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:56.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:56 smithi099 ceph-mon[21984]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:56.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:56 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-20T02:55:56.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:56 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:55:57.242 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:57 smithi044 ceph-mon[18790]: Deploying daemon osd.3 on smithi044 2024-08-20T02:55:57.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:57 smithi165 ceph-mon[22357]: Deploying daemon osd.3 on smithi044 2024-08-20T02:55:57.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:57 smithi099 ceph-mon[21984]: Deploying daemon osd.3 on smithi044 2024-08-20T02:55:58.266 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:55:58 smithi044 ceph-mon[18790]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:58.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:55:58 smithi165 ceph-mon[22357]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:55:58.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:55:58 smithi099 ceph-mon[21984]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:00.359 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:00 smithi044 ceph-mon[18790]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:00.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:00 smithi165 ceph-mon[22357]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:00.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:00 smithi099 ceph-mon[21984]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:01.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:01 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:01.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:01 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:01.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:01 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:01.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:01 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:01.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:01 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:01.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:01 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:01.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:01 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:01 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:01 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:02.348 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:02 smithi044 ceph-mon[18790]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:02.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:02 smithi165 ceph-mon[22357]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:02.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:02 smithi099 ceph-mon[21984]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:04.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:04 smithi099 ceph-mon[21984]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:04 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:04 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:04 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:04 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:04 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:04.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:04 smithi044 ceph-mon[18790]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:04.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:04 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:04.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:04 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:04.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:04 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:04.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:04 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:04.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:04 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:04 smithi165 ceph-mon[22357]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:04 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:04 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:04 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:04.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:04 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:04.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:04 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:04.963 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 3 on host 'smithi044' 2024-08-20T02:56:04.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.960+0000 7f8b95ffb640 1 -- 172.21.15.44:0/3955912358 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8b980ff2e0 con 0x7f8b880b3b90 2024-08-20T02:56:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.962+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/3955912358 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8b880b3b90 msgr2=0x7f8b880b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.962+0000 7f8b9f1d6640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8b880b3b90 0x7f8b880b6050 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b98100e20 tx=0x7f8b8400a2d0 comp rx=0 tx=0).stop 2024-08-20T02:56:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.962+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/3955912358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98102a90 msgr2=0x7f8b98198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.962+0000 7f8b9f1d6640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98102a90 0x7f8b98198e30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00c920 tx=0x7f8b8c00cd20 comp rx=0 tx=0).stop 2024-08-20T02:56:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.962+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/3955912358 shutdown_connections 2024-08-20T02:56:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.962+0000 7f8b9f1d6640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8b880b3b90 0x7f8b880b6050 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.962+0000 7f8b9f1d6640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b98108e20 0x7f8b9819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.962+0000 7f8b9f1d6640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b981084a0 0x7f8b98199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:04.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.962+0000 7f8b9f1d6640 1 --2- 172.21.15.44:0/3955912358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b98102a90 0x7f8b98198e30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:04.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.962+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/3955912358 >> 172.21.15.44:0/3955912358 conn(0x7f8b980fe540 msgr2=0x7f8b980febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:56:04.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.963+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/3955912358 shutdown_connections 2024-08-20T02:56:04.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:04.963+0000 7f8b9f1d6640 1 -- 172.21.15.44:0/3955912358 wait complete. 2024-08-20T02:56:05.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:05 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:05.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:05 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:05.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:05 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:05.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:05 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:05.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:05 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:05.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:05 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:05.605 DEBUG:teuthology.orchestra.run.smithi044:osd.3> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.3.service 2024-08-20T02:56:05.607 INFO:tasks.cephadm:Deploying osd.4 on smithi099 with /dev/vg_nvme/lv_3... 2024-08-20T02:56:05.607 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-20T02:56:05.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:05 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:05.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:05 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:05.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:05 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:05.865 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T02:56:06.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:06 smithi044 ceph-mon[18790]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:06.307 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:06 smithi099 ceph-mon[21984]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:06.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:06 smithi165 ceph-mon[22357]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:07.536 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 02:56:07 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3[43469]: 2024-08-20T02:56:07.320+0000 7f9400ac7640 -1 osd.3 0 log_to_monitors true 2024-08-20T02:56:08.359 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:08 smithi099 ceph-mon[21984]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:08.360 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:08 smithi099 ceph-mon[21984]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-20T02:56:08.362 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:56:08.387 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_3 2024-08-20T02:56:08.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:08 smithi044 ceph-mon[18790]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:08.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:08 smithi044 ceph-mon[18790]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-20T02:56:08.634 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T02:56:08.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:08 smithi165 ceph-mon[22357]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:08.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:08 smithi165 ceph-mon[22357]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-20T02:56:09.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.095+0000 7f4243173640 1 -- 172.21.15.99:0/1850450639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c10f0b0 msgr2=0x7f423c1114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:09.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.095+0000 7f4243173640 1 --2- 172.21.15.99:0/1850450639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c10f0b0 0x7f423c1114a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4234009ca0 tx=0x7f423402a8c0 comp rx=0 tx=0).stop 2024-08-20T02:56:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.095+0000 7f4243173640 1 -- 172.21.15.99:0/1850450639 shutdown_connections 2024-08-20T02:56:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.095+0000 7f4243173640 1 --2- 172.21.15.99:0/1850450639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c10f0b0 0x7f423c1114a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.095+0000 7f4243173640 1 --2- 172.21.15.99:0/1850450639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f423c075b10 0x7f423c10eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.095+0000 7f4243173640 1 --2- 172.21.15.99:0/1850450639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423c0751f0 0x7f423c0755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.095+0000 7f4243173640 1 -- 172.21.15.99:0/1850450639 >> 172.21.15.99:0/1850450639 conn(0x7f423c0fe550 msgr2=0x7f423c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:56:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.095+0000 7f4243173640 1 -- 172.21.15.99:0/1850450639 shutdown_connections 2024-08-20T02:56:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.095+0000 7f4243173640 1 -- 172.21.15.99:0/1850450639 wait complete. 2024-08-20T02:56:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f4243173640 1 Processor -- start 2024-08-20T02:56:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f4243173640 1 -- start start 2024-08-20T02:56:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f4243173640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423c0751f0 0x7f423c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:56:09.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f4243173640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f423c075b10 0x7f423c19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:56:09.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f4243173640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c10f0b0 0x7f423c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:56:09.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f4243173640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f423c114c90 con 0x7f423c0751f0 2024-08-20T02:56:09.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f4243173640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f423c114b10 con 0x7f423c10f0b0 2024-08-20T02:56:09.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f4243173640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f423c114e10 con 0x7f423c075b10 2024-08-20T02:56:09.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f4241710640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c10f0b0 0x7f423c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:56:09.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f4241710640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c10f0b0 0x7f423c1a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50742/0 (socket says 172.21.15.99:50742) 2024-08-20T02:56:09.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f4241710640 1 -- 172.21.15.99:0/1245413537 learned_addr learned my addr 172.21.15.99:0/1245413537 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-08-20T02:56:09.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f4240f0f640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423c0751f0 0x7f423c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:56:09.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.096+0000 7f423bfff640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f423c075b10 0x7f423c19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:56:09.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.097+0000 7f4241710640 1 -- 172.21.15.99:0/1245413537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f423c075b10 msgr2=0x7f423c19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.097+0000 7f4241710640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f423c075b10 0x7f423c19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.097+0000 7f4241710640 1 -- 172.21.15.99:0/1245413537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423c0751f0 msgr2=0x7f423c19b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.097+0000 7f4241710640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423c0751f0 0x7f423c19b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.097+0000 7f4241710640 1 -- 172.21.15.99:0/1245413537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c1a1330 con 0x7f423c10f0b0 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.097+0000 7f4240f0f640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423c0751f0 0x7f423c19b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.097+0000 7f4241710640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c10f0b0 0x7f423c1a0b60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f423402ade0 tx=0x7f42340319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.097+0000 7f4239ffb640 1 -- 172.21.15.99:0/1245413537 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234003bf0 con 0x7f423c10f0b0 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.097+0000 7f4239ffb640 1 -- 172.21.15.99:0/1245413537 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4234003d90 con 0x7f423c10f0b0 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.097+0000 7f4243173640 1 -- 172.21.15.99:0/1245413537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c1a15c0 con 0x7f423c10f0b0 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.097+0000 7f4239ffb640 1 -- 172.21.15.99:0/1245413537 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423402fbd0 con 0x7f423c10f0b0 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.097+0000 7f4243173640 1 -- 172.21.15.99:0/1245413537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c1a7ce0 con 0x7f423c10f0b0 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.098+0000 7f4243173640 1 -- 172.21.15.99:0/1245413537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f423c0755d0 con 0x7f423c10f0b0 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.100+0000 7f4239ffb640 1 -- 172.21.15.99:0/1245413537 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f423402fd70 con 0x7f423c10f0b0 2024-08-20T02:56:09.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.101+0000 7f4239ffb640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f422c0b3ca0 0x7f422c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:56:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.101+0000 7f4239ffb640 1 -- 172.21.15.99:0/1245413537 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3588+0+0 (secure 0 0 0) 0x7f4234104240 con 0x7f423c10f0b0 2024-08-20T02:56:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.101+0000 7f4240f0f640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f422c0b3ca0 0x7f422c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:56:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.102+0000 7f4240f0f640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f422c0b3ca0 0x7f422c0b6160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f42300058c0 tx=0x7f4230005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:56:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.104+0000 7f4239ffb640 1 -- 172.21.15.99:0/1245413537 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f42340cca60 con 0x7f423c10f0b0 2024-08-20T02:56:09.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:09.295+0000 7f4243173640 1 -- 172.21.15.99:0/1245413537 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f423c100160 con 0x7f422c0b3ca0 2024-08-20T02:56:09.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-20T02:56:09.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: osdmap e24: 4 total, 3 up, 4 in 2024-08-20T02:56:09.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]} : dispatch 2024-08-20T02:56:09.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:56:09.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:09.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:09.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:09.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:09.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:09.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:09.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:09.298 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:09.299 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:09 smithi099 ceph-mon[21984]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-08-20T02:56:09.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-20T02:56:09.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: osdmap e24: 4 total, 3 up, 4 in 2024-08-20T02:56:09.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]} : dispatch 2024-08-20T02:56:09.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:56:09.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:09.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:09.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:09.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:09.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:09.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:09.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:09.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:09.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-mon[18790]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-08-20T02:56:09.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-20T02:56:09.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: osdmap e24: 4 total, 3 up, 4 in 2024-08-20T02:56:09.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]} : dispatch 2024-08-20T02:56:09.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:56:09.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:09.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:09.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:09.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:09.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:09.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:09.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:09.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:09.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:09 smithi165 ceph-mon[22357]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-08-20T02:56:10.035 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3[43469]: 2024-08-20T02:56:09.576+0000 7f93fca4a640 -1 osd.3 0 waiting for initial osdmap 2024-08-20T02:56:10.036 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 02:56:09 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3[43469]: 2024-08-20T02:56:09.581+0000 7f93f4869640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T02:56:10.528 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:10.528 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: Detected new or changed devices on smithi044 2024-08-20T02:56:10.528 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi044 to 8546M 2024-08-20T02:56:10.528 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: osdmap e25: 4 total, 3 up, 4 in 2024-08-20T02:56:10.528 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:56:10.528 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:56:10.528 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:56:10.528 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:56:10.528 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:10.528 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' 2024-08-20T02:56:10.529 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702] boot 2024-08-20T02:56:10.529 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: osdmap e26: 4 total, 4 up, 4 in 2024-08-20T02:56:10.529 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:56:10.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:10.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: Detected new or changed devices on smithi044 2024-08-20T02:56:10.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi044 to 8546M 2024-08-20T02:56:10.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: osdmap e25: 4 total, 3 up, 4 in 2024-08-20T02:56:10.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:56:10.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:56:10.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:56:10.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:56:10.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:10.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' 2024-08-20T02:56:10.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702] boot 2024-08-20T02:56:10.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: osdmap e26: 4 total, 4 up, 4 in 2024-08-20T02:56:10.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:56:10.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-08-20T02:56:10.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: Detected new or changed devices on smithi044 2024-08-20T02:56:10.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi044 to 8546M 2024-08-20T02:56:10.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: osdmap e25: 4 total, 3 up, 4 in 2024-08-20T02:56:10.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:56:10.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:56:10.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:56:10.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:56:10.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:10.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: from='osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702]' entity='osd.3' 2024-08-20T02:56:10.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: osd.3 [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702] boot 2024-08-20T02:56:10.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: osdmap e26: 4 total, 4 up, 4 in 2024-08-20T02:56:10.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T02:56:11.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:11 smithi044 ceph-mon[18790]: purged_snaps scrub starts 2024-08-20T02:56:11.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:11 smithi044 ceph-mon[18790]: purged_snaps scrub ok 2024-08-20T02:56:11.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:11 smithi044 ceph-mon[18790]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:56:11.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:11 smithi165 ceph-mon[22357]: purged_snaps scrub starts 2024-08-20T02:56:11.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:11 smithi165 ceph-mon[22357]: purged_snaps scrub ok 2024-08-20T02:56:11.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:11 smithi165 ceph-mon[22357]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:56:11.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:11 smithi099 ceph-mon[21984]: purged_snaps scrub starts 2024-08-20T02:56:11.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:11 smithi099 ceph-mon[21984]: purged_snaps scrub ok 2024-08-20T02:56:11.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:11 smithi099 ceph-mon[21984]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:56:12.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:12 smithi099 ceph-mon[21984]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:12.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:12 smithi099 ceph-mon[21984]: osdmap e27: 4 total, 4 up, 4 in 2024-08-20T02:56:12.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:12 smithi099 ceph-mon[21984]: from='client.? 172.21.15.99:0/2151446194' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4edebfdb-af10-49ee-b724-2a63b9219221"} : dispatch 2024-08-20T02:56:12.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:12 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4edebfdb-af10-49ee-b724-2a63b9219221"} : dispatch 2024-08-20T02:56:12.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:12 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4edebfdb-af10-49ee-b724-2a63b9219221"}]': finished 2024-08-20T02:56:12.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:12 smithi099 ceph-mon[21984]: osdmap e28: 5 total, 4 up, 5 in 2024-08-20T02:56:12.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:12 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T02:56:12.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:12 smithi099 ceph-mon[21984]: from='client.? 172.21.15.99:0/2857843775' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:56:12.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:12 smithi044 ceph-mon[18790]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:12.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:12 smithi044 ceph-mon[18790]: osdmap e27: 4 total, 4 up, 4 in 2024-08-20T02:56:12.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:12 smithi044 ceph-mon[18790]: from='client.? 172.21.15.99:0/2151446194' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4edebfdb-af10-49ee-b724-2a63b9219221"} : dispatch 2024-08-20T02:56:12.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:12 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4edebfdb-af10-49ee-b724-2a63b9219221"} : dispatch 2024-08-20T02:56:12.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:12 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4edebfdb-af10-49ee-b724-2a63b9219221"}]': finished 2024-08-20T02:56:12.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:12 smithi044 ceph-mon[18790]: osdmap e28: 5 total, 4 up, 5 in 2024-08-20T02:56:12.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:12 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T02:56:12.610 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:12 smithi044 ceph-mon[18790]: from='client.? 172.21.15.99:0/2857843775' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:56:12.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:12 smithi165 ceph-mon[22357]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:12.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:12 smithi165 ceph-mon[22357]: osdmap e27: 4 total, 4 up, 4 in 2024-08-20T02:56:12.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:12 smithi165 ceph-mon[22357]: from='client.? 172.21.15.99:0/2151446194' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4edebfdb-af10-49ee-b724-2a63b9219221"} : dispatch 2024-08-20T02:56:12.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:12 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4edebfdb-af10-49ee-b724-2a63b9219221"} : dispatch 2024-08-20T02:56:12.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:12 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4edebfdb-af10-49ee-b724-2a63b9219221"}]': finished 2024-08-20T02:56:12.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:12 smithi165 ceph-mon[22357]: osdmap e28: 5 total, 4 up, 5 in 2024-08-20T02:56:12.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:12 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T02:56:12.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:12 smithi165 ceph-mon[22357]: from='client.? 172.21.15.99:0/2857843775' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:56:14.438 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:14 smithi165 ceph-mon[22357]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:14.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:14 smithi044 ceph-mon[18790]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:14.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:14 smithi099 ceph-mon[21984]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:16.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:16 smithi099 ceph-mon[21984]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-08-20T02:56:16.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:16 smithi044 ceph-mon[18790]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-08-20T02:56:16.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:16 smithi165 ceph-mon[22357]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-08-20T02:56:18.451 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:18 smithi099 ceph-mon[21984]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-08-20T02:56:18.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:18 smithi044 ceph-mon[18790]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-08-20T02:56:18.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:18 smithi165 ceph-mon[22357]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-08-20T02:56:20.452 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:20 smithi099 ceph-mon[21984]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T02:56:20.452 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:20 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-20T02:56:20.452 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:20 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:20.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:20 smithi044 ceph-mon[18790]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T02:56:20.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:20 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-20T02:56:20.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:20 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:20.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:20 smithi165 ceph-mon[22357]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T02:56:20.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:20 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-20T02:56:20.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:20 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:21.466 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:21 smithi099 ceph-mon[21984]: Deploying daemon osd.4 on smithi099 2024-08-20T02:56:21.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:21 smithi044 ceph-mon[18790]: Deploying daemon osd.4 on smithi099 2024-08-20T02:56:21.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:21 smithi165 ceph-mon[22357]: Deploying daemon osd.4 on smithi099 2024-08-20T02:56:22.496 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:22 smithi099 ceph-mon[21984]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-08-20T02:56:22.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:22 smithi044 ceph-mon[18790]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-08-20T02:56:22.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:22 smithi165 ceph-mon[22357]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-08-20T02:56:23.525 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:23 smithi099 ceph-mon[21984]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-08-20T02:56:23.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:23 smithi044 ceph-mon[18790]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-08-20T02:56:23.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:23 smithi165 ceph-mon[22357]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-08-20T02:56:25.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:25 smithi099 ceph-mon[21984]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:56:25.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:25.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:25.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:25.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:25 smithi044 ceph-mon[18790]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:56:25.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:25.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:25.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:25.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:25 smithi165 ceph-mon[22357]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:56:25.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:25.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:25.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:27.629 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:27 smithi099 ceph-mon[21984]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:27.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:27 smithi044 ceph-mon[18790]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:27.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:27 smithi165 ceph-mon[22357]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:29.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:29.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:29.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:29.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:29.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:29.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:29.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:29.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:29.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:29.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:29.149 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:29.149 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:29.149 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:29.149 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:29.149 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:29.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.187+0000 7f4239ffb640 1 -- 172.21.15.99:0/1245413537 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f423c100160 con 0x7f422c0b3ca0 2024-08-20T02:56:29.190 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 4 on host 'smithi099' 2024-08-20T02:56:29.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.189+0000 7f4243173640 1 -- 172.21.15.99:0/1245413537 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f422c0b3ca0 msgr2=0x7f422c0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:29.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.189+0000 7f4243173640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f422c0b3ca0 0x7f422c0b6160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f42300058c0 tx=0x7f4230005850 comp rx=0 tx=0).stop 2024-08-20T02:56:29.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.189+0000 7f4243173640 1 -- 172.21.15.99:0/1245413537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c10f0b0 msgr2=0x7f423c1a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:29.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.189+0000 7f4243173640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c10f0b0 0x7f423c1a0b60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f423402ade0 tx=0x7f42340319f0 comp rx=0 tx=0).stop 2024-08-20T02:56:29.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.189+0000 7f4243173640 1 -- 172.21.15.99:0/1245413537 shutdown_connections 2024-08-20T02:56:29.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.189+0000 7f4243173640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f422c0b3ca0 0x7f422c0b6160 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:29.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.189+0000 7f4243173640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c10f0b0 0x7f423c1a0b60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:29.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.189+0000 7f4243173640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f423c075b10 0x7f423c19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:29.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.189+0000 7f4243173640 1 --2- 172.21.15.99:0/1245413537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f423c0751f0 0x7f423c19b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:29.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.189+0000 7f4243173640 1 -- 172.21.15.99:0/1245413537 >> 172.21.15.99:0/1245413537 conn(0x7f423c0fe550 msgr2=0x7f423c1112d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:56:29.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.189+0000 7f4243173640 1 -- 172.21.15.99:0/1245413537 shutdown_connections 2024-08-20T02:56:29.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:56:29.189+0000 7f4243173640 1 -- 172.21.15.99:0/1245413537 wait complete. 2024-08-20T02:56:29.882 DEBUG:teuthology.orchestra.run.smithi099:osd.4> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.4.service 2024-08-20T02:56:29.884 INFO:tasks.cephadm:Deploying osd.5 on smithi165 with /dev/vg_nvme/lv_3... 2024-08-20T02:56:29.884 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-20T02:56:30.141 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:56:30.257 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:30 smithi165 ceph-mon[22357]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:30.257 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:30 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:30.257 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:30 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:30.257 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:30 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:30 smithi099 ceph-mon[21984]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:30 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:30 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:30.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:30 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:30.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:30 smithi044 ceph-mon[18790]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:30.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:30 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:30.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:30 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:30.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:30 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:31.665 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 02:56:31 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4[34618]: 2024-08-20T02:56:31.208+0000 7fa4a15ea640 -1 osd.4 0 log_to_monitors true 2024-08-20T02:56:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:32 smithi099 ceph-mon[21984]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:32.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:32 smithi099 ceph-mon[21984]: from='osd.4 [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T02:56:32.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:32 smithi099 ceph-mon[21984]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T02:56:32.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:32 smithi044 ceph-mon[18790]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:32.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:32 smithi044 ceph-mon[18790]: from='osd.4 [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T02:56:32.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:32 smithi044 ceph-mon[18790]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T02:56:32.604 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:32 smithi165 ceph-mon[22357]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:32.604 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:32 smithi165 ceph-mon[22357]: from='osd.4 [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T02:56:32.605 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:32 smithi165 ceph-mon[22357]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T02:56:32.607 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:56:32.632 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_3 2024-08-20T02:56:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:56:33.234 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-20T02:56:33.234 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: osdmap e29: 5 total, 4 up, 5 in 2024-08-20T02:56:33.234 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T02:56:33.234 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: from='osd.4 [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:56:33.234 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:56:33.234 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:33.234 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:33.234 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:33.234 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:33.235 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:33.235 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:33.235 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:33.235 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:33 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:33.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.317+0000 7fc387b1e640 1 -- 172.21.15.165:0/1122350075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc380101990 msgr2=0x7fc380101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:33.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.317+0000 7fc387b1e640 1 --2- 172.21.15.165:0/1122350075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc380101990 0x7fc380101d70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009900 tx=0x7fc37c02a410 comp rx=0 tx=0).stop 2024-08-20T02:56:33.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.317+0000 7fc387b1e640 1 -- 172.21.15.165:0/1122350075 shutdown_connections 2024-08-20T02:56:33.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.317+0000 7fc387b1e640 1 --2- 172.21.15.165:0/1122350075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc380107d20 0x7fc38010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:33.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.317+0000 7fc387b1e640 1 --2- 172.21.15.165:0/1122350075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3801073a0 0x7fc3801077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:33.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.317+0000 7fc387b1e640 1 --2- 172.21.15.165:0/1122350075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc380101990 0x7fc380101d70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:33.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.317+0000 7fc387b1e640 1 -- 172.21.15.165:0/1122350075 >> 172.21.15.165:0/1122350075 conn(0x7fc3800fd400 msgr2=0x7fc3800ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:56:33.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.318+0000 7fc387b1e640 1 -- 172.21.15.165:0/1122350075 shutdown_connections 2024-08-20T02:56:33.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.318+0000 7fc387b1e640 1 -- 172.21.15.165:0/1122350075 wait complete. 2024-08-20T02:56:33.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.318+0000 7fc387b1e640 1 Processor -- start 2024-08-20T02:56:33.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.318+0000 7fc387b1e640 1 -- start start 2024-08-20T02:56:33.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc387b1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc380101990 0x7fc380199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:56:33.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc387b1e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3801073a0 0x7fc38019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:56:33.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc387b1e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc380107d20 0x7fc3801a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:56:33.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc387b1e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3801119e0 con 0x7fc380101990 2024-08-20T02:56:33.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc387b1e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc380111860 con 0x7fc3801073a0 2024-08-20T02:56:33.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc387b1e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc380111b60 con 0x7fc380107d20 2024-08-20T02:56:33.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc3860bb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc380107d20 0x7fc3801a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:56:33.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc3850b9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3801073a0 0x7fc38019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc3860bb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc380107d20 0x7fc3801a4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43690/0 (socket says 172.21.15.165:43690) 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc3860bb640 1 -- 172.21.15.165:0/2188680861 learned_addr learned my addr 172.21.15.165:0/2188680861 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc3850b9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3801073a0 0x7fc38019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.165:43660/0 (socket says 172.21.15.165:43660) 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc3858ba640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc380101990 0x7fc380199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc3860bb640 1 -- 172.21.15.165:0/2188680861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3801073a0 msgr2=0x7fc38019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc3860bb640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3801073a0 0x7fc38019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc3860bb640 1 -- 172.21.15.165:0/2188680861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc380101990 msgr2=0x7fc380199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc3860bb640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc380101990 0x7fc380199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc3860bb640 1 -- 172.21.15.165:0/2188680861 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3801a5280 con 0x7fc380107d20 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc3860bb640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc380107d20 0x7fc3801a4d40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc37800eaa0 tx=0x7fc37800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc376ffd640 1 -- 172.21.15.165:0/2188680861 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378004d20 con 0x7fc380107d20 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc376ffd640 1 -- 172.21.15.165:0/2188680861 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc378005020 con 0x7fc380107d20 2024-08-20T02:56:33.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc387b1e640 1 -- 172.21.15.165:0/2188680861 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3801a5450 con 0x7fc380107d20 2024-08-20T02:56:33.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.319+0000 7fc376ffd640 1 -- 172.21.15.165:0/2188680861 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3780116b0 con 0x7fc380107d20 2024-08-20T02:56:33.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.320+0000 7fc387b1e640 1 -- 172.21.15.165:0/2188680861 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3801a5930 con 0x7fc380107d20 2024-08-20T02:56:33.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.321+0000 7fc387b1e640 1 -- 172.21.15.165:0/2188680861 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc380101d70 con 0x7fc380107d20 2024-08-20T02:56:33.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.322+0000 7fc376ffd640 1 -- 172.21.15.165:0/2188680861 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fc3780051d0 con 0x7fc380107d20 2024-08-20T02:56:33.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.323+0000 7fc376ffd640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc36c0b3c20 0x7fc36c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:56:33.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.323+0000 7fc376ffd640 1 -- 172.21.15.165:0/2188680861 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4036+0+0 (secure 0 0 0) 0x7fc3780e30b0 con 0x7fc380107d20 2024-08-20T02:56:33.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.323+0000 7fc3858ba640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc36c0b3c20 0x7fc36c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:56:33.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.324+0000 7fc3858ba640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc36c0b3c20 0x7fc36c0b60e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc37c031440 tx=0x7fc37c005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:56:33.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.329+0000 7fc376ffd640 1 -- 172.21.15.165:0/2188680861 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fc378016030 con 0x7fc380107d20 2024-08-20T02:56:33.415 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4[34618]: 2024-08-20T02:56:33.164+0000 7fa49d56d640 -1 osd.4 0 waiting for initial osdmap 2024-08-20T02:56:33.415 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4[34618]: 2024-08-20T02:56:33.170+0000 7fa499394640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T02:56:33.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-20T02:56:33.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: osdmap e29: 5 total, 4 up, 5 in 2024-08-20T02:56:33.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T02:56:33.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: from='osd.4 [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:56:33.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:56:33.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:33.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:33.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:33.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:33.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:33.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:33.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:33.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:33 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:33.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:33.519+0000 7fc387b1e640 1 -- 172.21.15.165:0/2188680861 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc3800632c0 con 0x7fc36c0b3c20 2024-08-20T02:56:33.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-20T02:56:33.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: osdmap e29: 5 total, 4 up, 5 in 2024-08-20T02:56:33.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T02:56:33.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: from='osd.4 [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:56:33.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:56:33.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:33.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:33.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:33.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:33.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:33.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:33.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:33.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:33 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:34 smithi165 ceph-mon[22357]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:34 smithi165 ceph-mon[22357]: Detected new or changed devices on smithi099 2024-08-20T02:56:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:34 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi099 to 8544M 2024-08-20T02:56:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:34 smithi165 ceph-mon[22357]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-08-20T02:56:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:34 smithi165 ceph-mon[22357]: osdmap e30: 5 total, 4 up, 5 in 2024-08-20T02:56:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T02:56:34.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:56:34.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:56:34.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:34 smithi044 ceph-mon[18790]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:34 smithi044 ceph-mon[18790]: Detected new or changed devices on smithi099 2024-08-20T02:56:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:34 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi099 to 8544M 2024-08-20T02:56:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:34 smithi044 ceph-mon[18790]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-08-20T02:56:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:34 smithi044 ceph-mon[18790]: osdmap e30: 5 total, 4 up, 5 in 2024-08-20T02:56:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T02:56:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:56:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:56:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:34.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:34 smithi099 ceph-mon[21984]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-08-20T02:56:34.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:34 smithi099 ceph-mon[21984]: Detected new or changed devices on smithi099 2024-08-20T02:56:34.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:34 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi099 to 8544M 2024-08-20T02:56:34.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:34 smithi099 ceph-mon[21984]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-08-20T02:56:34.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:34 smithi099 ceph-mon[21984]: osdmap e30: 5 total, 4 up, 5 in 2024-08-20T02:56:34.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T02:56:34.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:56:34.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:56:34.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:35.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:35 smithi044 ceph-mon[18790]: purged_snaps scrub starts 2024-08-20T02:56:35.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:35 smithi044 ceph-mon[18790]: purged_snaps scrub ok 2024-08-20T02:56:35.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:35 smithi044 ceph-mon[18790]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:56:35.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:35 smithi044 ceph-mon[18790]: osd.4 [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325] boot 2024-08-20T02:56:35.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:35 smithi044 ceph-mon[18790]: osdmap e31: 5 total, 5 up, 5 in 2024-08-20T02:56:35.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T02:56:35.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:35 smithi165 ceph-mon[22357]: purged_snaps scrub starts 2024-08-20T02:56:35.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:35 smithi165 ceph-mon[22357]: purged_snaps scrub ok 2024-08-20T02:56:35.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:35 smithi165 ceph-mon[22357]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:56:35.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:35 smithi165 ceph-mon[22357]: osd.4 [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325] boot 2024-08-20T02:56:35.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:35 smithi165 ceph-mon[22357]: osdmap e31: 5 total, 5 up, 5 in 2024-08-20T02:56:35.563 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T02:56:35.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:35 smithi099 ceph-mon[21984]: purged_snaps scrub starts 2024-08-20T02:56:35.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:35 smithi099 ceph-mon[21984]: purged_snaps scrub ok 2024-08-20T02:56:35.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:35 smithi099 ceph-mon[21984]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:56:35.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:35 smithi099 ceph-mon[21984]: osd.4 [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325] boot 2024-08-20T02:56:35.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:35 smithi099 ceph-mon[21984]: osdmap e31: 5 total, 5 up, 5 in 2024-08-20T02:56:35.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T02:56:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:36 smithi165 ceph-mon[22357]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:36 smithi165 ceph-mon[22357]: osdmap e32: 5 total, 5 up, 5 in 2024-08-20T02:56:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:36 smithi165 ceph-mon[22357]: from='client.? 172.21.15.165:0/2728347275' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d22c21f-cc43-4ef5-a75c-d4fc1a361c06"} : dispatch 2024-08-20T02:56:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:36 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d22c21f-cc43-4ef5-a75c-d4fc1a361c06"} : dispatch 2024-08-20T02:56:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:36 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d22c21f-cc43-4ef5-a75c-d4fc1a361c06"}]': finished 2024-08-20T02:56:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:36 smithi165 ceph-mon[22357]: osdmap e33: 6 total, 5 up, 6 in 2024-08-20T02:56:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:36.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:36 smithi044 ceph-mon[18790]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:36 smithi044 ceph-mon[18790]: osdmap e32: 5 total, 5 up, 5 in 2024-08-20T02:56:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:36 smithi044 ceph-mon[18790]: from='client.? 172.21.15.165:0/2728347275' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d22c21f-cc43-4ef5-a75c-d4fc1a361c06"} : dispatch 2024-08-20T02:56:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:36 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d22c21f-cc43-4ef5-a75c-d4fc1a361c06"} : dispatch 2024-08-20T02:56:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:36 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d22c21f-cc43-4ef5-a75c-d4fc1a361c06"}]': finished 2024-08-20T02:56:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:36 smithi044 ceph-mon[18790]: osdmap e33: 6 total, 5 up, 6 in 2024-08-20T02:56:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:36.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:36 smithi099 ceph-mon[21984]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:36.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:36 smithi099 ceph-mon[21984]: osdmap e32: 5 total, 5 up, 5 in 2024-08-20T02:56:36.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:36 smithi099 ceph-mon[21984]: from='client.? 172.21.15.165:0/2728347275' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d22c21f-cc43-4ef5-a75c-d4fc1a361c06"} : dispatch 2024-08-20T02:56:36.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:36 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d22c21f-cc43-4ef5-a75c-d4fc1a361c06"} : dispatch 2024-08-20T02:56:36.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:36 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d22c21f-cc43-4ef5-a75c-d4fc1a361c06"}]': finished 2024-08-20T02:56:36.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:36 smithi099 ceph-mon[21984]: osdmap e33: 6 total, 5 up, 6 in 2024-08-20T02:56:36.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:37.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:37 smithi044 ceph-mon[18790]: from='client.? 172.21.15.165:0/1100307455' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:56:37.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:37 smithi165 ceph-mon[22357]: from='client.? 172.21.15.165:0/1100307455' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:56:37.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:37 smithi099 ceph-mon[21984]: from='client.? 172.21.15.165:0/1100307455' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:56:38.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:38 smithi044 ceph-mon[18790]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:38.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:38 smithi165 ceph-mon[22357]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:38.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:38 smithi099 ceph-mon[21984]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:40.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:40 smithi044 ceph-mon[18790]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:40.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:40 smithi165 ceph-mon[22357]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:40.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:40 smithi099 ceph-mon[21984]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:42.460 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:42 smithi165 ceph-mon[22357]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:42.467 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:42 smithi099 ceph-mon[21984]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:42.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:42 smithi044 ceph-mon[18790]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:44.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:44 smithi044 ceph-mon[18790]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:44.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:44 smithi165 ceph-mon[22357]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:44.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:44 smithi099 ceph-mon[21984]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:45.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:45 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-20T02:56:45.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:45 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:45.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:45 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-20T02:56:45.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:45 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:45.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:45 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-20T02:56:45.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:45 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:46.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:46 smithi165 ceph-mon[22357]: Deploying daemon osd.5 on smithi165 2024-08-20T02:56:46.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:46 smithi165 ceph-mon[22357]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:46.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:46 smithi044 ceph-mon[18790]: Deploying daemon osd.5 on smithi165 2024-08-20T02:56:46.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:46 smithi044 ceph-mon[18790]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:46.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:46 smithi099 ceph-mon[21984]: Deploying daemon osd.5 on smithi165 2024-08-20T02:56:46.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:46 smithi099 ceph-mon[21984]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:48.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:48 smithi165 ceph-mon[22357]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:48.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:48 smithi044 ceph-mon[18790]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:48.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:48 smithi099 ceph-mon[21984]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:49.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:49 smithi044 ceph-mon[18790]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:49.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:49 smithi165 ceph-mon[22357]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:49.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:49 smithi099 ceph-mon[21984]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:50 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:50 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:50 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:50.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:50 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:50.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:50 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:50.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:50 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:50.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:50 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:50.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:50 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:50.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:50 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:51.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:51 smithi044 ceph-mon[18790]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:51.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:51 smithi165 ceph-mon[22357]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:51.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:51 smithi099 ceph-mon[21984]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:52.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.410+0000 7fc376ffd640 1 -- 172.21.15.165:0/2188680861 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc3800632c0 con 0x7fc36c0b3c20 2024-08-20T02:56:52.413 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 5 on host 'smithi165' 2024-08-20T02:56:52.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.412+0000 7fc387b1e640 1 -- 172.21.15.165:0/2188680861 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc36c0b3c20 msgr2=0x7fc36c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:52.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.412+0000 7fc387b1e640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc36c0b3c20 0x7fc36c0b60e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc37c031440 tx=0x7fc37c005b50 comp rx=0 tx=0).stop 2024-08-20T02:56:52.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.412+0000 7fc387b1e640 1 -- 172.21.15.165:0/2188680861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc380107d20 msgr2=0x7fc3801a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:52.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.412+0000 7fc387b1e640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc380107d20 0x7fc3801a4d40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc37800eaa0 tx=0x7fc37800eea0 comp rx=0 tx=0).stop 2024-08-20T02:56:52.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.412+0000 7fc387b1e640 1 -- 172.21.15.165:0/2188680861 shutdown_connections 2024-08-20T02:56:52.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.412+0000 7fc387b1e640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc36c0b3c20 0x7fc36c0b60e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:52.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.412+0000 7fc387b1e640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc380107d20 0x7fc3801a4d40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:52.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.412+0000 7fc387b1e640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3801073a0 0x7fc38019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:52.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.412+0000 7fc387b1e640 1 --2- 172.21.15.165:0/2188680861 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc380101990 0x7fc380199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:52.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.412+0000 7fc387b1e640 1 -- 172.21.15.165:0/2188680861 >> 172.21.15.165:0/2188680861 conn(0x7fc3800fd400 msgr2=0x7fc3800ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:56:52.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.413+0000 7fc387b1e640 1 -- 172.21.15.165:0/2188680861 shutdown_connections 2024-08-20T02:56:52.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:56:52.413+0000 7fc387b1e640 1 -- 172.21.15.165:0/2188680861 wait complete. 2024-08-20T02:56:52.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:52 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:52.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:52 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:52.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:52 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:52.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:52 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:52.973 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:52 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:52.974 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:52 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:52.974 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:52 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:52.974 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:52 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:53.008 DEBUG:teuthology.orchestra.run.smithi165:osd.5> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.5.service 2024-08-20T02:56:53.010 INFO:tasks.cephadm:Deploying osd.6 on smithi044 with /dev/vg_nvme/lv_2... 2024-08-20T02:56:53.010 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-20T02:56:53.041 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:52 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:53.041 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:52 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:53.041 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:52 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:53.041 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:52 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:53.042 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:52 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:53.042 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:52 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:53.042 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:52 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:53.042 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:52 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:53.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:52 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:53.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:52 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:53.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:52 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:53.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:52 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:53.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:52 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:53.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:52 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:56:53.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:52 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:53.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:52 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:53.275 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:56:53.979 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:53 smithi044 ceph-mon[18790]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:54.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:53 smithi165 ceph-mon[22357]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:54.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:53 smithi099 ceph-mon[21984]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:55.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 02:56:54 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T02:56:54.853+0000 7f94f8b9a640 -1 osd.5 0 log_to_monitors true 2024-08-20T02:56:56.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:55 smithi044 ceph-mon[18790]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:56.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:55 smithi044 ceph-mon[18790]: from='osd.5 [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T02:56:56.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:55 smithi044 ceph-mon[18790]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T02:56:56.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:55 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:56.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:55 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:56.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:55 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:56.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:55 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:56.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:55 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:56.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:55 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:56.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:55 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:56.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:55 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:56.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:55 smithi165 ceph-mon[22357]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:56.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:55 smithi165 ceph-mon[22357]: from='osd.5 [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T02:56:56.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:55 smithi165 ceph-mon[22357]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T02:56:56.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:55 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:56.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:55 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:56.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:55 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:56.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:55 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:56.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:55 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:56.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:55 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:56.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:55 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:56.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:55 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:56.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:55 smithi099 ceph-mon[21984]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:56.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:55 smithi099 ceph-mon[21984]: from='osd.5 [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T02:56:56.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:55 smithi099 ceph-mon[21984]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T02:56:56.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:55 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:56.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:55 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:56.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:55 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:56.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:55 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T02:56:56.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:55 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:56.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:55 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:56.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:55 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:56:56.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:55 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:56:56.185 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:56:56.211 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_2 2024-08-20T02:56:56.504 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:56:56.899 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:56 smithi044 ceph-mon[18790]: Detected new or changed devices on smithi165 2024-08-20T02:56:56.899 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:56 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi165 to 8544M 2024-08-20T02:56:56.899 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:56 smithi044 ceph-mon[18790]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-20T02:56:56.899 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:56 smithi044 ceph-mon[18790]: from='osd.5 [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:56:56.900 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:56 smithi044 ceph-mon[18790]: osdmap e34: 6 total, 5 up, 6 in 2024-08-20T02:56:56.900 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:56 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:56.900 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:56 smithi044 ceph-mon[18790]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:56:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.980+0000 7fef23b70640 1 -- 172.21.15.44:0/975426625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef1c101990 msgr2=0x7fef1c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.980+0000 7fef23b70640 1 --2- 172.21.15.44:0/975426625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef1c101990 0x7fef1c101d70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef18009ca0 tx=0x7fef1802a8c0 comp rx=0 tx=0).stop 2024-08-20T02:56:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.980+0000 7fef23b70640 1 -- 172.21.15.44:0/975426625 shutdown_connections 2024-08-20T02:56:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.980+0000 7fef23b70640 1 --2- 172.21.15.44:0/975426625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef1c107d20 0x7fef1c10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.980+0000 7fef23b70640 1 --2- 172.21.15.44:0/975426625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef1c1073a0 0x7fef1c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.980+0000 7fef23b70640 1 --2- 172.21.15.44:0/975426625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef1c101990 0x7fef1c101d70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.980+0000 7fef23b70640 1 -- 172.21.15.44:0/975426625 >> 172.21.15.44:0/975426625 conn(0x7fef1c0fd400 msgr2=0x7fef1c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:56:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.980+0000 7fef23b70640 1 -- 172.21.15.44:0/975426625 shutdown_connections 2024-08-20T02:56:56.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.980+0000 7fef23b70640 1 -- 172.21.15.44:0/975426625 wait complete. 2024-08-20T02:56:57.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.980+0000 7fef23b70640 1 Processor -- start 2024-08-20T02:56:57.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.980+0000 7fef23b70640 1 -- start start 2024-08-20T02:56:57.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef23b70640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef1c101990 0x7fef1c103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:56:57.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef23b70640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef1c1073a0 0x7fef1c103790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:56:57.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef23b70640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef1c107d20 0x7fef1c103cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:56:57.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef23b70640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c111a30 con 0x7fef1c107d20 2024-08-20T02:56:57.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef23b70640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c1118b0 con 0x7fef1c101990 2024-08-20T02:56:57.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef23b70640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c111bb0 con 0x7fef1c1073a0 2024-08-20T02:56:57.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef2210d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef1c107d20 0x7fef1c103cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:56:57.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef2210d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef1c107d20 0x7fef1c103cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:59904/0 (socket says 172.21.15.44:59904) 2024-08-20T02:56:57.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef2210d640 1 -- 172.21.15.44:0/2368508108 learned_addr learned my addr 172.21.15.44:0/2368508108 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef2110b640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef1c1073a0 0x7fef1c103790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef2190c640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef1c101990 0x7fef1c103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef2210d640 1 -- 172.21.15.44:0/2368508108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef1c1073a0 msgr2=0x7fef1c103790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef2210d640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef1c1073a0 0x7fef1c103790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef2210d640 1 -- 172.21.15.44:0/2368508108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef1c101990 msgr2=0x7fef1c103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef2210d640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef1c101990 0x7fef1c103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef2210d640 1 -- 172.21.15.44:0/2368508108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1c1a7360 con 0x7fef1c107d20 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef2210d640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef1c107d20 0x7fef1c103cd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef1400c970 tx=0x7fef1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef12ffd640 1 -- 172.21.15.44:0/2368508108 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14007bb0 con 0x7fef1c107d20 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef23b70640 1 -- 172.21.15.44:0/2368508108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1c1a7650 con 0x7fef1c107d20 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef12ffd640 1 -- 172.21.15.44:0/2368508108 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef14007d50 con 0x7fef1c107d20 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.981+0000 7fef12ffd640 1 -- 172.21.15.44:0/2368508108 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14018770 con 0x7fef1c107d20 2024-08-20T02:56:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.982+0000 7fef23b70640 1 -- 172.21.15.44:0/2368508108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef1c1a7b40 con 0x7fef1c107d20 2024-08-20T02:56:57.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.982+0000 7fef23b70640 1 -- 172.21.15.44:0/2368508108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef1c101d70 con 0x7fef1c107d20 2024-08-20T02:56:57.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.984+0000 7fef12ffd640 1 -- 172.21.15.44:0/2368508108 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fef1400f990 con 0x7fef1c107d20 2024-08-20T02:56:57.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.984+0000 7fef12ffd640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fef080b3c20 0x7fef080b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:56:57.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.985+0000 7fef2190c640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fef080b3c20 0x7fef080b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:56:57.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.985+0000 7fef12ffd640 1 -- 172.21.15.44:0/2368508108 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4468+0+0 (secure 0 0 0) 0x7fef140e2f10 con 0x7fef1c107d20 2024-08-20T02:56:57.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.985+0000 7fef2190c640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fef080b3c20 0x7fef080b60e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef18006010 tx=0x7fef18009820 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:56:57.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:56.987+0000 7fef12ffd640 1 -- 172.21.15.44:0/2368508108 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fef14014030 con 0x7fef1c107d20 2024-08-20T02:56:57.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:56 smithi165 ceph-mon[22357]: Detected new or changed devices on smithi165 2024-08-20T02:56:57.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:56 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi165 to 8544M 2024-08-20T02:56:57.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:56 smithi165 ceph-mon[22357]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-20T02:56:57.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:56 smithi165 ceph-mon[22357]: from='osd.5 [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:56:57.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:56 smithi165 ceph-mon[22357]: osdmap e34: 6 total, 5 up, 6 in 2024-08-20T02:56:57.132 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:56 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:57.133 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:56 smithi165 ceph-mon[22357]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:56:57.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:56 smithi099 ceph-mon[21984]: Detected new or changed devices on smithi165 2024-08-20T02:56:57.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:56 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi165 to 8544M 2024-08-20T02:56:57.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:56 smithi099 ceph-mon[21984]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-20T02:56:57.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:56 smithi099 ceph-mon[21984]: from='osd.5 [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:56:57.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:56 smithi099 ceph-mon[21984]: osdmap e34: 6 total, 5 up, 6 in 2024-08-20T02:56:57.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:56 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:57.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:56 smithi099 ceph-mon[21984]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:56:57.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:56:57.183+0000 7fef23b70640 1 -- 172.21.15.44:0/2368508108 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fef1c06a880 con 0x7fef080b3c20 2024-08-20T02:56:57.386 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 02:56:57 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T02:56:57.219+0000 7f94f4b1d640 -1 osd.5 0 waiting for initial osdmap 2024-08-20T02:56:57.386 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 02:56:57 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T02:56:57.227+0000 7f94ec93c640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T02:56:57.861 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:57 smithi044 ceph-mon[18790]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:57.861 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:57 smithi044 ceph-mon[18790]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-08-20T02:56:57.861 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:57 smithi044 ceph-mon[18790]: osdmap e35: 6 total, 5 up, 6 in 2024-08-20T02:56:57.861 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:57.861 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:57.861 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:57 smithi044 ceph-mon[18790]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:56:57.861 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:56:57.862 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:56:57.862 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:57.862 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:57 smithi044 ceph-mon[18790]: from='osd.5 ' entity='osd.5' 2024-08-20T02:56:58.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:57 smithi165 ceph-mon[22357]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:58.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:57 smithi165 ceph-mon[22357]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-08-20T02:56:58.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:57 smithi165 ceph-mon[22357]: osdmap e35: 6 total, 5 up, 6 in 2024-08-20T02:56:58.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:58.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:58.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:57 smithi165 ceph-mon[22357]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:56:58.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:56:58.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:56:58.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:58.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:57 smithi165 ceph-mon[22357]: from='osd.5 ' entity='osd.5' 2024-08-20T02:56:58.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:57 smithi099 ceph-mon[21984]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:56:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:57 smithi099 ceph-mon[21984]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-08-20T02:56:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:57 smithi099 ceph-mon[21984]: osdmap e35: 6 total, 5 up, 6 in 2024-08-20T02:56:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:57 smithi099 ceph-mon[21984]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:56:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:56:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:56:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:56:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:57 smithi099 ceph-mon[21984]: from='osd.5 ' entity='osd.5' 2024-08-20T02:56:58.897 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:58 smithi044 ceph-mon[18790]: purged_snaps scrub starts 2024-08-20T02:56:58.897 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:58 smithi044 ceph-mon[18790]: purged_snaps scrub ok 2024-08-20T02:56:58.897 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:58 smithi044 ceph-mon[18790]: osd.5 [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255] boot 2024-08-20T02:56:58.897 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:58 smithi044 ceph-mon[18790]: osdmap e36: 6 total, 6 up, 6 in 2024-08-20T02:56:58.897 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:59.148 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:58 smithi165 ceph-mon[22357]: purged_snaps scrub starts 2024-08-20T02:56:59.149 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:58 smithi165 ceph-mon[22357]: purged_snaps scrub ok 2024-08-20T02:56:59.149 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:58 smithi165 ceph-mon[22357]: osd.5 [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255] boot 2024-08-20T02:56:59.149 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:58 smithi165 ceph-mon[22357]: osdmap e36: 6 total, 6 up, 6 in 2024-08-20T02:56:59.149 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:56:59.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:58 smithi099 ceph-mon[21984]: purged_snaps scrub starts 2024-08-20T02:56:59.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:58 smithi099 ceph-mon[21984]: purged_snaps scrub ok 2024-08-20T02:56:59.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:58 smithi099 ceph-mon[21984]: osd.5 [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255] boot 2024-08-20T02:56:59.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:58 smithi099 ceph-mon[21984]: osdmap e36: 6 total, 6 up, 6 in 2024-08-20T02:56:59.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T02:57:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:59 smithi165 ceph-mon[22357]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:57:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:59 smithi165 ceph-mon[22357]: osdmap e37: 6 total, 6 up, 6 in 2024-08-20T02:57:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:59 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/95259782' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03df0c2f-1b7c-4242-888c-e047322fd8f8"} : dispatch 2024-08-20T02:57:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:59 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/95259782' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03df0c2f-1b7c-4242-888c-e047322fd8f8"}]': finished 2024-08-20T02:57:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:59 smithi165 ceph-mon[22357]: osdmap e38: 7 total, 6 up, 7 in 2024-08-20T02:57:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:56:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:00.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:59 smithi099 ceph-mon[21984]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:57:00.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:59 smithi099 ceph-mon[21984]: osdmap e37: 6 total, 6 up, 6 in 2024-08-20T02:57:00.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:59 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/95259782' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03df0c2f-1b7c-4242-888c-e047322fd8f8"} : dispatch 2024-08-20T02:57:00.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:59 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/95259782' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03df0c2f-1b7c-4242-888c-e047322fd8f8"}]': finished 2024-08-20T02:57:00.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:59 smithi099 ceph-mon[21984]: osdmap e38: 7 total, 6 up, 7 in 2024-08-20T02:57:00.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:56:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:00.235 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:59 smithi044 ceph-mon[18790]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-08-20T02:57:00.235 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:59 smithi044 ceph-mon[18790]: osdmap e37: 6 total, 6 up, 6 in 2024-08-20T02:57:00.236 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:59 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/95259782' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03df0c2f-1b7c-4242-888c-e047322fd8f8"} : dispatch 2024-08-20T02:57:00.236 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:59 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/95259782' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03df0c2f-1b7c-4242-888c-e047322fd8f8"}]': finished 2024-08-20T02:57:00.236 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:59 smithi044 ceph-mon[18790]: osdmap e38: 7 total, 6 up, 7 in 2024-08-20T02:57:00.236 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:56:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:01.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:00 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1078754367' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:57:01.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:00 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1078754367' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:57:01.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:00 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1078754367' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:57:02.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:01 smithi165 ceph-mon[22357]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:02.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:01 smithi099 ceph-mon[21984]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:02.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:01 smithi044 ceph-mon[18790]: pgmap v117: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:04.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:03 smithi044 ceph-mon[18790]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:04.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:03 smithi165 ceph-mon[22357]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:04.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:03 smithi099 ceph-mon[21984]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:06.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:05 smithi165 ceph-mon[22357]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-08-20T02:57:06.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:05 smithi099 ceph-mon[21984]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-08-20T02:57:06.274 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:05 smithi044 ceph-mon[18790]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-08-20T02:57:08.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:07 smithi165 ceph-mon[22357]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T02:57:08.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:07 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-20T02:57:08.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:07 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:08.140 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:07 smithi044 ceph-mon[18790]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T02:57:08.140 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:07 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-20T02:57:08.140 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:07 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:08.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:07 smithi099 ceph-mon[21984]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T02:57:08.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:07 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-20T02:57:08.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:07 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:09.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:08 smithi165 ceph-mon[22357]: Deploying daemon osd.6 on smithi044 2024-08-20T02:57:09.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:08 smithi099 ceph-mon[21984]: Deploying daemon osd.6 on smithi044 2024-08-20T02:57:09.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:08 smithi044 ceph-mon[18790]: Deploying daemon osd.6 on smithi044 2024-08-20T02:57:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:09 smithi165 ceph-mon[22357]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-08-20T02:57:10.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:09 smithi099 ceph-mon[21984]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-08-20T02:57:10.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:09 smithi044 ceph-mon[18790]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-08-20T02:57:12.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:11 smithi165 ceph-mon[22357]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-08-20T02:57:12.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:11 smithi099 ceph-mon[21984]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-08-20T02:57:12.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:11 smithi044 ceph-mon[18790]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-08-20T02:57:14.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:13 smithi044 ceph-mon[18790]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:57:14.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:57:14.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:14.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:14.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:13 smithi165 ceph-mon[22357]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:57:14.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:57:14.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:14.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:14.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:13 smithi099 ceph-mon[21984]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:57:14.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:57:14.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:14.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:16.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:15 smithi165 ceph-mon[22357]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:57:16.155 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:15 smithi099 ceph-mon[21984]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:57:16.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:15 smithi044 ceph-mon[18790]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:57:17.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.208+0000 7fef12ffd640 1 -- 172.21.15.44:0/2368508108 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fef1c06a880 con 0x7fef080b3c20 2024-08-20T02:57:17.210 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 6 on host 'smithi044' 2024-08-20T02:57:17.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.210+0000 7fef23b70640 1 -- 172.21.15.44:0/2368508108 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fef080b3c20 msgr2=0x7fef080b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:57:17.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.210+0000 7fef23b70640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fef080b3c20 0x7fef080b60e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef18006010 tx=0x7fef18009820 comp rx=0 tx=0).stop 2024-08-20T02:57:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.210+0000 7fef23b70640 1 -- 172.21.15.44:0/2368508108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef1c107d20 msgr2=0x7fef1c103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:57:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.210+0000 7fef23b70640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef1c107d20 0x7fef1c103cd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef1400c970 tx=0x7fef1400ce30 comp rx=0 tx=0).stop 2024-08-20T02:57:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.210+0000 7fef23b70640 1 -- 172.21.15.44:0/2368508108 shutdown_connections 2024-08-20T02:57:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.210+0000 7fef23b70640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fef080b3c20 0x7fef080b60e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.210+0000 7fef23b70640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef1c107d20 0x7fef1c103cd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.210+0000 7fef23b70640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef1c1073a0 0x7fef1c103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.210+0000 7fef23b70640 1 --2- 172.21.15.44:0/2368508108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef1c101990 0x7fef1c103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.210+0000 7fef23b70640 1 -- 172.21.15.44:0/2368508108 >> 172.21.15.44:0/2368508108 conn(0x7fef1c0fd400 msgr2=0x7fef1c0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:57:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.210+0000 7fef23b70640 1 -- 172.21.15.44:0/2368508108 shutdown_connections 2024-08-20T02:57:17.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:57:17.210+0000 7fef23b70640 1 -- 172.21.15.44:0/2368508108 wait complete. 2024-08-20T02:57:17.330 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:17.330 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:17.330 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:17.330 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:57:17.330 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:17 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:17.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:17.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:17.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:17.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:57:17.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:17 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:17.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:17.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:57:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:17 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:17.965 DEBUG:teuthology.orchestra.run.smithi044:osd.6> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.6.service 2024-08-20T02:57:17.967 INFO:tasks.cephadm:Deploying osd.7 on smithi099 with /dev/vg_nvme/lv_2... 2024-08-20T02:57:17.968 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-20T02:57:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:18 smithi044 ceph-mon[18790]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:18 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:57:18.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:18 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:18.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:18 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:18.225 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T02:57:18.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:18 smithi099 ceph-mon[21984]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:18.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:18 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:57:18.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:18 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:18.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:18 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:18.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:18 smithi165 ceph-mon[22357]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:18.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:18 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:57:18.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:18 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:18.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:18 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:19.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 02:57:18 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T02:57:18.661+0000 7f26e8ec4640 -1 osd.6 0 log_to_monitors true 2024-08-20T02:57:19.367 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:19 smithi044 ceph-mon[18790]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-20T02:57:19.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:19 smithi165 ceph-mon[22357]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-20T02:57:19.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:19 smithi099 ceph-mon[21984]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-20T02:57:20.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:20 smithi165 ceph-mon[22357]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:20.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:20 smithi165 ceph-mon[22357]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-20T02:57:20.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:20 smithi165 ceph-mon[22357]: osdmap e39: 7 total, 6 up, 7 in 2024-08-20T02:57:20.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:20 smithi165 ceph-mon[22357]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]} : dispatch 2024-08-20T02:57:20.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:20 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:20.402 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:20 smithi099 ceph-mon[21984]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:20.633 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:20 smithi099 ceph-mon[21984]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-20T02:57:20.633 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:20 smithi099 ceph-mon[21984]: osdmap e39: 7 total, 6 up, 7 in 2024-08-20T02:57:20.633 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:20 smithi099 ceph-mon[21984]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]} : dispatch 2024-08-20T02:57:20.633 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:20 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:20.634 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:20 smithi044 ceph-mon[18790]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:20.634 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:20 smithi044 ceph-mon[18790]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-20T02:57:20.634 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:20 smithi044 ceph-mon[18790]: osdmap e39: 7 total, 6 up, 7 in 2024-08-20T02:57:20.634 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:20 smithi044 ceph-mon[18790]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]} : dispatch 2024-08-20T02:57:20.634 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:20 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:20.786 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 02:57:20 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T02:57:20.506+0000 7f26e4e47640 -1 osd.6 0 waiting for initial osdmap 2024-08-20T02:57:20.786 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 02:57:20 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T02:57:20.516+0000 7f26dcc66640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T02:57:21.137 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:57:21.161 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_2 2024-08-20T02:57:21.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-08-20T02:57:21.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: osdmap e40: 7 total, 6 up, 7 in 2024-08-20T02:57:21.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:21.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:21.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' 2024-08-20T02:57:21.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:21.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:21.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:21.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:21.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:21.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:21.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:21.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:57:21.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:21.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-08-20T02:57:21.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: osdmap e40: 7 total, 6 up, 7 in 2024-08-20T02:57:21.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:21.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:21.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' 2024-08-20T02:57:21.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:21.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:21.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:21.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:21.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:21.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:21.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:21.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:57:21.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:21.403 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T02:57:21.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-08-20T02:57:21.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: osdmap e40: 7 total, 6 up, 7 in 2024-08-20T02:57:21.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:21.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:21.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986]' entity='osd.6' 2024-08-20T02:57:21.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:21.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:21.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:21.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:21.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:21.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:21.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:21.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:57:21.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:21.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.937+0000 7f1fb0c73640 1 -- 172.21.15.99:0/483989255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fac10ce20 msgr2=0x7f1fac10f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:57:21.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.937+0000 7f1fb0c73640 1 --2- 172.21.15.99:0/483989255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fac10ce20 0x7f1fac10f280 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009be0 tx=0x7f1fa402a6c0 comp rx=0 tx=0).stop 2024-08-20T02:57:21.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.937+0000 7f1fb0c73640 1 -- 172.21.15.99:0/483989255 shutdown_connections 2024-08-20T02:57:21.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.937+0000 7f1fb0c73640 1 --2- 172.21.15.99:0/483989255 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fac10ce20 0x7f1fac10f280 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:21.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.937+0000 7f1fb0c73640 1 --2- 172.21.15.99:0/483989255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fac1042e0 0x7f1fac10c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:21.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.937+0000 7f1fb0c73640 1 --2- 172.21.15.99:0/483989255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fac1039c0 0x7f1fac103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:21.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.937+0000 7f1fb0c73640 1 -- 172.21.15.99:0/483989255 >> 172.21.15.99:0/483989255 conn(0x7f1fac0fd420 msgr2=0x7f1fac0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:57:21.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.937+0000 7f1fb0c73640 1 -- 172.21.15.99:0/483989255 shutdown_connections 2024-08-20T02:57:21.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.937+0000 7f1fb0c73640 1 -- 172.21.15.99:0/483989255 wait complete. 2024-08-20T02:57:21.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1fb0c73640 1 Processor -- start 2024-08-20T02:57:21.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1fb0c73640 1 -- start start 2024-08-20T02:57:21.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1fb0c73640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fac1039c0 0x7f1fac101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:57:21.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1fb0c73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fac1042e0 0x7f1fac101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:57:21.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1fb0c73640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fac10ce20 0x7f1fac1020f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:57:21.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1fb0c73640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac112a90 con 0x7f1fac1042e0 2024-08-20T02:57:21.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1fb0c73640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac112910 con 0x7f1fac10ce20 2024-08-20T02:57:21.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1fb0c73640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac112c10 con 0x7f1fac1039c0 2024-08-20T02:57:21.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1faad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fac10ce20 0x7f1fac1020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:57:21.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1faad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fac10ce20 0x7f1fac1020f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53072/0 (socket says 172.21.15.99:53072) 2024-08-20T02:57:21.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1faad9d640 1 -- 172.21.15.99:0/397676369 learned_addr learned my addr 172.21.15.99:0/397676369 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-08-20T02:57:21.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1fa3fff640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fac1042e0 0x7f1fac101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:57:21.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1faa59c640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fac1039c0 0x7f1fac101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:57:21.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1faad9d640 1 -- 172.21.15.99:0/397676369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fac1039c0 msgr2=0x7f1fac101670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:57:21.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1faad9d640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fac1039c0 0x7f1fac101670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:21.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1faad9d640 1 -- 172.21.15.99:0/397676369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fac1042e0 msgr2=0x7f1fac101bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:57:21.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1faad9d640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fac1042e0 0x7f1fac101bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:21.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.938+0000 7f1faad9d640 1 -- 172.21.15.99:0/397676369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fac1a7340 con 0x7f1fac10ce20 2024-08-20T02:57:21.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.939+0000 7f1faad9d640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fac10ce20 0x7f1fac1020f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4004af0 tx=0x7f1fa4003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:57:21.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.939+0000 7f1fa37fe640 1 -- 172.21.15.99:0/397676369 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4041020 con 0x7f1fac10ce20 2024-08-20T02:57:21.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.939+0000 7f1fb0c73640 1 -- 172.21.15.99:0/397676369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fac1a75d0 con 0x7f1fac10ce20 2024-08-20T02:57:21.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.939+0000 7f1fa37fe640 1 -- 172.21.15.99:0/397676369 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fa4003e10 con 0x7f1fac10ce20 2024-08-20T02:57:21.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.939+0000 7f1fa37fe640 1 -- 172.21.15.99:0/397676369 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4036040 con 0x7f1fac10ce20 2024-08-20T02:57:21.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.939+0000 7f1fb0c73640 1 -- 172.21.15.99:0/397676369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fac1a7b00 con 0x7f1fac10ce20 2024-08-20T02:57:21.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.940+0000 7f1fb0c73640 1 -- 172.21.15.99:0/397676369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fac103da0 con 0x7f1fac10ce20 2024-08-20T02:57:21.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.941+0000 7f1fa37fe640 1 -- 172.21.15.99:0/397676369 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f1fa4044020 con 0x7f1fac10ce20 2024-08-20T02:57:21.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.942+0000 7f1fa37fe640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f980b3c20 0x7f1f980b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:57:21.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.942+0000 7f1fa37fe640 1 -- 172.21.15.99:0/397676369 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5180+0+0 (secure 0 0 0) 0x7f1fa4103ff0 con 0x7f1fac10ce20 2024-08-20T02:57:21.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.942+0000 7f1faa59c640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f980b3c20 0x7f1f980b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:57:21.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.943+0000 7f1faa59c640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f980b3c20 0x7f1f980b60e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c005fd0 tx=0x7f1f9c009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:57:21.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:21.945+0000 7f1fa37fe640 1 -- 172.21.15.99:0/397676369 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f1fa404b020 con 0x7f1fac10ce20 2024-08-20T02:57:22.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:22.130+0000 7f1fb0c73640 1 -- 172.21.15.99:0/397676369 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1fac0ff030 con 0x7f1f980b3c20 2024-08-20T02:57:22.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:22 smithi165 ceph-mon[22357]: purged_snaps scrub starts 2024-08-20T02:57:22.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:22 smithi165 ceph-mon[22357]: purged_snaps scrub ok 2024-08-20T02:57:22.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:22 smithi165 ceph-mon[22357]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:22.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:22 smithi165 ceph-mon[22357]: Detected new or changed devices on smithi044 2024-08-20T02:57:22.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:22 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi044 to 5697M 2024-08-20T02:57:22.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:22 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:22.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:22 smithi165 ceph-mon[22357]: osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986] boot 2024-08-20T02:57:22.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:22 smithi165 ceph-mon[22357]: osdmap e41: 7 total, 7 up, 7 in 2024-08-20T02:57:22.388 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:22 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:22.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:22 smithi099 ceph-mon[21984]: purged_snaps scrub starts 2024-08-20T02:57:22.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:22 smithi099 ceph-mon[21984]: purged_snaps scrub ok 2024-08-20T02:57:22.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:22 smithi099 ceph-mon[21984]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:22.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:22 smithi099 ceph-mon[21984]: Detected new or changed devices on smithi044 2024-08-20T02:57:22.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:22 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi044 to 5697M 2024-08-20T02:57:22.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:22 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:22.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:22 smithi099 ceph-mon[21984]: osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986] boot 2024-08-20T02:57:22.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:22 smithi099 ceph-mon[21984]: osdmap e41: 7 total, 7 up, 7 in 2024-08-20T02:57:22.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:22 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:22.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:22 smithi044 ceph-mon[18790]: purged_snaps scrub starts 2024-08-20T02:57:22.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:22 smithi044 ceph-mon[18790]: purged_snaps scrub ok 2024-08-20T02:57:22.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:22 smithi044 ceph-mon[18790]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-08-20T02:57:22.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:22 smithi044 ceph-mon[18790]: Detected new or changed devices on smithi044 2024-08-20T02:57:22.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:22 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi044 to 5697M 2024-08-20T02:57:22.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:22 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:22.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:22 smithi044 ceph-mon[18790]: osd.6 [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986] boot 2024-08-20T02:57:22.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:22 smithi044 ceph-mon[18790]: osdmap e41: 7 total, 7 up, 7 in 2024-08-20T02:57:22.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:22 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T02:57:23.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:23 smithi165 ceph-mon[22357]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:57:23.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:23 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:57:23.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:23 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:57:23.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:23 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:23.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:23 smithi165 ceph-mon[22357]: osdmap e42: 7 total, 7 up, 7 in 2024-08-20T02:57:23.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:23 smithi099 ceph-mon[21984]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:57:23.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:23 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:57:23.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:23 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:57:23.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:23 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:23.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:23 smithi099 ceph-mon[21984]: osdmap e42: 7 total, 7 up, 7 in 2024-08-20T02:57:23.437 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:23 smithi044 ceph-mon[18790]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:57:23.437 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:23 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:57:23.437 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:23 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:57:23.437 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:23 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:23.437 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:23 smithi044 ceph-mon[18790]: osdmap e42: 7 total, 7 up, 7 in 2024-08-20T02:57:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:24 smithi165 ceph-mon[22357]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:24.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:24 smithi099 ceph-mon[21984]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:24.453 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:24 smithi044 ceph-mon[18790]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:25.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:25 smithi165 ceph-mon[22357]: from='client.? 172.21.15.99:0/189031419' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84bbc683-9b13-448e-897c-114458dae33b"} : dispatch 2024-08-20T02:57:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:25 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84bbc683-9b13-448e-897c-114458dae33b"} : dispatch 2024-08-20T02:57:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:25 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84bbc683-9b13-448e-897c-114458dae33b"}]': finished 2024-08-20T02:57:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:25 smithi165 ceph-mon[22357]: osdmap e43: 8 total, 7 up, 8 in 2024-08-20T02:57:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:25 smithi165 ceph-mon[22357]: from='client.? 172.21.15.99:0/418005706' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:57:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:25 smithi099 ceph-mon[21984]: from='client.? 172.21.15.99:0/189031419' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84bbc683-9b13-448e-897c-114458dae33b"} : dispatch 2024-08-20T02:57:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:25 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84bbc683-9b13-448e-897c-114458dae33b"} : dispatch 2024-08-20T02:57:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:25 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84bbc683-9b13-448e-897c-114458dae33b"}]': finished 2024-08-20T02:57:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:25 smithi099 ceph-mon[21984]: osdmap e43: 8 total, 7 up, 8 in 2024-08-20T02:57:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:25.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:25 smithi099 ceph-mon[21984]: from='client.? 172.21.15.99:0/418005706' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:57:25.538 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:25 smithi044 ceph-mon[18790]: from='client.? 172.21.15.99:0/189031419' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84bbc683-9b13-448e-897c-114458dae33b"} : dispatch 2024-08-20T02:57:25.538 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:25 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84bbc683-9b13-448e-897c-114458dae33b"} : dispatch 2024-08-20T02:57:25.538 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:25 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84bbc683-9b13-448e-897c-114458dae33b"}]': finished 2024-08-20T02:57:25.538 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:25 smithi044 ceph-mon[18790]: osdmap e43: 8 total, 7 up, 8 in 2024-08-20T02:57:25.539 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:25.539 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:25 smithi044 ceph-mon[18790]: from='client.? 172.21.15.99:0/418005706' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:57:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:26 smithi165 ceph-mon[22357]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:26.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:26 smithi099 ceph-mon[21984]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:26.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:26 smithi044 ceph-mon[18790]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:28.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:28 smithi165 ceph-mon[22357]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:28.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:28 smithi099 ceph-mon[21984]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:28.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:28 smithi044 ceph-mon[18790]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:30.255 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:30 smithi099 ceph-mon[21984]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:30.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:30 smithi165 ceph-mon[22357]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:30 smithi044 ceph-mon[18790]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:32.344 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:32 smithi099 ceph-mon[21984]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:32 smithi165 ceph-mon[22357]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:32.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:32 smithi044 ceph-mon[18790]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:33.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:33 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-20T02:57:33.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:33 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:33.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:33 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-20T02:57:33.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:33 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:33.539 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:33 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-20T02:57:33.539 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:33 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:34 smithi165 ceph-mon[22357]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:34 smithi165 ceph-mon[22357]: Deploying daemon osd.7 on smithi099 2024-08-20T02:57:34.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:34 smithi099 ceph-mon[21984]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:34.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:34 smithi099 ceph-mon[21984]: Deploying daemon osd.7 on smithi099 2024-08-20T02:57:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:34 smithi044 ceph-mon[18790]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:34 smithi044 ceph-mon[18790]: Deploying daemon osd.7 on smithi099 2024-08-20T02:57:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:36 smithi099 ceph-mon[21984]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:36 smithi044 ceph-mon[18790]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:36 smithi165 ceph-mon[22357]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:38.354 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:38 smithi099 ceph-mon[21984]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:38.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:38 smithi044 ceph-mon[18790]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:38.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:38 smithi165 ceph-mon[22357]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:39.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:39 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:57:39.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:39 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:39.332 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:39 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:39.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:39 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:57:39.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:39 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:39.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:39 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:39.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:39 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:57:39.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:39 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:39.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:39 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:40.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:40 smithi099 ceph-mon[21984]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:40.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:40 smithi044 ceph-mon[18790]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:40.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:40 smithi165 ceph-mon[22357]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:42.329 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:42 smithi099 ceph-mon[21984]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:42.330 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:42 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:42.330 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:42 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:42.330 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:42 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:42.330 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:42 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:57:42.330 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:42 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:42.371 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 7 on host 'smithi099' 2024-08-20T02:57:42.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.368+0000 7f1fa37fe640 1 -- 172.21.15.99:0/397676369 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1fac0ff030 con 0x7f1f980b3c20 2024-08-20T02:57:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.370+0000 7f1fb0c73640 1 -- 172.21.15.99:0/397676369 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f980b3c20 msgr2=0x7f1f980b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:57:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.370+0000 7f1fb0c73640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f980b3c20 0x7f1f980b60e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c005fd0 tx=0x7f1f9c009260 comp rx=0 tx=0).stop 2024-08-20T02:57:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.370+0000 7f1fb0c73640 1 -- 172.21.15.99:0/397676369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fac10ce20 msgr2=0x7f1fac1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:57:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.370+0000 7f1fb0c73640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fac10ce20 0x7f1fac1020f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4004af0 tx=0x7f1fa4003c20 comp rx=0 tx=0).stop 2024-08-20T02:57:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.370+0000 7f1fb0c73640 1 -- 172.21.15.99:0/397676369 shutdown_connections 2024-08-20T02:57:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.370+0000 7f1fb0c73640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f980b3c20 0x7f1f980b60e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.370+0000 7f1fb0c73640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fac10ce20 0x7f1fac1020f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.370+0000 7f1fb0c73640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fac1042e0 0x7f1fac101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.370+0000 7f1fb0c73640 1 --2- 172.21.15.99:0/397676369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fac1039c0 0x7f1fac101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.370+0000 7f1fb0c73640 1 -- 172.21.15.99:0/397676369 >> 172.21.15.99:0/397676369 conn(0x7f1fac0fd420 msgr2=0x7f1fac10f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:57:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.370+0000 7f1fb0c73640 1 -- 172.21.15.99:0/397676369 shutdown_connections 2024-08-20T02:57:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:57:42.370+0000 7f1fb0c73640 1 -- 172.21.15.99:0/397676369 wait complete. 2024-08-20T02:57:42.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:42 smithi044 ceph-mon[18790]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:42.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:42 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:42.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:42 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:42.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:42 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:42.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:42 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:57:42.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:42 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:42.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:42 smithi165 ceph-mon[22357]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:42.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:42 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:42.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:42 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:42.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:42 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:42.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:42 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:57:42.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:42 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:43.061 DEBUG:teuthology.orchestra.run.smithi099:osd.7> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.7.service 2024-08-20T02:57:43.064 INFO:tasks.cephadm:Deploying osd.8 on smithi165 with /dev/vg_nvme/lv_2... 2024-08-20T02:57:43.065 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-20T02:57:43.307 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:57:43.307 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:43.307 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:43.325 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:57:43.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:57:43.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:43.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:43.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:57:43.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:43.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:44 smithi099 ceph-mon[21984]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:44.463 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:44 smithi165 ceph-mon[22357]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:44.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:44 smithi044 ceph-mon[18790]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:44.607 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 02:57:44 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T02:57:44.451+0000 7fe866e3d640 -1 osd.7 0 log_to_monitors true 2024-08-20T02:57:45.413 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:45 smithi165 ceph-mon[22357]: from='osd.7 [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T02:57:45.413 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:45 smithi165 ceph-mon[22357]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T02:57:45.470 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:45 smithi099 ceph-mon[21984]: from='osd.7 [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T02:57:45.470 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:45 smithi099 ceph-mon[21984]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T02:57:45.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:45 smithi044 ceph-mon[18790]: from='osd.7 [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T02:57:45.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:45 smithi044 ceph-mon[18790]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T02:57:45.979 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:57:46.007 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_2 2024-08-20T02:57:46.260 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:57:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:46 smithi165 ceph-mon[22357]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:46 smithi165 ceph-mon[22357]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-20T02:57:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:46 smithi165 ceph-mon[22357]: osdmap e44: 8 total, 7 up, 8 in 2024-08-20T02:57:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:46 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:46 smithi165 ceph-mon[22357]: from='osd.7 [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:57:46.369 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:46 smithi165 ceph-mon[22357]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:57:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:46 smithi099 ceph-mon[21984]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:46 smithi099 ceph-mon[21984]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-20T02:57:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:46 smithi099 ceph-mon[21984]: osdmap e44: 8 total, 7 up, 8 in 2024-08-20T02:57:46.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:46 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:46.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:46 smithi099 ceph-mon[21984]: from='osd.7 [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:57:46.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:46 smithi099 ceph-mon[21984]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:57:46.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:46 smithi044 ceph-mon[18790]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:46.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:46 smithi044 ceph-mon[18790]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-20T02:57:46.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:46 smithi044 ceph-mon[18790]: osdmap e44: 8 total, 7 up, 8 in 2024-08-20T02:57:46.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:46 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:46.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:46 smithi044 ceph-mon[18790]: from='osd.7 [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:57:46.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:46 smithi044 ceph-mon[18790]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:57:46.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.741+0000 7f55df3b1640 1 -- 172.21.15.165:0/4152222709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d8106c10 msgr2=0x7f55d8107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:57:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.741+0000 7f55df3b1640 1 --2- 172.21.15.165:0/4152222709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d8106c10 0x7f55d8107060 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009ca0 tx=0x7f55d002a8c0 comp rx=0 tx=0).stop 2024-08-20T02:57:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.741+0000 7f55df3b1640 1 -- 172.21.15.165:0/4152222709 shutdown_connections 2024-08-20T02:57:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.741+0000 7f55df3b1640 1 --2- 172.21.15.165:0/4152222709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d8106c10 0x7f55d8107060 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.741+0000 7f55df3b1640 1 --2- 172.21.15.165:0/4152222709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d81020f0 0x7f55d81065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.741+0000 7f55df3b1640 1 --2- 172.21.15.165:0/4152222709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55d81017d0 0x7f55d8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.741+0000 7f55df3b1640 1 -- 172.21.15.165:0/4152222709 >> 172.21.15.165:0/4152222709 conn(0x7f55d80fd440 msgr2=0x7f55d80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:57:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.741+0000 7f55df3b1640 1 -- 172.21.15.165:0/4152222709 shutdown_connections 2024-08-20T02:57:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.741+0000 7f55df3b1640 1 -- 172.21.15.165:0/4152222709 wait complete. 2024-08-20T02:57:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.742+0000 7f55df3b1640 1 Processor -- start 2024-08-20T02:57:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.742+0000 7f55df3b1640 1 -- start start 2024-08-20T02:57:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.742+0000 7f55df3b1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d81017d0 0x7f55d819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:57:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.742+0000 7f55df3b1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55d81020f0 0x7f55d819fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:57:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.742+0000 7f55df3b1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d8106c10 0x7f55d81a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:57:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.742+0000 7f55df3b1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8119130 con 0x7f55d81017d0 2024-08-20T02:57:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.742+0000 7f55df3b1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8118fb0 con 0x7f55d81020f0 2024-08-20T02:57:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.742+0000 7f55df3b1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81192b0 con 0x7f55d8106c10 2024-08-20T02:57:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.742+0000 7f55dd94e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d8106c10 0x7f55d81a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:57:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.742+0000 7f55dd94e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d8106c10 0x7f55d81a4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39664/0 (socket says 172.21.15.165:39664) 2024-08-20T02:57:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.742+0000 7f55dd94e640 1 -- 172.21.15.165:0/3571124610 learned_addr learned my addr 172.21.15.165:0/3571124610 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-08-20T02:57:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55dd14d640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d81017d0 0x7f55d819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:57:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55dc94c640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55d81020f0 0x7f55d819fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:57:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55dd94e640 1 -- 172.21.15.165:0/3571124610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55d81020f0 msgr2=0x7f55d819fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:57:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55dd94e640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55d81020f0 0x7f55d819fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55dd94e640 1 -- 172.21.15.165:0/3571124610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d81017d0 msgr2=0x7f55d819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:57:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55dd94e640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d81017d0 0x7f55d819f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:57:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55dd94e640 1 -- 172.21.15.165:0/3571124610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d81a56d0 con 0x7f55d8106c10 2024-08-20T02:57:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55dd94e640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d8106c10 0x7f55d81a4f00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f55d0005cc0 tx=0x7f55d002f380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:57:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55ce7fc640 1 -- 172.21.15.165:0/3571124610 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0003bf0 con 0x7f55d8106c10 2024-08-20T02:57:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55ce7fc640 1 -- 172.21.15.165:0/3571124610 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d0003d90 con 0x7f55d8106c10 2024-08-20T02:57:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55df3b1640 1 -- 172.21.15.165:0/3571124610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d81a5960 con 0x7f55d8106c10 2024-08-20T02:57:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55ce7fc640 1 -- 172.21.15.165:0/3571124610 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d002fbc0 con 0x7f55d8106c10 2024-08-20T02:57:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.743+0000 7f55df3b1640 1 -- 172.21.15.165:0/3571124610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d81ac100 con 0x7f55d8106c10 2024-08-20T02:57:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.744+0000 7f55df3b1640 1 -- 172.21.15.165:0/3571124610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d81065c0 con 0x7f55d8106c10 2024-08-20T02:57:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.746+0000 7f55ce7fc640 1 -- 172.21.15.165:0/3571124610 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f55d002fd60 con 0x7f55d8106c10 2024-08-20T02:57:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.746+0000 7f55ce7fc640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55c40b3c20 0x7f55c40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:57:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.746+0000 7f55ce7fc640 1 -- 172.21.15.165:0/3571124610 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5365+0+0 (secure 0 0 0) 0x7f55d0104a40 con 0x7f55d8106c10 2024-08-20T02:57:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.747+0000 7f55dd14d640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55c40b3c20 0x7f55c40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:57:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.747+0000 7f55dd14d640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55c40b3c20 0x7f55c40b60e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55d4005fd0 tx=0x7f55d4005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:57:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.749+0000 7f55ce7fc640 1 -- 172.21.15.165:0/3571124610 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f55d00ccc10 con 0x7f55d8106c10 2024-08-20T02:57:46.914 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 02:57:46 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T02:57:46.562+0000 7fe862dc0640 -1 osd.7 0 waiting for initial osdmap 2024-08-20T02:57:46.915 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 02:57:46 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T02:57:46.576+0000 7fe85abdf640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T02:57:46.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:57:46.933+0000 7f55df3b1640 1 -- 172.21.15.165:0/3571124610 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f55d8105c60 con 0x7f55c40b3c20 2024-08-20T02:57:47.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-08-20T02:57:47.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: osdmap e45: 8 total, 7 up, 8 in 2024-08-20T02:57:47.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:47.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:47.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: Detected new or changed devices on smithi099 2024-08-20T02:57:47.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:47.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:47.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:47.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:47.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:47.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi099 to 5696M 2024-08-20T02:57:47.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:47.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:47.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:57:47.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:47.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='osd.7 ' entity='osd.7' 2024-08-20T02:57:47.349 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: osdmap e46: 8 total, 7 up, 8 in 2024-08-20T02:57:47.349 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:47.349 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:57:47.349 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:57:47.349 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:47.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-08-20T02:57:47.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: osdmap e45: 8 total, 7 up, 8 in 2024-08-20T02:57:47.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:47.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:47.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: Detected new or changed devices on smithi099 2024-08-20T02:57:47.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:47.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:47.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi099 to 5696M 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='osd.7 ' entity='osd.7' 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: osdmap e46: 8 total, 7 up, 8 in 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:57:47.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:47.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-08-20T02:57:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: osdmap e45: 8 total, 7 up, 8 in 2024-08-20T02:57:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: Detected new or changed devices on smithi099 2024-08-20T02:57:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T02:57:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi099 to 5696M 2024-08-20T02:57:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:47.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:47.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:57:47.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:57:47.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='osd.7 ' entity='osd.7' 2024-08-20T02:57:47.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: osdmap e46: 8 total, 7 up, 8 in 2024-08-20T02:57:47.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:47.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:57:47.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:57:47.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:48.279 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:48 smithi165 ceph-mon[22357]: purged_snaps scrub starts 2024-08-20T02:57:48.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:48 smithi165 ceph-mon[22357]: purged_snaps scrub ok 2024-08-20T02:57:48.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:48 smithi165 ceph-mon[22357]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:48.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:48 smithi165 ceph-mon[22357]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:57:48.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:48 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:48.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:48 smithi165 ceph-mon[22357]: osd.7 [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564] boot 2024-08-20T02:57:48.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:48 smithi165 ceph-mon[22357]: osdmap e47: 8 total, 8 up, 8 in 2024-08-20T02:57:48.280 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:48 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:48.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:48 smithi044 ceph-mon[18790]: purged_snaps scrub starts 2024-08-20T02:57:48.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:48 smithi044 ceph-mon[18790]: purged_snaps scrub ok 2024-08-20T02:57:48.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:48 smithi044 ceph-mon[18790]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:48.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:48 smithi044 ceph-mon[18790]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:57:48.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:48 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:48.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:48 smithi044 ceph-mon[18790]: osd.7 [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564] boot 2024-08-20T02:57:48.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:48 smithi044 ceph-mon[18790]: osdmap e47: 8 total, 8 up, 8 in 2024-08-20T02:57:48.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:48 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:48.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:48 smithi099 ceph-mon[21984]: purged_snaps scrub starts 2024-08-20T02:57:48.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:48 smithi099 ceph-mon[21984]: purged_snaps scrub ok 2024-08-20T02:57:48.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:48 smithi099 ceph-mon[21984]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-08-20T02:57:48.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:48 smithi099 ceph-mon[21984]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:57:48.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:48 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:48.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:48 smithi099 ceph-mon[21984]: osd.7 [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564] boot 2024-08-20T02:57:48.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:48 smithi099 ceph-mon[21984]: osdmap e47: 8 total, 8 up, 8 in 2024-08-20T02:57:48.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:48 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T02:57:49.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:49 smithi165 ceph-mon[22357]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:57:49.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:49 smithi165 ceph-mon[22357]: osdmap e48: 8 total, 8 up, 8 in 2024-08-20T02:57:49.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:49 smithi165 ceph-mon[22357]: from='client.? 172.21.15.165:0/1296955391' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14d7ffd3-db05-4228-a12e-4c69ef65ef8f"} : dispatch 2024-08-20T02:57:49.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:49 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14d7ffd3-db05-4228-a12e-4c69ef65ef8f"} : dispatch 2024-08-20T02:57:49.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:49 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d7ffd3-db05-4228-a12e-4c69ef65ef8f"}]': finished 2024-08-20T02:57:49.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:49 smithi165 ceph-mon[22357]: osdmap e49: 9 total, 8 up, 9 in 2024-08-20T02:57:49.871 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:49 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:57:49.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:49 smithi099 ceph-mon[21984]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:57:49.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:49 smithi099 ceph-mon[21984]: osdmap e48: 8 total, 8 up, 8 in 2024-08-20T02:57:49.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:49 smithi099 ceph-mon[21984]: from='client.? 172.21.15.165:0/1296955391' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14d7ffd3-db05-4228-a12e-4c69ef65ef8f"} : dispatch 2024-08-20T02:57:49.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:49 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14d7ffd3-db05-4228-a12e-4c69ef65ef8f"} : dispatch 2024-08-20T02:57:49.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:49 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d7ffd3-db05-4228-a12e-4c69ef65ef8f"}]': finished 2024-08-20T02:57:49.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:49 smithi099 ceph-mon[21984]: osdmap e49: 9 total, 8 up, 9 in 2024-08-20T02:57:49.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:49 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:57:50.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:49 smithi044 ceph-mon[18790]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:57:50.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:49 smithi044 ceph-mon[18790]: osdmap e48: 8 total, 8 up, 8 in 2024-08-20T02:57:50.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:49 smithi044 ceph-mon[18790]: from='client.? 172.21.15.165:0/1296955391' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14d7ffd3-db05-4228-a12e-4c69ef65ef8f"} : dispatch 2024-08-20T02:57:50.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:49 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "14d7ffd3-db05-4228-a12e-4c69ef65ef8f"} : dispatch 2024-08-20T02:57:50.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:49 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d7ffd3-db05-4228-a12e-4c69ef65ef8f"}]': finished 2024-08-20T02:57:50.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:49 smithi044 ceph-mon[18790]: osdmap e49: 9 total, 8 up, 9 in 2024-08-20T02:57:50.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:49 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:57:50.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:50 smithi165 ceph-mon[22357]: from='client.? 172.21.15.165:0/286699835' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:57:50.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:50 smithi099 ceph-mon[21984]: from='client.? 172.21.15.165:0/286699835' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:57:51.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:50 smithi044 ceph-mon[18790]: from='client.? 172.21.15.165:0/286699835' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:57:51.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:51 smithi165 ceph-mon[22357]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:57:51.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:51 smithi099 ceph-mon[21984]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:57:52.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:51 smithi044 ceph-mon[18790]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:57:53.847 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:53 smithi044 ceph-mon[18790]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:57:53.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:53 smithi165 ceph-mon[22357]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:57:53.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:53 smithi099 ceph-mon[21984]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:57:55.849 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:55 smithi165 ceph-mon[22357]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-08-20T02:57:55.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:55 smithi099 ceph-mon[21984]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-08-20T02:57:56.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:55 smithi044 ceph-mon[18790]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-08-20T02:57:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:57 smithi165 ceph-mon[22357]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T02:57:57.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:57 smithi099 ceph-mon[21984]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T02:57:58.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:57 smithi044 ceph-mon[18790]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T02:57:58.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-20T02:57:58.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:58.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:58 smithi165 ceph-mon[22357]: Deploying daemon osd.8 on smithi165 2024-08-20T02:57:58.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-20T02:57:58.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:58.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:58 smithi099 ceph-mon[21984]: Deploying daemon osd.8 on smithi165 2024-08-20T02:57:59.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-20T02:57:59.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:57:59.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:58 smithi044 ceph-mon[18790]: Deploying daemon osd.8 on smithi165 2024-08-20T02:57:59.884 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:57:59 smithi165 ceph-mon[22357]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-08-20T02:57:59.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:57:59 smithi099 ceph-mon[21984]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-08-20T02:58:00.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:57:59 smithi044 ceph-mon[18790]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-08-20T02:58:01.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:01 smithi165 ceph-mon[22357]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-08-20T02:58:01.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:01 smithi099 ceph-mon[21984]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-08-20T02:58:02.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:01 smithi044 ceph-mon[18790]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-08-20T02:58:03.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:03 smithi165 ceph-mon[22357]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:58:03.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:03.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:03.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:03.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:03 smithi099 ceph-mon[21984]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:58:03.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:03.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:03.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:04.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:03 smithi044 ceph-mon[18790]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:58:04.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:04.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:04.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:05.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:05 smithi165 ceph-mon[22357]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:58:05.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:05 smithi099 ceph-mon[21984]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:58:06.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:05 smithi044 ceph-mon[18790]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:58:06.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.627+0000 7f55ce7fc640 1 -- 172.21.15.165:0/3571124610 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f55d8105c60 con 0x7f55c40b3c20 2024-08-20T02:58:06.630 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 8 on host 'smithi165' 2024-08-20T02:58:06.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.629+0000 7f55df3b1640 1 -- 172.21.15.165:0/3571124610 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55c40b3c20 msgr2=0x7f55c40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:58:06.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.629+0000 7f55df3b1640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55c40b3c20 0x7f55c40b60e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55d4005fd0 tx=0x7f55d4005850 comp rx=0 tx=0).stop 2024-08-20T02:58:06.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.629+0000 7f55df3b1640 1 -- 172.21.15.165:0/3571124610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d8106c10 msgr2=0x7f55d81a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:58:06.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.629+0000 7f55df3b1640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d8106c10 0x7f55d81a4f00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f55d0005cc0 tx=0x7f55d002f380 comp rx=0 tx=0).stop 2024-08-20T02:58:06.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.630+0000 7f55df3b1640 1 -- 172.21.15.165:0/3571124610 shutdown_connections 2024-08-20T02:58:06.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.630+0000 7f55df3b1640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55c40b3c20 0x7f55c40b60e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:06.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.630+0000 7f55df3b1640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55d8106c10 0x7f55d81a4f00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:06.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.630+0000 7f55df3b1640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55d81020f0 0x7f55d819fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:06.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.630+0000 7f55df3b1640 1 --2- 172.21.15.165:0/3571124610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55d81017d0 0x7f55d819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:06.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.630+0000 7f55df3b1640 1 -- 172.21.15.165:0/3571124610 >> 172.21.15.165:0/3571124610 conn(0x7f55d80fd440 msgr2=0x7f55d8104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:58:06.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.630+0000 7f55df3b1640 1 -- 172.21.15.165:0/3571124610 shutdown_connections 2024-08-20T02:58:06.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:58:06.630+0000 7f55df3b1640 1 -- 172.21.15.165:0/3571124610 wait complete. 2024-08-20T02:58:07.296 DEBUG:teuthology.orchestra.run.smithi165:osd.8> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.8.service 2024-08-20T02:58:07.298 INFO:tasks.cephadm:Deploying osd.9 on smithi044 with /dev/vg_nvme/lv_1... 2024-08-20T02:58:07.298 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-20T02:58:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:58:07.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:07 smithi165 ceph-mon[22357]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:58:07.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:07 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:07.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:07 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:07.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:07 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:07.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:07 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:07.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:07 smithi099 ceph-mon[21984]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:58:07.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:07 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:07.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:07 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:07.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:07 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:07.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:07 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:07.924 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:07 smithi044 ceph-mon[18790]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:58:07.924 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:07 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:07.925 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:07 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:07.925 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:07 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:07.925 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:07 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:09.387 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 02:58:09 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T02:58:09.142+0000 7f0204d5f640 -1 osd.8 0 log_to_monitors true 2024-08-20T02:58:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:09 smithi165 ceph-mon[22357]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:58:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:09 smithi165 ceph-mon[22357]: from='osd.8 [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T02:58:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:09 smithi165 ceph-mon[22357]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T02:58:09.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:09 smithi099 ceph-mon[21984]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:58:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:09 smithi099 ceph-mon[21984]: from='osd.8 [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T02:58:09.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:09 smithi099 ceph-mon[21984]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T02:58:09.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:09 smithi044 ceph-mon[18790]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:58:09.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:09 smithi044 ceph-mon[18790]: from='osd.8 [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T02:58:09.938 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:09 smithi044 ceph-mon[18790]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T02:58:10.351 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:58:10.378 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi044:vg_nvme/lv_1 2024-08-20T02:58:10.642 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-20T02:58:10.642 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: osdmap e50: 9 total, 8 up, 9 in 2024-08-20T02:58:10.642 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:58:10.643 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='osd.8 [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:58:10.643 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:58:10.643 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.643 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.643 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:10.643 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:10.643 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:10.643 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.643 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:10.643 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:10.650 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:58:10.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-20T02:58:10.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: osdmap e50: 9 total, 8 up, 9 in 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='osd.8 [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:10.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:10.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:10.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-20T02:58:10.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: osdmap e50: 9 total, 8 up, 9 in 2024-08-20T02:58:10.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:58:10.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='osd.8 [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:58:10.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:58:10.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:10.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:10.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:10.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:10.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:10.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:10.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:10.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:10.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.921 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:10.921 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:10.921 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:10.922 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:10.922 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:11.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.143+0000 7f510b688640 1 -- 172.21.15.44:0/421628659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5104101990 msgr2=0x7f5104101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:58:11.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.143+0000 7f510b688640 1 --2- 172.21.15.44:0/421628659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5104101990 0x7f5104101d70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5100009900 tx=0x7f510002a410 comp rx=0 tx=0).stop 2024-08-20T02:58:11.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.143+0000 7f510b688640 1 -- 172.21.15.44:0/421628659 shutdown_connections 2024-08-20T02:58:11.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.143+0000 7f510b688640 1 --2- 172.21.15.44:0/421628659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5104107d20 0x7f510410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:11.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.143+0000 7f510b688640 1 --2- 172.21.15.44:0/421628659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51041073a0 0x7f51041077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.143+0000 7f510b688640 1 --2- 172.21.15.44:0/421628659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5104101990 0x7f5104101d70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.143+0000 7f510b688640 1 -- 172.21.15.44:0/421628659 >> 172.21.15.44:0/421628659 conn(0x7f51040fd400 msgr2=0x7f51040ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:58:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.143+0000 7f510b688640 1 -- 172.21.15.44:0/421628659 shutdown_connections 2024-08-20T02:58:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.143+0000 7f510b688640 1 -- 172.21.15.44:0/421628659 wait complete. 2024-08-20T02:58:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.143+0000 7f510b688640 1 Processor -- start 2024-08-20T02:58:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.143+0000 7f510b688640 1 -- start start 2024-08-20T02:58:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f510b688640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5104101990 0x7f5104198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:58:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f510b688640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51041073a0 0x7f5104199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:58:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f510b688640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5104107d20 0x7f510419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:58:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f510b688640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f51041119e0 con 0x7f51041073a0 2024-08-20T02:58:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f510b688640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5104111860 con 0x7f5104107d20 2024-08-20T02:58:11.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f510b688640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5104111b60 con 0x7f5104101990 2024-08-20T02:58:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5108c23640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51041073a0 0x7f5104199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:58:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5108c23640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51041073a0 0x7f5104199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53670/0 (socket says 172.21.15.44:53670) 2024-08-20T02:58:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5108c23640 1 -- 172.21.15.44:0/326160431 learned_addr learned my addr 172.21.15.44:0/326160431 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:58:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5109424640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5104101990 0x7f5104198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:58:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5109c25640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5104107d20 0x7f510419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:58:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5108c23640 1 -- 172.21.15.44:0/326160431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5104101990 msgr2=0x7f5104198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:58:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5108c23640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5104101990 0x7f5104198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5108c23640 1 -- 172.21.15.44:0/326160431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5104107d20 msgr2=0x7f510419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:58:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5108c23640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5104107d20 0x7f510419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5108c23640 1 -- 172.21.15.44:0/326160431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510419ef90 con 0x7f51041073a0 2024-08-20T02:58:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5109424640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5104101990 0x7f5104198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:58:11.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5109c25640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5104107d20 0x7f510419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f5108c23640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51041073a0 0x7f5104199420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f50f800c920 tx=0x7f50f800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f50f67fc640 1 -- 172.21.15.44:0/326160431 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8007c70 con 0x7f51041073a0 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f510b688640 1 -- 172.21.15.44:0/326160431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510419f280 con 0x7f51041073a0 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f50f67fc640 1 -- 172.21.15.44:0/326160431 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50f8007e10 con 0x7f51041073a0 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f50f67fc640 1 -- 172.21.15.44:0/326160431 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8018820 con 0x7f51041073a0 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.144+0000 7f510b688640 1 -- 172.21.15.44:0/326160431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51041a5940 con 0x7f51041073a0 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.145+0000 7f510b688640 1 -- 172.21.15.44:0/326160431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5104101d70 con 0x7f51041073a0 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.147+0000 7f50f67fc640 1 -- 172.21.15.44:0/326160431 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f50f800f450 con 0x7f51041073a0 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.147+0000 7f50f67fc640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f50f00b3c20 0x7f50f00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.147+0000 7f5109424640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f50f00b3c20 0x7f50f00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.148+0000 7f50f67fc640 1 -- 172.21.15.44:0/326160431 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5764+0+0 (secure 0 0 0) 0x7f50f80e3470 con 0x7f51041073a0 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.148+0000 7f5109424640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f50f00b3c20 0x7f50f00b60e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5100031440 tx=0x7f51000095f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:58:11.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.152+0000 7f50f67fc640 1 -- 172.21.15.44:0/326160431 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f50f8014030 con 0x7f51041073a0 2024-08-20T02:58:11.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:11.345+0000 7f510b688640 1 -- 172.21.15.44:0/326160431 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5104105340 con 0x7f50f00b3c20 2024-08-20T02:58:11.386 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T02:58:11.036+0000 7f0200ce2640 -1 osd.8 0 waiting for initial osdmap 2024-08-20T02:58:11.386 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T02:58:11.048+0000 7f01f8b01640 -1 osd.8 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T02:58:11.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:58:11.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: Detected new or changed devices on smithi165 2024-08-20T02:58:11.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi165 to 5696M 2024-08-20T02:58:11.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-08-20T02:58:11.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: osdmap e51: 9 total, 8 up, 9 in 2024-08-20T02:58:11.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:58:11.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:58:11.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: from='osd.8 ' entity='osd.8' 2024-08-20T02:58:11.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:58:11.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:58:11.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:11.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: osd.8 [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688] boot 2024-08-20T02:58:11.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: osdmap e52: 9 total, 9 up, 9 in 2024-08-20T02:58:11.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:58:11.955 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:58:11.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: Detected new or changed devices on smithi165 2024-08-20T02:58:11.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi165 to 5696M 2024-08-20T02:58:11.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-08-20T02:58:11.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: osdmap e51: 9 total, 8 up, 9 in 2024-08-20T02:58:11.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:58:11.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:58:11.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: from='osd.8 ' entity='osd.8' 2024-08-20T02:58:11.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:58:11.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:58:11.956 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:11.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: osd.8 [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688] boot 2024-08-20T02:58:11.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: osdmap e52: 9 total, 9 up, 9 in 2024-08-20T02:58:11.957 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:58:12.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-08-20T02:58:12.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: Detected new or changed devices on smithi165 2024-08-20T02:58:12.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi165 to 5696M 2024-08-20T02:58:12.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-08-20T02:58:12.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: osdmap e51: 9 total, 8 up, 9 in 2024-08-20T02:58:12.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:58:12.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:58:12.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: from='osd.8 ' entity='osd.8' 2024-08-20T02:58:12.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:58:12.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:58:12.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:12.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: osd.8 [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688] boot 2024-08-20T02:58:12.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: osdmap e52: 9 total, 9 up, 9 in 2024-08-20T02:58:12.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T02:58:12.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:12 smithi044 ceph-mon[18790]: purged_snaps scrub starts 2024-08-20T02:58:12.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:12 smithi044 ceph-mon[18790]: purged_snaps scrub ok 2024-08-20T02:58:12.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:12 smithi044 ceph-mon[18790]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:58:12.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:12 smithi044 ceph-mon[18790]: osdmap e53: 9 total, 9 up, 9 in 2024-08-20T02:58:12.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:12 smithi099 ceph-mon[21984]: purged_snaps scrub starts 2024-08-20T02:58:12.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:12 smithi099 ceph-mon[21984]: purged_snaps scrub ok 2024-08-20T02:58:12.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:12 smithi099 ceph-mon[21984]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:58:12.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:12 smithi099 ceph-mon[21984]: osdmap e53: 9 total, 9 up, 9 in 2024-08-20T02:58:13.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:12 smithi165 ceph-mon[22357]: purged_snaps scrub starts 2024-08-20T02:58:13.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:12 smithi165 ceph-mon[22357]: purged_snaps scrub ok 2024-08-20T02:58:13.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:12 smithi165 ceph-mon[22357]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:58:13.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:12 smithi165 ceph-mon[22357]: osdmap e53: 9 total, 9 up, 9 in 2024-08-20T02:58:13.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:13 smithi044 ceph-mon[18790]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:13.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:13 smithi044 ceph-mon[18790]: osdmap e54: 9 total, 9 up, 9 in 2024-08-20T02:58:13.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:13 smithi099 ceph-mon[21984]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:13.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:13 smithi099 ceph-mon[21984]: osdmap e54: 9 total, 9 up, 9 in 2024-08-20T02:58:14.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:13 smithi165 ceph-mon[22357]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:14.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:13 smithi165 ceph-mon[22357]: osdmap e54: 9 total, 9 up, 9 in 2024-08-20T02:58:14.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:14 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/104035717' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1ae468a5-3d72-4eb0-a1a6-cd87eae740b5"} : dispatch 2024-08-20T02:58:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:14 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/104035717' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ae468a5-3d72-4eb0-a1a6-cd87eae740b5"}]': finished 2024-08-20T02:58:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:14 smithi044 ceph-mon[18790]: osdmap e55: 10 total, 9 up, 10 in 2024-08-20T02:58:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:14 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:14.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:14 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/398378300' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:58:14.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:14 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/104035717' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1ae468a5-3d72-4eb0-a1a6-cd87eae740b5"} : dispatch 2024-08-20T02:58:14.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:14 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/104035717' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ae468a5-3d72-4eb0-a1a6-cd87eae740b5"}]': finished 2024-08-20T02:58:14.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:14 smithi099 ceph-mon[21984]: osdmap e55: 10 total, 9 up, 10 in 2024-08-20T02:58:14.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:14 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:14.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:14 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/398378300' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:58:15.037 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:14 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/104035717' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1ae468a5-3d72-4eb0-a1a6-cd87eae740b5"} : dispatch 2024-08-20T02:58:15.038 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:14 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/104035717' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ae468a5-3d72-4eb0-a1a6-cd87eae740b5"}]': finished 2024-08-20T02:58:15.038 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:14 smithi165 ceph-mon[22357]: osdmap e55: 10 total, 9 up, 10 in 2024-08-20T02:58:15.038 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:14 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:15.038 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:14 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/398378300' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:58:16.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:15 smithi165 ceph-mon[22357]: pgmap v171: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:16.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:15 smithi099 ceph-mon[21984]: pgmap v171: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:16.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:15 smithi044 ceph-mon[18790]: pgmap v171: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:18.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:17 smithi165 ceph-mon[22357]: pgmap v172: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:18.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:17 smithi099 ceph-mon[21984]: pgmap v172: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:18.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:17 smithi044 ceph-mon[18790]: pgmap v172: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:20.134 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:19 smithi044 ceph-mon[18790]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T02:58:20.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:19 smithi165 ceph-mon[22357]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T02:58:20.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:19 smithi099 ceph-mon[21984]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T02:58:22.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:21 smithi044 ceph-mon[18790]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-20T02:58:22.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:21 smithi165 ceph-mon[22357]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-20T02:58:22.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:21 smithi099 ceph-mon[21984]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-20T02:58:23.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:22 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-20T02:58:23.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:22 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:23.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:22 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-20T02:58:23.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:22 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:23.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:22 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-20T02:58:23.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:22 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:24.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:23 smithi044 ceph-mon[18790]: Deploying daemon osd.9 on smithi044 2024-08-20T02:58:24.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:23 smithi044 ceph-mon[18790]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-08-20T02:58:24.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:23 smithi099 ceph-mon[21984]: Deploying daemon osd.9 on smithi044 2024-08-20T02:58:24.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:23 smithi099 ceph-mon[21984]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-08-20T02:58:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:23 smithi165 ceph-mon[22357]: Deploying daemon osd.9 on smithi044 2024-08-20T02:58:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:23 smithi165 ceph-mon[22357]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-08-20T02:58:26.093 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:25 smithi044 ceph-mon[18790]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-08-20T02:58:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:25 smithi165 ceph-mon[22357]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-08-20T02:58:26.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:25 smithi099 ceph-mon[21984]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-08-20T02:58:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:27 smithi044 ceph-mon[18790]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:58:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:27 smithi099 ceph-mon[21984]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:58:28.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:27 smithi165 ceph-mon[22357]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:58:29.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:29.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:29.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:29.185 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:29.186 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:29.186 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:29.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:29.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:29.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:30.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:29 smithi044 ceph-mon[18790]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:58:30.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:29 smithi165 ceph-mon[22357]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:58:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:29 smithi099 ceph-mon[21984]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T02:58:31.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.560+0000 7f50f67fc640 1 -- 172.21.15.44:0/326160431 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5104105340 con 0x7f50f00b3c20 2024-08-20T02:58:31.563 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 9 on host 'smithi044' 2024-08-20T02:58:31.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.562+0000 7f510b688640 1 -- 172.21.15.44:0/326160431 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f50f00b3c20 msgr2=0x7f50f00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:58:31.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.562+0000 7f510b688640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f50f00b3c20 0x7f50f00b60e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5100031440 tx=0x7f51000095f0 comp rx=0 tx=0).stop 2024-08-20T02:58:31.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.563+0000 7f510b688640 1 -- 172.21.15.44:0/326160431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51041073a0 msgr2=0x7f5104199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:58:31.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.563+0000 7f510b688640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51041073a0 0x7f5104199420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f50f800c920 tx=0x7f50f800cde0 comp rx=0 tx=0).stop 2024-08-20T02:58:31.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.563+0000 7f510b688640 1 -- 172.21.15.44:0/326160431 shutdown_connections 2024-08-20T02:58:31.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.563+0000 7f510b688640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f50f00b3c20 0x7f50f00b60e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:31.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.563+0000 7f510b688640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5104107d20 0x7f510419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:31.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.563+0000 7f510b688640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51041073a0 0x7f5104199420 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:31.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.563+0000 7f510b688640 1 --2- 172.21.15.44:0/326160431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5104101990 0x7f5104198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:31.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.563+0000 7f510b688640 1 -- 172.21.15.44:0/326160431 >> 172.21.15.44:0/326160431 conn(0x7f51040fd400 msgr2=0x7f51040ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:58:31.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.563+0000 7f510b688640 1 -- 172.21.15.44:0/326160431 shutdown_connections 2024-08-20T02:58:31.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:58:31.563+0000 7f510b688640 1 -- 172.21.15.44:0/326160431 wait complete. 2024-08-20T02:58:32.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:31 smithi165 ceph-mon[22357]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:32.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:31 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:31 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:31 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:32.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:31 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:32.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:31 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:31 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:32.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:31 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:31 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:31 smithi099 ceph-mon[21984]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:32.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:31 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:31 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:31 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:32.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:31 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:32.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:31 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:31 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:32.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:31 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:31 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:31 smithi044 ceph-mon[18790]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:32.245 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:31 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.246 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:31 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.246 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:31 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:32.246 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:31 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:32.246 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:31 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.246 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:31 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:32.246 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:31 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.246 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:31 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:32.273 DEBUG:teuthology.orchestra.run.smithi044:osd.9> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.9.service 2024-08-20T02:58:32.279 INFO:tasks.cephadm:Deploying osd.10 on smithi099 with /dev/vg_nvme/lv_1... 2024-08-20T02:58:32.279 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-20T02:58:32.540 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T02:58:33.703 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 02:58:33 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T02:58:33.477+0000 7f8353692640 -1 osd.9 0 log_to_monitors true 2024-08-20T02:58:34.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:33 smithi044 ceph-mon[18790]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:34.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:33 smithi044 ceph-mon[18790]: from='osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-20T02:58:34.092 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:33 smithi099 ceph-mon[21984]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:34.093 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:33 smithi099 ceph-mon[21984]: from='osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-20T02:58:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:33 smithi165 ceph-mon[22357]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:34.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:33 smithi165 ceph-mon[22357]: from='osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-20T02:58:35.113 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 02:58:34 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T02:58:34.889+0000 7f834f615640 -1 osd.9 0 waiting for initial osdmap 2024-08-20T02:58:35.113 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 02:58:34 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T02:58:34.902+0000 7f834b43c640 -1 osd.9 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T02:58:35.114 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:34 smithi044 ceph-mon[18790]: from='osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-20T02:58:35.114 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:34 smithi044 ceph-mon[18790]: osdmap e56: 10 total, 9 up, 10 in 2024-08-20T02:58:35.114 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:34 smithi044 ceph-mon[18790]: from='osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]} : dispatch 2024-08-20T02:58:35.114 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:35.114 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:34 smithi044 ceph-mon[18790]: from='osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-08-20T02:58:35.115 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:34 smithi044 ceph-mon[18790]: osdmap e57: 10 total, 9 up, 10 in 2024-08-20T02:58:35.115 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:35.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:34 smithi099 ceph-mon[21984]: from='osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-20T02:58:35.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:34 smithi099 ceph-mon[21984]: osdmap e56: 10 total, 9 up, 10 in 2024-08-20T02:58:35.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:34 smithi099 ceph-mon[21984]: from='osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]} : dispatch 2024-08-20T02:58:35.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:35.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:34 smithi099 ceph-mon[21984]: from='osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-08-20T02:58:35.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:34 smithi099 ceph-mon[21984]: osdmap e57: 10 total, 9 up, 10 in 2024-08-20T02:58:35.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-08-20T02:58:35.315 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_1 2024-08-20T02:58:35.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:34 smithi165 ceph-mon[22357]: from='osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-20T02:58:35.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:34 smithi165 ceph-mon[22357]: osdmap e56: 10 total, 9 up, 10 in 2024-08-20T02:58:35.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:34 smithi165 ceph-mon[22357]: from='osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]} : dispatch 2024-08-20T02:58:35.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:35.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:34 smithi165 ceph-mon[22357]: from='osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-08-20T02:58:35.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:34 smithi165 ceph-mon[22357]: osdmap e57: 10 total, 9 up, 10 in 2024-08-20T02:58:35.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:35.563 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T02:58:35.964 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:35.965 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:35.965 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: Detected new or changed devices on smithi044 2024-08-20T02:58:35.965 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:35.965 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:35.965 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:35.965 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:35.965 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:35.966 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:35.966 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi044 to 4273M 2024-08-20T02:58:35.966 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:35.966 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:35.966 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:35.966 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:35.966 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:36.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.045+0000 7fc80da08640 1 -- 172.21.15.99:0/2539938722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081042c0 msgr2=0x7fc80810c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:58:36.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.045+0000 7fc80da08640 1 --2- 172.21.15.99:0/2539938722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081042c0 0x7fc80810c7b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009900 tx=0x7fc7fc02a410 comp rx=0 tx=0).stop 2024-08-20T02:58:36.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.045+0000 7fc80da08640 1 -- 172.21.15.99:0/2539938722 shutdown_connections 2024-08-20T02:58:36.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.045+0000 7fc80da08640 1 --2- 172.21.15.99:0/2539938722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc80810ce00 0x7fc80810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:36.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.045+0000 7fc80da08640 1 --2- 172.21.15.99:0/2539938722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081042c0 0x7fc80810c7b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:36.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.045+0000 7fc80da08640 1 --2- 172.21.15.99:0/2539938722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8081039a0 0x7fc808103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:36.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.045+0000 7fc80da08640 1 -- 172.21.15.99:0/2539938722 >> 172.21.15.99:0/2539938722 conn(0x7fc8080fd400 msgr2=0x7fc8080ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:58:36.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.045+0000 7fc80da08640 1 -- 172.21.15.99:0/2539938722 shutdown_connections 2024-08-20T02:58:36.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.045+0000 7fc80da08640 1 -- 172.21.15.99:0/2539938722 wait complete. 2024-08-20T02:58:36.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.045+0000 7fc80da08640 1 Processor -- start 2024-08-20T02:58:36.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.045+0000 7fc80da08640 1 -- start start 2024-08-20T02:58:36.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc80da08640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081039a0 0x7fc808101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:58:36.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc80da08640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8081042c0 0x7fc808101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:58:36.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc80da08640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc80810ce00 0x7fc8081020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:58:36.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc80da08640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc808112b30 con 0x7fc80810ce00 2024-08-20T02:58:36.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc80da08640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc8081129b0 con 0x7fc8081039a0 2024-08-20T02:58:36.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc80da08640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc808112cb0 con 0x7fc8081042c0 2024-08-20T02:58:36.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc806ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081039a0 0x7fc808101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:58:36.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc806ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081039a0 0x7fc808101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54522/0 (socket says 172.21.15.99:54522) 2024-08-20T02:58:36.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc806ffd640 1 -- 172.21.15.99:0/1748001055 learned_addr learned my addr 172.21.15.99:0/1748001055 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-08-20T02:58:36.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc8077fe640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc80810ce00 0x7fc8081020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:58:36.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc8067fc640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8081042c0 0x7fc808101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:58:36.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc806ffd640 1 -- 172.21.15.99:0/1748001055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8081042c0 msgr2=0x7fc808101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:58:36.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc806ffd640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8081042c0 0x7fc808101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:36.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc806ffd640 1 -- 172.21.15.99:0/1748001055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc80810ce00 msgr2=0x7fc8081020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:58:36.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc806ffd640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc80810ce00 0x7fc8081020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:36.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc806ffd640 1 -- 172.21.15.99:0/1748001055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8081a7500 con 0x7fc8081039a0 2024-08-20T02:58:36.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc8067fc640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8081042c0 0x7fc808101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T02:58:36.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc806ffd640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081039a0 0x7fc808101650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800ba50 tx=0x7fc7f800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:58:36.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc80ca06640 1 -- 172.21.15.99:0/1748001055 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8019070 con 0x7fc8081039a0 2024-08-20T02:58:36.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc80ca06640 1 -- 172.21.15.99:0/1748001055 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f800cbe0 con 0x7fc8081039a0 2024-08-20T02:58:36.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc80ca06640 1 -- 172.21.15.99:0/1748001055 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80075d0 con 0x7fc8081039a0 2024-08-20T02:58:36.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc80da08640 1 -- 172.21.15.99:0/1748001055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8081a77f0 con 0x7fc8081039a0 2024-08-20T02:58:36.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.046+0000 7fc80da08640 1 -- 172.21.15.99:0/1748001055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8081a7e00 con 0x7fc8081039a0 2024-08-20T02:58:36.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.047+0000 7fc80da08640 1 -- 172.21.15.99:0/1748001055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc808103e00 con 0x7fc8081039a0 2024-08-20T02:58:36.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.048+0000 7fc80ca06640 1 -- 172.21.15.99:0/1748001055 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fc7f801c070 con 0x7fc8081039a0 2024-08-20T02:58:36.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.049+0000 7fc80ca06640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc7f00b3ca0 0x7fc7f00b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:58:36.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.049+0000 7fc80ca06640 1 -- 172.21.15.99:0/1748001055 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6476+0+0 (secure 0 0 0) 0x7fc7f80e2f60 con 0x7fc8081039a0 2024-08-20T02:58:36.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.050+0000 7fc8067fc640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc7f00b3ca0 0x7fc7f00b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:58:36.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.050+0000 7fc8067fc640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc7f00b3ca0 0x7fc7f00b6160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc005ab0 tx=0x7fc7fc031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:58:36.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.052+0000 7fc80ca06640 1 -- 172.21.15.99:0/1748001055 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fc7f80aa690 con 0x7fc8081039a0 2024-08-20T02:58:36.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:36.235+0000 7fc80da08640 1 -- 172.21.15.99:0/1748001055 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc8080632c0 con 0x7fc7f00b3ca0 2024-08-20T02:58:36.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:36.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:36.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: Detected new or changed devices on smithi044 2024-08-20T02:58:36.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:36.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:36.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:36.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:36.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:36.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:36.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi044 to 4273M 2024-08-20T02:58:36.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:36.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:36.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:36.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:36.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: Detected new or changed devices on smithi044 2024-08-20T02:58:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T02:58:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi044 to 4273M 2024-08-20T02:58:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:36.388 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:37.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:36 smithi099 ceph-mon[21984]: purged_snaps scrub starts 2024-08-20T02:58:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:36 smithi099 ceph-mon[21984]: purged_snaps scrub ok 2024-08-20T02:58:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:36 smithi099 ceph-mon[21984]: osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093] boot 2024-08-20T02:58:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:36 smithi099 ceph-mon[21984]: osdmap e58: 10 total, 10 up, 10 in 2024-08-20T02:58:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:58:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:58:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:36 smithi044 ceph-mon[18790]: purged_snaps scrub starts 2024-08-20T02:58:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:36 smithi044 ceph-mon[18790]: purged_snaps scrub ok 2024-08-20T02:58:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:36 smithi044 ceph-mon[18790]: osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093] boot 2024-08-20T02:58:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:36 smithi044 ceph-mon[18790]: osdmap e58: 10 total, 10 up, 10 in 2024-08-20T02:58:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:58:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:58:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:37.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:36 smithi165 ceph-mon[22357]: purged_snaps scrub starts 2024-08-20T02:58:37.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:36 smithi165 ceph-mon[22357]: purged_snaps scrub ok 2024-08-20T02:58:37.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:36 smithi165 ceph-mon[22357]: osd.9 [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093] boot 2024-08-20T02:58:37.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:36 smithi165 ceph-mon[22357]: osdmap e58: 10 total, 10 up, 10 in 2024-08-20T02:58:37.390 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T02:58:37.391 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:58:37.391 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:58:37.391 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:38.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:37 smithi099 ceph-mon[21984]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:58:38.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:37 smithi099 ceph-mon[21984]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:38.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:37 smithi099 ceph-mon[21984]: osdmap e59: 10 total, 10 up, 10 in 2024-08-20T02:58:38.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:37 smithi044 ceph-mon[18790]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:58:38.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:37 smithi044 ceph-mon[18790]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:38.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:37 smithi044 ceph-mon[18790]: osdmap e59: 10 total, 10 up, 10 in 2024-08-20T02:58:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:37 smithi165 ceph-mon[22357]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:58:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:37 smithi165 ceph-mon[22357]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-08-20T02:58:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:37 smithi165 ceph-mon[22357]: osdmap e59: 10 total, 10 up, 10 in 2024-08-20T02:58:39.117 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:38 smithi099 ceph-mon[21984]: from='client.? 172.21.15.99:0/1400354214' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1"} : dispatch 2024-08-20T02:58:39.117 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:38 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1"} : dispatch 2024-08-20T02:58:39.117 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:38 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1"}]': finished 2024-08-20T02:58:39.118 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:38 smithi099 ceph-mon[21984]: osdmap e60: 11 total, 10 up, 11 in 2024-08-20T02:58:39.118 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:38 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:58:39.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:38 smithi044 ceph-mon[18790]: from='client.? 172.21.15.99:0/1400354214' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1"} : dispatch 2024-08-20T02:58:39.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:38 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1"} : dispatch 2024-08-20T02:58:39.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:38 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1"}]': finished 2024-08-20T02:58:39.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:38 smithi044 ceph-mon[18790]: osdmap e60: 11 total, 10 up, 11 in 2024-08-20T02:58:39.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:38 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:58:39.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:38 smithi165 ceph-mon[22357]: from='client.? 172.21.15.99:0/1400354214' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1"} : dispatch 2024-08-20T02:58:39.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:38 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1"} : dispatch 2024-08-20T02:58:39.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:38 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1"}]': finished 2024-08-20T02:58:39.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:38 smithi165 ceph-mon[22357]: osdmap e60: 11 total, 10 up, 11 in 2024-08-20T02:58:39.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:38 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:58:40.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:39 smithi044 ceph-mon[18790]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:40.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:39 smithi044 ceph-mon[18790]: from='client.? 172.21.15.99:0/2793266896' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:58:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:39 smithi165 ceph-mon[22357]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:39 smithi165 ceph-mon[22357]: from='client.? 172.21.15.99:0/2793266896' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:58:40.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:39 smithi099 ceph-mon[21984]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:40.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:39 smithi099 ceph-mon[21984]: from='client.? 172.21.15.99:0/2793266896' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:58:42.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:41 smithi044 ceph-mon[18790]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:41 smithi165 ceph-mon[22357]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:42.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:41 smithi099 ceph-mon[21984]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:43 smithi099 ceph-mon[21984]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:44.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:43 smithi044 ceph-mon[18790]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:44.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:43 smithi165 ceph-mon[22357]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:46.048 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:45 smithi099 ceph-mon[21984]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:46.188 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:45 smithi165 ceph-mon[22357]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:46.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:45 smithi044 ceph-mon[18790]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:48.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:47 smithi099 ceph-mon[21984]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:48.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-20T02:58:48.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:48.111 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:47 smithi099 ceph-mon[21984]: Deploying daemon osd.10 on smithi099 2024-08-20T02:58:48.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:47 smithi044 ceph-mon[18790]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:48.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-20T02:58:48.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:48.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:47 smithi044 ceph-mon[18790]: Deploying daemon osd.10 on smithi099 2024-08-20T02:58:48.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:47 smithi165 ceph-mon[22357]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:48.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-20T02:58:48.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:48.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:47 smithi165 ceph-mon[22357]: Deploying daemon osd.10 on smithi099 2024-08-20T02:58:50.126 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:49 smithi099 ceph-mon[21984]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:49 smithi044 ceph-mon[18790]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:49 smithi165 ceph-mon[22357]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:52.205 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:51 smithi099 ceph-mon[21984]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:52.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:51 smithi044 ceph-mon[18790]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:52.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:51 smithi165 ceph-mon[22357]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:53.197 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:52 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:53.197 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:52 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:53.198 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:52 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:53.200 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:52 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:53.200 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:52 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:53.200 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:52 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:53.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:52 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:53.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:52 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:53.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:52 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:54.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:53 smithi099 ceph-mon[21984]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:54.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:53 smithi044 ceph-mon[18790]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:54.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:53 smithi165 ceph-mon[22357]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:56.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:55 smithi044 ceph-mon[18790]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:56.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:55 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:56.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:55 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:56.319 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:55 smithi099 ceph-mon[21984]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:56.319 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:55 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:56.320 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:55 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:56.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.357+0000 7fc80ca06640 1 -- 172.21.15.99:0/1748001055 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc8080632c0 con 0x7fc7f00b3ca0 2024-08-20T02:58:56.360 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 10 on host 'smithi099' 2024-08-20T02:58:56.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.360+0000 7fc80da08640 1 -- 172.21.15.99:0/1748001055 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc7f00b3ca0 msgr2=0x7fc7f00b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:58:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.360+0000 7fc80da08640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc7f00b3ca0 0x7fc7f00b6160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc005ab0 tx=0x7fc7fc031400 comp rx=0 tx=0).stop 2024-08-20T02:58:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.360+0000 7fc80da08640 1 -- 172.21.15.99:0/1748001055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081039a0 msgr2=0x7fc808101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:58:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.360+0000 7fc80da08640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081039a0 0x7fc808101650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800ba50 tx=0x7fc7f800bf10 comp rx=0 tx=0).stop 2024-08-20T02:58:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.360+0000 7fc80da08640 1 -- 172.21.15.99:0/1748001055 shutdown_connections 2024-08-20T02:58:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.360+0000 7fc80da08640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc7f00b3ca0 0x7fc7f00b6160 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.360+0000 7fc80da08640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc80810ce00 0x7fc8081020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.360+0000 7fc80da08640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8081042c0 0x7fc808101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.360+0000 7fc80da08640 1 --2- 172.21.15.99:0/1748001055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8081039a0 0x7fc808101650 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:58:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.360+0000 7fc80da08640 1 -- 172.21.15.99:0/1748001055 >> 172.21.15.99:0/1748001055 conn(0x7fc8080fd400 msgr2=0x7fc8080fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:58:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.360+0000 7fc80da08640 1 -- 172.21.15.99:0/1748001055 shutdown_connections 2024-08-20T02:58:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-08-20T02:58:56.360+0000 7fc80da08640 1 -- 172.21.15.99:0/1748001055 wait complete. 2024-08-20T02:58:56.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:55 smithi165 ceph-mon[22357]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:56.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:55 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:56.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:55 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:57.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:56 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:57.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:56 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:57.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:56 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:57.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:56 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:57.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:56 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:57.067 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:56 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:57.093 DEBUG:teuthology.orchestra.run.smithi099:osd.10> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.10.service 2024-08-20T02:58:57.095 INFO:tasks.cephadm:Deploying osd.11 on smithi165 with /dev/vg_nvme/lv_1... 2024-08-20T02:58:57.096 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-20T02:58:57.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:56 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:57.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:56 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:57.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:56 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:57.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:56 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:57.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:56 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:57.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:56 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:57.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:56 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:58:57.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:56 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:58:57.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:56 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:57.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:56 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:58:57.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:56 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:57.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:56 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:58:57.355 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:58:58.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:57 smithi165 ceph-mon[22357]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:58.253 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:57 smithi099 ceph-mon[21984]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:58.253 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 02:58:58 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T02:58:58.141+0000 7f7c5e1a9640 -1 osd.10 0 log_to_monitors true 2024-08-20T02:58:58.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:57 smithi044 ceph-mon[18790]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:58:59.104 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:58 smithi165 ceph-mon[22357]: from='osd.10 [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T02:58:59.104 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:58 smithi165 ceph-mon[22357]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T02:58:59.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:58 smithi044 ceph-mon[18790]: from='osd.10 [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T02:58:59.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:58 smithi044 ceph-mon[18790]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T02:58:59.299 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:58 smithi099 ceph-mon[21984]: from='osd.10 [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T02:58:59.299 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:58 smithi099 ceph-mon[21984]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T02:59:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:59 smithi165 ceph-mon[22357]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:59:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:59 smithi165 ceph-mon[22357]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-20T02:59:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:59 smithi165 ceph-mon[22357]: from='osd.10 [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:59:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:59 smithi165 ceph-mon[22357]: osdmap e61: 11 total, 10 up, 11 in 2024-08-20T02:59:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:59:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:58:59 smithi165 ceph-mon[22357]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:59:00.180 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-08-20T02:59:00.206 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi165:vg_nvme/lv_1 2024-08-20T02:59:00.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:59 smithi044 ceph-mon[18790]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:59:00.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:59 smithi044 ceph-mon[18790]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-20T02:59:00.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:59 smithi044 ceph-mon[18790]: from='osd.10 [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:59:00.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:59 smithi044 ceph-mon[18790]: osdmap e61: 11 total, 10 up, 11 in 2024-08-20T02:59:00.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:59:00.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:58:59 smithi044 ceph-mon[18790]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:59:00.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:59 smithi099 ceph-mon[21984]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:59:00.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:59 smithi099 ceph-mon[21984]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-20T02:59:00.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:59 smithi099 ceph-mon[21984]: from='osd.10 [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:59:00.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:59 smithi099 ceph-mon[21984]: osdmap e61: 11 total, 10 up, 11 in 2024-08-20T02:59:00.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:59:00.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:58:59 smithi099 ceph-mon[21984]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]} : dispatch 2024-08-20T02:59:00.415 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 02:58:59 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T02:58:59.969+0000 7f7c5a12c640 -1 osd.10 0 waiting for initial osdmap 2024-08-20T02:59:00.415 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 02:58:59 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T02:58:59.985+0000 7f7c55f53640 -1 osd.10 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T02:59:00.497 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:59:01.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.997+0000 7f55a9a4a640 1 -- 172.21.15.165:0/4220835627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55a4107d50 msgr2=0x7f55a410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:01.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.997+0000 7f55a9a4a640 1 --2- 172.21.15.165:0/4220835627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55a4107d50 0x7f55a410e550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f559c009cc0 tx=0x7f559c02cad0 comp rx=0 tx=0).stop 2024-08-20T02:59:01.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.997+0000 7f55a9a4a640 1 -- 172.21.15.165:0/4220835627 shutdown_connections 2024-08-20T02:59:01.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.997+0000 7f55a9a4a640 1 --2- 172.21.15.165:0/4220835627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55a4107d50 0x7f55a410e550 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:01.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.998+0000 7f55a9a4a640 1 --2- 172.21.15.165:0/4220835627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a41073d0 0x7f55a4107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:01.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.998+0000 7f55a9a4a640 1 --2- 172.21.15.165:0/4220835627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a41019c0 0x7f55a4101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:01.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.998+0000 7f55a9a4a640 1 -- 172.21.15.165:0/4220835627 >> 172.21.15.165:0/4220835627 conn(0x7f55a40fd410 msgr2=0x7f55a40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:01.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.998+0000 7f55a9a4a640 1 -- 172.21.15.165:0/4220835627 shutdown_connections 2024-08-20T02:59:01.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.998+0000 7f55a9a4a640 1 -- 172.21.15.165:0/4220835627 wait complete. 2024-08-20T02:59:01.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.998+0000 7f55a9a4a640 1 Processor -- start 2024-08-20T02:59:01.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.998+0000 7f55a9a4a640 1 -- start start 2024-08-20T02:59:01.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a9a4a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55a41019c0 0x7f55a4197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:01.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a9a4a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a41073d0 0x7f55a4197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:01.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a9a4a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a4107d50 0x7f55a419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:01.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a9a4a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55a4110910 con 0x7f55a4107d50 2024-08-20T02:59:01.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a9a4a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55a4110790 con 0x7f55a41073d0 2024-08-20T02:59:01.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a9a4a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55a4110a90 con 0x7f55a41019c0 2024-08-20T02:59:01.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a2ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55a41019c0 0x7f55a4197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:01.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a2ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55a41019c0 0x7f55a4197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39518/0 (socket says 172.21.15.165:39518) 2024-08-20T02:59:01.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a2ffd640 1 -- 172.21.15.165:0/1202486797 learned_addr learned my addr 172.21.15.165:0/1202486797 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-08-20T02:59:01.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f559a7fc640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a41073d0 0x7f55a4197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:01.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a37fe640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a4107d50 0x7f55a419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:01.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a2ffd640 1 -- 172.21.15.165:0/1202486797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a41073d0 msgr2=0x7f55a4197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:01.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a2ffd640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a41073d0 0x7f55a4197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:01.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a2ffd640 1 -- 172.21.15.165:0/1202486797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a4107d50 msgr2=0x7f55a419c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:01.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a2ffd640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a4107d50 0x7f55a419c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:01.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:00.999+0000 7f55a2ffd640 1 -- 172.21.15.165:0/1202486797 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a419d100 con 0x7f55a41019c0 2024-08-20T02:59:01.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.000+0000 7f55a2ffd640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55a41019c0 0x7f55a4197050 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f559400d950 tx=0x7f559400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:01.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.000+0000 7f55a8a48640 1 -- 172.21.15.165:0/1202486797 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594017070 con 0x7f55a41019c0 2024-08-20T02:59:01.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.000+0000 7f55a8a48640 1 -- 172.21.15.165:0/1202486797 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5594009c30 con 0x7f55a41019c0 2024-08-20T02:59:01.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.000+0000 7f55a9a4a640 1 -- 172.21.15.165:0/1202486797 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a419d3f0 con 0x7f55a41019c0 2024-08-20T02:59:01.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.000+0000 7f55a8a48640 1 -- 172.21.15.165:0/1202486797 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559400f620 con 0x7f55a41019c0 2024-08-20T02:59:01.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.000+0000 7f55a9a4a640 1 -- 172.21.15.165:0/1202486797 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a41a3ab0 con 0x7f55a41019c0 2024-08-20T02:59:01.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.001+0000 7f55a9a4a640 1 -- 172.21.15.165:0/1202486797 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a4101da0 con 0x7f55a41019c0 2024-08-20T02:59:01.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.002+0000 7f55a8a48640 1 -- 172.21.15.165:0/1202486797 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f5594010460 con 0x7f55a41019c0 2024-08-20T02:59:01.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.003+0000 7f55a8a48640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55900b3c20 0x7f55900b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:01.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.003+0000 7f55a8a48640 1 -- 172.21.15.165:0/1202486797 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6942+0+0 (secure 0 0 0) 0x7f55940e3330 con 0x7f55a41019c0 2024-08-20T02:59:01.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.003+0000 7f559a7fc640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55900b3c20 0x7f55900b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:01.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.004+0000 7f559a7fc640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55900b3c20 0x7f55900b60e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f55a41069c0 tx=0x7f558c00b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:01.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.007+0000 7f55a8a48640 1 -- 172.21.15.165:0/1202486797 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f55940aaee0 con 0x7f55a41019c0 2024-08-20T02:59:01.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:01.190+0000 7f55a9a4a640 1 -- 172.21.15.165:0/1202486797 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f55a406ae90 con 0x7f55900b3c20 2024-08-20T02:59:01.193 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-08-20T02:59:01.193 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: osdmap e62: 11 total, 10 up, 11 in 2024-08-20T02:59:01.193 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:59:01.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:59:01.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: Detected new or changed devices on smithi099 2024-08-20T02:59:01.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:01.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:01.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:01.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:01.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:01.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:01.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi099 to 4272M 2024-08-20T02:59:01.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:01.194 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:01.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:59:01.195 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:01.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-08-20T02:59:01.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: osdmap e62: 11 total, 10 up, 11 in 2024-08-20T02:59:01.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:59:01.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:59:01.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: Detected new or changed devices on smithi099 2024-08-20T02:59:01.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi099 to 4272M 2024-08-20T02:59:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:59:01.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-08-20T02:59:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: osdmap e62: 11 total, 10 up, 11 in 2024-08-20T02:59:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:59:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:59:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: Detected new or changed devices on smithi099 2024-08-20T02:59:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:01.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:01.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi099 to 4272M 2024-08-20T02:59:01.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:01.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:01.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:59:01.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:02.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:01 smithi044 ceph-mon[18790]: purged_snaps scrub starts 2024-08-20T02:59:02.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:01 smithi044 ceph-mon[18790]: purged_snaps scrub ok 2024-08-20T02:59:02.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:01 smithi044 ceph-mon[18790]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:59:02.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:01 smithi044 ceph-mon[18790]: osd.10 [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038] boot 2024-08-20T02:59:02.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:01 smithi044 ceph-mon[18790]: osdmap e63: 11 total, 11 up, 11 in 2024-08-20T02:59:02.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:01 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:59:02.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:01 smithi044 ceph-mon[18790]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:59:02.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:01 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:59:02.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:01 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:59:02.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:01 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:02.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:01 smithi044 ceph-mon[18790]: osdmap e64: 11 total, 11 up, 11 in 2024-08-20T02:59:02.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:01 smithi165 ceph-mon[22357]: purged_snaps scrub starts 2024-08-20T02:59:02.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:01 smithi165 ceph-mon[22357]: purged_snaps scrub ok 2024-08-20T02:59:02.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:01 smithi165 ceph-mon[22357]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:59:02.303 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:01 smithi165 ceph-mon[22357]: osd.10 [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038] boot 2024-08-20T02:59:02.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:01 smithi165 ceph-mon[22357]: osdmap e63: 11 total, 11 up, 11 in 2024-08-20T02:59:02.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:01 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:59:02.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:01 smithi165 ceph-mon[22357]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:59:02.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:01 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:59:02.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:01 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:59:02.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:01 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:02.304 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:01 smithi165 ceph-mon[22357]: osdmap e64: 11 total, 11 up, 11 in 2024-08-20T02:59:02.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:01 smithi099 ceph-mon[21984]: purged_snaps scrub starts 2024-08-20T02:59:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:01 smithi099 ceph-mon[21984]: purged_snaps scrub ok 2024-08-20T02:59:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:01 smithi099 ceph-mon[21984]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-08-20T02:59:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:01 smithi099 ceph-mon[21984]: osd.10 [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038] boot 2024-08-20T02:59:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:01 smithi099 ceph-mon[21984]: osdmap e63: 11 total, 11 up, 11 in 2024-08-20T02:59:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:01 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T02:59:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:01 smithi099 ceph-mon[21984]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi165:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:59:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:01 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T02:59:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:01 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T02:59:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:01 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:01 smithi099 ceph-mon[21984]: osdmap e64: 11 total, 11 up, 11 in 2024-08-20T02:59:03.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:03 smithi165 ceph-mon[22357]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:03.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:03 smithi165 ceph-mon[22357]: osdmap e65: 11 total, 11 up, 11 in 2024-08-20T02:59:03.903 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:03 smithi044 ceph-mon[18790]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:03.904 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:03 smithi044 ceph-mon[18790]: osdmap e65: 11 total, 11 up, 11 in 2024-08-20T02:59:03.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:03 smithi099 ceph-mon[21984]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:03.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:03 smithi099 ceph-mon[21984]: osdmap e65: 11 total, 11 up, 11 in 2024-08-20T02:59:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:04 smithi044 ceph-mon[18790]: from='client.? 172.21.15.165:0/2403350964' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "50715410-1fc2-4818-b600-604a4b7cce9a"} : dispatch 2024-08-20T02:59:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:04 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "50715410-1fc2-4818-b600-604a4b7cce9a"} : dispatch 2024-08-20T02:59:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:04 smithi044 ceph-mon[18790]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50715410-1fc2-4818-b600-604a4b7cce9a"}]': finished 2024-08-20T02:59:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:04 smithi044 ceph-mon[18790]: osdmap e66: 12 total, 11 up, 12 in 2024-08-20T02:59:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:04 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:04 smithi044 ceph-mon[18790]: from='client.? 172.21.15.165:0/1961818941' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:59:04.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:04 smithi165 ceph-mon[22357]: from='client.? 172.21.15.165:0/2403350964' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "50715410-1fc2-4818-b600-604a4b7cce9a"} : dispatch 2024-08-20T02:59:04.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:04 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "50715410-1fc2-4818-b600-604a4b7cce9a"} : dispatch 2024-08-20T02:59:04.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:04 smithi165 ceph-mon[22357]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50715410-1fc2-4818-b600-604a4b7cce9a"}]': finished 2024-08-20T02:59:04.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:04 smithi165 ceph-mon[22357]: osdmap e66: 12 total, 11 up, 12 in 2024-08-20T02:59:04.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:04 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:04.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:04 smithi165 ceph-mon[22357]: from='client.? 172.21.15.165:0/1961818941' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:59:04.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:04 smithi099 ceph-mon[21984]: from='client.? 172.21.15.165:0/2403350964' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "50715410-1fc2-4818-b600-604a4b7cce9a"} : dispatch 2024-08-20T02:59:04.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:04 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "50715410-1fc2-4818-b600-604a4b7cce9a"} : dispatch 2024-08-20T02:59:04.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:04 smithi099 ceph-mon[21984]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50715410-1fc2-4818-b600-604a4b7cce9a"}]': finished 2024-08-20T02:59:04.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:04 smithi099 ceph-mon[21984]: osdmap e66: 12 total, 11 up, 12 in 2024-08-20T02:59:04.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:04 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:04.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:04 smithi099 ceph-mon[21984]: from='client.? 172.21.15.165:0/1961818941' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T02:59:06.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:05 smithi044 ceph-mon[18790]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-20T02:59:06.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:05 smithi165 ceph-mon[22357]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-20T02:59:06.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:05 smithi099 ceph-mon[21984]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-20T02:59:08.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:07 smithi044 ceph-mon[18790]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-08-20T02:59:08.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:07 smithi165 ceph-mon[22357]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-08-20T02:59:08.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:07 smithi099 ceph-mon[21984]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-08-20T02:59:10.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:09 smithi044 ceph-mon[18790]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-08-20T02:59:10.107 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:09 smithi165 ceph-mon[22357]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-08-20T02:59:10.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:09 smithi099 ceph-mon[21984]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-08-20T02:59:12.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:11 smithi044 ceph-mon[18790]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T02:59:12.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:11 smithi165 ceph-mon[22357]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T02:59:12.166 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:11 smithi099 ceph-mon[21984]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T02:59:12.990 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:12 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-20T02:59:12.990 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:12 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:13.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:12 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-20T02:59:13.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:12 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:13.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:12 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-20T02:59:13.066 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:12 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:14.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:13 smithi044 ceph-mon[18790]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-08-20T02:59:14.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:13 smithi044 ceph-mon[18790]: Deploying daemon osd.11 on smithi165 2024-08-20T02:59:14.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:13 smithi165 ceph-mon[22357]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-08-20T02:59:14.053 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:13 smithi165 ceph-mon[22357]: Deploying daemon osd.11 on smithi165 2024-08-20T02:59:14.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:13 smithi099 ceph-mon[21984]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-08-20T02:59:14.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:13 smithi099 ceph-mon[21984]: Deploying daemon osd.11 on smithi165 2024-08-20T02:59:16.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:15 smithi044 ceph-mon[18790]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:16.069 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:15 smithi165 ceph-mon[22357]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:16.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:15 smithi099 ceph-mon[21984]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:17.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:17 smithi165 ceph-mon[22357]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:18.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:17 smithi044 ceph-mon[18790]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:18.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:17 smithi099 ceph-mon[21984]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:18.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:18 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:59:18.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:18 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:18.868 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:18 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:19.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:18 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:59:19.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:18 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:19.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:18 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:19.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:18 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:59:19.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:18 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:19.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:18 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:20.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:20 smithi099 ceph-mon[21984]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:20.427 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:20 smithi165 ceph-mon[22357]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:20.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:20 smithi044 ceph-mon[18790]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:21.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.681+0000 7f55a8a48640 1 -- 172.21.15.165:0/1202486797 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f55a406ae90 con 0x7f55900b3c20 2024-08-20T02:59:21.685 INFO:teuthology.orchestra.run.smithi165.stdout:Created osd(s) 11 on host 'smithi165' 2024-08-20T02:59:21.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.684+0000 7f55a9a4a640 1 -- 172.21.15.165:0/1202486797 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55900b3c20 msgr2=0x7f55900b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:21.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.684+0000 7f55a9a4a640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55900b3c20 0x7f55900b60e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f55a41069c0 tx=0x7f558c00b390 comp rx=0 tx=0).stop 2024-08-20T02:59:21.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.684+0000 7f55a9a4a640 1 -- 172.21.15.165:0/1202486797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55a41019c0 msgr2=0x7f55a4197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:21.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.684+0000 7f55a9a4a640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55a41019c0 0x7f55a4197050 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f559400d950 tx=0x7f559400de10 comp rx=0 tx=0).stop 2024-08-20T02:59:21.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.684+0000 7f55a9a4a640 1 -- 172.21.15.165:0/1202486797 shutdown_connections 2024-08-20T02:59:21.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.684+0000 7f55a9a4a640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55900b3c20 0x7f55900b60e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:21.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.684+0000 7f55a9a4a640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55a4107d50 0x7f55a419c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:21.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.684+0000 7f55a9a4a640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55a41073d0 0x7f55a4197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:21.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.684+0000 7f55a9a4a640 1 --2- 172.21.15.165:0/1202486797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55a41019c0 0x7f55a4197050 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:21.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.684+0000 7f55a9a4a640 1 -- 172.21.15.165:0/1202486797 >> 172.21.15.165:0/1202486797 conn(0x7f55a40fd410 msgr2=0x7f55a40ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:21.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.684+0000 7f55a9a4a640 1 -- 172.21.15.165:0/1202486797 shutdown_connections 2024-08-20T02:59:21.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:21.684+0000 7f55a9a4a640 1 -- 172.21.15.165:0/1202486797 wait complete. 2024-08-20T02:59:21.977 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:21 smithi044 ceph-mon[18790]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:21.977 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:21.977 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:21.977 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:21.977 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:59:21.978 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:21.978 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:59:21.978 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:21.978 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:21 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:22.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:21 smithi165 ceph-mon[22357]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:22.347 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:22.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:22.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:22.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:59:22.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:22.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:59:22.348 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:22.349 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:21 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:22.374 DEBUG:teuthology.orchestra.run.smithi165:osd.11> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.11.service 2024-08-20T02:59:22.377 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-08-20T02:59:22.377 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-20T02:59:22.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:21 smithi099 ceph-mon[21984]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:22.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:22.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:22.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:22.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:59:22.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:22.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:59:22.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:22.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:21 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:22.644 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:59:23.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.124+0000 7f6cdcec2640 1 -- 172.21.15.44:0/708891196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cd81073a0 msgr2=0x7f6cd81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:23.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.124+0000 7f6cdcec2640 1 --2- 172.21.15.44:0/708891196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cd81073a0 0x7f6cd81077e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009900 tx=0x7f6cc002a410 comp rx=0 tx=0).stop 2024-08-20T02:59:23.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.125+0000 7f6cdcec2640 1 -- 172.21.15.44:0/708891196 shutdown_connections 2024-08-20T02:59:23.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.125+0000 7f6cdcec2640 1 --2- 172.21.15.44:0/708891196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd8107d20 0x7f6cd810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:23.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.125+0000 7f6cdcec2640 1 --2- 172.21.15.44:0/708891196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cd81073a0 0x7f6cd81077e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:23.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.125+0000 7f6cdcec2640 1 --2- 172.21.15.44:0/708891196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cd8101990 0x7f6cd8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:23.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.125+0000 7f6cdcec2640 1 -- 172.21.15.44:0/708891196 >> 172.21.15.44:0/708891196 conn(0x7f6cd80fd400 msgr2=0x7f6cd80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:23.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.125+0000 7f6cdcec2640 1 -- 172.21.15.44:0/708891196 shutdown_connections 2024-08-20T02:59:23.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.125+0000 7f6cdcec2640 1 -- 172.21.15.44:0/708891196 wait complete. 2024-08-20T02:59:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.125+0000 7f6cdcec2640 1 Processor -- start 2024-08-20T02:59:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cdcec2640 1 -- start start 2024-08-20T02:59:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cdcec2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd8101990 0x7f6cd80fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cdcec2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cd81073a0 0x7f6cd80fde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cdcec2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cd8107d20 0x7f6cd80fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cdcec2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd81119e0 con 0x7f6cd81073a0 2024-08-20T02:59:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cdcec2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd8111860 con 0x7f6cd8101990 2024-08-20T02:59:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cdcec2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd8111b60 con 0x7f6cd8107d20 2024-08-20T02:59:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cd5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cd81073a0 0x7f6cd80fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cd5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cd81073a0 0x7f6cd80fde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46996/0 (socket says 172.21.15.44:46996) 2024-08-20T02:59:23.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cd5d9b640 1 -- 172.21.15.44:0/3815597073 learned_addr learned my addr 172.21.15.44:0/3815597073 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cd6d9d640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cd8107d20 0x7f6cd80fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cd659c640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd8101990 0x7f6cd80fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cd5d9b640 1 -- 172.21.15.44:0/3815597073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cd8107d20 msgr2=0x7f6cd80fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cd5d9b640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cd8107d20 0x7f6cd80fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cd5d9b640 1 -- 172.21.15.44:0/3815597073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd8101990 msgr2=0x7f6cd80fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cd5d9b640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd8101990 0x7f6cd80fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cd5d9b640 1 -- 172.21.15.44:0/3815597073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd81a7390 con 0x7f6cd81073a0 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cd659c640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd8101990 0x7f6cd80fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.126+0000 7f6cd5d9b640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cd81073a0 0x7f6cd80fde40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0005940 tx=0x7f6cc00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.127+0000 7f6ccf7fe640 1 -- 172.21.15.44:0/3815597073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc003f030 con 0x7f6cd81073a0 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.127+0000 7f6cdcec2640 1 -- 172.21.15.44:0/3815597073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd81a7620 con 0x7f6cd81073a0 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.127+0000 7f6ccf7fe640 1 -- 172.21.15.44:0/3815597073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cc0003cb0 con 0x7f6cd81073a0 2024-08-20T02:59:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.127+0000 7f6ccf7fe640 1 -- 172.21.15.44:0/3815597073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0035070 con 0x7f6cd81073a0 2024-08-20T02:59:23.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.127+0000 7f6cdcec2640 1 -- 172.21.15.44:0/3815597073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd81a7b00 con 0x7f6cd81073a0 2024-08-20T02:59:23.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.128+0000 7f6cdcec2640 1 -- 172.21.15.44:0/3815597073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd8101d70 con 0x7f6cd81073a0 2024-08-20T02:59:23.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.129+0000 7f6ccf7fe640 1 -- 172.21.15.44:0/3815597073 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f6cc0042030 con 0x7f6cd81073a0 2024-08-20T02:59:23.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.130+0000 7f6ccf7fe640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6cc40b3c20 0x7f6cc40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:23.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.130+0000 7f6cd659c640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6cc40b3c20 0x7f6cc40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:23.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.130+0000 7f6ccf7fe640 1 -- 172.21.15.44:0/3815597073 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7024+0+0 (secure 0 0 0) 0x7f6cc0104010 con 0x7f6cd81073a0 2024-08-20T02:59:23.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.130+0000 7f6cd659c640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6cc40b3c20 0x7f6cc40b60e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80058c0 tx=0x7f6cc8005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:23.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.134+0000 7f6ccf7fe640 1 -- 172.21.15.44:0/3815597073 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f6cc00cbaf0 con 0x7f6cd81073a0 2024-08-20T02:59:23.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.321+0000 7f6cdcec2640 1 -- 172.21.15.44:0/3815597073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6cd81a7e80 con 0x7f6cd81073a0 2024-08-20T02:59:23.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.322+0000 7f6ccf7fe640 1 -- 172.21.15.44:0/3815597073 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7f6cc00cb3f0 con 0x7f6cd81073a0 2024-08-20T02:59:23.324 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:59:23.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.324+0000 7f6cdcec2640 1 -- 172.21.15.44:0/3815597073 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6cc40b3c20 msgr2=0x7f6cc40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.324+0000 7f6cdcec2640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6cc40b3c20 0x7f6cc40b60e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80058c0 tx=0x7f6cc8005850 comp rx=0 tx=0).stop 2024-08-20T02:59:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.324+0000 7f6cdcec2640 1 -- 172.21.15.44:0/3815597073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cd81073a0 msgr2=0x7f6cd80fde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.324+0000 7f6cdcec2640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cd81073a0 0x7f6cd80fde40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0005940 tx=0x7f6cc00042f0 comp rx=0 tx=0).stop 2024-08-20T02:59:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.324+0000 7f6cdcec2640 1 -- 172.21.15.44:0/3815597073 shutdown_connections 2024-08-20T02:59:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.324+0000 7f6cdcec2640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6cc40b3c20 0x7f6cc40b60e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.324+0000 7f6cdcec2640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cd8107d20 0x7f6cd80fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.324+0000 7f6cdcec2640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6cd81073a0 0x7f6cd80fde40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.324+0000 7f6cdcec2640 1 --2- 172.21.15.44:0/3815597073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cd8101990 0x7f6cd80fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:23.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.324+0000 7f6cdcec2640 1 -- 172.21.15.44:0/3815597073 >> 172.21.15.44:0/3815597073 conn(0x7f6cd80fd400 msgr2=0x7f6cd806a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:23.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.324+0000 7f6cdcec2640 1 -- 172.21.15.44:0/3815597073 shutdown_connections 2024-08-20T02:59:23.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:23.324+0000 7f6cdcec2640 1 -- 172.21.15.44:0/3815597073 wait complete. 2024-08-20T02:59:24.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:23 smithi044 ceph-mon[18790]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:24.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:23 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3815597073' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T02:59:24.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:23 smithi044 ceph-mon[18790]: from='osd.11 [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T02:59:24.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:23 smithi044 ceph-mon[18790]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T02:59:24.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:23 smithi165 ceph-mon[22357]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:24.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:23 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3815597073' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T02:59:24.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:23 smithi165 ceph-mon[22357]: from='osd.11 [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T02:59:24.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:23 smithi165 ceph-mon[22357]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T02:59:24.137 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 02:59:23 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T02:59:23.965+0000 7f7017b50640 -1 osd.11 0 log_to_monitors true 2024-08-20T02:59:24.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:23 smithi099 ceph-mon[21984]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:24.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:23 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3815597073' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T02:59:24.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:23 smithi099 ceph-mon[21984]: from='osd.11 [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T02:59:24.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:23 smithi099 ceph-mon[21984]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T02:59:24.208 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1724122740,"num_in_osds":12,"osd_in_since":1724122743,"num_remapped_pgs":0} 2024-08-20T02:59:25.210 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-20T02:59:25.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:24 smithi044 ceph-mon[18790]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-20T02:59:25.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:24 smithi044 ceph-mon[18790]: osdmap e67: 12 total, 11 up, 12 in 2024-08-20T02:59:25.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:24 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:25.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:24 smithi044 ceph-mon[18790]: from='osd.11 [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:59:25.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:24 smithi044 ceph-mon[18790]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:59:25.255 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:24 smithi165 ceph-mon[22357]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-20T02:59:25.256 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:24 smithi165 ceph-mon[22357]: osdmap e67: 12 total, 11 up, 12 in 2024-08-20T02:59:25.256 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:24 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:25.256 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:24 smithi165 ceph-mon[22357]: from='osd.11 [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:59:25.256 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:24 smithi165 ceph-mon[22357]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:59:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:24 smithi099 ceph-mon[21984]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-20T02:59:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:24 smithi099 ceph-mon[21984]: osdmap e67: 12 total, 11 up, 12 in 2024-08-20T02:59:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:24 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:24 smithi099 ceph-mon[21984]: from='osd.11 [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:59:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:24 smithi099 ceph-mon[21984]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]} : dispatch 2024-08-20T02:59:25.486 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:59:25.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T02:59:25.401+0000 7f7013ad3640 -1 osd.11 0 waiting for initial osdmap 2024-08-20T02:59:25.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T02:59:25.412+0000 7f700b8f2640 -1 osd.11 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T02:59:26.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 -- 172.21.15.44:0/1210247932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35701084e0 msgr2=0x7f3570108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:26.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 --2- 172.21.15.44:0/1210247932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35701084e0 0x7f3570108920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3564009900 tx=0x7f356402a410 comp rx=0 tx=0).stop 2024-08-20T02:59:26.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 -- 172.21.15.44:0/1210247932 shutdown_connections 2024-08-20T02:59:26.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 --2- 172.21.15.44:0/1210247932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3570108e60 0x7f357010f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:26.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 --2- 172.21.15.44:0/1210247932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35701084e0 0x7f3570108920 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:26.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 --2- 172.21.15.44:0/1210247932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3570102ad0 0x7f3570102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:26.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 -- 172.21.15.44:0/1210247932 >> 172.21.15.44:0/1210247932 conn(0x7f35700fe540 msgr2=0x7f3570100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:26.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 -- 172.21.15.44:0/1210247932 shutdown_connections 2024-08-20T02:59:26.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 -- 172.21.15.44:0/1210247932 wait complete. 2024-08-20T02:59:26.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 Processor -- start 2024-08-20T02:59:26.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 -- start start 2024-08-20T02:59:26.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3570102ad0 0x7f35700751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:26.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.958+0000 7f3577e79640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35701084e0 0x7f3570075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:26.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3577e79640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3570108e60 0x7f3570075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:26.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3577e79640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3570112b20 con 0x7f3570102ad0 2024-08-20T02:59:26.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3577e79640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35701129a0 con 0x7f3570108e60 2024-08-20T02:59:26.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3577e79640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3570112ca0 con 0x7f35701084e0 2024-08-20T02:59:26.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3575c15640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3570102ad0 0x7f35700751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:26.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3575c15640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3570102ad0 0x7f35700751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47006/0 (socket says 172.21.15.44:47006) 2024-08-20T02:59:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3575c15640 1 -- 172.21.15.44:0/179362181 learned_addr learned my addr 172.21.15.44:0/179362181 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:59:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3576416640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3570108e60 0x7f3570075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3575414640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35701084e0 0x7f3570075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3575c15640 1 -- 172.21.15.44:0/179362181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35701084e0 msgr2=0x7f3570075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3575c15640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35701084e0 0x7f3570075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3575c15640 1 -- 172.21.15.44:0/179362181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3570108e60 msgr2=0x7f3570075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3575c15640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3570108e60 0x7f3570075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3575c15640 1 -- 172.21.15.44:0/179362181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35701a7470 con 0x7f3570102ad0 2024-08-20T02:59:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3576416640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3570108e60 0x7f3570075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:59:26.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3575c15640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3570102ad0 0x7f35700751d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f356000ba50 tx=0x7f356000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f356effd640 1 -- 172.21.15.44:0/179362181 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560019070 con 0x7f3570102ad0 2024-08-20T02:59:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3577e79640 1 -- 172.21.15.44:0/179362181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35701a7760 con 0x7f3570102ad0 2024-08-20T02:59:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f356effd640 1 -- 172.21.15.44:0/179362181 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f356000cbe0 con 0x7f3570102ad0 2024-08-20T02:59:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f356effd640 1 -- 172.21.15.44:0/179362181 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35600075d0 con 0x7f3570102ad0 2024-08-20T02:59:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.959+0000 7f3577e79640 1 -- 172.21.15.44:0/179362181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35701a7cf0 con 0x7f3570102ad0 2024-08-20T02:59:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.960+0000 7f3577e79640 1 -- 172.21.15.44:0/179362181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3570102eb0 con 0x7f3570102ad0 2024-08-20T02:59:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.962+0000 7f356effd640 1 -- 172.21.15.44:0/179362181 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f356001c070 con 0x7f3570102ad0 2024-08-20T02:59:26.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.963+0000 7f356effd640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f35580b3c20 0x7f35580b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:26.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.963+0000 7f3575414640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f35580b3c20 0x7f35580b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:26.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.963+0000 7f356effd640 1 -- 172.21.15.44:0/179362181 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7062+0+0 (secure 0 0 0) 0x7f35600e32e0 con 0x7f3570102ad0 2024-08-20T02:59:26.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.964+0000 7f3575414640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f35580b3c20 0x7f35580b60e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3564031440 tx=0x7f35640095f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:26.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:25.966+0000 7f356effd640 1 -- 172.21.15.44:0/179362181 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f35600aae10 con 0x7f3570102ad0 2024-08-20T02:59:26.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.153+0000 7f3577e79640 1 -- 172.21.15.44:0/179362181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f35700764d0 con 0x7f3570102ad0 2024-08-20T02:59:26.156 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:26.156 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-08-20T02:59:26.157 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: osdmap e68: 12 total, 11 up, 12 in 2024-08-20T02:59:26.157 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:26.157 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:26.157 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:26.157 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:26.157 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:26.157 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:26.157 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:26.158 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:26.158 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:26.158 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:26.158 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:59:26.158 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:26.158 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:25 smithi044 ceph-mon[18790]: from='osd.11 ' entity='osd.11' 2024-08-20T02:59:26.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.154+0000 7f356effd640 1 -- 172.21.15.44:0/179362181 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f35700764d0 con 0x7f3570102ad0 2024-08-20T02:59:26.159 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:59:26.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.156+0000 7f3577e79640 1 -- 172.21.15.44:0/179362181 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f35580b3c20 msgr2=0x7f35580b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:26.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.156+0000 7f3577e79640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f35580b3c20 0x7f35580b60e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3564031440 tx=0x7f35640095f0 comp rx=0 tx=0).stop 2024-08-20T02:59:26.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.156+0000 7f3577e79640 1 -- 172.21.15.44:0/179362181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3570102ad0 msgr2=0x7f35700751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:26.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.156+0000 7f3577e79640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3570102ad0 0x7f35700751d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f356000ba50 tx=0x7f356000bf10 comp rx=0 tx=0).stop 2024-08-20T02:59:26.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.156+0000 7f3577e79640 1 -- 172.21.15.44:0/179362181 shutdown_connections 2024-08-20T02:59:26.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.156+0000 7f3577e79640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f35580b3c20 0x7f35580b60e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:26.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.156+0000 7f3577e79640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3570108e60 0x7f3570075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:26.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.156+0000 7f3577e79640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35701084e0 0x7f3570075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:26.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.156+0000 7f3577e79640 1 --2- 172.21.15.44:0/179362181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3570102ad0 0x7f35700751d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:26.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.156+0000 7f3577e79640 1 -- 172.21.15.44:0/179362181 >> 172.21.15.44:0/179362181 conn(0x7f35700fe540 msgr2=0x7f3570100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:26.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.156+0000 7f3577e79640 1 -- 172.21.15.44:0/179362181 shutdown_connections 2024-08-20T02:59:26.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:26.156+0000 7f3577e79640 1 -- 172.21.15.44:0/179362181 wait complete. 2024-08-20T02:59:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: osdmap e68: 12 total, 11 up, 12 in 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:59:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:26.388 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:25 smithi165 ceph-mon[22357]: from='osd.11 ' entity='osd.11' 2024-08-20T02:59:26.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:26.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi165", "root=default"]}]': finished 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: osdmap e68: 12 total, 11 up, 12 in 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:59:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:26.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:25 smithi099 ceph-mon[21984]: from='osd.11 ' entity='osd.11' 2024-08-20T02:59:26.887 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1724122740,"num_in_osds":12,"osd_in_since":1724122743,"num_remapped_pgs":0} 2024-08-20T02:59:27.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:26 smithi044 ceph-mon[18790]: Detected new or changed devices on smithi165 2024-08-20T02:59:27.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:26 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi165 to 4272M 2024-08-20T02:59:27.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:26 smithi044 ceph-mon[18790]: purged_snaps scrub starts 2024-08-20T02:59:27.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:26 smithi044 ceph-mon[18790]: purged_snaps scrub ok 2024-08-20T02:59:27.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:26 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:27.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:26 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/179362181' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T02:59:27.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:26 smithi044 ceph-mon[18790]: osd.11 [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415] boot 2024-08-20T02:59:27.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:26 smithi044 ceph-mon[18790]: osdmap e69: 12 total, 12 up, 12 in 2024-08-20T02:59:27.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:26 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:27.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:26 smithi165 ceph-mon[22357]: Detected new or changed devices on smithi165 2024-08-20T02:59:27.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:26 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi165 to 4272M 2024-08-20T02:59:27.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:26 smithi165 ceph-mon[22357]: purged_snaps scrub starts 2024-08-20T02:59:27.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:26 smithi165 ceph-mon[22357]: purged_snaps scrub ok 2024-08-20T02:59:27.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:26 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:27.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:26 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/179362181' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T02:59:27.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:26 smithi165 ceph-mon[22357]: osd.11 [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415] boot 2024-08-20T02:59:27.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:26 smithi165 ceph-mon[22357]: osdmap e69: 12 total, 12 up, 12 in 2024-08-20T02:59:27.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:26 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:27.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:26 smithi099 ceph-mon[21984]: Detected new or changed devices on smithi165 2024-08-20T02:59:27.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:26 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi165 to 4272M 2024-08-20T02:59:27.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:26 smithi099 ceph-mon[21984]: purged_snaps scrub starts 2024-08-20T02:59:27.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:26 smithi099 ceph-mon[21984]: purged_snaps scrub ok 2024-08-20T02:59:27.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:26 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:27.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:26 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/179362181' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T02:59:27.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:26 smithi099 ceph-mon[21984]: osd.11 [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415] boot 2024-08-20T02:59:27.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:26 smithi099 ceph-mon[21984]: osdmap e69: 12 total, 12 up, 12 in 2024-08-20T02:59:27.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:26 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T02:59:27.889 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-20T02:59:28.156 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:59:28.276 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:27 smithi044 ceph-mon[18790]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:28.276 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:27 smithi044 ceph-mon[18790]: osdmap e70: 12 total, 12 up, 12 in 2024-08-20T02:59:28.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:28 smithi165 ceph-mon[22357]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:28.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:28 smithi165 ceph-mon[22357]: osdmap e70: 12 total, 12 up, 12 in 2024-08-20T02:59:28.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:27 smithi099 ceph-mon[21984]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-08-20T02:59:28.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:27 smithi099 ceph-mon[21984]: osdmap e70: 12 total, 12 up, 12 in 2024-08-20T02:59:28.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.709+0000 7f3754ca5640 1 -- 172.21.15.44:0/2978731194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f375010f000 msgr2=0x7f3750111480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:28.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.709+0000 7f3754ca5640 1 --2- 172.21.15.44:0/2978731194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f375010f000 0x7f3750111480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3748009ca0 tx=0x7f374802a8c0 comp rx=0 tx=0).stop 2024-08-20T02:59:28.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.709+0000 7f3754ca5640 1 -- 172.21.15.44:0/2978731194 shutdown_connections 2024-08-20T02:59:28.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.709+0000 7f3754ca5640 1 --2- 172.21.15.44:0/2978731194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f375010f000 0x7f3750111480 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:28.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.709+0000 7f3754ca5640 1 --2- 172.21.15.44:0/2978731194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3750101e40 0x7f375010ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:28.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.709+0000 7f3754ca5640 1 --2- 172.21.15.44:0/2978731194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3750101520 0x7f3750101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:28.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.709+0000 7f3754ca5640 1 -- 172.21.15.44:0/2978731194 >> 172.21.15.44:0/2978731194 conn(0x7f3750076c80 msgr2=0x7f3750077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:28.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.709+0000 7f3754ca5640 1 -- 172.21.15.44:0/2978731194 shutdown_connections 2024-08-20T02:59:28.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.709+0000 7f3754ca5640 1 -- 172.21.15.44:0/2978731194 wait complete. 2024-08-20T02:59:28.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.710+0000 7f3754ca5640 1 Processor -- start 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.710+0000 7f3754ca5640 1 -- start start 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.710+0000 7f3754ca5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3750101520 0x7f375019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.710+0000 7f3754ca5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3750101e40 0x7f375019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.710+0000 7f3754ca5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f375010f000 0x7f37501a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.710+0000 7f3754ca5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3750114cf0 con 0x7f3750101520 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.710+0000 7f3754ca5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3750114b70 con 0x7f375010f000 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.710+0000 7f3754ca5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3750114e70 con 0x7f3750101e40 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.710+0000 7f374e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3750101520 0x7f375019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f374e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3750101520 0x7f375019b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53552/0 (socket says 172.21.15.44:53552) 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f374e59c640 1 -- 172.21.15.44:0/516869676 learned_addr learned my addr 172.21.15.44:0/516869676 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f374ed9d640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f375010f000 0x7f37501a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f374dd9b640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3750101e40 0x7f375019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:28.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f374e59c640 1 -- 172.21.15.44:0/516869676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3750101e40 msgr2=0x7f375019b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f374e59c640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3750101e40 0x7f375019b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f374e59c640 1 -- 172.21.15.44:0/516869676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f375010f000 msgr2=0x7f37501a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f374e59c640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f375010f000 0x7f37501a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f374e59c640 1 -- 172.21.15.44:0/516869676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37501a1350 con 0x7f3750101520 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f374dd9b640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3750101e40 0x7f375019b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f374ed9d640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f375010f000 0x7f37501a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f374e59c640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3750101520 0x7f375019b2a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f374000b4a0 tx=0x7f374000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f373f7fe640 1 -- 172.21.15.44:0/516869676 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740014070 con 0x7f3750101520 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f3754ca5640 1 -- 172.21.15.44:0/516869676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37501a1640 con 0x7f3750101520 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f373f7fe640 1 -- 172.21.15.44:0/516869676 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3740010ac0 con 0x7f3750101520 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f373f7fe640 1 -- 172.21.15.44:0/516869676 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740017c30 con 0x7f3750101520 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.711+0000 7f3754ca5640 1 -- 172.21.15.44:0/516869676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37501a7ed0 con 0x7f3750101520 2024-08-20T02:59:28.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.712+0000 7f3754ca5640 1 -- 172.21.15.44:0/516869676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3750101900 con 0x7f3750101520 2024-08-20T02:59:28.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.714+0000 7f373f7fe640 1 -- 172.21.15.44:0/516869676 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f3740017070 con 0x7f3750101520 2024-08-20T02:59:28.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.714+0000 7f373f7fe640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f37380b3ca0 0x7f37380b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:28.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.714+0000 7f374dd9b640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f37380b3ca0 0x7f37380b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:28.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.714+0000 7f373f7fe640 1 -- 172.21.15.44:0/516869676 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f37400e2fe0 con 0x7f3750101520 2024-08-20T02:59:28.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.715+0000 7f374dd9b640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f37380b3ca0 0x7f37380b6160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f37501117e0 tx=0x7f3744005e70 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:28.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.717+0000 7f373f7fe640 1 -- 172.21.15.44:0/516869676 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f37400aa9a0 con 0x7f3750101520 2024-08-20T02:59:28.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.901+0000 7f3754ca5640 1 -- 172.21.15.44:0/516869676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f375019c3d0 con 0x7f3750101520 2024-08-20T02:59:28.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.901+0000 7f373f7fe640 1 -- 172.21.15.44:0/516869676 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f375019c3d0 con 0x7f3750101520 2024-08-20T02:59:28.904 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:59:28.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.903+0000 7f3754ca5640 1 -- 172.21.15.44:0/516869676 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f37380b3ca0 msgr2=0x7f37380b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:28.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.903+0000 7f3754ca5640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f37380b3ca0 0x7f37380b6160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f37501117e0 tx=0x7f3744005e70 comp rx=0 tx=0).stop 2024-08-20T02:59:28.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.903+0000 7f3754ca5640 1 -- 172.21.15.44:0/516869676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3750101520 msgr2=0x7f375019b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:28.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.903+0000 7f3754ca5640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3750101520 0x7f375019b2a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f374000b4a0 tx=0x7f374000b960 comp rx=0 tx=0).stop 2024-08-20T02:59:28.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.904+0000 7f3754ca5640 1 -- 172.21.15.44:0/516869676 shutdown_connections 2024-08-20T02:59:28.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.904+0000 7f3754ca5640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f37380b3ca0 0x7f37380b6160 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:28.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.904+0000 7f3754ca5640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f375010f000 0x7f37501a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:28.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.904+0000 7f3754ca5640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3750101e40 0x7f375019b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:28.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.904+0000 7f3754ca5640 1 --2- 172.21.15.44:0/516869676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3750101520 0x7f375019b2a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:28.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.904+0000 7f3754ca5640 1 -- 172.21.15.44:0/516869676 >> 172.21.15.44:0/516869676 conn(0x7f3750076c80 msgr2=0x7f375010f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:28.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.904+0000 7f3754ca5640 1 -- 172.21.15.44:0/516869676 shutdown_connections 2024-08-20T02:59:28.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:28.904+0000 7f3754ca5640 1 -- 172.21.15.44:0/516869676 wait complete. 2024-08-20T02:59:29.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:29 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/516869676' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T02:59:29.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:29 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/516869676' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T02:59:29.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:29 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/516869676' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T02:59:29.629 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1724122766,"num_in_osds":12,"osd_in_since":1724122743,"num_remapped_pgs":0} 2024-08-20T02:59:29.630 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T02:59:29.880 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:59:30.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:30 smithi044 ceph-mon[18790]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:30.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.324+0000 7f26282f8640 1 -- 172.21.15.44:0/1490237185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f262010ce40 msgr2=0x7f262010f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:30.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.324+0000 7f26282f8640 1 --2- 172.21.15.44:0/1490237185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f262010ce40 0x7f262010f2a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2618009900 tx=0x7f261802a410 comp rx=0 tx=0).stop 2024-08-20T02:59:30.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.324+0000 7f26282f8640 1 -- 172.21.15.44:0/1490237185 shutdown_connections 2024-08-20T02:59:30.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.324+0000 7f26282f8640 1 --2- 172.21.15.44:0/1490237185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f262010ce40 0x7f262010f2a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:30.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.324+0000 7f26282f8640 1 --2- 172.21.15.44:0/1490237185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2620104300 0x7f262010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:30.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.324+0000 7f26282f8640 1 --2- 172.21.15.44:0/1490237185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26201039e0 0x7f2620103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:30.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.324+0000 7f26282f8640 1 -- 172.21.15.44:0/1490237185 >> 172.21.15.44:0/1490237185 conn(0x7f26200fd440 msgr2=0x7f26200ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:30.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.325+0000 7f26282f8640 1 -- 172.21.15.44:0/1490237185 shutdown_connections 2024-08-20T02:59:30.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.325+0000 7f26282f8640 1 -- 172.21.15.44:0/1490237185 wait complete. 2024-08-20T02:59:30.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.325+0000 7f26282f8640 1 Processor -- start 2024-08-20T02:59:30.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.325+0000 7f26282f8640 1 -- start start 2024-08-20T02:59:30.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.325+0000 7f26282f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26201039e0 0x7f2620198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:30.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.325+0000 7f26282f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2620104300 0x7f26201994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:30.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.325+0000 7f26282f8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f262010ce40 0x7f262019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:30.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.325+0000 7f26282f8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2620112b40 con 0x7f26201039e0 2024-08-20T02:59:30.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.325+0000 7f26282f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26201129c0 con 0x7f2620104300 2024-08-20T02:59:30.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.325+0000 7f26282f8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2620112cc0 con 0x7f262010ce40 2024-08-20T02:59:30.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f2626094640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26201039e0 0x7f2620198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f2625893640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2620104300 0x7f26201994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f2625893640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2620104300 0x7f26201994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:33834/0 (socket says 172.21.15.44:33834) 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f2626094640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26201039e0 0x7f2620198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53570/0 (socket says 172.21.15.44:53570) 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f2625893640 1 -- 172.21.15.44:0/1590146846 learned_addr learned my addr 172.21.15.44:0/1590146846 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f2626895640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f262010ce40 0x7f262019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f2626094640 1 -- 172.21.15.44:0/1590146846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f262010ce40 msgr2=0x7f262019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f2626094640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f262010ce40 0x7f262019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f2626094640 1 -- 172.21.15.44:0/1590146846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2620104300 msgr2=0x7f26201994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f2626094640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2620104300 0x7f26201994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f2626094640 1 -- 172.21.15.44:0/1590146846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262019f040 con 0x7f26201039e0 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f2626094640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26201039e0 0x7f2620198f90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f261000bd70 tx=0x7f261000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f261f7fe640 1 -- 172.21.15.44:0/1590146846 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610019070 con 0x7f26201039e0 2024-08-20T02:59:30.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f261f7fe640 1 -- 172.21.15.44:0/1590146846 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26100148f0 con 0x7f26201039e0 2024-08-20T02:59:30.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f26282f8640 1 -- 172.21.15.44:0/1590146846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262019f330 con 0x7f26201039e0 2024-08-20T02:59:30.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f261f7fe640 1 -- 172.21.15.44:0/1590146846 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2610012860 con 0x7f26201039e0 2024-08-20T02:59:30.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.326+0000 7f26282f8640 1 -- 172.21.15.44:0/1590146846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26201a5a70 con 0x7f26201039e0 2024-08-20T02:59:30.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.327+0000 7f26282f8640 1 -- 172.21.15.44:0/1590146846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2620103dc0 con 0x7f26201039e0 2024-08-20T02:59:30.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.329+0000 7f261f7fe640 1 -- 172.21.15.44:0/1590146846 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f261001c070 con 0x7f26201039e0 2024-08-20T02:59:30.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.330+0000 7f261f7fe640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f26080b3c20 0x7f26080b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:30.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.330+0000 7f2625893640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f26080b3c20 0x7f26080b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:30.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.330+0000 7f261f7fe640 1 -- 172.21.15.44:0/1590146846 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f26100e35f0 con 0x7f26201039e0 2024-08-20T02:59:30.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.330+0000 7f2625893640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f26080b3c20 0x7f26080b60e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f26200ffc90 tx=0x7f261400a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:30.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.333+0000 7f261f7fe640 1 -- 172.21.15.44:0/1590146846 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f26100aa920 con 0x7f26201039e0 2024-08-20T02:59:30.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:30 smithi165 ceph-mon[22357]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:30 smithi099 ceph-mon[21984]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:30.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.520+0000 7f26282f8640 1 -- 172.21.15.44:0/1590146846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26201a5e40 con 0x7f26201039e0 2024-08-20T02:59:30.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.521+0000 7f261f7fe640 1 -- 172.21.15.44:0/1590146846 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18738 (secure 0 0 0) 0x7f26100aee70 con 0x7f26201039e0 2024-08-20T02:59:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:59:30.523 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":70,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","created":"2024-08-20T02:52:10.979264+0000","modified":"2024-08-20T02:59:27.406323+0000","last_up_change":"2024-08-20T02:59:26.404917+0000","last_in_change":"2024-08-20T02:59:03.707380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T02:55:46.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7708f8c-a799-4b8f-ab1e-78482d73ca1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6803","nonce":858898316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6805","nonce":858898316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6809","nonce":858898316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6807","nonce":858898316}]},"public_addr":"172.21.15.44:6803/858898316","cluster_addr":"172.21.15.44:6805/858898316","heartbeat_back_addr":"172.21.15.44:6809/858898316","heartbeat_front_addr":"172.21.15.44:6807/858898316","state":["exists","up"]},{"osd":1,"uuid":"12f07986-60bf-498f-a012-cd7ea607b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6801","nonce":4252911663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6803","nonce":4252911663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6807","nonce":4252911663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6805","nonce":4252911663}]},"public_addr":"172.21.15.99:6801/4252911663","cluster_addr":"172.21.15.99:6803/4252911663","heartbeat_back_addr":"172.21.15.99:6807/4252911663","heartbeat_front_addr":"172.21.15.99:6805/4252911663","state":["exists","up"]},{"osd":2,"uuid":"0f189000-ba7f-4412-b0d3-75e8265f6685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6801","nonce":1492740465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6803","nonce":1492740465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6807","nonce":1492740465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6805","nonce":1492740465}]},"public_addr":"172.21.15.165:6801/1492740465","cluster_addr":"172.21.15.165:6803/1492740465","heartbeat_back_addr":"172.21.15.165:6807/1492740465","heartbeat_front_addr":"172.21.15.165:6805/1492740465","state":["exists","up"]},{"osd":3,"uuid":"8f9b042c-d6b5-4eee-aa29-ffc41a48c20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6811","nonce":3423983702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6813","nonce":3423983702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6817","nonce":3423983702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6815","nonce":3423983702}]},"public_addr":"172.21.15.44:6811/3423983702","cluster_addr":"172.21.15.44:6813/3423983702","heartbeat_back_addr":"172.21.15.44:6817/3423983702","heartbeat_front_addr":"172.21.15.44:6815/3423983702","state":["exists","up"]},{"osd":4,"uuid":"4edebfdb-af10-49ee-b724-2a63b9219221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6809","nonce":3099617325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6811","nonce":3099617325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6815","nonce":3099617325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6813","nonce":3099617325}]},"public_addr":"172.21.15.99:6809/3099617325","cluster_addr":"172.21.15.99:6811/3099617325","heartbeat_back_addr":"172.21.15.99:6815/3099617325","heartbeat_front_addr":"172.21.15.99:6813/3099617325","state":["exists","up"]},{"osd":5,"uuid":"6d22c21f-cc43-4ef5-a75c-d4fc1a361c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6809","nonce":523245255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6811","nonce":523245255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6815","nonce":523245255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6813","nonce":523245255}]},"public_addr":"172.21.15.165:6809/523245255","cluster_addr":"172.21.15.165:6811/523245255","heartbeat_back_addr":"172.21.15.165:6815/523245255","heartbeat_front_addr":"172.21.15.165:6813/523245255","state":["exists","up"]},{"osd":6,"uuid":"03df0c2f-1b7c-4242-888c-e047322fd8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6819","nonce":1039098986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6821","nonce":1039098986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6825","nonce":1039098986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6823","nonce":1039098986}]},"public_addr":"172.21.15.44:6819/1039098986","cluster_addr":"172.21.15.44:6821/1039098986","heartbeat_back_addr":"172.21.15.44:6825/1039098986","heartbeat_front_addr":"172.21.15.44:6823/1039098986","state":["exists","up"]},{"osd":7,"uuid":"84bbc683-9b13-448e-897c-114458dae33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6817","nonce":1438613564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6819","nonce":1438613564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6823","nonce":1438613564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6821","nonce":1438613564}]},"public_addr":"172.21.15.99:6817/1438613564","cluster_addr":"172.21.15.99:6819/1438613564","heartbeat_back_addr":"172.21.15.99:6823/1438613564","heartbeat_front_addr":"172.21.15.99:6821/1438613564","state":["exists","up"]},{"osd":8,"uuid":"14d7ffd3-db05-4228-a12e-4c69ef65ef8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6817","nonce":632957688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6819","nonce":632957688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6823","nonce":632957688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6821","nonce":632957688}]},"public_addr":"172.21.15.165:6817/632957688","cluster_addr":"172.21.15.165:6819/632957688","heartbeat_back_addr":"172.21.15.165:6823/632957688","heartbeat_front_addr":"172.21.15.165:6821/632957688","state":["exists","up"]},{"osd":9,"uuid":"1ae468a5-3d72-4eb0-a1a6-cd87eae740b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6827","nonce":2426940093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6829","nonce":2426940093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6833","nonce":2426940093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6831","nonce":2426940093}]},"public_addr":"172.21.15.44:6827/2426940093","cluster_addr":"172.21.15.44:6829/2426940093","heartbeat_back_addr":"172.21.15.44:6833/2426940093","heartbeat_front_addr":"172.21.15.44:6831/2426940093","state":["exists","up"]},{"osd":10,"uuid":"fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6825","nonce":3275793038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6827","nonce":3275793038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6831","nonce":3275793038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6829","nonce":3275793038}]},"public_addr":"172.21.15.99:6825/3275793038","cluster_addr":"172.21.15.99:6827/3275793038","heartbeat_back_addr":"172.21.15.99:6831/3275793038","heartbeat_front_addr":"172.21.15.99:6829/3275793038","state":["exists","up"]},{"osd":11,"uuid":"50715410-1fc2-4818-b600-604a4b7cce9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6825","nonce":107688415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6827","nonce":107688415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6831","nonce":107688415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6829","nonce":107688415}]},"public_addr":"172.21.15.165:6825/107688415","cluster_addr":"172.21.15.165:6827/107688415","heartbeat_back_addr":"172.21.15.165:6831/107688415","heartbeat_front_addr":"172.21.15.165:6829/107688415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:54:56.697075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:20.486846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:43.353376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:08.333879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:32.186492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:55.891366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:19.681421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:45.475926+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:10.109795+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:34.444169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:59.147646+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:59:25.403071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2597827045":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/3582612619":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/1916809746":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2919033419":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:6800/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2331034359":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2572185146":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/4176802819":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/3654439062":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6801/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/2376733275":"2024-08-21T02:53:58.312659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T02:59:30.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.523+0000 7f26282f8640 1 -- 172.21.15.44:0/1590146846 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f26080b3c20 msgr2=0x7f26080b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:30.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.523+0000 7f26282f8640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f26080b3c20 0x7f26080b60e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f26200ffc90 tx=0x7f261400a2d0 comp rx=0 tx=0).stop 2024-08-20T02:59:30.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.523+0000 7f26282f8640 1 -- 172.21.15.44:0/1590146846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26201039e0 msgr2=0x7f2620198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:30.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.523+0000 7f26282f8640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26201039e0 0x7f2620198f90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f261000bd70 tx=0x7f261000be60 comp rx=0 tx=0).stop 2024-08-20T02:59:30.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.523+0000 7f26282f8640 1 -- 172.21.15.44:0/1590146846 shutdown_connections 2024-08-20T02:59:30.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.523+0000 7f26282f8640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f26080b3c20 0x7f26080b60e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:30.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.523+0000 7f26282f8640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f262010ce40 0x7f262019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:30.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.523+0000 7f26282f8640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2620104300 0x7f26201994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:30.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.523+0000 7f26282f8640 1 --2- 172.21.15.44:0/1590146846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26201039e0 0x7f2620198f90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:30.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.523+0000 7f26282f8640 1 -- 172.21.15.44:0/1590146846 >> 172.21.15.44:0/1590146846 conn(0x7f26200fd440 msgr2=0x7f262010f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:30.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.523+0000 7f26282f8640 1 -- 172.21.15.44:0/1590146846 shutdown_connections 2024-08-20T02:59:30.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:30.523+0000 7f26282f8640 1 -- 172.21.15.44:0/1590146846 wait complete. 2024-08-20T02:59:31.152 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:31 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1590146846' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T02:59:31.181 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-20T02:55:46.770770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-20T02:59:31.182 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-08-20T02:59:31.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:31 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1590146846' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T02:59:31.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:31 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1590146846' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T02:59:31.431 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:59:31.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.921+0000 7f7110386640 1 -- 172.21.15.44:0/2739720322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71081073e0 msgr2=0x7f7108107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.921+0000 7f7110386640 1 --2- 172.21.15.44:0/2739720322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71081073e0 0x7f7108107820 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009900 tx=0x7f70f802a410 comp rx=0 tx=0).stop 2024-08-20T02:59:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.921+0000 7f7110386640 1 -- 172.21.15.44:0/2739720322 shutdown_connections 2024-08-20T02:59:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.921+0000 7f7110386640 1 --2- 172.21.15.44:0/2739720322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7108107d60 0x7f710810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.921+0000 7f7110386640 1 --2- 172.21.15.44:0/2739720322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71081073e0 0x7f7108107820 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.921+0000 7f7110386640 1 --2- 172.21.15.44:0/2739720322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71081019d0 0x7f7108101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.921+0000 7f7110386640 1 -- 172.21.15.44:0/2739720322 >> 172.21.15.44:0/2739720322 conn(0x7f71080fd440 msgr2=0x7f71080ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.921+0000 7f7110386640 1 -- 172.21.15.44:0/2739720322 shutdown_connections 2024-08-20T02:59:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.921+0000 7f7110386640 1 -- 172.21.15.44:0/2739720322 wait complete. 2024-08-20T02:59:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f7110386640 1 Processor -- start 2024-08-20T02:59:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f7110386640 1 -- start start 2024-08-20T02:59:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f7110386640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71081019d0 0x7f710810f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:31.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f7110386640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71081073e0 0x7f710810f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f7110386640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7108107d60 0x7f710810faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f7110386640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7108112b40 con 0x7f7108107d60 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f7110386640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71081129c0 con 0x7f71081019d0 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f7110386640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7108112cc0 con 0x7f71081073e0 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f710e923640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7108107d60 0x7f710810faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f710e923640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7108107d60 0x7f710810faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53576/0 (socket says 172.21.15.44:53576) 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f710e923640 1 -- 172.21.15.44:0/777977962 learned_addr learned my addr 172.21.15.44:0/777977962 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f710e122640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71081019d0 0x7f710810f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f710d921640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71081073e0 0x7f710810f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f710e923640 1 -- 172.21.15.44:0/777977962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71081073e0 msgr2=0x7f710810f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f710e923640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71081073e0 0x7f710810f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f710e923640 1 -- 172.21.15.44:0/777977962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71081019d0 msgr2=0x7f710810f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:31.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f710e923640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71081019d0 0x7f710810f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f710e923640 1 -- 172.21.15.44:0/777977962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71081a7330 con 0x7f7108107d60 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f710e122640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71081019d0 0x7f710810f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.922+0000 7f710e923640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7108107d60 0x7f710810faf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f710000cce0 tx=0x7f710000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.923+0000 7f70ff7fe640 1 -- 172.21.15.44:0/777977962 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710000f450 con 0x7f7108107d60 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.923+0000 7f70ff7fe640 1 -- 172.21.15.44:0/777977962 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f710000faa0 con 0x7f7108107d60 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.923+0000 7f7110386640 1 -- 172.21.15.44:0/777977962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71081a7620 con 0x7f7108107d60 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.923+0000 7f70ff7fe640 1 -- 172.21.15.44:0/777977962 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100018b70 con 0x7f7108107d60 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.923+0000 7f7110386640 1 -- 172.21.15.44:0/777977962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71081a7b30 con 0x7f7108107d60 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.924+0000 7f7110386640 1 -- 172.21.15.44:0/777977962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7108101db0 con 0x7f7108107d60 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.925+0000 7f70ff7fe640 1 -- 172.21.15.44:0/777977962 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f7100018d10 con 0x7f7108107d60 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.926+0000 7f70ff7fe640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f70f40b3ca0 0x7f70f40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.926+0000 7f710e122640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f70f40b3ca0 0x7f70f40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:31.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.926+0000 7f70ff7fe640 1 -- 172.21.15.44:0/777977962 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f71000e3e10 con 0x7f7108107d60 2024-08-20T02:59:31.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.926+0000 7f710e122640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f70f40b3ca0 0x7f70f40b6160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f71040058c0 tx=0x7f7104005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:31.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:31.929+0000 7f70ff7fe640 1 -- 172.21.15.44:0/777977962 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7100014030 con 0x7f7108107d60 2024-08-20T02:59:32.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.116+0000 7f7110386640 1 -- 172.21.15.44:0/777977962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71081a7f00 con 0x7f7108107d60 2024-08-20T02:59:32.118 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:32 smithi044 ceph-mon[18790]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:32.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.116+0000 7f70ff7fe640 1 -- 172.21.15.44:0/777977962 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7f71000ab940 con 0x7f7108107d60 2024-08-20T02:59:32.119 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-08-20T02:59:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.118+0000 7f7110386640 1 -- 172.21.15.44:0/777977962 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f70f40b3ca0 msgr2=0x7f70f40b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.118+0000 7f7110386640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f70f40b3ca0 0x7f70f40b6160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f71040058c0 tx=0x7f7104005850 comp rx=0 tx=0).stop 2024-08-20T02:59:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.118+0000 7f7110386640 1 -- 172.21.15.44:0/777977962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7108107d60 msgr2=0x7f710810faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.118+0000 7f7110386640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7108107d60 0x7f710810faf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f710000cce0 tx=0x7f710000a620 comp rx=0 tx=0).stop 2024-08-20T02:59:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.118+0000 7f7110386640 1 -- 172.21.15.44:0/777977962 shutdown_connections 2024-08-20T02:59:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.118+0000 7f7110386640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f70f40b3ca0 0x7f70f40b6160 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.118+0000 7f7110386640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7108107d60 0x7f710810faf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.118+0000 7f7110386640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71081073e0 0x7f710810f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.118+0000 7f7110386640 1 --2- 172.21.15.44:0/777977962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71081019d0 0x7f710810f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.118+0000 7f7110386640 1 -- 172.21.15.44:0/777977962 >> 172.21.15.44:0/777977962 conn(0x7f71080fd440 msgr2=0x7f71080ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.118+0000 7f7110386640 1 -- 172.21.15.44:0/777977962 shutdown_connections 2024-08-20T02:59:32.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:32.118+0000 7f7110386640 1 -- 172.21.15.44:0/777977962 wait complete. 2024-08-20T02:59:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:32 smithi165 ceph-mon[22357]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:32 smithi099 ceph-mon[21984]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:32.951 INFO:tasks.cephadm:Adding mds.a on smithi044 2024-08-20T02:59:32.951 INFO:tasks.cephadm:Adding mds.d on smithi044 2024-08-20T02:59:32.951 INFO:tasks.cephadm:Adding mds.g on smithi044 2024-08-20T02:59:32.951 INFO:tasks.cephadm:Adding mds.j on smithi044 2024-08-20T02:59:32.951 INFO:tasks.cephadm:Adding mds.b on smithi099 2024-08-20T02:59:32.951 INFO:tasks.cephadm:Adding mds.e on smithi099 2024-08-20T02:59:32.951 INFO:tasks.cephadm:Adding mds.h on smithi099 2024-08-20T02:59:32.951 INFO:tasks.cephadm:Adding mds.k on smithi099 2024-08-20T02:59:32.951 INFO:tasks.cephadm:Adding mds.c on smithi165 2024-08-20T02:59:32.951 INFO:tasks.cephadm:Adding mds.f on smithi165 2024-08-20T02:59:32.952 INFO:tasks.cephadm:Adding mds.i on smithi165 2024-08-20T02:59:32.952 INFO:tasks.cephadm:Adding mds.l on smithi165 2024-08-20T02:59:32.952 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph orch apply mds all '12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l' 2024-08-20T02:59:33.214 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T02:59:33.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:33 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/777977962' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T02:59:33.322 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:33 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/777977962' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T02:59:33.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:33 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/777977962' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T02:59:33.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.718+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/1943996548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f08101990 msgr2=0x7f6f08101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.718+0000 7f6f0ed2a640 1 --2- 172.21.15.165:0/1943996548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f08101990 0x7f6f08101d70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009ca0 tx=0x7f6ef802a8c0 comp rx=0 tx=0).stop 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.718+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/1943996548 shutdown_connections 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.718+0000 7f6f0ed2a640 1 --2- 172.21.15.165:0/1943996548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f08107d20 0x7f6f0810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.718+0000 7f6f0ed2a640 1 --2- 172.21.15.165:0/1943996548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f081073a0 0x7f6f081077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.718+0000 7f6f0ed2a640 1 --2- 172.21.15.165:0/1943996548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f08101990 0x7f6f08101d70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.718+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/1943996548 >> 172.21.15.165:0/1943996548 conn(0x7f6f080fd400 msgr2=0x7f6f080ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.718+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/1943996548 shutdown_connections 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.718+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/1943996548 wait complete. 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.719+0000 7f6f0ed2a640 1 Processor -- start 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.719+0000 7f6f0ed2a640 1 -- start start 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0ed2a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f08101990 0x7f6f08199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0ed2a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f081073a0 0x7f6f0819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:33.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0ed2a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f08107d20 0x7f6f0819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:33.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0ed2a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6f081119e0 con 0x7f6f08107d20 2024-08-20T02:59:33.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0ed2a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08111860 con 0x7f6f081073a0 2024-08-20T02:59:33.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0ed2a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6f08111b60 con 0x7f6f08101990 2024-08-20T02:59:33.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0cac6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f08101990 0x7f6f08199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:33.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0cac6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f08101990 0x7f6f08199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50070/0 (socket says 172.21.15.165:50070) 2024-08-20T02:59:33.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0cac6640 1 -- 172.21.15.165:0/3716771098 learned_addr learned my addr 172.21.15.165:0/3716771098 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-08-20T02:59:33.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f07fff640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f081073a0 0x7f6f0819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:33.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0d2c7640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f08107d20 0x7f6f0819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:33.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0cac6640 1 -- 172.21.15.165:0/3716771098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f081073a0 msgr2=0x7f6f0819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:33.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0cac6640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f081073a0 0x7f6f0819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:33.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0cac6640 1 -- 172.21.15.165:0/3716771098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f08107d20 msgr2=0x7f6f0819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:33.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0cac6640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f08107d20 0x7f6f0819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0cac6640 1 -- 172.21.15.165:0/3716771098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f081a5170 con 0x7f6f08101990 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0cac6640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f08101990 0x7f6f08199d70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8006010 tx=0x7f6ef80319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f05ffb640 1 -- 172.21.15.165:0/3716771098 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8003bf0 con 0x7f6f08101990 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f05ffb640 1 -- 172.21.15.165:0/3716771098 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef8003d90 con 0x7f6f08101990 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/3716771098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f081a5400 con 0x7f6f08101990 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.720+0000 7f6f05ffb640 1 -- 172.21.15.165:0/3716771098 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef802fc20 con 0x7f6f08101990 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.721+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/3716771098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f081a5940 con 0x7f6f08101990 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.722+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/3716771098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f08101d70 con 0x7f6f08101990 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.724+0000 7f6f05ffb640 1 -- 172.21.15.165:0/3716771098 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f6ef80046a0 con 0x7f6f08101990 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.725+0000 7f6f05ffb640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6ef40b3c20 0x7f6ef40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.725+0000 7f6f05ffb640 1 -- 172.21.15.165:0/3716771098 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f6ef8104ed0 con 0x7f6f08101990 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.725+0000 7f6f07fff640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6ef40b3c20 0x7f6ef40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.726+0000 7f6f07fff640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6ef40b3c20 0x7f6ef40b60e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f0806b3d0 tx=0x7f6ef000b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.727+0000 7f6f05ffb640 1 -- 172.21.15.165:0/3716771098 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f6ef80cc8f0 con 0x7f6f08101990 2024-08-20T02:59:33.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.924+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/3716771098 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}) -- 0x7f6f080632c0 con 0x7f6ef40b3c20 2024-08-20T02:59:33.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.932+0000 7f6f05ffb640 1 -- 172.21.15.165:0/3716771098 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6f080632c0 con 0x7f6ef40b3c20 2024-08-20T02:59:33.935 INFO:teuthology.orchestra.run.smithi165.stdout:Scheduled mds.all update... 2024-08-20T02:59:33.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.934+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/3716771098 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6ef40b3c20 msgr2=0x7f6ef40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:33.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.934+0000 7f6f0ed2a640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6ef40b3c20 0x7f6ef40b60e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f0806b3d0 tx=0x7f6ef000b410 comp rx=0 tx=0).stop 2024-08-20T02:59:33.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.934+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/3716771098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f08101990 msgr2=0x7f6f08199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:33.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.934+0000 7f6f0ed2a640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f08101990 0x7f6f08199d70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8006010 tx=0x7f6ef80319f0 comp rx=0 tx=0).stop 2024-08-20T02:59:33.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.934+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/3716771098 shutdown_connections 2024-08-20T02:59:33.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.934+0000 7f6f0ed2a640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6ef40b3c20 0x7f6ef40b60e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:33.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.934+0000 7f6f0ed2a640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6f08107d20 0x7f6f0819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:33.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.934+0000 7f6f0ed2a640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f081073a0 0x7f6f0819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:33.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.934+0000 7f6f0ed2a640 1 --2- 172.21.15.165:0/3716771098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6f08101990 0x7f6f08199d70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:33.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.934+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/3716771098 >> 172.21.15.165:0/3716771098 conn(0x7f6f080fd400 msgr2=0x7f6f080ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:33.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.934+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/3716771098 shutdown_connections 2024-08-20T02:59:33.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-08-20T02:59:33.934+0000 7f6f0ed2a640 1 -- 172.21.15.165:0/3716771098 wait complete. 2024-08-20T02:59:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:34 smithi165 ceph-mon[22357]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:59:34.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:34.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:59:34.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:34.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:34.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:34.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:34.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:34 smithi044 ceph-mon[18790]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:34.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:34.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:59:34.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:34.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:59:34.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:34.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:34.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:34.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:34.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:34 smithi099 ceph-mon[21984]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T02:59:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T02:59:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:34.707 DEBUG:teuthology.orchestra.run.smithi044:mds.a> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.a.service 2024-08-20T02:59:34.710 DEBUG:teuthology.orchestra.run.smithi044:mds.d> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.d.service 2024-08-20T02:59:34.755 DEBUG:teuthology.orchestra.run.smithi044:mds.g> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.g.service 2024-08-20T02:59:34.799 DEBUG:teuthology.orchestra.run.smithi044:mds.j> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.j.service 2024-08-20T02:59:34.842 DEBUG:teuthology.orchestra.run.smithi099:mds.b> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.b.service 2024-08-20T02:59:34.845 DEBUG:teuthology.orchestra.run.smithi099:mds.e> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.e.service 2024-08-20T02:59:34.888 DEBUG:teuthology.orchestra.run.smithi099:mds.h> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.h.service 2024-08-20T02:59:34.931 DEBUG:teuthology.orchestra.run.smithi099:mds.k> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.k.service 2024-08-20T02:59:34.974 DEBUG:teuthology.orchestra.run.smithi165:mds.c> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.c.service 2024-08-20T02:59:34.977 DEBUG:teuthology.orchestra.run.smithi165:mds.f> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.f.service 2024-08-20T02:59:35.021 DEBUG:teuthology.orchestra.run.smithi165:mds.i> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.i.service 2024-08-20T02:59:35.064 DEBUG:teuthology.orchestra.run.smithi165:mds.l> sudo journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.l.service 2024-08-20T02:59:35.108 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-08-20T02:59:35.109 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T02:59:35.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:35 smithi099 ceph-mon[21984]: from='client.24538 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:59:35.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:35 smithi099 ceph-mon[21984]: Saving service mds.all spec with placement smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-08-20T02:59:35.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:35 smithi099 ceph-mon[21984]: Deploying daemon mds.l on smithi165 2024-08-20T02:59:35.233 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:35 smithi165 ceph-mon[22357]: from='client.24538 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:59:35.233 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:35 smithi165 ceph-mon[22357]: Saving service mds.all spec with placement smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-08-20T02:59:35.233 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:35 smithi165 ceph-mon[22357]: Deploying daemon mds.l on smithi165 2024-08-20T02:59:35.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:35 smithi044 ceph-mon[18790]: from='client.24538 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T02:59:35.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:35 smithi044 ceph-mon[18790]: Saving service mds.all spec with placement smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-08-20T02:59:35.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:35 smithi044 ceph-mon[18790]: Deploying daemon mds.l on smithi165 2024-08-20T02:59:35.364 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:59:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.828+0000 7feacdc9f640 1 -- 172.21.15.44:0/3069788631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feac81074d0 msgr2=0x7feac810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.828+0000 7feacdc9f640 1 --2- 172.21.15.44:0/3069788631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feac81074d0 0x7feac810dcd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feac0009ca0 tx=0x7feac002a8c0 comp rx=0 tx=0).stop 2024-08-20T02:59:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.828+0000 7feacdc9f640 1 -- 172.21.15.44:0/3069788631 shutdown_connections 2024-08-20T02:59:35.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.828+0000 7feacdc9f640 1 --2- 172.21.15.44:0/3069788631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feac81074d0 0x7feac810dcd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.828+0000 7feacdc9f640 1 --2- 172.21.15.44:0/3069788631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feac8106b50 0x7feac8106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.828+0000 7feacdc9f640 1 --2- 172.21.15.44:0/3069788631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac8101520 0x7feac8101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.828+0000 7feacdc9f640 1 -- 172.21.15.44:0/3069788631 >> 172.21.15.44:0/3069788631 conn(0x7feac8076c80 msgr2=0x7feac8077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.828+0000 7feacdc9f640 1 -- 172.21.15.44:0/3069788631 shutdown_connections 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.828+0000 7feacdc9f640 1 -- 172.21.15.44:0/3069788631 wait complete. 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.828+0000 7feacdc9f640 1 Processor -- start 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feacdc9f640 1 -- start start 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feacdc9f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feac8101520 0x7feac8198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feacdc9f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feac8106b50 0x7feac81994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feacdc9f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac81074d0 0x7feac819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feacdc9f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feac81108d0 con 0x7feac8101520 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feacdc9f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feac8110750 con 0x7feac81074d0 2024-08-20T02:59:35.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feacdc9f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feac8110a50 con 0x7feac8106b50 2024-08-20T02:59:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feac77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feac8101520 0x7feac8198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feac77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feac8101520 0x7feac8198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53604/0 (socket says 172.21.15.44:53604) 2024-08-20T02:59:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feac77fe640 1 -- 172.21.15.44:0/2761119421 learned_addr learned my addr 172.21.15.44:0/2761119421 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:59:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feac6ffd640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feac8106b50 0x7feac81994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feac7fff640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac81074d0 0x7feac819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feac77fe640 1 -- 172.21.15.44:0/2761119421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feac8106b50 msgr2=0x7feac81994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feac77fe640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feac8106b50 0x7feac81994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feac77fe640 1 -- 172.21.15.44:0/2761119421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac81074d0 msgr2=0x7feac819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feac77fe640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac81074d0 0x7feac819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feac77fe640 1 -- 172.21.15.44:0/2761119421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac819f030 con 0x7feac8101520 2024-08-20T02:59:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feac77fe640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feac8101520 0x7feac8198f80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feab800d950 tx=0x7feab800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:35.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.829+0000 7feac4ff9640 1 -- 172.21.15.44:0/2761119421 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab8017070 con 0x7feac8101520 2024-08-20T02:59:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.830+0000 7feacdc9f640 1 -- 172.21.15.44:0/2761119421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac819f320 con 0x7feac8101520 2024-08-20T02:59:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.830+0000 7feac4ff9640 1 -- 172.21.15.44:0/2761119421 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feab8009c30 con 0x7feac8101520 2024-08-20T02:59:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.830+0000 7feac4ff9640 1 -- 172.21.15.44:0/2761119421 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab800f620 con 0x7feac8101520 2024-08-20T02:59:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.830+0000 7feacdc9f640 1 -- 172.21.15.44:0/2761119421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac81a59e0 con 0x7feac8101520 2024-08-20T02:59:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.831+0000 7feacdc9f640 1 -- 172.21.15.44:0/2761119421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feac8101900 con 0x7feac8101520 2024-08-20T02:59:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.832+0000 7feac4ff9640 1 -- 172.21.15.44:0/2761119421 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7feab8010460 con 0x7feac8101520 2024-08-20T02:59:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.833+0000 7feac4ff9640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feab00b3c20 0x7feab00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.833+0000 7feac6ffd640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feab00b3c20 0x7feab00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.833+0000 7feac4ff9640 1 -- 172.21.15.44:0/2761119421 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7feab80e3560 con 0x7feac8101520 2024-08-20T02:59:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.833+0000 7feac6ffd640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feab00b3c20 0x7feab00b60e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feac8103300 tx=0x7feabc010040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:35.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:35.837+0000 7feac4ff9640 1 -- 172.21.15.44:0/2761119421 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7feab80aaee0 con 0x7feac8101520 2024-08-20T02:59:36.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.060+0000 7feacdc9f640 1 -- 172.21.15.44:0/2761119421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feac802e340 con 0x7feac8101520 2024-08-20T02:59:36.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.060+0000 7feac4ff9640 1 -- 172.21.15.44:0/2761119421 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7feab8009de0 con 0x7feac8101520 2024-08-20T02:59:36.063 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:59:36.064 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":1,"btime":"2024-08-20T02:52:10:979066+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-08-20T02:59:36.064 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 1 2024-08-20T02:59:36.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.062+0000 7feacdc9f640 1 -- 172.21.15.44:0/2761119421 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feab00b3c20 msgr2=0x7feab00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:36.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.062+0000 7feacdc9f640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feab00b3c20 0x7feab00b60e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feac8103300 tx=0x7feabc010040 comp rx=0 tx=0).stop 2024-08-20T02:59:36.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.062+0000 7feacdc9f640 1 -- 172.21.15.44:0/2761119421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feac8101520 msgr2=0x7feac8198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:36.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.062+0000 7feacdc9f640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feac8101520 0x7feac8198f80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feab800d950 tx=0x7feab800de10 comp rx=0 tx=0).stop 2024-08-20T02:59:36.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.062+0000 7feacdc9f640 1 -- 172.21.15.44:0/2761119421 shutdown_connections 2024-08-20T02:59:36.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.062+0000 7feacdc9f640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feab00b3c20 0x7feab00b60e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:36.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.062+0000 7feacdc9f640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feac81074d0 0x7feac819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:36.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.062+0000 7feacdc9f640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feac8106b50 0x7feac81994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:36.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.062+0000 7feacdc9f640 1 --2- 172.21.15.44:0/2761119421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feac8101520 0x7feac8198f80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:36.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.062+0000 7feacdc9f640 1 -- 172.21.15.44:0/2761119421 >> 172.21.15.44:0/2761119421 conn(0x7feac8076c80 msgr2=0x7feac80ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:36.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.062+0000 7feacdc9f640 1 -- 172.21.15.44:0/2761119421 shutdown_connections 2024-08-20T02:59:36.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:36.062+0000 7feacdc9f640 1 -- 172.21.15.44:0/2761119421 wait complete. 2024-08-20T02:59:36.186 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:36 smithi165 ceph-mon[22357]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:36.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:36 smithi044 ceph-mon[18790]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:36 smithi099 ceph-mon[21984]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:36.582 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 02:59:36 smithi165 systemd[1]: Starting Ceph mds.l for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T02:59:36.840 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 02:59:36 smithi165 podman[53531]: 2024-08-20 02:59:36.482712111 +0000 UTC m=+0.015911924 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T02:59:36.840 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 02:59:36 smithi165 podman[53531]: 2024-08-20 02:59:36.60036119 +0000 UTC m=+0.133560998 container create 99599d5447be1c8f27eafb400bc7a50057dd3a2537cc9b0110059976645c1a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-20T02:59:36.841 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 02:59:36 smithi165 podman[53531]: 2024-08-20 02:59:36.838998849 +0000 UTC m=+0.372198656 container init 99599d5447be1c8f27eafb400bc7a50057dd3a2537cc9b0110059976645c1a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T02:59:37.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 02:59:36 smithi165 podman[53531]: 2024-08-20 02:59:36.841709429 +0000 UTC m=+0.374909236 container start 99599d5447be1c8f27eafb400bc7a50057dd3a2537cc9b0110059976645c1a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T02:59:37.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 02:59:36 smithi165 ceph-mds[53550]: -- 172.21.15.165:0/1910239955 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ce9c639860 con 0x55ce9c617000 2024-08-20T02:59:37.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 02:59:36 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l[53546]: starting mds.l at 2024-08-20T02:59:37.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 02:59:36 smithi165 bash[53531]: 99599d5447be1c8f27eafb400bc7a50057dd3a2537cc9b0110059976645c1a86 2024-08-20T02:59:37.137 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 02:59:36 smithi165 systemd[1]: Started Ceph mds.l for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:59:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:37 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2761119421' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:37 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.165:6832/1910345968,v1:172.21.15.165:6833/1910345968] up:boot 2024-08-20T02:59:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:37 smithi165 ceph-mon[22357]: fsmap 1 up:standby 2024-08-20T02:59:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-20T02:59:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:37.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:37.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:37.138 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:37 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2761119421' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:37 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.165:6832/1910345968,v1:172.21.15.165:6833/1910345968] up:boot 2024-08-20T02:59:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:37 smithi044 ceph-mon[18790]: fsmap 1 up:standby 2024-08-20T02:59:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-20T02:59:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:37.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:37.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:37.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:37 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2761119421' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:37.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:37 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.165:6832/1910345968,v1:172.21.15.165:6833/1910345968] up:boot 2024-08-20T02:59:37.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:37 smithi099 ceph-mon[21984]: fsmap 1 up:standby 2024-08-20T02:59:37.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-20T02:59:37.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:37.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:37.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:37.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:37.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:37.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:38.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:38 smithi044 ceph-mon[18790]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:38.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:38 smithi044 ceph-mon[18790]: Deploying daemon mds.k on smithi099 2024-08-20T02:59:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:38 smithi165 ceph-mon[22357]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:38 smithi165 ceph-mon[22357]: Deploying daemon mds.k on smithi099 2024-08-20T02:59:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:38 smithi099 ceph-mon[21984]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:38.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:38 smithi099 ceph-mon[21984]: Deploying daemon mds.k on smithi099 2024-08-20T02:59:38.858 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T02:59:39.124 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:59:39.165 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 02:59:38 smithi099 systemd[1]: Starting Ceph mds.k for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T02:59:39.428 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 02:59:39 smithi099 podman[52818]: 2024-08-20 02:59:39.074550127 +0000 UTC m=+0.015409998 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T02:59:39.428 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 02:59:39 smithi099 podman[52818]: 2024-08-20 02:59:39.182523556 +0000 UTC m=+0.123383460 container create b692a8f7e019459dbba23929498832959f7efdb2dbb88b448c2d8663c4019953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-08-20T02:59:39.428 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 02:59:39 smithi099 podman[52818]: 2024-08-20 02:59:39.426909386 +0000 UTC m=+0.367769244 container init b692a8f7e019459dbba23929498832959f7efdb2dbb88b448c2d8663c4019953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-08-20T02:59:39.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.598+0000 7f51e2db7640 1 -- 172.21.15.44:0/4038224932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51dc104250 msgr2=0x7f51dc10c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:39.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.598+0000 7f51e2db7640 1 --2- 172.21.15.44:0/4038224932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51dc104250 0x7f51dc10c830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009900 tx=0x7f51c802a410 comp rx=0 tx=0).stop 2024-08-20T02:59:39.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.598+0000 7f51e2db7640 1 -- 172.21.15.44:0/4038224932 shutdown_connections 2024-08-20T02:59:39.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.598+0000 7f51e2db7640 1 --2- 172.21.15.44:0/4038224932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51dc10ce00 0x7f51dc10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.598+0000 7f51e2db7640 1 --2- 172.21.15.44:0/4038224932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51dc104250 0x7f51dc10c830 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.598+0000 7f51e2db7640 1 --2- 172.21.15.44:0/4038224932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51dc103850 0x7f51dc103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.598+0000 7f51e2db7640 1 -- 172.21.15.44:0/4038224932 >> 172.21.15.44:0/4038224932 conn(0x7f51dc0fd400 msgr2=0x7f51dc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.598+0000 7f51e2db7640 1 -- 172.21.15.44:0/4038224932 shutdown_connections 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.598+0000 7f51e2db7640 1 -- 172.21.15.44:0/4038224932 wait complete. 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e2db7640 1 Processor -- start 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e2db7640 1 -- start start 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e2db7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51dc103850 0x7f51dc198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e2db7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51dc104250 0x7f51dc199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e2db7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51dc10ce00 0x7f51dc19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e2db7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc112a60 con 0x7f51dc103850 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e2db7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc1128e0 con 0x7f51dc10ce00 2024-08-20T02:59:39.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e2db7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc112be0 con 0x7f51dc104250 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e0b53640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51dc103850 0x7f51dc198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e0b53640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51dc103850 0x7f51dc198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46690/0 (socket says 172.21.15.44:46690) 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e0b53640 1 -- 172.21.15.44:0/3233672966 learned_addr learned my addr 172.21.15.44:0/3233672966 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e1354640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51dc10ce00 0x7f51dc19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51dbfff640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51dc104250 0x7f51dc199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e0b53640 1 -- 172.21.15.44:0/3233672966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51dc104250 msgr2=0x7f51dc199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.599+0000 7f51e0b53640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51dc104250 0x7f51dc199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.600+0000 7f51e0b53640 1 -- 172.21.15.44:0/3233672966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51dc10ce00 msgr2=0x7f51dc19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.600+0000 7f51e0b53640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51dc10ce00 0x7f51dc19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.600+0000 7f51e0b53640 1 -- 172.21.15.44:0/3233672966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc19f000 con 0x7f51dc103850 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.600+0000 7f51dbfff640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51dc104250 0x7f51dc199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.600+0000 7f51e0b53640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51dc103850 0x7f51dc198f50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f51d000e9f0 tx=0x7f51d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:39.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.600+0000 7f51d9ffb640 1 -- 172.21.15.44:0/3233672966 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d000cd80 con 0x7f51dc103850 2024-08-20T02:59:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.600+0000 7f51d9ffb640 1 -- 172.21.15.44:0/3233672966 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51d00108e0 con 0x7f51dc103850 2024-08-20T02:59:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.600+0000 7f51d9ffb640 1 -- 172.21.15.44:0/3233672966 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d0018980 con 0x7f51dc103850 2024-08-20T02:59:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.600+0000 7f51e2db7640 1 -- 172.21.15.44:0/3233672966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51dc19f2f0 con 0x7f51dc103850 2024-08-20T02:59:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.600+0000 7f51e2db7640 1 -- 172.21.15.44:0/3233672966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51dc1a59b0 con 0x7f51dc103850 2024-08-20T02:59:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.601+0000 7f51e2db7640 1 -- 172.21.15.44:0/3233672966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51dc103c30 con 0x7f51dc103850 2024-08-20T02:59:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.603+0000 7f51d9ffb640 1 -- 172.21.15.44:0/3233672966 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f51d0010430 con 0x7f51dc103850 2024-08-20T02:59:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.604+0000 7f51d9ffb640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f51cc0b3c20 0x7f51cc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.604+0000 7f51dbfff640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f51cc0b3c20 0x7f51cc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.604+0000 7f51d9ffb640 1 -- 172.21.15.44:0/3233672966 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f51d00e4110 con 0x7f51dc103850 2024-08-20T02:59:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.604+0000 7f51dbfff640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f51cc0b3c20 0x7f51cc0b60e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f51c8031440 tx=0x7f51c8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:39.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.607+0000 7f51d9ffb640 1 -- 172.21.15.44:0/3233672966 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f51d00abb30 con 0x7f51dc103850 2024-08-20T02:59:39.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.830+0000 7f51e2db7640 1 -- 172.21.15.44:0/3233672966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51dc02e340 con 0x7f51dc103850 2024-08-20T02:59:39.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.830+0000 7f51d9ffb640 1 -- 172.21.15.44:0/3233672966 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f51d00ab430 con 0x7f51dc103850 2024-08-20T02:59:39.833 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:59:39.833 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":2,"btime":"2024-08-20T02:59:36:870016+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24550,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-20T02:59:39.833 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 2 2024-08-20T02:59:39.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.832+0000 7f51e2db7640 1 -- 172.21.15.44:0/3233672966 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f51cc0b3c20 msgr2=0x7f51cc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:39.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.832+0000 7f51e2db7640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f51cc0b3c20 0x7f51cc0b60e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f51c8031440 tx=0x7f51c8005a20 comp rx=0 tx=0).stop 2024-08-20T02:59:39.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.832+0000 7f51e2db7640 1 -- 172.21.15.44:0/3233672966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51dc103850 msgr2=0x7f51dc198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:39.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.832+0000 7f51e2db7640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51dc103850 0x7f51dc198f50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f51d000e9f0 tx=0x7f51d000eeb0 comp rx=0 tx=0).stop 2024-08-20T02:59:39.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.833+0000 7f51e2db7640 1 -- 172.21.15.44:0/3233672966 shutdown_connections 2024-08-20T02:59:39.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.833+0000 7f51e2db7640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f51cc0b3c20 0x7f51cc0b60e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:39.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.833+0000 7f51e2db7640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51dc10ce00 0x7f51dc19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:39.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.833+0000 7f51e2db7640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51dc104250 0x7f51dc199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:39.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.833+0000 7f51e2db7640 1 --2- 172.21.15.44:0/3233672966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51dc103850 0x7f51dc198f50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:39.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.833+0000 7f51e2db7640 1 -- 172.21.15.44:0/3233672966 >> 172.21.15.44:0/3233672966 conn(0x7f51dc0fd400 msgr2=0x7f51dc0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:39.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.833+0000 7f51e2db7640 1 -- 172.21.15.44:0/3233672966 shutdown_connections 2024-08-20T02:59:39.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:39.833+0000 7f51e2db7640 1 -- 172.21.15.44:0/3233672966 wait complete. 2024-08-20T02:59:39.915 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 02:59:39 smithi099 podman[52818]: 2024-08-20 02:59:39.429900275 +0000 UTC m=+0.370760134 container start b692a8f7e019459dbba23929498832959f7efdb2dbb88b448c2d8663c4019953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T02:59:39.915 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 02:59:39 smithi099 ceph-mds[52837]: -- 172.21.15.99:0/360284 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565213583860 con 0x565213561000 2024-08-20T02:59:39.915 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 02:59:39 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-k[52833]: starting mds.k at 2024-08-20T02:59:39.915 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 02:59:39 smithi099 bash[52818]: b692a8f7e019459dbba23929498832959f7efdb2dbb88b448c2d8663c4019953 2024-08-20T02:59:39.915 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 02:59:39 smithi099 systemd[1]: Started Ceph mds.k for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:59:40.275 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:40 smithi044 ceph-mon[18790]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:40.275 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:40.276 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:40.276 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:40.276 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:40.276 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:40.276 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:40.276 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:40 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3233672966' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:40 smithi165 ceph-mon[22357]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:40.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:40 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3233672966' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:40.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:40 smithi099 ceph-mon[21984]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:40.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:40.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:40.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:40.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:40.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:40.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:40.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:40 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3233672966' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:41.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:41 smithi099 ceph-mon[21984]: Deploying daemon mds.j on smithi044 2024-08-20T02:59:41.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:41 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.99:6832/2092228440,v1:172.21.15.99:6833/2092228440] up:boot 2024-08-20T02:59:41.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:41 smithi099 ceph-mon[21984]: fsmap 2 up:standby 2024-08-20T02:59:41.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:41 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-20T02:59:41.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:41 smithi044 ceph-mon[18790]: Deploying daemon mds.j on smithi044 2024-08-20T02:59:41.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:41 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.99:6832/2092228440,v1:172.21.15.99:6833/2092228440] up:boot 2024-08-20T02:59:41.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:41 smithi044 ceph-mon[18790]: fsmap 2 up:standby 2024-08-20T02:59:41.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:41 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-20T02:59:41.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:41 smithi165 ceph-mon[22357]: Deploying daemon mds.j on smithi044 2024-08-20T02:59:41.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:41 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.99:6832/2092228440,v1:172.21.15.99:6833/2092228440] up:boot 2024-08-20T02:59:41.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:41 smithi165 ceph-mon[22357]: fsmap 2 up:standby 2024-08-20T02:59:41.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:41 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-20T02:59:42.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:42 smithi044 ceph-mon[18790]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:42.286 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 02:59:42 smithi044 systemd[1]: Starting Ceph mds.j for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T02:59:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:42 smithi165 ceph-mon[22357]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:42.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:42 smithi099 ceph-mon[21984]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:42.466 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T02:59:42.681 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 02:59:42 smithi044 podman[62260]: 2024-08-20 02:59:42.318820402 +0000 UTC m=+0.016293666 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T02:59:42.682 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 02:59:42 smithi044 podman[62260]: 2024-08-20 02:59:42.432264624 +0000 UTC m=+0.129737877 container create 48ce209b7fab2e767801d5befff2c5f7fbdf0f4a56f46efae923003a6d1f49ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-j, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-08-20T02:59:42.682 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 02:59:42 smithi044 podman[62260]: 2024-08-20 02:59:42.651913215 +0000 UTC m=+0.349386463 container init 48ce209b7fab2e767801d5befff2c5f7fbdf0f4a56f46efae923003a6d1f49ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-08-20T02:59:42.682 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 02:59:42 smithi044 podman[62260]: 2024-08-20 02:59:42.655902694 +0000 UTC m=+0.353375951 container start 48ce209b7fab2e767801d5befff2c5f7fbdf0f4a56f46efae923003a6d1f49ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-20T02:59:42.834 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:59:42.955 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 02:59:42 smithi044 ceph-mds[62310]: -- 172.21.15.44:0/663694941 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c30e1d860 con 0x557c30dfa800 2024-08-20T02:59:42.955 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 02:59:42 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-j[62294]: starting mds.j at 2024-08-20T02:59:42.955 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 02:59:42 smithi044 bash[62260]: 48ce209b7fab2e767801d5befff2c5f7fbdf0f4a56f46efae923003a6d1f49ea 2024-08-20T02:59:42.955 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 02:59:42 smithi044 systemd[1]: Started Ceph mds.j for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:59:43.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.329+0000 7ff2daa89640 1 -- 172.21.15.44:0/4214091706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d4108e30 msgr2=0x7ff2d410f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:43.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.329+0000 7ff2daa89640 1 --2- 172.21.15.44:0/4214091706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d4108e30 0x7ff2d410f630 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009ca0 tx=0x7ff2cc02a8c0 comp rx=0 tx=0).stop 2024-08-20T02:59:43.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.329+0000 7ff2daa89640 1 -- 172.21.15.44:0/4214091706 shutdown_connections 2024-08-20T02:59:43.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.329+0000 7ff2daa89640 1 --2- 172.21.15.44:0/4214091706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d4108e30 0x7ff2d410f630 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:43.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.329+0000 7ff2daa89640 1 --2- 172.21.15.44:0/4214091706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2d41084b0 0x7ff2d41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.329+0000 7ff2daa89640 1 --2- 172.21.15.44:0/4214091706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2d4102aa0 0x7ff2d4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.329+0000 7ff2daa89640 1 -- 172.21.15.44:0/4214091706 >> 172.21.15.44:0/4214091706 conn(0x7ff2d40fe550 msgr2=0x7ff2d4100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.329+0000 7ff2daa89640 1 -- 172.21.15.44:0/4214091706 shutdown_connections 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.329+0000 7ff2daa89640 1 -- 172.21.15.44:0/4214091706 wait complete. 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.329+0000 7ff2daa89640 1 Processor -- start 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2daa89640 1 -- start start 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2daa89640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2d4102aa0 0x7ff2d40751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2daa89640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2d41084b0 0x7ff2d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2daa89640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d4108e30 0x7ff2d4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2daa89640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4112af0 con 0x7ff2d4108e30 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2daa89640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4112970 con 0x7ff2d4102aa0 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2daa89640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4112c70 con 0x7ff2d41084b0 2024-08-20T02:59:43.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2d9026640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d4108e30 0x7ff2d4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:43.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2d9026640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d4108e30 0x7ff2d4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46738/0 (socket says 172.21.15.44:46738) 2024-08-20T02:59:43.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2d9026640 1 -- 172.21.15.44:0/10160331 learned_addr learned my addr 172.21.15.44:0/10160331 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:59:43.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2d8825640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2d4102aa0 0x7ff2d40751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:43.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2d3fff640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2d41084b0 0x7ff2d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:43.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2d9026640 1 -- 172.21.15.44:0/10160331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2d41084b0 msgr2=0x7ff2d4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:43.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2d9026640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2d41084b0 0x7ff2d4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:43.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2d9026640 1 -- 172.21.15.44:0/10160331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2d4102aa0 msgr2=0x7ff2d40751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:43.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2d9026640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2d4102aa0 0x7ff2d40751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:43.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2d9026640 1 -- 172.21.15.44:0/10160331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d41a7440 con 0x7ff2d4108e30 2024-08-20T02:59:43.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2d3fff640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2d41084b0 0x7ff2d4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:59:43.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.330+0000 7ff2d9026640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d4108e30 0x7ff2d4075c50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc005cc0 tx=0x7ff2cc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:43.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.331+0000 7ff2d1ffb640 1 -- 172.21.15.44:0/10160331 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc003bf0 con 0x7ff2d4108e30 2024-08-20T02:59:43.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.331+0000 7ff2d1ffb640 1 -- 172.21.15.44:0/10160331 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2cc003d90 con 0x7ff2d4108e30 2024-08-20T02:59:43.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.331+0000 7ff2daa89640 1 -- 172.21.15.44:0/10160331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d41a76d0 con 0x7ff2d4108e30 2024-08-20T02:59:43.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.331+0000 7ff2d1ffb640 1 -- 172.21.15.44:0/10160331 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc02fbc0 con 0x7ff2d4108e30 2024-08-20T02:59:43.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.331+0000 7ff2daa89640 1 -- 172.21.15.44:0/10160331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d41a7c10 con 0x7ff2d4108e30 2024-08-20T02:59:43.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.332+0000 7ff2daa89640 1 -- 172.21.15.44:0/10160331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d4102e80 con 0x7ff2d4108e30 2024-08-20T02:59:43.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.333+0000 7ff2d1ffb640 1 -- 172.21.15.44:0/10160331 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7ff2cc02fd60 con 0x7ff2d4108e30 2024-08-20T02:59:43.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.334+0000 7ff2d1ffb640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2c40b3c20 0x7ff2c40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:43.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.334+0000 7ff2d8825640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2c40b3c20 0x7ff2c40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:43.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.334+0000 7ff2d1ffb640 1 -- 172.21.15.44:0/10160331 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7ff2cc105120 con 0x7ff2d4108e30 2024-08-20T02:59:43.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.335+0000 7ff2d8825640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2c40b3c20 0x7ff2c40b60e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800bbe0 tx=0x7ff2c800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:43.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.337+0000 7ff2d1ffb640 1 -- 172.21.15.44:0/10160331 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff2cc0ccb40 con 0x7ff2d4108e30 2024-08-20T02:59:43.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.565+0000 7ff2daa89640 1 -- 172.21.15.44:0/10160331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2d40764d0 con 0x7ff2d4108e30 2024-08-20T02:59:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.565+0000 7ff2d1ffb640 1 -- 172.21.15.44:0/10160331 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7ff2d40764d0 con 0x7ff2d4108e30 2024-08-20T02:59:43.568 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:59:43.568 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":4,"btime":"2024-08-20T02:59:43:047458+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24550,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-20T02:59:43.568 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 4 2024-08-20T02:59:43.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.568+0000 7ff2daa89640 1 -- 172.21.15.44:0/10160331 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2c40b3c20 msgr2=0x7ff2c40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:43.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.568+0000 7ff2daa89640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2c40b3c20 0x7ff2c40b60e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800bbe0 tx=0x7ff2c800b390 comp rx=0 tx=0).stop 2024-08-20T02:59:43.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.568+0000 7ff2daa89640 1 -- 172.21.15.44:0/10160331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d4108e30 msgr2=0x7ff2d4075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:43.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.568+0000 7ff2daa89640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d4108e30 0x7ff2d4075c50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc005cc0 tx=0x7ff2cc02f380 comp rx=0 tx=0).stop 2024-08-20T02:59:43.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.568+0000 7ff2daa89640 1 -- 172.21.15.44:0/10160331 shutdown_connections 2024-08-20T02:59:43.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.568+0000 7ff2daa89640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2c40b3c20 0x7ff2c40b60e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:43.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.568+0000 7ff2daa89640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2d4108e30 0x7ff2d4075c50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:43.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.568+0000 7ff2daa89640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2d41084b0 0x7ff2d4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:43.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.568+0000 7ff2daa89640 1 --2- 172.21.15.44:0/10160331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2d4102aa0 0x7ff2d40751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:43.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.568+0000 7ff2daa89640 1 -- 172.21.15.44:0/10160331 >> 172.21.15.44:0/10160331 conn(0x7ff2d40fe550 msgr2=0x7ff2d4100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:43.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.568+0000 7ff2daa89640 1 -- 172.21.15.44:0/10160331 shutdown_connections 2024-08-20T02:59:43.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:43.568+0000 7ff2daa89640 1 -- 172.21.15.44:0/10160331 wait complete. 2024-08-20T02:59:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:43 smithi165 ceph-mon[22357]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:43 smithi165 ceph-mon[22357]: Deploying daemon mds.d on smithi044 2024-08-20T02:59:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:43 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.44:6834/1875462475,v1:172.21.15.44:6835/1875462475] up:boot 2024-08-20T02:59:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:43 smithi165 ceph-mon[22357]: fsmap 3 up:standby 2024-08-20T02:59:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-20T02:59:44.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:43 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/10160331' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:44.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:43 smithi099 ceph-mon[21984]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:44.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:43 smithi099 ceph-mon[21984]: Deploying daemon mds.d on smithi044 2024-08-20T02:59:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:43 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.44:6834/1875462475,v1:172.21.15.44:6835/1875462475] up:boot 2024-08-20T02:59:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:43 smithi099 ceph-mon[21984]: fsmap 3 up:standby 2024-08-20T02:59:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-20T02:59:44.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:43 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/10160331' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:44.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:43 smithi044 ceph-mon[18790]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:44.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:44.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:44.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:44.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:44.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:44.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:44.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:43 smithi044 ceph-mon[18790]: Deploying daemon mds.d on smithi044 2024-08-20T02:59:44.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:43 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.44:6834/1875462475,v1:172.21.15.44:6835/1875462475] up:boot 2024-08-20T02:59:44.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:43 smithi044 ceph-mon[18790]: fsmap 3 up:standby 2024-08-20T02:59:44.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-20T02:59:44.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:43 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/10160331' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:45.988 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:45 smithi044 ceph-mon[18790]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:46.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:45 smithi165 ceph-mon[22357]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:46.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:45 smithi099 ceph-mon[21984]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:46.364 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T02:59:46.623 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:59:47.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.070+0000 7feb8a433640 1 -- 172.21.15.44:0/3567536059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841039e0 msgr2=0x7feb84103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:47.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.070+0000 7feb8a433640 1 --2- 172.21.15.44:0/3567536059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841039e0 0x7feb84103dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb74009900 tx=0x7feb7402a410 comp rx=0 tx=0).stop 2024-08-20T02:59:47.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.070+0000 7feb8a433640 1 -- 172.21.15.44:0/3567536059 shutdown_connections 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.070+0000 7feb8a433640 1 --2- 172.21.15.44:0/3567536059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb8410ce40 0x7feb8410f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.070+0000 7feb8a433640 1 --2- 172.21.15.44:0/3567536059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb84104300 0x7feb8410c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.070+0000 7feb8a433640 1 --2- 172.21.15.44:0/3567536059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841039e0 0x7feb84103dc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.070+0000 7feb8a433640 1 -- 172.21.15.44:0/3567536059 >> 172.21.15.44:0/3567536059 conn(0x7feb840fd440 msgr2=0x7feb840ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.071+0000 7feb8a433640 1 -- 172.21.15.44:0/3567536059 shutdown_connections 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.071+0000 7feb8a433640 1 -- 172.21.15.44:0/3567536059 wait complete. 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.071+0000 7feb8a433640 1 Processor -- start 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.071+0000 7feb8a433640 1 -- start start 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.071+0000 7feb8a433640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841039e0 0x7feb84198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.071+0000 7feb8a433640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84104300 0x7feb84199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.071+0000 7feb8a433640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb8410ce40 0x7feb8419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.071+0000 7feb8a433640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feb84112ab0 con 0x7feb841039e0 2024-08-20T02:59:47.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.071+0000 7feb8a433640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb84112930 con 0x7feb84104300 2024-08-20T02:59:47.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.071+0000 7feb8a433640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7feb84112c30 con 0x7feb8410ce40 2024-08-20T02:59:47.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb83fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841039e0 0x7feb84198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:47.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb83fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841039e0 0x7feb84198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46784/0 (socket says 172.21.15.44:46784) 2024-08-20T02:59:47.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb83fff640 1 -- 172.21.15.44:0/29757294 learned_addr learned my addr 172.21.15.44:0/29757294 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:59:47.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb837fe640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84104300 0x7feb84199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:47.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb889d0640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb8410ce40 0x7feb8419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:47.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb83fff640 1 -- 172.21.15.44:0/29757294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb8410ce40 msgr2=0x7feb8419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:47.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb83fff640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb8410ce40 0x7feb8419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:47.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb83fff640 1 -- 172.21.15.44:0/29757294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84104300 msgr2=0x7feb84199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:47.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb83fff640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84104300 0x7feb84199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:47.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb83fff640 1 -- 172.21.15.44:0/29757294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8419f070 con 0x7feb841039e0 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb83fff640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841039e0 0x7feb84198fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feb74005940 tx=0x7feb74004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb817fa640 1 -- 172.21.15.44:0/29757294 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7403f030 con 0x7feb841039e0 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb8a433640 1 -- 172.21.15.44:0/29757294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8419f300 con 0x7feb841039e0 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb817fa640 1 -- 172.21.15.44:0/29757294 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb74003aa0 con 0x7feb841039e0 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb817fa640 1 -- 172.21.15.44:0/29757294 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb74035070 con 0x7feb841039e0 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.072+0000 7feb8a433640 1 -- 172.21.15.44:0/29757294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb841a5a20 con 0x7feb841039e0 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.073+0000 7feb8a433640 1 -- 172.21.15.44:0/29757294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb84103dc0 con 0x7feb841039e0 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.076+0000 7feb817fa640 1 -- 172.21.15.44:0/29757294 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7feb74042030 con 0x7feb841039e0 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.076+0000 7feb817fa640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feb6c0b3c20 0x7feb6c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.076+0000 7feb837fe640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feb6c0b3c20 0x7feb6c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.076+0000 7feb817fa640 1 -- 172.21.15.44:0/29757294 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7feb74104220 con 0x7feb841039e0 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.077+0000 7feb837fe640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feb6c0b3c20 0x7feb6c0b60e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb8410f4e0 tx=0x7feb78005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:47.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.079+0000 7feb817fa640 1 -- 172.21.15.44:0/29757294 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7feb740cb550 con 0x7feb841039e0 2024-08-20T02:59:47.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.299+0000 7feb8a433640 1 -- 172.21.15.44:0/29757294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb841a5c80 con 0x7feb841039e0 2024-08-20T02:59:47.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.300+0000 7feb817fa640 1 -- 172.21.15.44:0/29757294 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7feb740d2e20 con 0x7feb841039e0 2024-08-20T02:59:47.302 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:59:47.302 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":5,"btime":"2024-08-20T02:59:46:811810+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24550,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-20T02:59:47.303 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 5 2024-08-20T02:59:47.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.302+0000 7feb8a433640 1 -- 172.21.15.44:0/29757294 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feb6c0b3c20 msgr2=0x7feb6c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:47.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.302+0000 7feb8a433640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feb6c0b3c20 0x7feb6c0b60e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb8410f4e0 tx=0x7feb78005f90 comp rx=0 tx=0).stop 2024-08-20T02:59:47.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.302+0000 7feb8a433640 1 -- 172.21.15.44:0/29757294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841039e0 msgr2=0x7feb84198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:47.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.302+0000 7feb8a433640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841039e0 0x7feb84198fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feb74005940 tx=0x7feb74004270 comp rx=0 tx=0).stop 2024-08-20T02:59:47.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.302+0000 7feb8a433640 1 -- 172.21.15.44:0/29757294 shutdown_connections 2024-08-20T02:59:47.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.302+0000 7feb8a433640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feb6c0b3c20 0x7feb6c0b60e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:47.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.302+0000 7feb8a433640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb8410ce40 0x7feb8419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:47.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.302+0000 7feb8a433640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb84104300 0x7feb84199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:47.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.302+0000 7feb8a433640 1 --2- 172.21.15.44:0/29757294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feb841039e0 0x7feb84198fc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:47.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.302+0000 7feb8a433640 1 -- 172.21.15.44:0/29757294 >> 172.21.15.44:0/29757294 conn(0x7feb840fd440 msgr2=0x7feb840fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:47.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.302+0000 7feb8a433640 1 -- 172.21.15.44:0/29757294 shutdown_connections 2024-08-20T02:59:47.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:47.302+0000 7feb8a433640 1 -- 172.21.15.44:0/29757294 wait complete. 2024-08-20T02:59:47.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:47 smithi165 ceph-mon[22357]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:47.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:47.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:47.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:47 smithi165 ceph-mon[22357]: Deploying daemon mds.e on smithi099 2024-08-20T02:59:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:47 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.44:6836/3289736917,v1:172.21.15.44:6837/3289736917] up:boot 2024-08-20T02:59:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:47 smithi165 ceph-mon[22357]: fsmap 4 up:standby 2024-08-20T02:59:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:47 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-20T02:59:47.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:47 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/29757294' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:47.653 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:47 smithi044 ceph-mon[18790]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:47.653 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:47.653 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:47.654 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:47.654 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:47.654 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:47.654 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:47.654 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:47 smithi044 ceph-mon[18790]: Deploying daemon mds.e on smithi099 2024-08-20T02:59:47.654 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:47 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.44:6836/3289736917,v1:172.21.15.44:6837/3289736917] up:boot 2024-08-20T02:59:47.654 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:47 smithi044 ceph-mon[18790]: fsmap 4 up:standby 2024-08-20T02:59:47.654 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:47 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-20T02:59:47.654 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:47 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/29757294' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:47 smithi099 ceph-mon[21984]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:47 smithi099 ceph-mon[21984]: Deploying daemon mds.e on smithi099 2024-08-20T02:59:47.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:47 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.44:6836/3289736917,v1:172.21.15.44:6837/3289736917] up:boot 2024-08-20T02:59:47.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:47 smithi099 ceph-mon[21984]: fsmap 4 up:standby 2024-08-20T02:59:47.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:47 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-20T02:59:47.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:47 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/29757294' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:48.665 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 02:59:48 smithi099 podman[53132]: 2024-08-20 02:59:48.571728264 +0000 UTC m=+0.134532552 container create 595d45342c7041e680e1fbc231c6a59d95ead5d834489d2ec8f5389312181e41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-20T02:59:49.165 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 02:59:48 smithi099 podman[53132]: 2024-08-20 02:59:48.816284378 +0000 UTC m=+0.379084391 container init 595d45342c7041e680e1fbc231c6a59d95ead5d834489d2ec8f5389312181e41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-08-20T02:59:49.165 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 02:59:48 smithi099 podman[53132]: 2024-08-20 02:59:48.819065867 +0000 UTC m=+0.381865878 container start 595d45342c7041e680e1fbc231c6a59d95ead5d834489d2ec8f5389312181e41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-08-20T02:59:49.165 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 02:59:48 smithi099 ceph-mds[53151]: -- 172.21.15.99:0/4145251090 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56041a4a1520 con 0x56041a474800 2024-08-20T02:59:49.165 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 02:59:48 smithi099 ceph-mds[53151]: -- 172.21.15.99:0/4145251090 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56041a497860 con 0x56041a474800 2024-08-20T02:59:49.165 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 02:59:48 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-e[53147]: starting mds.e at 2024-08-20T02:59:49.165 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 02:59:48 smithi099 bash[53132]: 595d45342c7041e680e1fbc231c6a59d95ead5d834489d2ec8f5389312181e41 2024-08-20T02:59:49.165 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 02:59:48 smithi099 systemd[1]: Started Ceph mds.e for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:59:50.106 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T02:59:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:49 smithi044 ceph-mon[18790]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:49 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:49 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:49 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:49 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:49 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:50.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:49 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:50.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:49 smithi044 ceph-mon[18790]: Deploying daemon mds.f on smithi165 2024-08-20T02:59:50.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:49 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.99:6834/797787628,v1:172.21.15.99:6835/797787628] up:boot 2024-08-20T02:59:50.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:49 smithi044 ceph-mon[18790]: fsmap 5 up:standby 2024-08-20T02:59:50.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:49 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T02:59:50.372 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:59:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:49 smithi165 ceph-mon[22357]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:49 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:49 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:49 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:49 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:49 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:49 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:49 smithi165 ceph-mon[22357]: Deploying daemon mds.f on smithi165 2024-08-20T02:59:50.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:49 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.99:6834/797787628,v1:172.21.15.99:6835/797787628] up:boot 2024-08-20T02:59:50.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:49 smithi165 ceph-mon[22357]: fsmap 5 up:standby 2024-08-20T02:59:50.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:49 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T02:59:50.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:49 smithi099 ceph-mon[21984]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:50.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:49 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:50.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:49 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:50.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:49 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:50.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:49 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:50.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:49 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:50.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:49 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:50.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:49 smithi099 ceph-mon[21984]: Deploying daemon mds.f on smithi165 2024-08-20T02:59:50.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:49 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.99:6834/797787628,v1:172.21.15.99:6835/797787628] up:boot 2024-08-20T02:59:50.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:49 smithi099 ceph-mon[21984]: fsmap 5 up:standby 2024-08-20T02:59:50.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:49 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T02:59:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.863+0000 7f172cd5e640 1 -- 172.21.15.44:0/149943124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17281019d0 msgr2=0x7f1728101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.863+0000 7f172cd5e640 1 --2- 172.21.15.44:0/149943124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17281019d0 0x7f1728101db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1718009900 tx=0x7f171802a410 comp rx=0 tx=0).stop 2024-08-20T02:59:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.863+0000 7f172cd5e640 1 -- 172.21.15.44:0/149943124 shutdown_connections 2024-08-20T02:59:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.863+0000 7f172cd5e640 1 --2- 172.21.15.44:0/149943124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728107d60 0x7f172810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.863+0000 7f172cd5e640 1 --2- 172.21.15.44:0/149943124 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17281073e0 0x7f1728107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.863+0000 7f172cd5e640 1 --2- 172.21.15.44:0/149943124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17281019d0 0x7f1728101db0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.863+0000 7f172cd5e640 1 -- 172.21.15.44:0/149943124 >> 172.21.15.44:0/149943124 conn(0x7f17280fd440 msgr2=0x7f17280ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.863+0000 7f172cd5e640 1 -- 172.21.15.44:0/149943124 shutdown_connections 2024-08-20T02:59:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.863+0000 7f172cd5e640 1 -- 172.21.15.44:0/149943124 wait complete. 2024-08-20T02:59:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.864+0000 7f172cd5e640 1 Processor -- start 2024-08-20T02:59:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.864+0000 7f172cd5e640 1 -- start start 2024-08-20T02:59:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.864+0000 7f172cd5e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17281019d0 0x7f1728198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.864+0000 7f172cd5e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17281073e0 0x7f17281994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.864+0000 7f172cd5e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728107d60 0x7f172819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.864+0000 7f172cd5e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1728112b40 con 0x7f17281019d0 2024-08-20T02:59:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.864+0000 7f172cd5e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f17281129c0 con 0x7f17281073e0 2024-08-20T02:59:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.864+0000 7f172cd5e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1728112cc0 con 0x7f1728107d60 2024-08-20T02:59:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.864+0000 7f172659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17281019d0 0x7f1728198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.864+0000 7f172659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17281019d0 0x7f1728198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55938/0 (socket says 172.21.15.44:55938) 2024-08-20T02:59:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.864+0000 7f172659c640 1 -- 172.21.15.44:0/2284829145 learned_addr learned my addr 172.21.15.44:0/2284829145 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:59:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.864+0000 7f1726d9d640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728107d60 0x7f172819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.865+0000 7f171dd9b640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17281073e0 0x7f17281994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.865+0000 7f172659c640 1 -- 172.21.15.44:0/2284829145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728107d60 msgr2=0x7f172819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.865+0000 7f172659c640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728107d60 0x7f172819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.865+0000 7f172659c640 1 -- 172.21.15.44:0/2284829145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17281073e0 msgr2=0x7f17281994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.865+0000 7f172659c640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17281073e0 0x7f17281994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.865+0000 7f172659c640 1 -- 172.21.15.44:0/2284829145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172819f020 con 0x7f17281019d0 2024-08-20T02:59:50.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.865+0000 7f171dd9b640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17281073e0 0x7f17281994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T02:59:50.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.865+0000 7f172659c640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17281019d0 0x7f1728198f70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1718005940 tx=0x7f17180040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.866+0000 7f171ffff640 1 -- 172.21.15.44:0/2284829145 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171803f030 con 0x7f17281019d0 2024-08-20T02:59:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.866+0000 7f172cd5e640 1 -- 172.21.15.44:0/2284829145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172819f2b0 con 0x7f17281019d0 2024-08-20T02:59:50.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.866+0000 7f171ffff640 1 -- 172.21.15.44:0/2284829145 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17180044c0 con 0x7f17281019d0 2024-08-20T02:59:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.866+0000 7f171ffff640 1 -- 172.21.15.44:0/2284829145 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718035070 con 0x7f17281019d0 2024-08-20T02:59:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.866+0000 7f172cd5e640 1 -- 172.21.15.44:0/2284829145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17281a59d0 con 0x7f17281019d0 2024-08-20T02:59:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.867+0000 7f172cd5e640 1 -- 172.21.15.44:0/2284829145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1728101db0 con 0x7f17281019d0 2024-08-20T02:59:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.868+0000 7f171ffff640 1 -- 172.21.15.44:0/2284829145 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f1718042030 con 0x7f17281019d0 2024-08-20T02:59:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.869+0000 7f171ffff640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f17140b3ca0 0x7f17140b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.869+0000 7f171dd9b640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f17140b3ca0 0x7f17140b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.869+0000 7f171ffff640 1 -- 172.21.15.44:0/2284829145 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f1718103e50 con 0x7f17281019d0 2024-08-20T02:59:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.870+0000 7f171dd9b640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f17140b3ca0 0x7f17140b6160 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1728106f60 tx=0x7f1710005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:50.872+0000 7f171ffff640 1 -- 172.21.15.44:0/2284829145 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f17180cb870 con 0x7f17281019d0 2024-08-20T02:59:51.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.093+0000 7f172cd5e640 1 -- 172.21.15.44:0/2284829145 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f172819a0a0 con 0x7f17281019d0 2024-08-20T02:59:51.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.094+0000 7f171ffff640 1 -- 172.21.15.44:0/2284829145 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7f17180cb170 con 0x7f17281019d0 2024-08-20T02:59:51.096 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:59:51.097 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":6,"btime":"2024-08-20T02:59:49:389587+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24550,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24557,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-20T02:59:51.097 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 6 2024-08-20T02:59:51.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.096+0000 7f172cd5e640 1 -- 172.21.15.44:0/2284829145 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f17140b3ca0 msgr2=0x7f17140b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:51.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.096+0000 7f172cd5e640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f17140b3ca0 0x7f17140b6160 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1728106f60 tx=0x7f1710005f90 comp rx=0 tx=0).stop 2024-08-20T02:59:51.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.096+0000 7f172cd5e640 1 -- 172.21.15.44:0/2284829145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17281019d0 msgr2=0x7f1728198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:51.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.096+0000 7f172cd5e640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17281019d0 0x7f1728198f70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1718005940 tx=0x7f17180040c0 comp rx=0 tx=0).stop 2024-08-20T02:59:51.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.096+0000 7f172cd5e640 1 -- 172.21.15.44:0/2284829145 shutdown_connections 2024-08-20T02:59:51.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.096+0000 7f172cd5e640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f17140b3ca0 0x7f17140b6160 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:51.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.096+0000 7f172cd5e640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728107d60 0x7f172819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:51.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.096+0000 7f172cd5e640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17281073e0 0x7f17281994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:51.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.096+0000 7f172cd5e640 1 --2- 172.21.15.44:0/2284829145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17281019d0 0x7f1728198f70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:51.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.096+0000 7f172cd5e640 1 -- 172.21.15.44:0/2284829145 >> 172.21.15.44:0/2284829145 conn(0x7f17280fd440 msgr2=0x7f17280fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:51.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.096+0000 7f172cd5e640 1 -- 172.21.15.44:0/2284829145 shutdown_connections 2024-08-20T02:59:51.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:51.096+0000 7f172cd5e640 1 -- 172.21.15.44:0/2284829145 wait complete. 2024-08-20T02:59:51.122 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 02:59:50 smithi165 systemd[1]: Starting Ceph mds.f for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T02:59:51.386 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 02:59:51 smithi165 podman[53844]: 2024-08-20 02:59:51.119520491 +0000 UTC m=+0.109029531 container create 0783fe6121a0a108f9921c0bebd470f2d1da52b84b7b845660fa4394a93f7f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T02:59:51.386 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 02:59:51 smithi165 podman[53844]: 2024-08-20 02:59:51.02646429 +0000 UTC m=+0.015973342 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T02:59:51.387 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 02:59:51 smithi165 podman[53844]: 2024-08-20 02:59:51.363952743 +0000 UTC m=+0.353461784 container init 0783fe6121a0a108f9921c0bebd470f2d1da52b84b7b845660fa4394a93f7f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-20T02:59:51.387 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 02:59:51 smithi165 podman[53844]: 2024-08-20 02:59:51.366735264 +0000 UTC m=+0.356244312 container start 0783fe6121a0a108f9921c0bebd470f2d1da52b84b7b845660fa4394a93f7f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux ) 2024-08-20T02:59:51.886 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 02:59:51 smithi165 ceph-mds[53863]: -- 172.21.15.165:0/2926129308 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d497d31860 con 0x55d497d0f000 2024-08-20T02:59:51.886 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 02:59:51 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f[53859]: starting mds.f at 2024-08-20T02:59:51.886 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 02:59:51 smithi165 bash[53844]: 0783fe6121a0a108f9921c0bebd470f2d1da52b84b7b845660fa4394a93f7f82 2024-08-20T02:59:51.886 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 02:59:51 smithi165 systemd[1]: Started Ceph mds.f for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:59:52.227 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:51 smithi165 ceph-mon[22357]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:52.228 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:51 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2284829145' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:52.228 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:51 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:52.228 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:51 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:52.228 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:51 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:52.228 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:51 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:52.228 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:51 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:52.228 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:51 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:52.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:51 smithi044 ceph-mon[18790]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:52.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:51 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2284829145' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:52.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:51 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:52.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:51 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:52.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:51 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:52.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:51 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:52.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:51 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:52.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:51 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:52.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:51 smithi099 ceph-mon[21984]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:52.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:51 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2284829145' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:52.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:51 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:52.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:51 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:52.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:51 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:52.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:51 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:52.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:51 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:52.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:51 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:53.267 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:52 smithi165 ceph-mon[22357]: Deploying daemon mds.c on smithi165 2024-08-20T02:59:53.268 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:52 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] up:boot 2024-08-20T02:59:53.268 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:52 smithi165 ceph-mon[22357]: fsmap 6 up:standby 2024-08-20T02:59:53.268 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:52 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T02:59:53.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:52 smithi044 ceph-mon[18790]: Deploying daemon mds.c on smithi165 2024-08-20T02:59:53.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:52 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] up:boot 2024-08-20T02:59:53.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:52 smithi044 ceph-mon[18790]: fsmap 6 up:standby 2024-08-20T02:59:53.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:52 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T02:59:53.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:52 smithi099 ceph-mon[21984]: Deploying daemon mds.c on smithi165 2024-08-20T02:59:53.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:52 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] up:boot 2024-08-20T02:59:53.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:52 smithi099 ceph-mon[21984]: fsmap 6 up:standby 2024-08-20T02:59:53.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:52 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T02:59:53.636 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 02:59:53 smithi165 systemd[1]: Starting Ceph mds.c for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T02:59:53.886 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T02:59:53.953 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 02:59:53 smithi165 podman[54144]: 2024-08-20 02:59:53.559026046 +0000 UTC m=+0.016040687 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T02:59:53.954 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 02:59:53 smithi165 podman[54144]: 2024-08-20 02:59:53.682894721 +0000 UTC m=+0.139909388 container create 3c27108e7bb325e6f8cae18ef9534218e9a733eaa9da3848a1022c0bed44a0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T02:59:54.147 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:59:54.250 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:53 smithi099 ceph-mon[21984]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:54.268 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:53 smithi044 ceph-mon[18790]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:54.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:53 smithi165 ceph-mon[22357]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:54.386 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 02:59:53 smithi165 podman[54144]: 2024-08-20 02:59:53.952222133 +0000 UTC m=+0.409236758 container init 3c27108e7bb325e6f8cae18ef9534218e9a733eaa9da3848a1022c0bed44a0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-08-20T02:59:54.386 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 02:59:53 smithi165 podman[54144]: 2024-08-20 02:59:53.955016907 +0000 UTC m=+0.412031546 container start 3c27108e7bb325e6f8cae18ef9534218e9a733eaa9da3848a1022c0bed44a0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T02:59:54.386 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 02:59:53 smithi165 ceph-mds[54162]: -- 172.21.15.165:0/3250009736 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f78791860 con 0x556f7876f400 2024-08-20T02:59:54.387 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 02:59:53 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c[54158]: starting mds.c at 2024-08-20T02:59:54.387 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 02:59:54 smithi165 bash[54144]: 3c27108e7bb325e6f8cae18ef9534218e9a733eaa9da3848a1022c0bed44a0c4 2024-08-20T02:59:54.387 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 02:59:54 smithi165 systemd[1]: Started Ceph mds.c for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:59:54.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.632+0000 7f84dea8a640 1 -- 172.21.15.44:0/4029010522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84d8104230 msgr2=0x7f84d810c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:54.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.632+0000 7f84dea8a640 1 --2- 172.21.15.44:0/4029010522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84d8104230 0x7f84d810c810 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009900 tx=0x7f84c002a410 comp rx=0 tx=0).stop 2024-08-20T02:59:54.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.632+0000 7f84dea8a640 1 -- 172.21.15.44:0/4029010522 shutdown_connections 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.632+0000 7f84dea8a640 1 --2- 172.21.15.44:0/4029010522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84d810cde0 0x7f84d810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.632+0000 7f84dea8a640 1 --2- 172.21.15.44:0/4029010522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84d8104230 0x7f84d810c810 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.632+0000 7f84dea8a640 1 --2- 172.21.15.44:0/4029010522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84d8103830 0x7f84d8103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.632+0000 7f84dea8a640 1 -- 172.21.15.44:0/4029010522 >> 172.21.15.44:0/4029010522 conn(0x7f84d80fd400 msgr2=0x7f84d80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.632+0000 7f84dea8a640 1 -- 172.21.15.44:0/4029010522 shutdown_connections 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.632+0000 7f84dea8a640 1 -- 172.21.15.44:0/4029010522 wait complete. 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dea8a640 1 Processor -- start 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dea8a640 1 -- start start 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dea8a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84d8103830 0x7f84d8199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dea8a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84d8104230 0x7f84d8199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dea8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84d810cde0 0x7f84d819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dea8a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8112ac0 con 0x7f84d810cde0 2024-08-20T02:59:54.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dea8a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8112940 con 0x7f84d8104230 2024-08-20T02:59:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dea8a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8112c40 con 0x7f84d8103830 2024-08-20T02:59:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dd027640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84d810cde0 0x7f84d819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dd027640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84d810cde0 0x7f84d819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55950/0 (socket says 172.21.15.44:55950) 2024-08-20T02:59:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dd027640 1 -- 172.21.15.44:0/1102311633 learned_addr learned my addr 172.21.15.44:0/1102311633 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:59:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dc826640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84d8103830 0x7f84d8199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dd027640 1 -- 172.21.15.44:0/1102311633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84d8103830 msgr2=0x7f84d8199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dd027640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84d8103830 0x7f84d8199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dd027640 1 -- 172.21.15.44:0/1102311633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84d8104230 msgr2=0x7f84d8199550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T02:59:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dd027640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84d8104230 0x7f84d8199550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dd027640 1 -- 172.21.15.44:0/1102311633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d819f0c0 con 0x7f84d810cde0 2024-08-20T02:59:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.633+0000 7f84dd027640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84d810cde0 0x7f84d819e8f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f84d000b970 tx=0x7f84d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:54.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.634+0000 7f84d5ffb640 1 -- 172.21.15.44:0/1102311633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0019070 con 0x7f84d810cde0 2024-08-20T02:59:54.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.634+0000 7f84d5ffb640 1 -- 172.21.15.44:0/1102311633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d0014440 con 0x7f84d810cde0 2024-08-20T02:59:54.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.634+0000 7f84dea8a640 1 -- 172.21.15.44:0/1102311633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d819f3b0 con 0x7f84d810cde0 2024-08-20T02:59:54.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.634+0000 7f84d5ffb640 1 -- 172.21.15.44:0/1102311633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d00075d0 con 0x7f84d810cde0 2024-08-20T02:59:54.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.634+0000 7f84dea8a640 1 -- 172.21.15.44:0/1102311633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d81a5a70 con 0x7f84d810cde0 2024-08-20T02:59:54.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.635+0000 7f84dea8a640 1 -- 172.21.15.44:0/1102311633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d8103c10 con 0x7f84d810cde0 2024-08-20T02:59:54.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.636+0000 7f84d5ffb640 1 -- 172.21.15.44:0/1102311633 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f84d001c070 con 0x7f84d810cde0 2024-08-20T02:59:54.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.637+0000 7f84d5ffb640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f84c40b3c20 0x7f84c40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:54.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.637+0000 7f84dc826640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f84c40b3c20 0x7f84c40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:54.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.637+0000 7f84d5ffb640 1 -- 172.21.15.44:0/1102311633 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f84d00e2ed0 con 0x7f84d810cde0 2024-08-20T02:59:54.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.638+0000 7f84dc826640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f84c40b3c20 0x7f84c40b60e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f84c8005fd0 tx=0x7f84c800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:54.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.641+0000 7f84d5ffb640 1 -- 172.21.15.44:0/1102311633 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f84d00aa200 con 0x7f84d810cde0 2024-08-20T02:59:54.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.862+0000 7f84dea8a640 1 -- 172.21.15.44:0/1102311633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84d819a140 con 0x7f84d810cde0 2024-08-20T02:59:54.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.863+0000 7f84d5ffb640 1 -- 172.21.15.44:0/1102311633 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6391 (secure 0 0 0) 0x7f84d00ae750 con 0x7f84d810cde0 2024-08-20T02:59:54.865 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:59:54.865 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"btime":"2024-08-20T02:59:53:983270+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24550,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24557,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-08-20T02:59:54.866 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 8 2024-08-20T02:59:54.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.865+0000 7f84dea8a640 1 -- 172.21.15.44:0/1102311633 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f84c40b3c20 msgr2=0x7f84c40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:54.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.865+0000 7f84dea8a640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f84c40b3c20 0x7f84c40b60e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f84c8005fd0 tx=0x7f84c800b390 comp rx=0 tx=0).stop 2024-08-20T02:59:54.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.865+0000 7f84dea8a640 1 -- 172.21.15.44:0/1102311633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84d810cde0 msgr2=0x7f84d819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:54.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.865+0000 7f84dea8a640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84d810cde0 0x7f84d819e8f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f84d000b970 tx=0x7f84d000be30 comp rx=0 tx=0).stop 2024-08-20T02:59:54.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.865+0000 7f84dea8a640 1 -- 172.21.15.44:0/1102311633 shutdown_connections 2024-08-20T02:59:54.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.865+0000 7f84dea8a640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f84c40b3c20 0x7f84c40b60e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:54.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.865+0000 7f84dea8a640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f84d810cde0 0x7f84d819e8f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:54.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.865+0000 7f84dea8a640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84d8104230 0x7f84d8199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:54.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.865+0000 7f84dea8a640 1 --2- 172.21.15.44:0/1102311633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84d8103830 0x7f84d8199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:54.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.865+0000 7f84dea8a640 1 -- 172.21.15.44:0/1102311633 >> 172.21.15.44:0/1102311633 conn(0x7f84d80fd400 msgr2=0x7f84d80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:54.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.865+0000 7f84dea8a640 1 -- 172.21.15.44:0/1102311633 shutdown_connections 2024-08-20T02:59:54.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:54.865+0000 7f84dea8a640 1 -- 172.21.15.44:0/1102311633 wait complete. 2024-08-20T02:59:55.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:54 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] up:boot 2024-08-20T02:59:55.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:54 smithi099 ceph-mon[21984]: fsmap 7 up:standby 2024-08-20T02:59:55.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:54 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T02:59:55.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:54 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:55.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:54 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:55.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:54 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:55.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:54 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:55.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:54 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:55.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:54 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:55.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:54 smithi099 ceph-mon[21984]: Deploying daemon mds.h on smithi099 2024-08-20T02:59:55.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:54 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1102311633' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:55.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:54 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] up:boot 2024-08-20T02:59:55.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:54 smithi044 ceph-mon[18790]: fsmap 7 up:standby 2024-08-20T02:59:55.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:54 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T02:59:55.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:54 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:55.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:54 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:55.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:54 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:55.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:54 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:55.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:54 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:55.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:54 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:55.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:54 smithi044 ceph-mon[18790]: Deploying daemon mds.h on smithi099 2024-08-20T02:59:55.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:54 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1102311633' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:55.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:54 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] up:boot 2024-08-20T02:59:55.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:54 smithi165 ceph-mon[22357]: fsmap 7 up:standby 2024-08-20T02:59:55.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:54 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T02:59:55.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:54 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:55.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:54 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:55.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:54 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:55.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:54 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:55.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:54 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:55.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:54 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:55.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:54 smithi165 ceph-mon[22357]: Deploying daemon mds.h on smithi099 2024-08-20T02:59:55.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:54 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1102311633' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:56.378 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:56 smithi099 ceph-mon[21984]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:56.378 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 02:59:56 smithi099 systemd[1]: Starting Ceph mds.h for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T02:59:56.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:56 smithi165 ceph-mon[22357]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:56.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:56 smithi044 ceph-mon[18790]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:56.658 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 02:59:56 smithi099 podman[53433]: 2024-08-20 02:59:56.278523239 +0000 UTC m=+0.016110249 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T02:59:56.658 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 02:59:56 smithi099 podman[53433]: 2024-08-20 02:59:56.403229116 +0000 UTC m=+0.140816155 container create 02ec440a0d71d6169cc66d5761e117f7513a278ce0bc88df38a6acd69fa284d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T02:59:56.658 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 02:59:56 smithi099 podman[53433]: 2024-08-20 02:59:56.656178163 +0000 UTC m=+0.393765159 container init 02ec440a0d71d6169cc66d5761e117f7513a278ce0bc88df38a6acd69fa284d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T02:59:56.914 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 02:59:56 smithi099 podman[53433]: 2024-08-20 02:59:56.658987672 +0000 UTC m=+0.396574669 container start 02ec440a0d71d6169cc66d5761e117f7513a278ce0bc88df38a6acd69fa284d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T02:59:56.915 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 02:59:56 smithi099 ceph-mds[53452]: -- 172.21.15.99:0/1180171351 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56522b62f860 con 0x56522b60d400 2024-08-20T02:59:56.915 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 02:59:56 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h[53448]: starting mds.h at 2024-08-20T02:59:56.915 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 02:59:56 smithi099 bash[53433]: 02ec440a0d71d6169cc66d5761e117f7513a278ce0bc88df38a6acd69fa284d1 2024-08-20T02:59:56.915 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 02:59:56 smithi099 systemd[1]: Started Ceph mds.h for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T02:59:57.613 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T02:59:57.884 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T02:59:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:57 smithi165 ceph-mon[22357]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:57 smithi165 ceph-mon[22357]: Deploying daemon mds.i on smithi165 2024-08-20T02:59:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:57 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] up:boot 2024-08-20T02:59:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:57 smithi165 ceph-mon[22357]: fsmap 8 up:standby 2024-08-20T02:59:57.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:57 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T02:59:58.004 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:57 smithi044 ceph-mon[18790]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:58.004 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:58.005 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:58.005 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:58.005 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:58.005 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:58.005 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:58.005 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:57 smithi044 ceph-mon[18790]: Deploying daemon mds.i on smithi165 2024-08-20T02:59:58.005 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:57 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] up:boot 2024-08-20T02:59:58.005 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:57 smithi044 ceph-mon[18790]: fsmap 8 up:standby 2024-08-20T02:59:58.005 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:57 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T02:59:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:57 smithi099 ceph-mon[21984]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T02:59:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T02:59:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T02:59:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T02:59:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T02:59:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:57 smithi099 ceph-mon[21984]: Deploying daemon mds.i on smithi165 2024-08-20T02:59:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:57 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] up:boot 2024-08-20T02:59:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:57 smithi099 ceph-mon[21984]: fsmap 8 up:standby 2024-08-20T02:59:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:57 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T02:59:58.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.384+0000 7fd048e62640 1 -- 172.21.15.44:0/2248397781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0441073e0 msgr2=0x7fd044107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:58.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.384+0000 7fd048e62640 1 --2- 172.21.15.44:0/2248397781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0441073e0 0x7fd044107820 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd038009ca0 tx=0x7fd03802a8c0 comp rx=0 tx=0).stop 2024-08-20T02:59:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.384+0000 7fd048e62640 1 -- 172.21.15.44:0/2248397781 shutdown_connections 2024-08-20T02:59:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.384+0000 7fd048e62640 1 --2- 172.21.15.44:0/2248397781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd044107d60 0x7fd04410e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.384+0000 7fd048e62640 1 --2- 172.21.15.44:0/2248397781 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0441073e0 0x7fd044107820 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.384+0000 7fd048e62640 1 --2- 172.21.15.44:0/2248397781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0441019d0 0x7fd044101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.384+0000 7fd048e62640 1 -- 172.21.15.44:0/2248397781 >> 172.21.15.44:0/2248397781 conn(0x7fd0440fd440 msgr2=0x7fd0440ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.384+0000 7fd048e62640 1 -- 172.21.15.44:0/2248397781 shutdown_connections 2024-08-20T02:59:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.384+0000 7fd048e62640 1 -- 172.21.15.44:0/2248397781 wait complete. 2024-08-20T02:59:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd048e62640 1 Processor -- start 2024-08-20T02:59:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd048e62640 1 -- start start 2024-08-20T02:59:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd048e62640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0441019d0 0x7fd044199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd048e62640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0441073e0 0x7fd04419a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd048e62640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd044107d60 0x7fd04419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd048e62640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441119b0 con 0x7fd0441019d0 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd048e62640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd044111830 con 0x7fd044107d60 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd048e62640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd044111b30 con 0x7fd0441073e0 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd04259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0441019d0 0x7fd044199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd04259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0441019d0 0x7fd044199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43422/0 (socket says 172.21.15.44:43422) 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd04259c640 1 -- 172.21.15.44:0/2483840247 learned_addr learned my addr 172.21.15.44:0/2483840247 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd041d9b640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0441073e0 0x7fd04419a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd042d9d640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd044107d60 0x7fd04419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd04259c640 1 -- 172.21.15.44:0/2483840247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0441073e0 msgr2=0x7fd04419a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd04259c640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0441073e0 0x7fd04419a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd04259c640 1 -- 172.21.15.44:0/2483840247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd044107d60 msgr2=0x7fd04419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd04259c640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd044107d60 0x7fd04419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd04259c640 1 -- 172.21.15.44:0/2483840247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0441a5100 con 0x7fd0441019d0 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd041d9b640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0441073e0 0x7fd04419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.385+0000 7fd04259c640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0441019d0 0x7fd044199d20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c920 tx=0x7fd03400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.386+0000 7fd0337fe640 1 -- 172.21.15.44:0/2483840247 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034007c70 con 0x7fd0441019d0 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.386+0000 7fd048e62640 1 -- 172.21.15.44:0/2483840247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0441a53f0 con 0x7fd0441019d0 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.386+0000 7fd0337fe640 1 -- 172.21.15.44:0/2483840247 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd034007e10 con 0x7fd0441019d0 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.386+0000 7fd0337fe640 1 -- 172.21.15.44:0/2483840247 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034018820 con 0x7fd0441019d0 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.386+0000 7fd048e62640 1 -- 172.21.15.44:0/2483840247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0441a5930 con 0x7fd0441019d0 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.387+0000 7fd048e62640 1 -- 172.21.15.44:0/2483840247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd044101db0 con 0x7fd0441019d0 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.388+0000 7fd0337fe640 1 -- 172.21.15.44:0/2483840247 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fd03400f450 con 0x7fd0441019d0 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.389+0000 7fd0337fe640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd02c0b3c20 0x7fd02c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.389+0000 7fd041d9b640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd02c0b3c20 0x7fd02c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.389+0000 7fd0337fe640 1 -- 172.21.15.44:0/2483840247 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fd0340e3d20 con 0x7fd0441019d0 2024-08-20T02:59:58.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.389+0000 7fd041d9b640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd02c0b3c20 0x7fd02c0b60e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd04406bd90 tx=0x7fd03802f480 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T02:59:58.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.392+0000 7fd0337fe640 1 -- 172.21.15.44:0/2483840247 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd034014030 con 0x7fd0441019d0 2024-08-20T02:59:58.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.616+0000 7fd048e62640 1 -- 172.21.15.44:0/2483840247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd04406b5a0 con 0x7fd0441019d0 2024-08-20T02:59:58.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.616+0000 7fd0337fe640 1 -- 172.21.15.44:0/2483840247 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7213 (secure 0 0 0) 0x7fd0340ab7d0 con 0x7fd0441019d0 2024-08-20T02:59:58.620 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T02:59:58.620 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":9,"btime":"2024-08-20T02:59:57:119282+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24550,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24557,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24569,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-08-20T02:59:58.620 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 9 2024-08-20T02:59:58.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.619+0000 7fd048e62640 1 -- 172.21.15.44:0/2483840247 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd02c0b3c20 msgr2=0x7fd02c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:58.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.619+0000 7fd048e62640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd02c0b3c20 0x7fd02c0b60e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd04406bd90 tx=0x7fd03802f480 comp rx=0 tx=0).stop 2024-08-20T02:59:58.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.619+0000 7fd048e62640 1 -- 172.21.15.44:0/2483840247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0441019d0 msgr2=0x7fd044199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T02:59:58.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.619+0000 7fd048e62640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0441019d0 0x7fd044199d20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c920 tx=0x7fd03400cde0 comp rx=0 tx=0).stop 2024-08-20T02:59:58.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.619+0000 7fd048e62640 1 -- 172.21.15.44:0/2483840247 shutdown_connections 2024-08-20T02:59:58.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.619+0000 7fd048e62640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd02c0b3c20 0x7fd02c0b60e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:58.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.619+0000 7fd048e62640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd044107d60 0x7fd04419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:58.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.619+0000 7fd048e62640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0441073e0 0x7fd04419a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:58.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.619+0000 7fd048e62640 1 --2- 172.21.15.44:0/2483840247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0441019d0 0x7fd044199d20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T02:59:58.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.619+0000 7fd048e62640 1 -- 172.21.15.44:0/2483840247 >> 172.21.15.44:0/2483840247 conn(0x7fd0440fd440 msgr2=0x7fd0440ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T02:59:58.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.619+0000 7fd048e62640 1 -- 172.21.15.44:0/2483840247 shutdown_connections 2024-08-20T02:59:58.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T02:59:58.619+0000 7fd048e62640 1 -- 172.21.15.44:0/2483840247 wait complete. 2024-08-20T02:59:59.028 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:58 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2483840247' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:59.035 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:58 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2483840247' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:59.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:58 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2483840247' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T02:59:59.636 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 02:59:59 smithi165 systemd[1]: Started Ceph mds.i for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T03:00:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:59 smithi165 ceph-mon[22357]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T03:00:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T03:00:00.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 02:59:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:00.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:59 smithi099 ceph-mon[21984]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:00.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:00.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:00.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:00.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T03:00:00.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T03:00:00.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 02:59:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:00.180 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:59 smithi044 ceph-mon[18790]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:00.181 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:00.181 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:00.181 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:00.181 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T03:00:00.181 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T03:00:00.181 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 02:59:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:01.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:00 smithi165 ceph-mon[22357]: Deploying daemon mds.g on smithi044 2024-08-20T03:00:01.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:00 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] up:boot 2024-08-20T03:00:01.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:00 smithi165 ceph-mon[22357]: fsmap 9 up:standby 2024-08-20T03:00:01.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:00 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T03:00:01.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:00 smithi165 ceph-mon[22357]: overall HEALTH_OK 2024-08-20T03:00:01.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:00 smithi099 ceph-mon[21984]: Deploying daemon mds.g on smithi044 2024-08-20T03:00:01.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:00 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] up:boot 2024-08-20T03:00:01.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:00 smithi099 ceph-mon[21984]: fsmap 9 up:standby 2024-08-20T03:00:01.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:00 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T03:00:01.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:00 smithi099 ceph-mon[21984]: overall HEALTH_OK 2024-08-20T03:00:01.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:00 smithi044 ceph-mon[18790]: Deploying daemon mds.g on smithi044 2024-08-20T03:00:01.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:00 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] up:boot 2024-08-20T03:00:01.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:00 smithi044 ceph-mon[18790]: fsmap 9 up:standby 2024-08-20T03:00:01.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:00 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T03:00:01.195 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:00 smithi044 ceph-mon[18790]: overall HEALTH_OK 2024-08-20T03:00:01.427 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:00:01.452 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:00:01 smithi044 systemd[1]: Starting Ceph mds.g for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:00:01.785 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:00:01 smithi044 podman[63464]: 2024-08-20 03:00:01.53898921 +0000 UTC m=+0.017979539 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:00:01.786 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:00:01 smithi044 podman[63464]: 2024-08-20 03:00:01.67814195 +0000 UTC m=+0.157132306 container create 98b40491673a910fce95b8a51f96d1cbc9b5f56d0c8a0fab80a9c4c22a5b161d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-g, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-08-20T03:00:02.121 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:02.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:01 smithi165 ceph-mon[22357]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:02.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:01 smithi099 ceph-mon[21984]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:02.244 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:01 smithi044 ceph-mon[18790]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:02.244 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:00:01 smithi044 podman[63464]: 2024-08-20 03:00:01.939387432 +0000 UTC m=+0.418377757 container init 98b40491673a910fce95b8a51f96d1cbc9b5f56d0c8a0fab80a9c4c22a5b161d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-g, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-08-20T03:00:02.244 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:00:01 smithi044 podman[63464]: 2024-08-20 03:00:01.942589057 +0000 UTC m=+0.421579378 container start 98b40491673a910fce95b8a51f96d1cbc9b5f56d0c8a0fab80a9c4c22a5b161d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-g, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-08-20T03:00:02.245 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:00:01 smithi044 ceph-mds[63508]: -- 172.21.15.44:0/1156048887 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e469a25860 con 0x55e469a03000 2024-08-20T03:00:02.245 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:00:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-g[63491]: starting mds.g at 2024-08-20T03:00:02.245 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:00:02 smithi044 bash[63464]: 98b40491673a910fce95b8a51f96d1cbc9b5f56d0c8a0fab80a9c4c22a5b161d 2024-08-20T03:00:02.245 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:00:02 smithi044 systemd[1]: Started Ceph mds.g for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T03:00:02.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.591+0000 7fd47baf6640 1 -- 172.21.15.44:0/58270457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd474107d20 msgr2=0x7fd47410e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:02.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.591+0000 7fd47baf6640 1 --2- 172.21.15.44:0/58270457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd474107d20 0x7fd47410e520 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009ca0 tx=0x7fd46c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:02.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.591+0000 7fd47baf6640 1 -- 172.21.15.44:0/58270457 shutdown_connections 2024-08-20T03:00:02.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.591+0000 7fd47baf6640 1 --2- 172.21.15.44:0/58270457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd474107d20 0x7fd47410e520 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:02.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.591+0000 7fd47baf6640 1 --2- 172.21.15.44:0/58270457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4741073a0 0x7fd4741077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:02.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.591+0000 7fd47baf6640 1 --2- 172.21.15.44:0/58270457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd474101990 0x7fd474101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:02.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.591+0000 7fd47baf6640 1 -- 172.21.15.44:0/58270457 >> 172.21.15.44:0/58270457 conn(0x7fd4740fd400 msgr2=0x7fd4740ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:02.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.591+0000 7fd47baf6640 1 -- 172.21.15.44:0/58270457 shutdown_connections 2024-08-20T03:00:02.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.591+0000 7fd47baf6640 1 -- 172.21.15.44:0/58270457 wait complete. 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47baf6640 1 Processor -- start 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47baf6640 1 -- start start 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47baf6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd474101990 0x7fd4740fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47baf6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4741073a0 0x7fd4740fe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47baf6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd474107d20 0x7fd4740fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47baf6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741119a0 con 0x7fd474107d20 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47baf6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd474111820 con 0x7fd474101990 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47baf6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd474111b20 con 0x7fd4741073a0 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47a093640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd474107d20 0x7fd4740fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47a093640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd474107d20 0x7fd4740fe5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43460/0 (socket says 172.21.15.44:43460) 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47a093640 1 -- 172.21.15.44:0/2299873864 learned_addr learned my addr 172.21.15.44:0/2299873864 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd479091640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4741073a0 0x7fd4740fe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:02.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd479892640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd474101990 0x7fd4740fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47a093640 1 -- 172.21.15.44:0/2299873864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4741073a0 msgr2=0x7fd4740fe0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47a093640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4741073a0 0x7fd4740fe0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47a093640 1 -- 172.21.15.44:0/2299873864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd474101990 msgr2=0x7fd4740fdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47a093640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd474101990 0x7fd4740fdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd47a093640 1 -- 172.21.15.44:0/2299873864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4741a72e0 con 0x7fd474107d20 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.592+0000 7fd479091640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4741073a0 0x7fd4740fe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.593+0000 7fd47a093640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd474107d20 0x7fd4740fe5e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd46c005cc0 tx=0x7fd46c02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.593+0000 7fd46affd640 1 -- 172.21.15.44:0/2299873864 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c003cf0 con 0x7fd474107d20 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.593+0000 7fd47baf6640 1 -- 172.21.15.44:0/2299873864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4741a7570 con 0x7fd474107d20 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.593+0000 7fd46affd640 1 -- 172.21.15.44:0/2299873864 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd46c0044f0 con 0x7fd474107d20 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.593+0000 7fd46affd640 1 -- 172.21.15.44:0/2299873864 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c02fd10 con 0x7fd474107d20 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.593+0000 7fd47baf6640 1 -- 172.21.15.44:0/2299873864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4741a7b00 con 0x7fd474107d20 2024-08-20T03:00:02.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.594+0000 7fd47baf6640 1 -- 172.21.15.44:0/2299873864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd474101d70 con 0x7fd474107d20 2024-08-20T03:00:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.596+0000 7fd46affd640 1 -- 172.21.15.44:0/2299873864 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fd46c004040 con 0x7fd474107d20 2024-08-20T03:00:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.596+0000 7fd46affd640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4600b3c20 0x7fd4600b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.596+0000 7fd479892640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4600b3c20 0x7fd4600b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.596+0000 7fd46affd640 1 -- 172.21.15.44:0/2299873864 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fd46c105430 con 0x7fd474107d20 2024-08-20T03:00:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.597+0000 7fd479892640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4600b3c20 0x7fd4600b60e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd470005e10 tx=0x7fd470005da0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:02.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.599+0000 7fd46affd640 1 -- 172.21.15.44:0/2299873864 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd46c082e20 con 0x7fd474107d20 2024-08-20T03:00:02.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.828+0000 7fd47baf6640 1 -- 172.21.15.44:0/2299873864 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4740fee60 con 0x7fd474107d20 2024-08-20T03:00:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.828+0000 7fd46affd640 1 -- 172.21.15.44:0/2299873864 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8043 (secure 0 0 0) 0x7fd4740fee60 con 0x7fd474107d20 2024-08-20T03:00:02.832 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:02.832 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":10,"btime":"2024-08-20T02:59:59:840867+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24550,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24557,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24569,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24604,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-20T03:00:02.832 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 10 2024-08-20T03:00:02.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.830+0000 7fd47baf6640 1 -- 172.21.15.44:0/2299873864 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4600b3c20 msgr2=0x7fd4600b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:02.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.830+0000 7fd47baf6640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4600b3c20 0x7fd4600b60e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd470005e10 tx=0x7fd470005da0 comp rx=0 tx=0).stop 2024-08-20T03:00:02.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.830+0000 7fd47baf6640 1 -- 172.21.15.44:0/2299873864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd474107d20 msgr2=0x7fd4740fe5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:02.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.830+0000 7fd47baf6640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd474107d20 0x7fd4740fe5e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd46c005cc0 tx=0x7fd46c02f470 comp rx=0 tx=0).stop 2024-08-20T03:00:02.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.830+0000 7fd47baf6640 1 -- 172.21.15.44:0/2299873864 shutdown_connections 2024-08-20T03:00:02.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.830+0000 7fd47baf6640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4600b3c20 0x7fd4600b60e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:02.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.830+0000 7fd47baf6640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd474107d20 0x7fd4740fe5e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:02.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.830+0000 7fd47baf6640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4741073a0 0x7fd4740fe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:02.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.830+0000 7fd47baf6640 1 --2- 172.21.15.44:0/2299873864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd474101990 0x7fd4740fdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:02.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.830+0000 7fd47baf6640 1 -- 172.21.15.44:0/2299873864 >> 172.21.15.44:0/2299873864 conn(0x7fd4740fd400 msgr2=0x7fd47406aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:02.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.831+0000 7fd47baf6640 1 -- 172.21.15.44:0/2299873864 shutdown_connections 2024-08-20T03:00:02.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:02.831+0000 7fd47baf6640 1 -- 172.21.15.44:0/2299873864 wait complete. 2024-08-20T03:00:03.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:03.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:03.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:03.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T03:00:03.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T03:00:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:03 smithi165 ceph-mon[22357]: Deploying daemon mds.a on smithi044 2024-08-20T03:00:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:03 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2299873864' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:00:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:03 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.44:6838/335802127,v1:172.21.15.44:6839/335802127] up:boot 2024-08-20T03:00:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:03 smithi165 ceph-mon[22357]: fsmap 10 up:standby 2024-08-20T03:00:03.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T03:00:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T03:00:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T03:00:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:03 smithi099 ceph-mon[21984]: Deploying daemon mds.a on smithi044 2024-08-20T03:00:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:03 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2299873864' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:00:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:03 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.44:6838/335802127,v1:172.21.15.44:6839/335802127] up:boot 2024-08-20T03:00:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:03 smithi099 ceph-mon[21984]: fsmap 10 up:standby 2024-08-20T03:00:03.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T03:00:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T03:00:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T03:00:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:03 smithi044 ceph-mon[18790]: Deploying daemon mds.a on smithi044 2024-08-20T03:00:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:03 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2299873864' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:00:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:03 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.44:6838/335802127,v1:172.21.15.44:6839/335802127] up:boot 2024-08-20T03:00:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:03 smithi044 ceph-mon[18790]: fsmap 10 up:standby 2024-08-20T03:00:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T03:00:04.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:04 smithi165 ceph-mon[22357]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:04.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:04 smithi099 ceph-mon[21984]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:04.460 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:04 smithi044 ceph-mon[18790]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:05.036 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:00:04 smithi044 systemd[1]: Starting Ceph mds.a for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:00:05.417 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:00:05 smithi044 podman[63862]: 2024-08-20 03:00:05.049225425 +0000 UTC m=+0.016082362 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:00:05.417 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:00:05 smithi044 podman[63862]: 2024-08-20 03:00:05.152998697 +0000 UTC m=+0.119855633 container create 37ec51dad20bb89ce993ad9e18e7cd70eacd541b9d3a7ada6111a367a35f1670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-08-20T03:00:05.418 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:00:05 smithi044 podman[63862]: 2024-08-20 03:00:05.389239071 +0000 UTC m=+0.356096010 container init 37ec51dad20bb89ce993ad9e18e7cd70eacd541b9d3a7ada6111a367a35f1670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-a, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-20T03:00:05.418 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:00:05 smithi044 podman[63862]: 2024-08-20 03:00:05.392009467 +0000 UTC m=+0.358866406 container start 37ec51dad20bb89ce993ad9e18e7cd70eacd541b9d3a7ada6111a367a35f1670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T03:00:05.610 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:00:05.785 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:00:05 smithi044 ceph-mds[63880]: -- 172.21.15.44:0/1795758391 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d27bb4f860 con 0x55d27bb2c800 2024-08-20T03:00:05.786 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:00:05 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-a[63876]: starting mds.a at 2024-08-20T03:00:05.786 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:00:05 smithi044 bash[63862]: 37ec51dad20bb89ce993ad9e18e7cd70eacd541b9d3a7ada6111a367a35f1670 2024-08-20T03:00:05.786 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:00:05 smithi044 systemd[1]: Started Ceph mds.a for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T03:00:05.866 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:06.278 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:06 smithi044 ceph-mon[18790]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:06.278 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:06 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:06.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:06 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:06.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:06 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:06.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:06 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T03:00:06.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:06 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T03:00:06.279 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:06 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:06.386 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:06 smithi099 ceph-mon[21984]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:06.386 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:06 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:06.386 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:06 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:06.387 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:06 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:06.387 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:06 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T03:00:06.387 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:06 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T03:00:06.387 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:06 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:06.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:06 smithi165 ceph-mon[22357]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:06.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:06 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:06.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:06 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:06.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:06 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:06.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:06 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T03:00:06.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:06 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T03:00:06.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:06 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:06.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 -- 172.21.15.44:0/613557539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660101e50 msgr2=0x7fe66010eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:06.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 --2- 172.21.15.44:0/613557539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660101e50 0x7fe66010eba0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe650009900 tx=0x7fe65002a480 comp rx=0 tx=0).stop 2024-08-20T03:00:06.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 -- 172.21.15.44:0/613557539 shutdown_connections 2024-08-20T03:00:06.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 --2- 172.21.15.44:0/613557539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe66010f0e0 0x7fe6601114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:06.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 --2- 172.21.15.44:0/613557539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660101e50 0x7fe66010eba0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:06.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 --2- 172.21.15.44:0/613557539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe660101530 0x7fe660101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:06.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 -- 172.21.15.44:0/613557539 >> 172.21.15.44:0/613557539 conn(0x7fe6600fd400 msgr2=0x7fe6600ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:06.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 -- 172.21.15.44:0/613557539 shutdown_connections 2024-08-20T03:00:06.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 -- 172.21.15.44:0/613557539 wait complete. 2024-08-20T03:00:06.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 Processor -- start 2024-08-20T03:00:06.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 -- start start 2024-08-20T03:00:06.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660101530 0x7fe66019b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:06.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe660101e50 0x7fe66019b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:06.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.358+0000 7fe667bcc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe66010f0e0 0x7fe6601a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:06.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe667bcc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe660114c60 con 0x7fe660101530 2024-08-20T03:00:06.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe667bcc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe660114ae0 con 0x7fe66010f0e0 2024-08-20T03:00:06.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe667bcc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe660114de0 con 0x7fe660101e50 2024-08-20T03:00:06.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe665968640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660101530 0x7fe66019b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:06.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe665968640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660101530 0x7fe66019b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43508/0 (socket says 172.21.15.44:43508) 2024-08-20T03:00:06.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe665968640 1 -- 172.21.15.44:0/2926616822 learned_addr learned my addr 172.21.15.44:0/2926616822 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:06.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe666169640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe66010f0e0 0x7fe6601a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:06.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe665167640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe660101e50 0x7fe66019b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:06.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe665968640 1 -- 172.21.15.44:0/2926616822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe660101e50 msgr2=0x7fe66019b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:06.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe665968640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe660101e50 0x7fe66019b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:06.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe665968640 1 -- 172.21.15.44:0/2926616822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe66010f0e0 msgr2=0x7fe6601a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:06.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe665968640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe66010f0e0 0x7fe6601a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:06.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe665968640 1 -- 172.21.15.44:0/2926616822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6601a1200 con 0x7fe660101530 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe665968640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660101530 0x7fe66019b150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00eab0 tx=0x7fe65c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe656ffd640 1 -- 172.21.15.44:0/2926616822 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c00ce40 con 0x7fe660101530 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe656ffd640 1 -- 172.21.15.44:0/2926616822 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe65c011920 con 0x7fe660101530 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe667bcc640 1 -- 172.21.15.44:0/2926616822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6601a14f0 con 0x7fe660101530 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe656ffd640 1 -- 172.21.15.44:0/2926616822 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c010a30 con 0x7fe660101530 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.359+0000 7fe667bcc640 1 -- 172.21.15.44:0/2926616822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6601a7d80 con 0x7fe660101530 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.360+0000 7fe667bcc640 1 -- 172.21.15.44:0/2926616822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe660101910 con 0x7fe660101530 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.362+0000 7fe656ffd640 1 -- 172.21.15.44:0/2926616822 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fe65c010bd0 con 0x7fe660101530 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.362+0000 7fe656ffd640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe64c0b3ca0 0x7fe64c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.362+0000 7fe665167640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe64c0b3ca0 0x7fe64c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.363+0000 7fe656ffd640 1 -- 172.21.15.44:0/2926616822 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fe65c0e3e40 con 0x7fe660101530 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.363+0000 7fe665167640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe64c0b3ca0 0x7fe64c0b6160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe650031440 tx=0x7fe6500095f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:06.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.366+0000 7fe656ffd640 1 -- 172.21.15.44:0/2926616822 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe65c016030 con 0x7fe660101530 2024-08-20T03:00:06.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.587+0000 7fe667bcc640 1 -- 172.21.15.44:0/2926616822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6601a7f50 con 0x7fe660101530 2024-08-20T03:00:06.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.587+0000 7fe656ffd640 1 -- 172.21.15.44:0/2926616822 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9689 (secure 0 0 0) 0x7fe65c0ab860 con 0x7fe660101530 2024-08-20T03:00:06.591 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:06.591 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":12,"btime":"2024-08-20T03:00:06:122598+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24550,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24557,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24569,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24604,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-20T03:00:06.591 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 12 2024-08-20T03:00:06.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.589+0000 7fe667bcc640 1 -- 172.21.15.44:0/2926616822 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe64c0b3ca0 msgr2=0x7fe64c0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:06.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.589+0000 7fe667bcc640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe64c0b3ca0 0x7fe64c0b6160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe650031440 tx=0x7fe6500095f0 comp rx=0 tx=0).stop 2024-08-20T03:00:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.589+0000 7fe667bcc640 1 -- 172.21.15.44:0/2926616822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660101530 msgr2=0x7fe66019b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.589+0000 7fe667bcc640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660101530 0x7fe66019b150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00eab0 tx=0x7fe65c00ef70 comp rx=0 tx=0).stop 2024-08-20T03:00:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.589+0000 7fe667bcc640 1 -- 172.21.15.44:0/2926616822 shutdown_connections 2024-08-20T03:00:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.589+0000 7fe667bcc640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe64c0b3ca0 0x7fe64c0b6160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.589+0000 7fe667bcc640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe66010f0e0 0x7fe6601a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.589+0000 7fe667bcc640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe660101e50 0x7fe66019b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.589+0000 7fe667bcc640 1 --2- 172.21.15.44:0/2926616822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe660101530 0x7fe66019b150 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.589+0000 7fe667bcc640 1 -- 172.21.15.44:0/2926616822 >> 172.21.15.44:0/2926616822 conn(0x7fe6600fd400 msgr2=0x7fe6600ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.589+0000 7fe667bcc640 1 -- 172.21.15.44:0/2926616822 shutdown_connections 2024-08-20T03:00:06.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:06.589+0000 7fe667bcc640 1 -- 172.21.15.44:0/2926616822 wait complete. 2024-08-20T03:00:07.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:07 smithi044 ceph-mon[18790]: Deploying daemon mds.b on smithi099 2024-08-20T03:00:07.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:07 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.44:6840/370123956,v1:172.21.15.44:6841/370123956] up:boot 2024-08-20T03:00:07.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:07 smithi044 ceph-mon[18790]: fsmap 11 up:standby 2024-08-20T03:00:07.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:07 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-20T03:00:07.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:07 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2926616822' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:00:07.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:07 smithi165 ceph-mon[22357]: Deploying daemon mds.b on smithi099 2024-08-20T03:00:07.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:07 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.44:6840/370123956,v1:172.21.15.44:6841/370123956] up:boot 2024-08-20T03:00:07.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:07 smithi165 ceph-mon[22357]: fsmap 11 up:standby 2024-08-20T03:00:07.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:07 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-20T03:00:07.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:07 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2926616822' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:00:07.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:07 smithi099 ceph-mon[21984]: Deploying daemon mds.b on smithi099 2024-08-20T03:00:07.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:07 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.44:6840/370123956,v1:172.21.15.44:6841/370123956] up:boot 2024-08-20T03:00:07.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:07 smithi099 ceph-mon[21984]: fsmap 11 up:standby 2024-08-20T03:00:07.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:07 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-20T03:00:07.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:07 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2926616822' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:00:07.815 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:00:07 smithi099 systemd[1]: Starting Ceph mds.b for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:00:08.096 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:00:07 smithi099 podman[53736]: 2024-08-20 03:00:07.716006284 +0000 UTC m=+0.015624066 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:00:08.097 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:00:07 smithi099 podman[53736]: 2024-08-20 03:00:07.842497768 +0000 UTC m=+0.142115597 container create 2a1257101341a66597c6096b9e1f16659ec661031c76f2b1b3654c953ba2abc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:00:08.097 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:00:08 smithi099 podman[53736]: 2024-08-20 03:00:08.09520585 +0000 UTC m=+0.394823621 container init 2a1257101341a66597c6096b9e1f16659ec661031c76f2b1b3654c953ba2abc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:00:08.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:08 smithi099 ceph-mon[21984]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:08.415 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:00:08 smithi099 podman[53736]: 2024-08-20 03:00:08.098116799 +0000 UTC m=+0.397734570 container start 2a1257101341a66597c6096b9e1f16659ec661031c76f2b1b3654c953ba2abc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-08-20T03:00:08.415 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:00:08 smithi099 ceph-mds[53755]: -- 172.21.15.99:0/421340561 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564736319860 con 0x5647362f7000 2024-08-20T03:00:08.415 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:00:08 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-b[53751]: starting mds.b at 2024-08-20T03:00:08.415 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:00:08 smithi099 bash[53736]: 2a1257101341a66597c6096b9e1f16659ec661031c76f2b1b3654c953ba2abc1 2024-08-20T03:00:08.415 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:00:08 smithi099 systemd[1]: Started Ceph mds.b for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T03:00:08.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:08 smithi044 ceph-mon[18790]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:08.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:08 smithi165 ceph-mon[22357]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:09.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:09 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] up:boot 2024-08-20T03:00:09.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:09 smithi044 ceph-mon[18790]: fsmap 12 up:standby 2024-08-20T03:00:09.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T03:00:09.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:00:09.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:09 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.343 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:00:09.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:09 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] up:boot 2024-08-20T03:00:09.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:09 smithi165 ceph-mon[22357]: fsmap 12 up:standby 2024-08-20T03:00:09.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T03:00:09.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:00:09.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:09 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:09 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] up:boot 2024-08-20T03:00:09.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:09 smithi099 ceph-mon[21984]: fsmap 12 up:standby 2024-08-20T03:00:09.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T03:00:09.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:00:09.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:09 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:09.623 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.127+0000 7fe0cc508640 1 -- 172.21.15.44:0/3370225418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c41084e0 msgr2=0x7fe0c4108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.127+0000 7fe0cc508640 1 --2- 172.21.15.44:0/3370225418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c41084e0 0x7fe0c4108920 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009900 tx=0x7fe0b402a410 comp rx=0 tx=0).stop 2024-08-20T03:00:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.127+0000 7fe0cc508640 1 -- 172.21.15.44:0/3370225418 shutdown_connections 2024-08-20T03:00:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.127+0000 7fe0cc508640 1 --2- 172.21.15.44:0/3370225418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0c4108e60 0x7fe0c410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.127+0000 7fe0cc508640 1 --2- 172.21.15.44:0/3370225418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c41084e0 0x7fe0c4108920 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.127+0000 7fe0cc508640 1 --2- 172.21.15.44:0/3370225418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c4102ad0 0x7fe0c4102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.127+0000 7fe0cc508640 1 -- 172.21.15.44:0/3370225418 >> 172.21.15.44:0/3370225418 conn(0x7fe0c40fe540 msgr2=0x7fe0c4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.127+0000 7fe0cc508640 1 -- 172.21.15.44:0/3370225418 shutdown_connections 2024-08-20T03:00:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.127+0000 7fe0cc508640 1 -- 172.21.15.44:0/3370225418 wait complete. 2024-08-20T03:00:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.127+0000 7fe0cc508640 1 Processor -- start 2024-08-20T03:00:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0cc508640 1 -- start start 2024-08-20T03:00:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0cc508640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0c4102ad0 0x7fe0c4198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0cc508640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c41084e0 0x7fe0c4199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0cc508640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c4108e60 0x7fe0c419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0cc508640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4112ae0 con 0x7fe0c4108e60 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0cc508640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4112960 con 0x7fe0c41084e0 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0cc508640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4112c60 con 0x7fe0c4102ad0 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0caaa5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c4108e60 0x7fe0c419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0caaa5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c4108e60 0x7fe0c419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47780/0 (socket says 172.21.15.44:47780) 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0caaa5640 1 -- 172.21.15.44:0/2800637812 learned_addr learned my addr 172.21.15.44:0/2800637812 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0c9aa3640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c41084e0 0x7fe0c4199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0ca2a4640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0c4102ad0 0x7fe0c4198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0caaa5640 1 -- 172.21.15.44:0/2800637812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0c4102ad0 msgr2=0x7fe0c4198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0caaa5640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0c4102ad0 0x7fe0c4198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0caaa5640 1 -- 172.21.15.44:0/2800637812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c41084e0 msgr2=0x7fe0c4199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0caaa5640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c41084e0 0x7fe0c4199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0caaa5640 1 -- 172.21.15.44:0/2800637812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c419f080 con 0x7fe0c4108e60 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0caaa5640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c4108e60 0x7fe0c419e8b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00eaa0 tx=0x7fe0bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.128+0000 7fe0bb7fe640 1 -- 172.21.15.44:0/2800637812 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc00cdb0 con 0x7fe0c4108e60 2024-08-20T03:00:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.129+0000 7fe0bb7fe640 1 -- 172.21.15.44:0/2800637812 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0bc005020 con 0x7fe0c4108e60 2024-08-20T03:00:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.129+0000 7fe0bb7fe640 1 -- 172.21.15.44:0/2800637812 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0116d0 con 0x7fe0c4108e60 2024-08-20T03:00:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.129+0000 7fe0cc508640 1 -- 172.21.15.44:0/2800637812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c419f370 con 0x7fe0c4108e60 2024-08-20T03:00:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.129+0000 7fe0cc508640 1 -- 172.21.15.44:0/2800637812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c41a5c00 con 0x7fe0c4108e60 2024-08-20T03:00:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.130+0000 7fe0cc508640 1 -- 172.21.15.44:0/2800637812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c4102eb0 con 0x7fe0c4108e60 2024-08-20T03:00:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.131+0000 7fe0bb7fe640 1 -- 172.21.15.44:0/2800637812 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fe0bc0051d0 con 0x7fe0c4108e60 2024-08-20T03:00:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.131+0000 7fe0bb7fe640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0b00b3ca0 0x7fe0b00b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.132+0000 7fe0ca2a4640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0b00b3ca0 0x7fe0b00b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.132+0000 7fe0bb7fe640 1 -- 172.21.15.44:0/2800637812 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fe0bc0e3870 con 0x7fe0c4108e60 2024-08-20T03:00:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.132+0000 7fe0ca2a4640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0b00b3ca0 0x7fe0b00b6160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0005d00 tx=0x7fe0c000b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.136+0000 7fe0bb7fe640 1 -- 172.21.15.44:0/2800637812 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe0bc016030 con 0x7fe0c4108e60 2024-08-20T03:00:10.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.367+0000 7fe0cc508640 1 -- 172.21.15.44:0/2800637812 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0c410ac40 con 0x7fe0c4108e60 2024-08-20T03:00:10.371 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:10 smithi044 ceph-mon[18790]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:10.371 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:10.371 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:10.371 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:10.371 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:10 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:10.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.368+0000 7fe0bb7fe640 1 -- 172.21.15.44:0/2800637812 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10515 (secure 0 0 0) 0x7fe0bc0aac40 con 0x7fe0c4108e60 2024-08-20T03:00:10.372 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:10.372 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":13,"btime":"2024-08-20T03:00:08:133832+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14808,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24521,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24550,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24557,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24569,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24604,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-20T03:00:10.373 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 13 2024-08-20T03:00:10.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.371+0000 7fe0cc508640 1 -- 172.21.15.44:0/2800637812 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0b00b3ca0 msgr2=0x7fe0b00b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:10.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.371+0000 7fe0cc508640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0b00b3ca0 0x7fe0b00b6160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0005d00 tx=0x7fe0c000b410 comp rx=0 tx=0).stop 2024-08-20T03:00:10.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.371+0000 7fe0cc508640 1 -- 172.21.15.44:0/2800637812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c4108e60 msgr2=0x7fe0c419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.371+0000 7fe0cc508640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c4108e60 0x7fe0c419e8b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00eaa0 tx=0x7fe0bc00ef60 comp rx=0 tx=0).stop 2024-08-20T03:00:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.371+0000 7fe0cc508640 1 -- 172.21.15.44:0/2800637812 shutdown_connections 2024-08-20T03:00:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.371+0000 7fe0cc508640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0b00b3ca0 0x7fe0b00b6160 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.371+0000 7fe0cc508640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c4108e60 0x7fe0c419e8b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.371+0000 7fe0cc508640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c41084e0 0x7fe0c4199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.371+0000 7fe0cc508640 1 --2- 172.21.15.44:0/2800637812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0c4102ad0 0x7fe0c4198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.371+0000 7fe0cc508640 1 -- 172.21.15.44:0/2800637812 >> 172.21.15.44:0/2800637812 conn(0x7fe0c40fe540 msgr2=0x7fe0c4100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.371+0000 7fe0cc508640 1 -- 172.21.15.44:0/2800637812 shutdown_connections 2024-08-20T03:00:10.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:10.371+0000 7fe0cc508640 1 -- 172.21.15.44:0/2800637812 wait complete. 2024-08-20T03:00:10.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:10 smithi099 ceph-mon[21984]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:10.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:10.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:10.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:10.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:10 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:10.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:10 smithi165 ceph-mon[22357]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:10.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:10.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:10.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:10.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:10 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:11.090 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-08-20T03:00:11.606 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:11.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:11.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:11 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2800637812' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:00:11.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:11 smithi099 ceph-mon[21984]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:11.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:11.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:00:11.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:11.607 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:11 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:00:11.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:11.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:11.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:11 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2800637812' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:00:11.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:11 smithi165 ceph-mon[22357]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:11.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:11.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:00:11.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:11.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:11 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:00:11.662 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:11.663 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:11.663 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:11 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2800637812' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:00:11.663 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:11 smithi044 ceph-mon[18790]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:11.663 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:11.663 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:00:11.664 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:11.664 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:11 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:00:11.933 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:12.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.418+0000 7fa8e2acc640 1 -- 172.21.15.44:0/1969640982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc104300 msgr2=0x7fa8dc10c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:12.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.419+0000 7fa8e2acc640 1 --2- 172.21.15.44:0/1969640982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc104300 0x7fa8dc10c7f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009ca0 tx=0x7fa8cc02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:12.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.419+0000 7fa8e2acc640 1 -- 172.21.15.44:0/1969640982 shutdown_connections 2024-08-20T03:00:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.419+0000 7fa8e2acc640 1 --2- 172.21.15.44:0/1969640982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8dc10ce40 0x7fa8dc10f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.419+0000 7fa8e2acc640 1 --2- 172.21.15.44:0/1969640982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc104300 0x7fa8dc10c7f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.419+0000 7fa8e2acc640 1 --2- 172.21.15.44:0/1969640982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8dc1039e0 0x7fa8dc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.419+0000 7fa8e2acc640 1 -- 172.21.15.44:0/1969640982 >> 172.21.15.44:0/1969640982 conn(0x7fa8dc0fd440 msgr2=0x7fa8dc0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.419+0000 7fa8e2acc640 1 -- 172.21.15.44:0/1969640982 shutdown_connections 2024-08-20T03:00:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.419+0000 7fa8e2acc640 1 -- 172.21.15.44:0/1969640982 wait complete. 2024-08-20T03:00:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.419+0000 7fa8e2acc640 1 Processor -- start 2024-08-20T03:00:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.419+0000 7fa8e2acc640 1 -- start start 2024-08-20T03:00:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8e2acc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8dc1039e0 0x7fa8dc101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8e2acc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc104300 0x7fa8dc101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8e2acc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8dc10ce40 0x7fa8dc102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:12.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8e2acc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc112ab0 con 0x7fa8dc104300 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8e2acc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc112930 con 0x7fa8dc1039e0 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8e2acc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc112c30 con 0x7fa8dc10ce40 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8dbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc104300 0x7fa8dc101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8e0868640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8dc1039e0 0x7fa8dc101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8dbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc104300 0x7fa8dc101bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47798/0 (socket says 172.21.15.44:47798) 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8dbfff640 1 -- 172.21.15.44:0/647339621 learned_addr learned my addr 172.21.15.44:0/647339621 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8e1069640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8dc10ce40 0x7fa8dc102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8dbfff640 1 -- 172.21.15.44:0/647339621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8dc10ce40 msgr2=0x7fa8dc102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8dbfff640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8dc10ce40 0x7fa8dc102110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8dbfff640 1 -- 172.21.15.44:0/647339621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8dc1039e0 msgr2=0x7fa8dc101690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8dbfff640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8dc1039e0 0x7fa8dc101690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8dbfff640 1 -- 172.21.15.44:0/647339621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8dc1a7400 con 0x7fa8dc104300 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8e1069640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8dc10ce40 0x7fa8dc102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:00:12.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8dbfff640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc104300 0x7fa8dc101bd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc02ade0 tx=0x7fa8cc031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:12.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8d9ffb640 1 -- 172.21.15.44:0/647339621 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc003b70 con 0x7fa8dc104300 2024-08-20T03:00:12.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8d9ffb640 1 -- 172.21.15.44:0/647339621 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8cc003d10 con 0x7fa8dc104300 2024-08-20T03:00:12.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.420+0000 7fa8e2acc640 1 -- 172.21.15.44:0/647339621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8dc1a7690 con 0x7fa8dc104300 2024-08-20T03:00:12.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.421+0000 7fa8d9ffb640 1 -- 172.21.15.44:0/647339621 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc02fa50 con 0x7fa8dc104300 2024-08-20T03:00:12.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.421+0000 7fa8e2acc640 1 -- 172.21.15.44:0/647339621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8dc1a7bd0 con 0x7fa8dc104300 2024-08-20T03:00:12.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.422+0000 7fa8e2acc640 1 -- 172.21.15.44:0/647339621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8dc103dc0 con 0x7fa8dc104300 2024-08-20T03:00:12.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.423+0000 7fa8d9ffb640 1 -- 172.21.15.44:0/647339621 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fa8cc004580 con 0x7fa8dc104300 2024-08-20T03:00:12.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.424+0000 7fa8d9ffb640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8c80b3c20 0x7fa8c80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:12.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.424+0000 7fa8e0868640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8c80b3c20 0x7fa8c80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:12.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.424+0000 7fa8d9ffb640 1 -- 172.21.15.44:0/647339621 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fa8cc104f20 con 0x7fa8dc104300 2024-08-20T03:00:12.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.424+0000 7fa8e0868640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8c80b3c20 0x7fa8c80b60e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4005d00 tx=0x7fa8c400b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:12.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.427+0000 7fa8d9ffb640 1 -- 172.21.15.44:0/647339621 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa8cc0cc8a0 con 0x7fa8dc104300 2024-08-20T03:00:12.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.636+0000 7fa8e2acc640 1 -- 172.21.15.44:0/647339621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa8dc102990 con 0x7fa8dc104300 2024-08-20T03:00:12.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.636+0000 7fa8d9ffb640 1 -- 172.21.15.44:0/647339621 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7fa8cc0cc1a0 con 0x7fa8dc104300 2024-08-20T03:00:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:12.640 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-08-20T03:00:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:00:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-08-20T03:00:12.640 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-08-20T03:00:12.641 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:00:12.641 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-08-20T03:00:12.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.639+0000 7fa8e2acc640 1 -- 172.21.15.44:0/647339621 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8c80b3c20 msgr2=0x7fa8c80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:12.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.639+0000 7fa8e2acc640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8c80b3c20 0x7fa8c80b60e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4005d00 tx=0x7fa8c400b410 comp rx=0 tx=0).stop 2024-08-20T03:00:12.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.639+0000 7fa8e2acc640 1 -- 172.21.15.44:0/647339621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc104300 msgr2=0x7fa8dc101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.639+0000 7fa8e2acc640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc104300 0x7fa8dc101bd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc02ade0 tx=0x7fa8cc031770 comp rx=0 tx=0).stop 2024-08-20T03:00:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.639+0000 7fa8e2acc640 1 -- 172.21.15.44:0/647339621 shutdown_connections 2024-08-20T03:00:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.639+0000 7fa8e2acc640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8c80b3c20 0x7fa8c80b60e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.639+0000 7fa8e2acc640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8dc10ce40 0x7fa8dc102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.639+0000 7fa8e2acc640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8dc104300 0x7fa8dc101bd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.639+0000 7fa8e2acc640 1 --2- 172.21.15.44:0/647339621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8dc1039e0 0x7fa8dc101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.639+0000 7fa8e2acc640 1 -- 172.21.15.44:0/647339621 >> 172.21.15.44:0/647339621 conn(0x7fa8dc0fd440 msgr2=0x7fa8dc0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.639+0000 7fa8e2acc640 1 -- 172.21.15.44:0/647339621 shutdown_connections 2024-08-20T03:00:12.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:12.639+0000 7fa8e2acc640 1 -- 172.21.15.44:0/647339621 wait complete. 2024-08-20T03:00:13.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:13 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/647339621' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-20T03:00:13.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:13.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:00:13.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.400 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-20T03:00:13.400 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-20T03:00:13.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:13 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/647339621' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-20T03:00:13.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:13.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:00:13.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.651 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:13.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:13 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/647339621' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-20T03:00:13.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:13.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:00:13.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:00:13.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:00:14.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.153+0000 7f2da45bf640 1 -- 172.21.15.44:0/1175379671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d9c075b10 msgr2=0x7f2d9c10eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:14.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.153+0000 7f2da45bf640 1 --2- 172.21.15.44:0/1175379671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d9c075b10 0x7f2d9c10eb80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009900 tx=0x7f2d8c02a410 comp rx=0 tx=0).stop 2024-08-20T03:00:14.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.153+0000 7f2da45bf640 1 -- 172.21.15.44:0/1175379671 shutdown_connections 2024-08-20T03:00:14.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.153+0000 7f2da45bf640 1 --2- 172.21.15.44:0/1175379671 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d9c10f0c0 0x7f2d9c1114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:14.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.153+0000 7f2da45bf640 1 --2- 172.21.15.44:0/1175379671 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d9c075b10 0x7f2d9c10eb80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:14.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.153+0000 7f2da45bf640 1 --2- 172.21.15.44:0/1175379671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d9c0751f0 0x7f2d9c0755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:14.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.153+0000 7f2da45bf640 1 -- 172.21.15.44:0/1175379671 >> 172.21.15.44:0/1175379671 conn(0x7f2d9c0fe500 msgr2=0x7f2d9c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:14.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.153+0000 7f2da45bf640 1 -- 172.21.15.44:0/1175379671 shutdown_connections 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.153+0000 7f2da45bf640 1 -- 172.21.15.44:0/1175379671 wait complete. 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.154+0000 7f2da45bf640 1 Processor -- start 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.154+0000 7f2da45bf640 1 -- start start 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.154+0000 7f2da45bf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d9c0751f0 0x7f2d9c19b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.154+0000 7f2da45bf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d9c075b10 0x7f2d9c19b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.154+0000 7f2da45bf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d9c10f0c0 0x7f2d9c1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.154+0000 7f2da45bf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c114d00 con 0x7f2d9c075b10 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.154+0000 7f2da45bf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c114b80 con 0x7f2d9c10f0c0 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.154+0000 7f2da45bf640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c114e80 con 0x7f2d9c0751f0 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.154+0000 7f2da1b5a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d9c075b10 0x7f2d9c19b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.154+0000 7f2da1b5a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d9c075b10 0x7f2d9c19b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47812/0 (socket says 172.21.15.44:47812) 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.154+0000 7f2da1b5a640 1 -- 172.21.15.44:0/510211605 learned_addr learned my addr 172.21.15.44:0/510211605 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2da2b5c640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d9c10f0c0 0x7f2d9c1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:14.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2da235b640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d9c0751f0 0x7f2d9c19b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2da1b5a640 1 -- 172.21.15.44:0/510211605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d9c0751f0 msgr2=0x7f2d9c19b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2da1b5a640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d9c0751f0 0x7f2d9c19b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2da1b5a640 1 -- 172.21.15.44:0/510211605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d9c10f0c0 msgr2=0x7f2d9c1a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2da1b5a640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d9c10f0c0 0x7f2d9c1a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2da1b5a640 1 -- 172.21.15.44:0/510211605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9c1a1300 con 0x7f2d9c075b10 2024-08-20T03:00:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2da1b5a640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d9c075b10 0x7f2d9c19b790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c005940 tx=0x7f2d8c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2d937fe640 1 -- 172.21.15.44:0/510211605 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c03f030 con 0x7f2d9c075b10 2024-08-20T03:00:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2da45bf640 1 -- 172.21.15.44:0/510211605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9c1a1590 con 0x7f2d9c075b10 2024-08-20T03:00:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2d937fe640 1 -- 172.21.15.44:0/510211605 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d8c003aa0 con 0x7f2d9c075b10 2024-08-20T03:00:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2d937fe640 1 -- 172.21.15.44:0/510211605 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c035070 con 0x7f2d9c075b10 2024-08-20T03:00:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.155+0000 7f2da45bf640 1 -- 172.21.15.44:0/510211605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9c1a7cb0 con 0x7f2d9c075b10 2024-08-20T03:00:14.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.156+0000 7f2da45bf640 1 -- 172.21.15.44:0/510211605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d9c0755d0 con 0x7f2d9c075b10 2024-08-20T03:00:14.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.157+0000 7f2d937fe640 1 -- 172.21.15.44:0/510211605 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f2d8c042030 con 0x7f2d9c075b10 2024-08-20T03:00:14.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.158+0000 7f2d937fe640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2d880b3c20 0x7f2d880b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:14.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.158+0000 7f2da235b640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2d880b3c20 0x7f2d880b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:14.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.159+0000 7f2d937fe640 1 -- 172.21.15.44:0/510211605 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f2d8c103b30 con 0x7f2d9c075b10 2024-08-20T03:00:14.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.159+0000 7f2da235b640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2d880b3c20 0x7f2d880b60e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2d98005fd0 tx=0x7f2d98005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:14.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.162+0000 7f2d937fe640 1 -- 172.21.15.44:0/510211605 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2d8c0cb4b0 con 0x7f2d9c075b10 2024-08-20T03:00:14.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:14.349+0000 7f2da45bf640 1 -- 172.21.15.44:0/510211605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2d9c1a7fd0 con 0x7f2d9c075b10 2024-08-20T03:00:14.352 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:14 smithi044 ceph-mon[18790]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:14.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:14 smithi099 ceph-mon[21984]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:14.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:14 smithi165 ceph-mon[22357]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:15.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.200+0000 7f2d937fe640 1 -- 172.21.15.44:0/510211605 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f2d8c0cadb0 con 0x7f2d9c075b10 2024-08-20T03:00:15.204 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-08-20T03:00:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.204+0000 7f2da45bf640 1 -- 172.21.15.44:0/510211605 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2d880b3c20 msgr2=0x7f2d880b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.204+0000 7f2da45bf640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2d880b3c20 0x7f2d880b60e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2d98005fd0 tx=0x7f2d98005850 comp rx=0 tx=0).stop 2024-08-20T03:00:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.204+0000 7f2da45bf640 1 -- 172.21.15.44:0/510211605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d9c075b10 msgr2=0x7f2d9c19b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.204+0000 7f2da45bf640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d9c075b10 0x7f2d9c19b790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c005940 tx=0x7f2d8c004270 comp rx=0 tx=0).stop 2024-08-20T03:00:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.204+0000 7f2da45bf640 1 -- 172.21.15.44:0/510211605 shutdown_connections 2024-08-20T03:00:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.204+0000 7f2da45bf640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2d880b3c20 0x7f2d880b60e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.204+0000 7f2da45bf640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d9c10f0c0 0x7f2d9c1a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.204+0000 7f2da45bf640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d9c075b10 0x7f2d9c19b790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:15.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.204+0000 7f2da45bf640 1 --2- 172.21.15.44:0/510211605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d9c0751f0 0x7f2d9c19b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:15.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.204+0000 7f2da45bf640 1 -- 172.21.15.44:0/510211605 >> 172.21.15.44:0/510211605 conn(0x7f2d9c0fe500 msgr2=0x7f2d9c0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:15.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.204+0000 7f2da45bf640 1 -- 172.21.15.44:0/510211605 shutdown_connections 2024-08-20T03:00:15.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:15.204+0000 7f2da45bf640 1 -- 172.21.15.44:0/510211605 wait complete. 2024-08-20T03:00:15.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:15 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/510211605' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-20T03:00:15.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:15 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/510211605' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-20T03:00:15.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:15 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/510211605' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-20T03:00:15.891 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-20T03:00:16.152 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:16.215 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:16 smithi044 ceph-mon[18790]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:16.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:16 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/510211605' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-20T03:00:16.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:16 smithi044 ceph-mon[18790]: osdmap e71: 12 total, 12 up, 12 in 2024-08-20T03:00:16.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:16 smithi165 ceph-mon[22357]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:16.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:16 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/510211605' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-20T03:00:16.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:16 smithi165 ceph-mon[22357]: osdmap e71: 12 total, 12 up, 12 in 2024-08-20T03:00:16.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:16 smithi099 ceph-mon[21984]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:16.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:16 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/510211605' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-20T03:00:16.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:16 smithi099 ceph-mon[21984]: osdmap e71: 12 total, 12 up, 12 in 2024-08-20T03:00:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.670+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1765503679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b0810cf60 msgr2=0x7f0b0810f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.670+0000 7f0b0fc1a640 1 --2- 172.21.15.44:0/1765503679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b0810cf60 0x7f0b0810f350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009ca0 tx=0x7f0b0002a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.670+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1765503679 shutdown_connections 2024-08-20T03:00:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.670+0000 7f0b0fc1a640 1 --2- 172.21.15.44:0/1765503679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b0810cf60 0x7f0b0810f350 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.670+0000 7f0b0fc1a640 1 --2- 172.21.15.44:0/1765503679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b0810a630 0x7f0b0810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.670+0000 7f0b0fc1a640 1 --2- 172.21.15.44:0/1765503679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b08101520 0x7f0b08101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.670+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1765503679 >> 172.21.15.44:0/1765503679 conn(0x7f0b08076c80 msgr2=0x7f0b08077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.670+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1765503679 shutdown_connections 2024-08-20T03:00:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.670+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1765503679 wait complete. 2024-08-20T03:00:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0fc1a640 1 Processor -- start 2024-08-20T03:00:16.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0fc1a640 1 -- start start 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0fc1a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b08101520 0x7f0b0819b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0fc1a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b0810a630 0x7f0b0819b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0fc1a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0810cf60 0x7f0b081a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0fc1a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0b08112aa0 con 0x7f0b08101520 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0fc1a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b08112920 con 0x7f0b0810cf60 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0fc1a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0b08112c20 con 0x7f0b0810a630 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0d9b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b08101520 0x7f0b0819b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0d9b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b08101520 0x7f0b0819b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47844/0 (socket says 172.21.15.44:47844) 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0d9b6640 1 -- 172.21.15.44:0/1494865249 learned_addr learned my addr 172.21.15.44:0/1494865249 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0d1b5640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b0810a630 0x7f0b0819b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.671+0000 7f0b0e1b7640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0810cf60 0x7f0b081a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.672+0000 7f0b0d9b6640 1 -- 172.21.15.44:0/1494865249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b0810a630 msgr2=0x7f0b0819b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:16.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.672+0000 7f0b0d9b6640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b0810a630 0x7f0b0819b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:16.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.672+0000 7f0b0d9b6640 1 -- 172.21.15.44:0/1494865249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0810cf60 msgr2=0x7f0b081a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:16.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.672+0000 7f0b0d9b6640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0810cf60 0x7f0b081a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:16.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.672+0000 7f0b0d9b6640 1 -- 172.21.15.44:0/1494865249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b081a1290 con 0x7f0b08101520 2024-08-20T03:00:16.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.672+0000 7f0b0d9b6640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b08101520 0x7f0b0819b1e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400d950 tx=0x7f0b0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:16.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.672+0000 7f0afeffd640 1 -- 172.21.15.44:0/1494865249 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04017070 con 0x7f0b08101520 2024-08-20T03:00:16.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.672+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1494865249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b081a1580 con 0x7f0b08101520 2024-08-20T03:00:16.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.672+0000 7f0afeffd640 1 -- 172.21.15.44:0/1494865249 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b04009c30 con 0x7f0b08101520 2024-08-20T03:00:16.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.672+0000 7f0afeffd640 1 -- 172.21.15.44:0/1494865249 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0400f620 con 0x7f0b08101520 2024-08-20T03:00:16.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.672+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1494865249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b081a7e10 con 0x7f0b08101520 2024-08-20T03:00:16.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.673+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1494865249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b08101900 con 0x7f0b08101520 2024-08-20T03:00:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.675+0000 7f0afeffd640 1 -- 172.21.15.44:0/1494865249 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f0b04010460 con 0x7f0b08101520 2024-08-20T03:00:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.675+0000 7f0afeffd640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0af40b3c20 0x7f0af40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.675+0000 7f0b0d1b5640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0af40b3c20 0x7f0af40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.676+0000 7f0afeffd640 1 -- 172.21.15.44:0/1494865249 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7712+0+0 (secure 0 0 0) 0x7f0b040e3570 con 0x7f0b08101520 2024-08-20T03:00:16.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.676+0000 7f0b0d1b5640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0af40b3c20 0x7f0af40b60e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b0810c170 tx=0x7f0af8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:16.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.679+0000 7f0afeffd640 1 -- 172.21.15.44:0/1494865249 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0b04009de0 con 0x7f0b08101520 2024-08-20T03:00:16.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:16.864+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1494865249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0b081a8010 con 0x7f0b08101520 2024-08-20T03:00:17.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.217+0000 7f0afeffd640 1 -- 172.21.15.44:0/1494865249 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7f0b040b30f0 con 0x7f0b08101520 2024-08-20T03:00:17.220 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-08-20T03:00:17.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.219+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1494865249 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0af40b3c20 msgr2=0x7f0af40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:17.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.220+0000 7f0b0fc1a640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0af40b3c20 0x7f0af40b60e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b0810c170 tx=0x7f0af8005f90 comp rx=0 tx=0).stop 2024-08-20T03:00:17.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.220+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1494865249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b08101520 msgr2=0x7f0b0819b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:17.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.220+0000 7f0b0fc1a640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b08101520 0x7f0b0819b1e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400d950 tx=0x7f0b0400de10 comp rx=0 tx=0).stop 2024-08-20T03:00:17.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.220+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1494865249 shutdown_connections 2024-08-20T03:00:17.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.220+0000 7f0b0fc1a640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0af40b3c20 0x7f0af40b60e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:17.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.220+0000 7f0b0fc1a640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b0810cf60 0x7f0b081a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:17.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.220+0000 7f0b0fc1a640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b0810a630 0x7f0b0819b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:17.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.220+0000 7f0b0fc1a640 1 --2- 172.21.15.44:0/1494865249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b08101520 0x7f0b0819b1e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:17.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.220+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1494865249 >> 172.21.15.44:0/1494865249 conn(0x7f0b08076c80 msgr2=0x7f0b081089f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:17.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.220+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1494865249 shutdown_connections 2024-08-20T03:00:17.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:17.220+0000 7f0b0fc1a640 1 -- 172.21.15.44:0/1494865249 wait complete. 2024-08-20T03:00:17.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:17 smithi044 ceph-mon[18790]: osdmap e72: 12 total, 12 up, 12 in 2024-08-20T03:00:17.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:17 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1494865249' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-20T03:00:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:17 smithi165 ceph-mon[22357]: osdmap e72: 12 total, 12 up, 12 in 2024-08-20T03:00:17.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:17 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1494865249' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-20T03:00:17.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:17 smithi099 ceph-mon[21984]: osdmap e72: 12 total, 12 up, 12 in 2024-08-20T03:00:17.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:17 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1494865249' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-20T03:00:17.935 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-20T03:00:18.197 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:18.487 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:18 smithi044 ceph-mon[18790]: pgmap v249: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:18.487 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:18 smithi044 ceph-mon[18790]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-20T03:00:18.487 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:18 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1494865249' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-20T03:00:18.487 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:18 smithi044 ceph-mon[18790]: osdmap e73: 12 total, 12 up, 12 in 2024-08-20T03:00:18.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:18 smithi165 ceph-mon[22357]: pgmap v249: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:18.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:18 smithi165 ceph-mon[22357]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-20T03:00:18.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:18 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1494865249' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-20T03:00:18.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:18 smithi165 ceph-mon[22357]: osdmap e73: 12 total, 12 up, 12 in 2024-08-20T03:00:18.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:18 smithi099 ceph-mon[21984]: pgmap v249: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:18.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:18 smithi099 ceph-mon[21984]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-20T03:00:18.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:18 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1494865249' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-20T03:00:18.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:18 smithi099 ceph-mon[21984]: osdmap e73: 12 total, 12 up, 12 in 2024-08-20T03:00:18.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.701+0000 7fa97d0db640 1 -- 172.21.15.44:0/3585390163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9781039c0 msgr2=0x7fa978103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:18.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.701+0000 7fa97d0db640 1 --2- 172.21.15.44:0/3585390163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9781039c0 0x7fa978103da0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa968009ca0 tx=0x7fa96802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:18.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.701+0000 7fa97d0db640 1 -- 172.21.15.44:0/3585390163 shutdown_connections 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.701+0000 7fa97d0db640 1 --2- 172.21.15.44:0/3585390163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa97810ce20 0x7fa97810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.701+0000 7fa97d0db640 1 --2- 172.21.15.44:0/3585390163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9781042e0 0x7fa97810c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.701+0000 7fa97d0db640 1 --2- 172.21.15.44:0/3585390163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9781039c0 0x7fa978103da0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.701+0000 7fa97d0db640 1 -- 172.21.15.44:0/3585390163 >> 172.21.15.44:0/3585390163 conn(0x7fa9780fd400 msgr2=0x7fa9780ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.701+0000 7fa97d0db640 1 -- 172.21.15.44:0/3585390163 shutdown_connections 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.701+0000 7fa97d0db640 1 -- 172.21.15.44:0/3585390163 wait complete. 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.702+0000 7fa97d0db640 1 Processor -- start 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.702+0000 7fa97d0db640 1 -- start start 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa97d0db640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9781039c0 0x7fa978199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa97d0db640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9781042e0 0x7fa9781997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa97d0db640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa97810ce20 0x7fa97819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa97d0db640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa978112a90 con 0x7fa9781039c0 2024-08-20T03:00:18.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa97d0db640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa978112910 con 0x7fa97810ce20 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa97d0db640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa978112c10 con 0x7fa9781042e0 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa976d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9781039c0 0x7fa978199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa976d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9781039c0 0x7fa978199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45080/0 (socket says 172.21.15.44:45080) 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa976d9d640 1 -- 172.21.15.44:0/259292822 learned_addr learned my addr 172.21.15.44:0/259292822 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa97659c640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9781042e0 0x7fa9781997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa97759e640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa97810ce20 0x7fa97819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa976d9d640 1 -- 172.21.15.44:0/259292822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9781042e0 msgr2=0x7fa9781997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa976d9d640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9781042e0 0x7fa9781997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa976d9d640 1 -- 172.21.15.44:0/259292822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa97810ce20 msgr2=0x7fa97819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa976d9d640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa97810ce20 0x7fa97819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa976d9d640 1 -- 172.21.15.44:0/259292822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97819f330 con 0x7fa9781039c0 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa976d9d640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9781039c0 0x7fa978199280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa96802ade0 tx=0x7fa968031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:18.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa96ffff640 1 -- 172.21.15.44:0/259292822 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968003b70 con 0x7fa9781039c0 2024-08-20T03:00:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa96ffff640 1 -- 172.21.15.44:0/259292822 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa968003d10 con 0x7fa9781039c0 2024-08-20T03:00:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa97d0db640 1 -- 172.21.15.44:0/259292822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97819f5c0 con 0x7fa9781039c0 2024-08-20T03:00:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.703+0000 7fa96ffff640 1 -- 172.21.15.44:0/259292822 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96802fa50 con 0x7fa9781039c0 2024-08-20T03:00:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.704+0000 7fa97d0db640 1 -- 172.21.15.44:0/259292822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9781a5ce0 con 0x7fa9781039c0 2024-08-20T03:00:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.704+0000 7fa97d0db640 1 -- 172.21.15.44:0/259292822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa978103da0 con 0x7fa9781039c0 2024-08-20T03:00:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.706+0000 7fa96ffff640 1 -- 172.21.15.44:0/259292822 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fa968004580 con 0x7fa9781039c0 2024-08-20T03:00:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.706+0000 7fa96ffff640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa9640b3c20 0x7fa9640b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.707+0000 7fa97659c640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa9640b3c20 0x7fa9640b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.707+0000 7fa96ffff640 1 -- 172.21.15.44:0/259292822 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8094+0+0 (secure 0 0 0) 0x7fa968105130 con 0x7fa9781039c0 2024-08-20T03:00:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.707+0000 7fa97659c640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa9640b3c20 0x7fa9640b60e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa97810f4c0 tx=0x7fa960005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:18.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.710+0000 7fa96ffff640 1 -- 172.21.15.44:0/259292822 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa9680cc860 con 0x7fa9781039c0 2024-08-20T03:00:18.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:18.934+0000 7fa97d0db640 1 -- 172.21.15.44:0/259292822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa97819a3b0 con 0x7fa9781039c0 2024-08-20T03:00:19.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.236+0000 7fa96ffff640 1 -- 172.21.15.44:0/259292822 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa9680cc160 con 0x7fa9781039c0 2024-08-20T03:00:19.239 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 2 and data pool 3 2024-08-20T03:00:19.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.239+0000 7fa97d0db640 1 -- 172.21.15.44:0/259292822 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa9640b3c20 msgr2=0x7fa9640b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:19.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.239+0000 7fa97d0db640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa9640b3c20 0x7fa9640b60e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa97810f4c0 tx=0x7fa960005f90 comp rx=0 tx=0).stop 2024-08-20T03:00:19.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.239+0000 7fa97d0db640 1 -- 172.21.15.44:0/259292822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9781039c0 msgr2=0x7fa978199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:19.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.239+0000 7fa97d0db640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9781039c0 0x7fa978199280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa96802ade0 tx=0x7fa968031770 comp rx=0 tx=0).stop 2024-08-20T03:00:19.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.239+0000 7fa97d0db640 1 -- 172.21.15.44:0/259292822 shutdown_connections 2024-08-20T03:00:19.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.239+0000 7fa97d0db640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa9640b3c20 0x7fa9640b60e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:19.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.239+0000 7fa97d0db640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa97810ce20 0x7fa97819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:19.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.239+0000 7fa97d0db640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9781042e0 0x7fa9781997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:19.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.239+0000 7fa97d0db640 1 --2- 172.21.15.44:0/259292822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9781039c0 0x7fa978199280 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:19.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.239+0000 7fa97d0db640 1 -- 172.21.15.44:0/259292822 >> 172.21.15.44:0/259292822 conn(0x7fa9780fd400 msgr2=0x7fa9780fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:19.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.239+0000 7fa97d0db640 1 -- 172.21.15.44:0/259292822 shutdown_connections 2024-08-20T03:00:19.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:19.239+0000 7fa97d0db640 1 -- 172.21.15.44:0/259292822 wait complete. 2024-08-20T03:00:19.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:19 smithi044 ceph-mon[18790]: osdmap e74: 12 total, 12 up, 12 in 2024-08-20T03:00:19.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:19 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/259292822' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-20T03:00:19.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:19 smithi044 ceph-mon[18790]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-08-20T03:00:19.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:19 smithi165 ceph-mon[22357]: osdmap e74: 12 total, 12 up, 12 in 2024-08-20T03:00:19.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:19 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/259292822' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-20T03:00:19.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:19 smithi165 ceph-mon[22357]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-08-20T03:00:19.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:19 smithi099 ceph-mon[21984]: osdmap e74: 12 total, 12 up, 12 in 2024-08-20T03:00:19.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:19 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/259292822' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-20T03:00:19.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:19 smithi099 ceph-mon[21984]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-08-20T03:00:19.993 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T03:00:20.251 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:20.517 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:20 smithi044 ceph-mon[18790]: pgmap v252: 97 pgs: 14 creating+peering, 67 unknown, 16 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:20.517 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:20 smithi044 ceph-mon[18790]: osdmap e75: 12 total, 12 up, 12 in 2024-08-20T03:00:20.518 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:20 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/259292822' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-20T03:00:20.518 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:20 smithi044 ceph-mon[18790]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-08-20T03:00:20.518 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:20 smithi044 ceph-mon[18790]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-08-20T03:00:20.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:20 smithi165 ceph-mon[22357]: pgmap v252: 97 pgs: 14 creating+peering, 67 unknown, 16 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:20.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:20 smithi165 ceph-mon[22357]: osdmap e75: 12 total, 12 up, 12 in 2024-08-20T03:00:20.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:20 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/259292822' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-20T03:00:20.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:20 smithi165 ceph-mon[22357]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-08-20T03:00:20.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:20 smithi165 ceph-mon[22357]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-08-20T03:00:20.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:20 smithi099 ceph-mon[21984]: pgmap v252: 97 pgs: 14 creating+peering, 67 unknown, 16 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:20.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:20 smithi099 ceph-mon[21984]: osdmap e75: 12 total, 12 up, 12 in 2024-08-20T03:00:20.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:20 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/259292822' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-20T03:00:20.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:20 smithi099 ceph-mon[21984]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-08-20T03:00:20.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:20 smithi099 ceph-mon[21984]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-08-20T03:00:20.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.764+0000 7f4093f37640 1 -- 172.21.15.44:0/206429494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f408c108e40 msgr2=0x7f408c10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:20.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.764+0000 7f4093f37640 1 --2- 172.21.15.44:0/206429494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f408c108e40 0x7f408c10f640 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4084009ca0 tx=0x7f408402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:20.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.764+0000 7f4093f37640 1 -- 172.21.15.44:0/206429494 shutdown_connections 2024-08-20T03:00:20.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.764+0000 7f4093f37640 1 --2- 172.21.15.44:0/206429494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f408c108e40 0x7f408c10f640 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.764+0000 7f4093f37640 1 --2- 172.21.15.44:0/206429494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f408c1084c0 0x7f408c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.764+0000 7f4093f37640 1 --2- 172.21.15.44:0/206429494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f408c102ab0 0x7f408c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.764+0000 7f4093f37640 1 -- 172.21.15.44:0/206429494 >> 172.21.15.44:0/206429494 conn(0x7f408c0fe500 msgr2=0x7f408c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.764+0000 7f4093f37640 1 -- 172.21.15.44:0/206429494 shutdown_connections 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.764+0000 7f4093f37640 1 -- 172.21.15.44:0/206429494 wait complete. 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.764+0000 7f4093f37640 1 Processor -- start 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.764+0000 7f4093f37640 1 -- start start 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4093f37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f408c102ab0 0x7f408c199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4093f37640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f408c1084c0 0x7f408c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4093f37640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f408c108e40 0x7f408c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4093f37640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f408c112b10 con 0x7f408c102ab0 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4093f37640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f408c112990 con 0x7f408c1084c0 2024-08-20T03:00:20.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4093f37640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f408c112c90 con 0x7f408c108e40 2024-08-20T03:00:20.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4091cd3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f408c102ab0 0x7f408c199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:20.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4091cd3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f408c102ab0 0x7f408c199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45096/0 (socket says 172.21.15.44:45096) 2024-08-20T03:00:20.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4091cd3640 1 -- 172.21.15.44:0/1650071583 learned_addr learned my addr 172.21.15.44:0/1650071583 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:20.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f40914d2640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f408c1084c0 0x7f408c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:20.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f40924d4640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f408c108e40 0x7f408c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:20.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4091cd3640 1 -- 172.21.15.44:0/1650071583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f408c108e40 msgr2=0x7f408c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:20.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4091cd3640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f408c108e40 0x7f408c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:20.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4091cd3640 1 -- 172.21.15.44:0/1650071583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f408c1084c0 msgr2=0x7f408c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:20.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4091cd3640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f408c1084c0 0x7f408c1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:20.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4091cd3640 1 -- 172.21.15.44:0/1650071583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c19f110 con 0x7f408c102ab0 2024-08-20T03:00:20.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f40914d2640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f408c1084c0 0x7f408c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:00:20.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f40924d4640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f408c108e40 0x7f408c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4091cd3640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f408c102ab0 0x7f408c199060 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f408800d950 tx=0x7f408800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4082ffd640 1 -- 172.21.15.44:0/1650071583 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4088017030 con 0x7f408c102ab0 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4093f37640 1 -- 172.21.15.44:0/1650071583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408c19f400 con 0x7f408c102ab0 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4082ffd640 1 -- 172.21.15.44:0/1650071583 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4088004020 con 0x7f408c102ab0 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4082ffd640 1 -- 172.21.15.44:0/1650071583 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40880047a0 con 0x7f408c102ab0 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.765+0000 7f4093f37640 1 -- 172.21.15.44:0/1650071583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408c1a5ac0 con 0x7f408c102ab0 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.767+0000 7f4093f37640 1 -- 172.21.15.44:0/1650071583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f408c102e90 con 0x7f408c102ab0 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.768+0000 7f4082ffd640 1 -- 172.21.15.44:0/1650071583 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f4088009a30 con 0x7f408c102ab0 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.768+0000 7f4082ffd640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40780b3c20 0x7f40780b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.769+0000 7f40914d2640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40780b3c20 0x7f40780b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.769+0000 7f4082ffd640 1 -- 172.21.15.44:0/1650071583 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f40880e3780 con 0x7f408c102ab0 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.769+0000 7f40914d2640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40780b3c20 0x7f40780b60e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f408c1049e0 tx=0x7f407c005cb0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:20.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.773+0000 7f4082ffd640 1 -- 172.21.15.44:0/1650071583 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f40880aadf0 con 0x7f408c102ab0 2024-08-20T03:00:20.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.961+0000 7f4093f37640 1 -- 172.21.15.44:0/1650071583 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f408c19a190 con 0x7f408c102ab0 2024-08-20T03:00:20.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.962+0000 7f4082ffd640 1 -- 172.21.15.44:0/1650071583 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22549 (secure 0 0 0) 0x7f40880aa6f0 con 0x7f408c102ab0 2024-08-20T03:00:20.965 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:20.965 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":75,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","created":"2024-08-20T02:52:10.979264+0000","modified":"2024-08-20T03:00:18.935366+0000","last_up_change":"2024-08-20T02:59:26.404917+0000","last_in_change":"2024-08-20T02:59:03.707380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T02:55:46.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T03:00:14.354568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T03:00:16.869335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7708f8c-a799-4b8f-ab1e-78482d73ca1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6803","nonce":858898316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6805","nonce":858898316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6809","nonce":858898316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6807","nonce":858898316}]},"public_addr":"172.21.15.44:6803/858898316","cluster_addr":"172.21.15.44:6805/858898316","heartbeat_back_addr":"172.21.15.44:6809/858898316","heartbeat_front_addr":"172.21.15.44:6807/858898316","state":["exists","up"]},{"osd":1,"uuid":"12f07986-60bf-498f-a012-cd7ea607b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6801","nonce":4252911663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6803","nonce":4252911663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6807","nonce":4252911663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6805","nonce":4252911663}]},"public_addr":"172.21.15.99:6801/4252911663","cluster_addr":"172.21.15.99:6803/4252911663","heartbeat_back_addr":"172.21.15.99:6807/4252911663","heartbeat_front_addr":"172.21.15.99:6805/4252911663","state":["exists","up"]},{"osd":2,"uuid":"0f189000-ba7f-4412-b0d3-75e8265f6685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6801","nonce":1492740465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6803","nonce":1492740465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6807","nonce":1492740465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6805","nonce":1492740465}]},"public_addr":"172.21.15.165:6801/1492740465","cluster_addr":"172.21.15.165:6803/1492740465","heartbeat_back_addr":"172.21.15.165:6807/1492740465","heartbeat_front_addr":"172.21.15.165:6805/1492740465","state":["exists","up"]},{"osd":3,"uuid":"8f9b042c-d6b5-4eee-aa29-ffc41a48c20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6811","nonce":3423983702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6813","nonce":3423983702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6817","nonce":3423983702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6815","nonce":3423983702}]},"public_addr":"172.21.15.44:6811/3423983702","cluster_addr":"172.21.15.44:6813/3423983702","heartbeat_back_addr":"172.21.15.44:6817/3423983702","heartbeat_front_addr":"172.21.15.44:6815/3423983702","state":["exists","up"]},{"osd":4,"uuid":"4edebfdb-af10-49ee-b724-2a63b9219221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6809","nonce":3099617325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6811","nonce":3099617325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6815","nonce":3099617325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6813","nonce":3099617325}]},"public_addr":"172.21.15.99:6809/3099617325","cluster_addr":"172.21.15.99:6811/3099617325","heartbeat_back_addr":"172.21.15.99:6815/3099617325","heartbeat_front_addr":"172.21.15.99:6813/3099617325","state":["exists","up"]},{"osd":5,"uuid":"6d22c21f-cc43-4ef5-a75c-d4fc1a361c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6809","nonce":523245255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6811","nonce":523245255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6815","nonce":523245255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6813","nonce":523245255}]},"public_addr":"172.21.15.165:6809/523245255","cluster_addr":"172.21.15.165:6811/523245255","heartbeat_back_addr":"172.21.15.165:6815/523245255","heartbeat_front_addr":"172.21.15.165:6813/523245255","state":["exists","up"]},{"osd":6,"uuid":"03df0c2f-1b7c-4242-888c-e047322fd8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6819","nonce":1039098986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6821","nonce":1039098986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6825","nonce":1039098986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6823","nonce":1039098986}]},"public_addr":"172.21.15.44:6819/1039098986","cluster_addr":"172.21.15.44:6821/1039098986","heartbeat_back_addr":"172.21.15.44:6825/1039098986","heartbeat_front_addr":"172.21.15.44:6823/1039098986","state":["exists","up"]},{"osd":7,"uuid":"84bbc683-9b13-448e-897c-114458dae33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6817","nonce":1438613564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6819","nonce":1438613564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6823","nonce":1438613564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6821","nonce":1438613564}]},"public_addr":"172.21.15.99:6817/1438613564","cluster_addr":"172.21.15.99:6819/1438613564","heartbeat_back_addr":"172.21.15.99:6823/1438613564","heartbeat_front_addr":"172.21.15.99:6821/1438613564","state":["exists","up"]},{"osd":8,"uuid":"14d7ffd3-db05-4228-a12e-4c69ef65ef8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6817","nonce":632957688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6819","nonce":632957688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6823","nonce":632957688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6821","nonce":632957688}]},"public_addr":"172.21.15.165:6817/632957688","cluster_addr":"172.21.15.165:6819/632957688","heartbeat_back_addr":"172.21.15.165:6823/632957688","heartbeat_front_addr":"172.21.15.165:6821/632957688","state":["exists","up"]},{"osd":9,"uuid":"1ae468a5-3d72-4eb0-a1a6-cd87eae740b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6827","nonce":2426940093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6829","nonce":2426940093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6833","nonce":2426940093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6831","nonce":2426940093}]},"public_addr":"172.21.15.44:6827/2426940093","cluster_addr":"172.21.15.44:6829/2426940093","heartbeat_back_addr":"172.21.15.44:6833/2426940093","heartbeat_front_addr":"172.21.15.44:6831/2426940093","state":["exists","up"]},{"osd":10,"uuid":"fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6825","nonce":3275793038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6827","nonce":3275793038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6831","nonce":3275793038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6829","nonce":3275793038}]},"public_addr":"172.21.15.99:6825/3275793038","cluster_addr":"172.21.15.99:6827/3275793038","heartbeat_back_addr":"172.21.15.99:6831/3275793038","heartbeat_front_addr":"172.21.15.99:6829/3275793038","state":["exists","up"]},{"osd":11,"uuid":"50715410-1fc2-4818-b600-604a4b7cce9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6825","nonce":107688415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6827","nonce":107688415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6831","nonce":107688415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6829","nonce":107688415}]},"public_addr":"172.21.15.165:6825/107688415","cluster_addr":"172.21.15.165:6827/107688415","heartbeat_back_addr":"172.21.15.165:6831/107688415","heartbeat_front_addr":"172.21.15.165:6829/107688415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:54:56.697075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:20.486846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:43.353376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:08.333879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:32.186492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:55.891366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:19.681421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:45.475926+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:10.109795+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:34.444169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:59.147646+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:59:25.403071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2597827045":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/3582612619":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/1916809746":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2919033419":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:6800/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2331034359":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2572185146":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/4176802819":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/3654439062":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6801/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/2376733275":"2024-08-21T02:53:58.312659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T03:00:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.964+0000 7f4093f37640 1 -- 172.21.15.44:0/1650071583 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40780b3c20 msgr2=0x7f40780b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.964+0000 7f4093f37640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40780b3c20 0x7f40780b60e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f408c1049e0 tx=0x7f407c005cb0 comp rx=0 tx=0).stop 2024-08-20T03:00:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.964+0000 7f4093f37640 1 -- 172.21.15.44:0/1650071583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f408c102ab0 msgr2=0x7f408c199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.964+0000 7f4093f37640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f408c102ab0 0x7f408c199060 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f408800d950 tx=0x7f408800de10 comp rx=0 tx=0).stop 2024-08-20T03:00:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.964+0000 7f4093f37640 1 -- 172.21.15.44:0/1650071583 shutdown_connections 2024-08-20T03:00:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.964+0000 7f4093f37640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40780b3c20 0x7f40780b60e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.964+0000 7f4093f37640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f408c108e40 0x7f408c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.964+0000 7f4093f37640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f408c1084c0 0x7f408c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.964+0000 7f4093f37640 1 --2- 172.21.15.44:0/1650071583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f408c102ab0 0x7f408c199060 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.964+0000 7f4093f37640 1 -- 172.21.15.44:0/1650071583 >> 172.21.15.44:0/1650071583 conn(0x7f408c0fe500 msgr2=0x7f408c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:20.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.965+0000 7f4093f37640 1 -- 172.21.15.44:0/1650071583 shutdown_connections 2024-08-20T03:00:20.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:20.965+0000 7f4093f37640 1 -- 172.21.15.44:0/1650071583 wait complete. 2024-08-20T03:00:21.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:21 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] up:active 2024-08-20T03:00:21.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:21 smithi044 ceph-mon[18790]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-08-20T03:00:21.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:21 smithi044 ceph-mon[18790]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-08-20T03:00:21.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:21 smithi044 ceph-mon[18790]: pgmap v254: 97 pgs: 14 creating+peering, 27 unknown, 56 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:21.396 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:21 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1650071583' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:21.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:21 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] up:active 2024-08-20T03:00:21.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:21 smithi165 ceph-mon[22357]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-08-20T03:00:21.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:21 smithi165 ceph-mon[22357]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-08-20T03:00:21.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:21 smithi165 ceph-mon[22357]: pgmap v254: 97 pgs: 14 creating+peering, 27 unknown, 56 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:21.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:21 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1650071583' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:21.639 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T03:00:21.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:21 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] up:active 2024-08-20T03:00:21.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:21 smithi099 ceph-mon[21984]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-08-20T03:00:21.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:21 smithi099 ceph-mon[21984]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-08-20T03:00:21.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:21 smithi099 ceph-mon[21984]: pgmap v254: 97 pgs: 14 creating+peering, 27 unknown, 56 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:00:21.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:21 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1650071583' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:21.892 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:22.265 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:22 smithi044 ceph-mon[18790]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-20T03:00:22.266 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:22 smithi044 ceph-mon[18790]: Cluster is now healthy 2024-08-20T03:00:22.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.344+0000 7f73e333e640 1 -- 172.21.15.44:0/3668782089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73dc10a630 msgr2=0x7f73dc10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:22.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.344+0000 7f73e333e640 1 --2- 172.21.15.44:0/3668782089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73dc10a630 0x7f73dc10ca20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009900 tx=0x7f73cc02a410 comp rx=0 tx=0).stop 2024-08-20T03:00:22.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.344+0000 7f73e333e640 1 -- 172.21.15.44:0/3668782089 shutdown_connections 2024-08-20T03:00:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.344+0000 7f73e333e640 1 --2- 172.21.15.44:0/3668782089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73dc10cf60 0x7f73dc10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.344+0000 7f73e333e640 1 --2- 172.21.15.44:0/3668782089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73dc10a630 0x7f73dc10ca20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.344+0000 7f73e333e640 1 --2- 172.21.15.44:0/3668782089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73dc101520 0x7f73dc101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.344+0000 7f73e333e640 1 -- 172.21.15.44:0/3668782089 >> 172.21.15.44:0/3668782089 conn(0x7f73dc076c80 msgr2=0x7f73dc077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.344+0000 7f73e333e640 1 -- 172.21.15.44:0/3668782089 shutdown_connections 2024-08-20T03:00:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.344+0000 7f73e333e640 1 -- 172.21.15.44:0/3668782089 wait complete. 2024-08-20T03:00:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e333e640 1 Processor -- start 2024-08-20T03:00:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e333e640 1 -- start start 2024-08-20T03:00:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e333e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73dc101520 0x7f73dc198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e333e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73dc10a630 0x7f73dc1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:22.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e333e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73dc10cf60 0x7f73dc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e333e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc112aa0 con 0x7f73dc101520 2024-08-20T03:00:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e333e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc112920 con 0x7f73dc10cf60 2024-08-20T03:00:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e333e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc112c20 con 0x7f73dc10a630 2024-08-20T03:00:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e10da640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73dc101520 0x7f73dc198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e10da640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73dc101520 0x7f73dc198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45114/0 (socket says 172.21.15.44:45114) 2024-08-20T03:00:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e10da640 1 -- 172.21.15.44:0/2957878465 learned_addr learned my addr 172.21.15.44:0/2957878465 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e08d9640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73dc10a630 0x7f73dc1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.345+0000 7f73e18db640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73dc10cf60 0x7f73dc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.346+0000 7f73e10da640 1 -- 172.21.15.44:0/2957878465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73dc10a630 msgr2=0x7f73dc1994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.346+0000 7f73e10da640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73dc10a630 0x7f73dc1994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.346+0000 7f73e10da640 1 -- 172.21.15.44:0/2957878465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73dc10cf60 msgr2=0x7f73dc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:22.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.346+0000 7f73e10da640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73dc10cf60 0x7f73dc19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.346+0000 7f73e10da640 1 -- 172.21.15.44:0/2957878465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc19f020 con 0x7f73dc101520 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.346+0000 7f73e18db640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73dc10cf60 0x7f73dc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.346+0000 7f73e10da640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73dc101520 0x7f73dc198f70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f73d800d950 tx=0x7f73d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.346+0000 7f73d27fc640 1 -- 172.21.15.44:0/2957878465 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8017070 con 0x7f73dc101520 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.346+0000 7f73e333e640 1 -- 172.21.15.44:0/2957878465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73dc19f310 con 0x7f73dc101520 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.346+0000 7f73d27fc640 1 -- 172.21.15.44:0/2957878465 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73d8009c30 con 0x7f73dc101520 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.346+0000 7f73d27fc640 1 -- 172.21.15.44:0/2957878465 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d800f620 con 0x7f73dc101520 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.346+0000 7f73e333e640 1 -- 172.21.15.44:0/2957878465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73dc1a59d0 con 0x7f73dc101520 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.347+0000 7f73e333e640 1 -- 172.21.15.44:0/2957878465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73dc101900 con 0x7f73dc101520 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.348+0000 7f73d27fc640 1 -- 172.21.15.44:0/2957878465 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f73d8005030 con 0x7f73dc101520 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.349+0000 7f73d27fc640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f73c80b3c20 0x7f73c80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.349+0000 7f73e08d9640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f73c80b3c20 0x7f73c80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:22.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.349+0000 7f73d27fc640 1 -- 172.21.15.44:0/2957878465 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f73d80e3850 con 0x7f73dc101520 2024-08-20T03:00:22.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.350+0000 7f73e08d9640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f73c80b3c20 0x7f73c80b60e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f73cc031440 tx=0x7f73cc005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:22.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.352+0000 7f73d27fc640 1 -- 172.21.15.44:0/2957878465 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f73d80aaec0 con 0x7f73dc101520 2024-08-20T03:00:22.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.538+0000 7f73e333e640 1 -- 172.21.15.44:0/2957878465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73dc1a5da0 con 0x7f73dc101520 2024-08-20T03:00:22.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.539+0000 7f73d27fc640 1 -- 172.21.15.44:0/2957878465 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22549 (secure 0 0 0) 0x7f73d8009de0 con 0x7f73dc101520 2024-08-20T03:00:22.542 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:22.542 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":75,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","created":"2024-08-20T02:52:10.979264+0000","modified":"2024-08-20T03:00:18.935366+0000","last_up_change":"2024-08-20T02:59:26.404917+0000","last_in_change":"2024-08-20T02:59:03.707380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T02:55:46.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T03:00:14.354568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T03:00:16.869335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7708f8c-a799-4b8f-ab1e-78482d73ca1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6803","nonce":858898316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6805","nonce":858898316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6809","nonce":858898316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6807","nonce":858898316}]},"public_addr":"172.21.15.44:6803/858898316","cluster_addr":"172.21.15.44:6805/858898316","heartbeat_back_addr":"172.21.15.44:6809/858898316","heartbeat_front_addr":"172.21.15.44:6807/858898316","state":["exists","up"]},{"osd":1,"uuid":"12f07986-60bf-498f-a012-cd7ea607b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6801","nonce":4252911663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6803","nonce":4252911663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6807","nonce":4252911663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6805","nonce":4252911663}]},"public_addr":"172.21.15.99:6801/4252911663","cluster_addr":"172.21.15.99:6803/4252911663","heartbeat_back_addr":"172.21.15.99:6807/4252911663","heartbeat_front_addr":"172.21.15.99:6805/4252911663","state":["exists","up"]},{"osd":2,"uuid":"0f189000-ba7f-4412-b0d3-75e8265f6685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6801","nonce":1492740465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6803","nonce":1492740465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6807","nonce":1492740465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6805","nonce":1492740465}]},"public_addr":"172.21.15.165:6801/1492740465","cluster_addr":"172.21.15.165:6803/1492740465","heartbeat_back_addr":"172.21.15.165:6807/1492740465","heartbeat_front_addr":"172.21.15.165:6805/1492740465","state":["exists","up"]},{"osd":3,"uuid":"8f9b042c-d6b5-4eee-aa29-ffc41a48c20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6811","nonce":3423983702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6813","nonce":3423983702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6817","nonce":3423983702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6815","nonce":3423983702}]},"public_addr":"172.21.15.44:6811/3423983702","cluster_addr":"172.21.15.44:6813/3423983702","heartbeat_back_addr":"172.21.15.44:6817/3423983702","heartbeat_front_addr":"172.21.15.44:6815/3423983702","state":["exists","up"]},{"osd":4,"uuid":"4edebfdb-af10-49ee-b724-2a63b9219221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6809","nonce":3099617325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6811","nonce":3099617325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6815","nonce":3099617325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6813","nonce":3099617325}]},"public_addr":"172.21.15.99:6809/3099617325","cluster_addr":"172.21.15.99:6811/3099617325","heartbeat_back_addr":"172.21.15.99:6815/3099617325","heartbeat_front_addr":"172.21.15.99:6813/3099617325","state":["exists","up"]},{"osd":5,"uuid":"6d22c21f-cc43-4ef5-a75c-d4fc1a361c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6809","nonce":523245255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6811","nonce":523245255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6815","nonce":523245255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6813","nonce":523245255}]},"public_addr":"172.21.15.165:6809/523245255","cluster_addr":"172.21.15.165:6811/523245255","heartbeat_back_addr":"172.21.15.165:6815/523245255","heartbeat_front_addr":"172.21.15.165:6813/523245255","state":["exists","up"]},{"osd":6,"uuid":"03df0c2f-1b7c-4242-888c-e047322fd8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6819","nonce":1039098986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6821","nonce":1039098986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6825","nonce":1039098986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6823","nonce":1039098986}]},"public_addr":"172.21.15.44:6819/1039098986","cluster_addr":"172.21.15.44:6821/1039098986","heartbeat_back_addr":"172.21.15.44:6825/1039098986","heartbeat_front_addr":"172.21.15.44:6823/1039098986","state":["exists","up"]},{"osd":7,"uuid":"84bbc683-9b13-448e-897c-114458dae33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6817","nonce":1438613564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6819","nonce":1438613564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6823","nonce":1438613564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6821","nonce":1438613564}]},"public_addr":"172.21.15.99:6817/1438613564","cluster_addr":"172.21.15.99:6819/1438613564","heartbeat_back_addr":"172.21.15.99:6823/1438613564","heartbeat_front_addr":"172.21.15.99:6821/1438613564","state":["exists","up"]},{"osd":8,"uuid":"14d7ffd3-db05-4228-a12e-4c69ef65ef8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6817","nonce":632957688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6819","nonce":632957688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6823","nonce":632957688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6821","nonce":632957688}]},"public_addr":"172.21.15.165:6817/632957688","cluster_addr":"172.21.15.165:6819/632957688","heartbeat_back_addr":"172.21.15.165:6823/632957688","heartbeat_front_addr":"172.21.15.165:6821/632957688","state":["exists","up"]},{"osd":9,"uuid":"1ae468a5-3d72-4eb0-a1a6-cd87eae740b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6827","nonce":2426940093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6829","nonce":2426940093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6833","nonce":2426940093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6831","nonce":2426940093}]},"public_addr":"172.21.15.44:6827/2426940093","cluster_addr":"172.21.15.44:6829/2426940093","heartbeat_back_addr":"172.21.15.44:6833/2426940093","heartbeat_front_addr":"172.21.15.44:6831/2426940093","state":["exists","up"]},{"osd":10,"uuid":"fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6825","nonce":3275793038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6827","nonce":3275793038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6831","nonce":3275793038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6829","nonce":3275793038}]},"public_addr":"172.21.15.99:6825/3275793038","cluster_addr":"172.21.15.99:6827/3275793038","heartbeat_back_addr":"172.21.15.99:6831/3275793038","heartbeat_front_addr":"172.21.15.99:6829/3275793038","state":["exists","up"]},{"osd":11,"uuid":"50715410-1fc2-4818-b600-604a4b7cce9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6825","nonce":107688415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6827","nonce":107688415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6831","nonce":107688415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6829","nonce":107688415}]},"public_addr":"172.21.15.165:6825/107688415","cluster_addr":"172.21.15.165:6827/107688415","heartbeat_back_addr":"172.21.15.165:6831/107688415","heartbeat_front_addr":"172.21.15.165:6829/107688415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:54:56.697075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:20.486846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:43.353376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:08.333879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:32.186492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:55.891366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:19.681421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:45.475926+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:10.109795+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:34.444169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:59.147646+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:59:25.403071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2597827045":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/3582612619":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/1916809746":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2919033419":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:6800/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2331034359":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2572185146":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/4176802819":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/3654439062":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6801/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/2376733275":"2024-08-21T02:53:58.312659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T03:00:22.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.542+0000 7f73e333e640 1 -- 172.21.15.44:0/2957878465 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f73c80b3c20 msgr2=0x7f73c80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:22.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.542+0000 7f73e333e640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f73c80b3c20 0x7f73c80b60e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f73cc031440 tx=0x7f73cc005b50 comp rx=0 tx=0).stop 2024-08-20T03:00:22.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.542+0000 7f73e333e640 1 -- 172.21.15.44:0/2957878465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73dc101520 msgr2=0x7f73dc198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:22.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.542+0000 7f73e333e640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73dc101520 0x7f73dc198f70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f73d800d950 tx=0x7f73d800de10 comp rx=0 tx=0).stop 2024-08-20T03:00:22.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.542+0000 7f73e333e640 1 -- 172.21.15.44:0/2957878465 shutdown_connections 2024-08-20T03:00:22.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.542+0000 7f73e333e640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f73c80b3c20 0x7f73c80b60e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:22.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.542+0000 7f73e333e640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73dc10cf60 0x7f73dc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:22.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.542+0000 7f73e333e640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73dc10a630 0x7f73dc1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:22.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.542+0000 7f73e333e640 1 --2- 172.21.15.44:0/2957878465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73dc101520 0x7f73dc198f70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:22.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.542+0000 7f73e333e640 1 -- 172.21.15.44:0/2957878465 >> 172.21.15.44:0/2957878465 conn(0x7f73dc076c80 msgr2=0x7f73dc1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:22.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.542+0000 7f73e333e640 1 -- 172.21.15.44:0/2957878465 shutdown_connections 2024-08-20T03:00:22.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:22.542+0000 7f73e333e640 1 -- 172.21.15.44:0/2957878465 wait complete. 2024-08-20T03:00:22.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:22 smithi165 ceph-mon[22357]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-20T03:00:22.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:22 smithi165 ceph-mon[22357]: Cluster is now healthy 2024-08-20T03:00:22.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:22 smithi099 ceph-mon[21984]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-20T03:00:22.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:22 smithi099 ceph-mon[21984]: Cluster is now healthy 2024-08-20T03:00:23.328 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-08-20T03:00:23.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:23 smithi044 ceph-mon[18790]: pgmap v255: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-08-20T03:00:23.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:23 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2957878465' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:23.582 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:23.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:23 smithi165 ceph-mon[22357]: pgmap v255: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-08-20T03:00:23.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:23 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2957878465' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:23.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:23 smithi099 ceph-mon[21984]: pgmap v255: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-08-20T03:00:23.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:23 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2957878465' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:24.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.039+0000 7fd811416640 1 -- 172.21.15.44:0/2038520722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd80c107d60 msgr2=0x7fd80c10e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:24.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.039+0000 7fd811416640 1 --2- 172.21.15.44:0/2038520722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd80c107d60 0x7fd80c10e560 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd804009900 tx=0x7fd80402a410 comp rx=0 tx=0).stop 2024-08-20T03:00:24.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.039+0000 7fd811416640 1 -- 172.21.15.44:0/2038520722 shutdown_connections 2024-08-20T03:00:24.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.039+0000 7fd811416640 1 --2- 172.21.15.44:0/2038520722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd80c107d60 0x7fd80c10e560 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:24.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.039+0000 7fd811416640 1 --2- 172.21.15.44:0/2038520722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd80c1073e0 0x7fd80c107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:24.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.040+0000 7fd811416640 1 --2- 172.21.15.44:0/2038520722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd80c1019d0 0x7fd80c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:24.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.040+0000 7fd811416640 1 -- 172.21.15.44:0/2038520722 >> 172.21.15.44:0/2038520722 conn(0x7fd80c0fd440 msgr2=0x7fd80c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.040+0000 7fd811416640 1 -- 172.21.15.44:0/2038520722 shutdown_connections 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.040+0000 7fd811416640 1 -- 172.21.15.44:0/2038520722 wait complete. 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.040+0000 7fd811416640 1 Processor -- start 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.040+0000 7fd811416640 1 -- start start 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.040+0000 7fd811416640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd80c1019d0 0x7fd80c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.040+0000 7fd811416640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd80c1073e0 0x7fd80c199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.040+0000 7fd811416640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd80c107d60 0x7fd80c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.040+0000 7fd811416640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c1119e0 con 0x7fd80c1073e0 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.040+0000 7fd811416640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c111860 con 0x7fd80c1019d0 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.040+0000 7fd811416640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c111b60 con 0x7fd80c107d60 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd80a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd80c1073e0 0x7fd80c199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd80a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd80c1073e0 0x7fd80c199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45124/0 (socket says 172.21.15.44:45124) 2024-08-20T03:00:24.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd80a7fc640 1 -- 172.21.15.44:0/1584907248 learned_addr learned my addr 172.21.15.44:0/1584907248 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd80affd640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd80c1019d0 0x7fd80c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd80b7fe640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd80c107d60 0x7fd80c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd80a7fc640 1 -- 172.21.15.44:0/1584907248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd80c107d60 msgr2=0x7fd80c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd80a7fc640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd80c107d60 0x7fd80c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd80a7fc640 1 -- 172.21.15.44:0/1584907248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd80c1019d0 msgr2=0x7fd80c198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd80a7fc640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd80c1019d0 0x7fd80c198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd80a7fc640 1 -- 172.21.15.44:0/1584907248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c19eff0 con 0x7fd80c1073e0 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd80a7fc640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd80c1073e0 0x7fd80c199480 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400c890 tx=0x7fd7f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd803fff640 1 -- 172.21.15.44:0/1584907248 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4004220 con 0x7fd80c1073e0 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd803fff640 1 -- 172.21.15.44:0/1584907248 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7f40043c0 con 0x7fd80c1073e0 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd811416640 1 -- 172.21.15.44:0/1584907248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c19f2e0 con 0x7fd80c1073e0 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd803fff640 1 -- 172.21.15.44:0/1584907248 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4004a00 con 0x7fd80c1073e0 2024-08-20T03:00:24.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.041+0000 7fd811416640 1 -- 172.21.15.44:0/1584907248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c1a59a0 con 0x7fd80c1073e0 2024-08-20T03:00:24.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.042+0000 7fd811416640 1 -- 172.21.15.44:0/1584907248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80c101db0 con 0x7fd80c1073e0 2024-08-20T03:00:24.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.044+0000 7fd803fff640 1 -- 172.21.15.44:0/1584907248 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fd7f40079b0 con 0x7fd80c1073e0 2024-08-20T03:00:24.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.044+0000 7fd803fff640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd7f80b3c20 0x7fd7f80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:24.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.045+0000 7fd80affd640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd7f80b3c20 0x7fd7f80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:24.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.045+0000 7fd803fff640 1 -- 172.21.15.44:0/1584907248 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fd7f40e3d50 con 0x7fd80c1073e0 2024-08-20T03:00:24.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.045+0000 7fd80affd640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd7f80b3c20 0x7fd7f80b60e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc005fd0 tx=0x7fd7fc009760 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:24.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.047+0000 7fd803fff640 1 -- 172.21.15.44:0/1584907248 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd7f4014030 con 0x7fd80c1073e0 2024-08-20T03:00:24.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.271+0000 7fd811416640 1 -- 172.21.15.44:0/1584907248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd80c1a5d80 con 0x7fd80c1073e0 2024-08-20T03:00:24.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.323+0000 7fd803fff640 1 -- 172.21.15.44:0/1584907248 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fd7f40ab3c0 con 0x7fd80c1073e0 2024-08-20T03:00:24.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.325+0000 7fd811416640 1 -- 172.21.15.44:0/1584907248 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd7f80b3c20 msgr2=0x7fd7f80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:24.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.325+0000 7fd811416640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd7f80b3c20 0x7fd7f80b60e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc005fd0 tx=0x7fd7fc009760 comp rx=0 tx=0).stop 2024-08-20T03:00:24.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.325+0000 7fd811416640 1 -- 172.21.15.44:0/1584907248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd80c1073e0 msgr2=0x7fd80c199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:24.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.325+0000 7fd811416640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd80c1073e0 0x7fd80c199480 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400c890 tx=0x7fd7f400cd50 comp rx=0 tx=0).stop 2024-08-20T03:00:24.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.325+0000 7fd811416640 1 -- 172.21.15.44:0/1584907248 shutdown_connections 2024-08-20T03:00:24.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.325+0000 7fd811416640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd7f80b3c20 0x7fd7f80b60e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:24.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.325+0000 7fd811416640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd80c107d60 0x7fd80c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:24.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.325+0000 7fd811416640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd80c1073e0 0x7fd80c199480 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:24.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.325+0000 7fd811416640 1 --2- 172.21.15.44:0/1584907248 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd80c1019d0 0x7fd80c198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:24.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.325+0000 7fd811416640 1 -- 172.21.15.44:0/1584907248 >> 172.21.15.44:0/1584907248 conn(0x7fd80c0fd440 msgr2=0x7fd80c105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:24.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.326+0000 7fd811416640 1 -- 172.21.15.44:0/1584907248 shutdown_connections 2024-08-20T03:00:24.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:24.326+0000 7fd811416640 1 -- 172.21.15.44:0/1584907248 wait complete. 2024-08-20T03:00:24.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:24 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1584907248' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-20T03:00:24.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:24 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1584907248' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-20T03:00:24.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:24 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1584907248' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-20T03:00:25.027 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-08-20T03:00:25.027 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-08-20T03:00:25.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:00:24 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:00:24.853+0000 7f9a1477a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-20T03:00:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:25.582 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:25 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1584907248' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-20T03:00:25.582 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:25 smithi044 ceph-mon[18790]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-08-20T03:00:25.582 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:25 smithi044 ceph-mon[18790]: pgmap v256: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-08-20T03:00:25.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:25 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1584907248' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-20T03:00:25.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:25 smithi165 ceph-mon[22357]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-08-20T03:00:25.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:25 smithi165 ceph-mon[22357]: pgmap v256: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-08-20T03:00:25.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:25 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1584907248' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-20T03:00:25.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:25 smithi099 ceph-mon[21984]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-08-20T03:00:25.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:25 smithi099 ceph-mon[21984]: pgmap v256: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-08-20T03:00:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.786+0000 7f8ca11f3640 1 -- 172.21.15.44:0/2625975016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c9c1084c0 msgr2=0x7f8c9c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.786+0000 7f8ca11f3640 1 --2- 172.21.15.44:0/2625975016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c9c1084c0 0x7f8c9c108900 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009900 tx=0x7f8c9002a410 comp rx=0 tx=0).stop 2024-08-20T03:00:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.786+0000 7f8ca11f3640 1 -- 172.21.15.44:0/2625975016 shutdown_connections 2024-08-20T03:00:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.786+0000 7f8ca11f3640 1 --2- 172.21.15.44:0/2625975016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c9c108e40 0x7f8c9c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.786+0000 7f8ca11f3640 1 --2- 172.21.15.44:0/2625975016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c9c1084c0 0x7f8c9c108900 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.786+0000 7f8ca11f3640 1 --2- 172.21.15.44:0/2625975016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c9c102ab0 0x7f8c9c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.786+0000 7f8ca11f3640 1 -- 172.21.15.44:0/2625975016 >> 172.21.15.44:0/2625975016 conn(0x7f8c9c0fe540 msgr2=0x7f8c9c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:25.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.786+0000 7f8ca11f3640 1 -- 172.21.15.44:0/2625975016 shutdown_connections 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.786+0000 7f8ca11f3640 1 -- 172.21.15.44:0/2625975016 wait complete. 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.787+0000 7f8ca11f3640 1 Processor -- start 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.787+0000 7f8ca11f3640 1 -- start start 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.787+0000 7f8ca11f3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c9c102ab0 0x7f8c9c199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.787+0000 7f8ca11f3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c9c1084c0 0x7f8c9c199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.787+0000 7f8ca11f3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c9c108e40 0x7f8c9c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.787+0000 7f8ca11f3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c112b00 con 0x7f8c9c1084c0 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.787+0000 7f8ca11f3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c112980 con 0x7f8c9c102ab0 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.787+0000 7f8ca11f3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9c112c80 con 0x7f8c9c108e40 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.787+0000 7f8c9a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c9c1084c0 0x7f8c9c199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.787+0000 7f8c9a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c9c1084c0 0x7f8c9c199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45146/0 (socket says 172.21.15.44:45146) 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.787+0000 7f8c9a59c640 1 -- 172.21.15.44:0/3713276243 learned_addr learned my addr 172.21.15.44:0/3713276243 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8c9ad9d640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c9c102ab0 0x7f8c9c199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:25.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8c9b59e640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c9c108e40 0x7f8c9c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8c9a59c640 1 -- 172.21.15.44:0/3713276243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c9c108e40 msgr2=0x7f8c9c19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8c9a59c640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c9c108e40 0x7f8c9c19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8c9a59c640 1 -- 172.21.15.44:0/3713276243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c9c102ab0 msgr2=0x7f8c9c199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8c9a59c640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c9c102ab0 0x7f8c9c199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8c9a59c640 1 -- 172.21.15.44:0/3713276243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9c19f0c0 con 0x7f8c9c1084c0 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8c9b59e640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c9c108e40 0x7f8c9c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8c9a59c640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c9c1084c0 0x7f8c9c199550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c90005940 tx=0x7f8c900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8c83fff640 1 -- 172.21.15.44:0/3713276243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9003f030 con 0x7f8c9c1084c0 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8ca11f3640 1 -- 172.21.15.44:0/3713276243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c9c19f350 con 0x7f8c9c1084c0 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8c83fff640 1 -- 172.21.15.44:0/3713276243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c90003cb0 con 0x7f8c9c1084c0 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8c83fff640 1 -- 172.21.15.44:0/3713276243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90035070 con 0x7f8c9c1084c0 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.788+0000 7f8ca11f3640 1 -- 172.21.15.44:0/3713276243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c9c1a5a70 con 0x7f8c9c1084c0 2024-08-20T03:00:25.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.789+0000 7f8ca11f3640 1 -- 172.21.15.44:0/3713276243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c9c102e90 con 0x7f8c9c1084c0 2024-08-20T03:00:25.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.790+0000 7f8c83fff640 1 -- 172.21.15.44:0/3713276243 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f8c90042030 con 0x7f8c9c1084c0 2024-08-20T03:00:25.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.791+0000 7f8c83fff640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8c840b3c20 0x7f8c840b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:25.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.792+0000 7f8c9ad9d640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8c840b3c20 0x7f8c840b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:25.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.792+0000 7f8c83fff640 1 -- 172.21.15.44:0/3713276243 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f8c901047c0 con 0x7f8c9c1084c0 2024-08-20T03:00:25.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.792+0000 7f8c9ad9d640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8c840b3c20 0x7f8c840b60e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00bbe0 tx=0x7f8c8c00b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:25.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:25.795+0000 7f8c83fff640 1 -- 172.21.15.44:0/3713276243 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8c900cbdd0 con 0x7f8c9c1084c0 2024-08-20T03:00:26.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.017+0000 7f8ca11f3640 1 -- 172.21.15.44:0/3713276243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f8c9c02e340 con 0x7f8c9c1084c0 2024-08-20T03:00:26.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.342+0000 7f8c83fff640 1 -- 172.21.15.44:0/3713276243 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f8c900d40a0 con 0x7f8c9c1084c0 2024-08-20T03:00:26.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.345+0000 7f8ca11f3640 1 -- 172.21.15.44:0/3713276243 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8c840b3c20 msgr2=0x7f8c840b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:26.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.345+0000 7f8ca11f3640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8c840b3c20 0x7f8c840b60e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00bbe0 tx=0x7f8c8c00b390 comp rx=0 tx=0).stop 2024-08-20T03:00:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.345+0000 7f8ca11f3640 1 -- 172.21.15.44:0/3713276243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c9c1084c0 msgr2=0x7f8c9c199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.345+0000 7f8ca11f3640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c9c1084c0 0x7f8c9c199550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c90005940 tx=0x7f8c900042f0 comp rx=0 tx=0).stop 2024-08-20T03:00:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.345+0000 7f8ca11f3640 1 -- 172.21.15.44:0/3713276243 shutdown_connections 2024-08-20T03:00:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.345+0000 7f8ca11f3640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8c840b3c20 0x7f8c840b60e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.345+0000 7f8ca11f3640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c9c108e40 0x7f8c9c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.345+0000 7f8ca11f3640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c9c1084c0 0x7f8c9c199550 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.345+0000 7f8ca11f3640 1 --2- 172.21.15.44:0/3713276243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c9c102ab0 0x7f8c9c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.345+0000 7f8ca11f3640 1 -- 172.21.15.44:0/3713276243 >> 172.21.15.44:0/3713276243 conn(0x7f8c9c0fe540 msgr2=0x7f8c9c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.345+0000 7f8ca11f3640 1 -- 172.21.15.44:0/3713276243 shutdown_connections 2024-08-20T03:00:26.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:26.345+0000 7f8ca11f3640 1 -- 172.21.15.44:0/3713276243 wait complete. 2024-08-20T03:00:26.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:26 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3713276243' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-08-20T03:00:26.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:26 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3713276243' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-08-20T03:00:26.715 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:26 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3713276243' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-08-20T03:00:27.080 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-08-20T03:00:27.343 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:27.595 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:27 smithi044 ceph-mon[18790]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-20T03:00:27.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:27 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3713276243' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-08-20T03:00:27.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:27 smithi044 ceph-mon[18790]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-20T03:00:27.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:27 smithi044 ceph-mon[18790]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-08-20T03:00:27.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:27 smithi044 ceph-mon[18790]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-08-20T03:00:27.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:27 smithi044 ceph-mon[18790]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-08-20T03:00:27.596 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:27 smithi044 ceph-mon[18790]: pgmap v257: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-20T03:00:27.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:27 smithi165 ceph-mon[22357]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-20T03:00:27.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:27 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3713276243' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-08-20T03:00:27.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:27 smithi165 ceph-mon[22357]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-20T03:00:27.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:27 smithi165 ceph-mon[22357]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-08-20T03:00:27.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:27 smithi165 ceph-mon[22357]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-08-20T03:00:27.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:27 smithi165 ceph-mon[22357]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-08-20T03:00:27.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:27 smithi165 ceph-mon[22357]: pgmap v257: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-20T03:00:27.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:27 smithi099 ceph-mon[21984]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-20T03:00:27.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:27 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3713276243' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-08-20T03:00:27.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:27 smithi099 ceph-mon[21984]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-20T03:00:27.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:27 smithi099 ceph-mon[21984]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-08-20T03:00:27.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:27 smithi099 ceph-mon[21984]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-08-20T03:00:27.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:27 smithi099 ceph-mon[21984]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-08-20T03:00:27.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:27 smithi099 ceph-mon[21984]: pgmap v257: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-20T03:00:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.826+0000 7f861b6b9640 1 -- 172.21.15.44:0/2481806624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8614108e60 msgr2=0x7f861410f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.826+0000 7f861b6b9640 1 --2- 172.21.15.44:0/2481806624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8614108e60 0x7f861410f660 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f860c009ca0 tx=0x7f860c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.826+0000 7f861b6b9640 1 -- 172.21.15.44:0/2481806624 shutdown_connections 2024-08-20T03:00:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.826+0000 7f861b6b9640 1 --2- 172.21.15.44:0/2481806624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8614108e60 0x7f861410f660 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.826+0000 7f861b6b9640 1 --2- 172.21.15.44:0/2481806624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86141084e0 0x7f8614108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.826+0000 7f861b6b9640 1 --2- 172.21.15.44:0/2481806624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8614102ad0 0x7f8614102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.826+0000 7f861b6b9640 1 -- 172.21.15.44:0/2481806624 >> 172.21.15.44:0/2481806624 conn(0x7f86140fe540 msgr2=0x7f8614100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.826+0000 7f861b6b9640 1 -- 172.21.15.44:0/2481806624 shutdown_connections 2024-08-20T03:00:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.826+0000 7f861b6b9640 1 -- 172.21.15.44:0/2481806624 wait complete. 2024-08-20T03:00:27.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.826+0000 7f861b6b9640 1 Processor -- start 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f861b6b9640 1 -- start start 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f861b6b9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8614102ad0 0x7f86140751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f861b6b9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86141084e0 0x7f8614075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f861b6b9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8614108e60 0x7f8614075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f861b6b9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8614112b20 con 0x7f86141084e0 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f861b6b9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86141129a0 con 0x7f8614108e60 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f861b6b9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8614112ca0 con 0x7f8614102ad0 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f8618c54640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86141084e0 0x7f8614075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f8618c54640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86141084e0 0x7f8614075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57348/0 (socket says 172.21.15.44:57348) 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f8618c54640 1 -- 172.21.15.44:0/900242014 learned_addr learned my addr 172.21.15.44:0/900242014 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f8619455640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8614102ad0 0x7f86140751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f8619c56640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8614108e60 0x7f8614075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:27.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f8618c54640 1 -- 172.21.15.44:0/900242014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8614102ad0 msgr2=0x7f86140751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f8618c54640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8614102ad0 0x7f86140751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f8618c54640 1 -- 172.21.15.44:0/900242014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8614108e60 msgr2=0x7f8614075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f8618c54640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8614108e60 0x7f8614075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f8618c54640 1 -- 172.21.15.44:0/900242014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86141a7350 con 0x7f86141084e0 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.827+0000 7f8618c54640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86141084e0 0x7f8614075710 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f860400c920 tx=0x7f860400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.828+0000 7f860a7fc640 1 -- 172.21.15.44:0/900242014 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604007bb0 con 0x7f86141084e0 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.828+0000 7f860a7fc640 1 -- 172.21.15.44:0/900242014 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8604007d50 con 0x7f86141084e0 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.828+0000 7f861b6b9640 1 -- 172.21.15.44:0/900242014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86141a7640 con 0x7f86141084e0 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.828+0000 7f860a7fc640 1 -- 172.21.15.44:0/900242014 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604018730 con 0x7f86141084e0 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.828+0000 7f861b6b9640 1 -- 172.21.15.44:0/900242014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86141a7c00 con 0x7f86141084e0 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.829+0000 7f861b6b9640 1 -- 172.21.15.44:0/900242014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8614102eb0 con 0x7f86141084e0 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.830+0000 7f860a7fc640 1 -- 172.21.15.44:0/900242014 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f860400f450 con 0x7f86141084e0 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.831+0000 7f860a7fc640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86000b3c20 0x7f86000b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:27.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.831+0000 7f8619455640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86000b3c20 0x7f86000b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:27.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.831+0000 7f860a7fc640 1 -- 172.21.15.44:0/900242014 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f86040e3f90 con 0x7f86141084e0 2024-08-20T03:00:27.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.831+0000 7f8619455640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86000b3c20 0x7f86000b60e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8610009990 tx=0x7f8610009110 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:27.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:27.834+0000 7f860a7fc640 1 -- 172.21.15.44:0/900242014 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8604014030 con 0x7f86141084e0 2024-08-20T03:00:28.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.056+0000 7f861b6b9640 1 -- 172.21.15.44:0/900242014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f861402e340 con 0x7f86141084e0 2024-08-20T03:00:28.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.361+0000 7f860a7fc640 1 -- 172.21.15.44:0/900242014 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f86040ab690 con 0x7f86141084e0 2024-08-20T03:00:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.363+0000 7f861b6b9640 1 -- 172.21.15.44:0/900242014 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86000b3c20 msgr2=0x7f86000b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.363+0000 7f861b6b9640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86000b3c20 0x7f86000b60e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8610009990 tx=0x7f8610009110 comp rx=0 tx=0).stop 2024-08-20T03:00:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.363+0000 7f861b6b9640 1 -- 172.21.15.44:0/900242014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86141084e0 msgr2=0x7f8614075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.363+0000 7f861b6b9640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86141084e0 0x7f8614075710 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f860400c920 tx=0x7f860400cd20 comp rx=0 tx=0).stop 2024-08-20T03:00:28.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.363+0000 7f861b6b9640 1 -- 172.21.15.44:0/900242014 shutdown_connections 2024-08-20T03:00:28.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.363+0000 7f861b6b9640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86000b3c20 0x7f86000b60e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:28.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.363+0000 7f861b6b9640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8614108e60 0x7f8614075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:28.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.363+0000 7f861b6b9640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86141084e0 0x7f8614075710 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:28.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.363+0000 7f861b6b9640 1 --2- 172.21.15.44:0/900242014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8614102ad0 0x7f86140751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:28.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.363+0000 7f861b6b9640 1 -- 172.21.15.44:0/900242014 >> 172.21.15.44:0/900242014 conn(0x7f86140fe540 msgr2=0x7f8614100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:28.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.363+0000 7f861b6b9640 1 -- 172.21.15.44:0/900242014 shutdown_connections 2024-08-20T03:00:28.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:28.363+0000 7f861b6b9640 1 -- 172.21.15.44:0/900242014 wait complete. 2024-08-20T03:00:28.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:28 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] up:active 2024-08-20T03:00:28.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:28 smithi165 ceph-mon[22357]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-20T03:00:28.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:28 smithi165 ceph-mon[22357]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-08-20T03:00:28.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:28 smithi165 ceph-mon[22357]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-08-20T03:00:28.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:28 smithi165 ceph-mon[22357]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-08-20T03:00:28.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:28 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/900242014' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-20T03:00:28.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:28 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] up:active 2024-08-20T03:00:28.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:28 smithi099 ceph-mon[21984]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-20T03:00:28.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:28 smithi099 ceph-mon[21984]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-08-20T03:00:28.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:28 smithi099 ceph-mon[21984]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-08-20T03:00:28.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:28 smithi099 ceph-mon[21984]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-08-20T03:00:28.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:28 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/900242014' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-20T03:00:28.665 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:28 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] up:active 2024-08-20T03:00:28.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:28 smithi044 ceph-mon[18790]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-20T03:00:28.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:28 smithi044 ceph-mon[18790]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-08-20T03:00:28.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:28 smithi044 ceph-mon[18790]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-08-20T03:00:28.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:28 smithi044 ceph-mon[18790]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-08-20T03:00:28.666 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:28 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/900242014' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-20T03:00:29.013 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-08-20T03:00:29.269 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:29.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:29 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/900242014' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-20T03:00:29.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:29 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] up:active 2024-08-20T03:00:29.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:29 smithi165 ceph-mon[22357]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-08-20T03:00:29.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:29 smithi165 ceph-mon[22357]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-08-20T03:00:29.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:29 smithi165 ceph-mon[22357]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-08-20T03:00:29.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:29 smithi165 ceph-mon[22357]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-08-20T03:00:29.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:29 smithi165 ceph-mon[22357]: pgmap v258: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-08-20T03:00:29.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:29 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/900242014' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-20T03:00:29.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:29 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] up:active 2024-08-20T03:00:29.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:29 smithi099 ceph-mon[21984]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-08-20T03:00:29.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:29 smithi099 ceph-mon[21984]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-08-20T03:00:29.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:29 smithi099 ceph-mon[21984]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-08-20T03:00:29.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:29 smithi099 ceph-mon[21984]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-08-20T03:00:29.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:29 smithi099 ceph-mon[21984]: pgmap v258: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-08-20T03:00:29.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:29 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/900242014' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-20T03:00:29.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:29 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] up:active 2024-08-20T03:00:29.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:29 smithi044 ceph-mon[18790]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-08-20T03:00:29.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:29 smithi044 ceph-mon[18790]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-08-20T03:00:29.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:29 smithi044 ceph-mon[18790]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-08-20T03:00:29.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:29 smithi044 ceph-mon[18790]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-08-20T03:00:29.671 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:29 smithi044 ceph-mon[18790]: pgmap v258: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-08-20T03:00:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.753+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/962651690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f81073a0 msgr2=0x7fa8f81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:29.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.753+0000 7fa8fd3fd640 1 --2- 172.21.15.44:0/962651690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f81073a0 0x7fa8f81077e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009900 tx=0x7fa8e002a410 comp rx=0 tx=0).stop 2024-08-20T03:00:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.753+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/962651690 shutdown_connections 2024-08-20T03:00:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.753+0000 7fa8fd3fd640 1 --2- 172.21.15.44:0/962651690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f8107d20 0x7fa8f810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.753+0000 7fa8fd3fd640 1 --2- 172.21.15.44:0/962651690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f81073a0 0x7fa8f81077e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.753+0000 7fa8fd3fd640 1 --2- 172.21.15.44:0/962651690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f8101990 0x7fa8f8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.753+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/962651690 >> 172.21.15.44:0/962651690 conn(0x7fa8f80fd400 msgr2=0x7fa8f80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:29.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.753+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/962651690 shutdown_connections 2024-08-20T03:00:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.754+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/962651690 wait complete. 2024-08-20T03:00:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.754+0000 7fa8fd3fd640 1 Processor -- start 2024-08-20T03:00:29.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.754+0000 7fa8fd3fd640 1 -- start start 2024-08-20T03:00:29.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.754+0000 7fa8fd3fd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f8101990 0x7fa8f81990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.754+0000 7fa8fd3fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f81073a0 0x7fa8f8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.754+0000 7fa8fd3fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8107d20 0x7fa8f819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8fd3fd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f8111970 con 0x7fa8f8107d20 2024-08-20T03:00:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8fd3fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f81117f0 con 0x7fa8f81073a0 2024-08-20T03:00:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8fd3fd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f8111af0 con 0x7fa8f8101990 2024-08-20T03:00:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8107d20 0x7fa8f819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8107d20 0x7fa8f819e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57364/0 (socket says 172.21.15.44:57364) 2024-08-20T03:00:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8f77fe640 1 -- 172.21.15.44:0/2399227316 learned_addr learned my addr 172.21.15.44:0/2399227316 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8f67fc640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f81073a0 0x7fa8f8199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8f6ffd640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f8101990 0x7fa8f81990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:29.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8f77fe640 1 -- 172.21.15.44:0/2399227316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f8101990 msgr2=0x7fa8f81990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8f77fe640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f8101990 0x7fa8f81990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8f77fe640 1 -- 172.21.15.44:0/2399227316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f81073a0 msgr2=0x7fa8f8199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8f77fe640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f81073a0 0x7fa8f8199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8f77fe640 1 -- 172.21.15.44:0/2399227316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f819f180 con 0x7fa8f8107d20 2024-08-20T03:00:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8f77fe640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8107d20 0x7fa8f819e9b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000c970 tx=0x7fa8f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8effff640 1 -- 172.21.15.44:0/2399227316 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0007bb0 con 0x7fa8f8107d20 2024-08-20T03:00:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/2399227316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f819f470 con 0x7fa8f8107d20 2024-08-20T03:00:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/2399227316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f81a5b30 con 0x7fa8f8107d20 2024-08-20T03:00:29.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8effff640 1 -- 172.21.15.44:0/2399227316 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8f0007d50 con 0x7fa8f8107d20 2024-08-20T03:00:29.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.755+0000 7fa8effff640 1 -- 172.21.15.44:0/2399227316 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0004e60 con 0x7fa8f8107d20 2024-08-20T03:00:29.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.756+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/2399227316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d8005290 con 0x7fa8f8107d20 2024-08-20T03:00:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.759+0000 7fa8effff640 1 -- 172.21.15.44:0/2399227316 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7fa8f00046c0 con 0x7fa8f8107d20 2024-08-20T03:00:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.759+0000 7fa8effff640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8e40b3ca0 0x7fa8e40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.759+0000 7fa8f6ffd640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8e40b3ca0 0x7fa8e40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.759+0000 7fa8effff640 1 -- 172.21.15.44:0/2399227316 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fa8f00e3a50 con 0x7fa8f8107d20 2024-08-20T03:00:29.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.760+0000 7fa8f6ffd640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8e40b3ca0 0x7fa8e40b6160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8005fd0 tx=0x7fa8e80098b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:29.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.762+0000 7fa8effff640 1 -- 172.21.15.44:0/2399227316 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa8f0014030 con 0x7fa8f8107d20 2024-08-20T03:00:29.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:29.984+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/2399227316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa8d8005580 con 0x7fa8f8107d20 2024-08-20T03:00:30.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:00:29 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:00:29.853+0000 7f9a1477a640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-08-20T03:00:30.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:00:29 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:00:29.853+0000 7f9a1477a640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-08-20T03:00:30.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:00:29 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:00:29.853+0000 7f9a1477a640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-08-20T03:00:30.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.380+0000 7fa8effff640 1 -- 172.21.15.44:0/2399227316 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fa8f00ab0e0 con 0x7fa8f8107d20 2024-08-20T03:00:30.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.382+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/2399227316 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8e40b3ca0 msgr2=0x7fa8e40b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:30.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.382+0000 7fa8fd3fd640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8e40b3ca0 0x7fa8e40b6160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8005fd0 tx=0x7fa8e80098b0 comp rx=0 tx=0).stop 2024-08-20T03:00:30.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.382+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/2399227316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8107d20 msgr2=0x7fa8f819e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:30.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.382+0000 7fa8fd3fd640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8107d20 0x7fa8f819e9b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000c970 tx=0x7fa8f000ce30 comp rx=0 tx=0).stop 2024-08-20T03:00:30.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.382+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/2399227316 shutdown_connections 2024-08-20T03:00:30.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.382+0000 7fa8fd3fd640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8e40b3ca0 0x7fa8e40b6160 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:30.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.382+0000 7fa8fd3fd640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8107d20 0x7fa8f819e9b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:30.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.382+0000 7fa8fd3fd640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f81073a0 0x7fa8f8199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:30.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.382+0000 7fa8fd3fd640 1 --2- 172.21.15.44:0/2399227316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f8101990 0x7fa8f81990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:30.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.382+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/2399227316 >> 172.21.15.44:0/2399227316 conn(0x7fa8f80fd400 msgr2=0x7fa8f8104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:30.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.383+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/2399227316 shutdown_connections 2024-08-20T03:00:30.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:30.383+0000 7fa8fd3fd640 1 -- 172.21.15.44:0/2399227316 wait complete. 2024-08-20T03:00:30.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:30 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] up:active 2024-08-20T03:00:30.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:30 smithi165 ceph-mon[22357]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-08-20T03:00:30.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:30 smithi165 ceph-mon[22357]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-20T03:00:30.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:30 smithi165 ceph-mon[22357]: Cluster is now healthy 2024-08-20T03:00:30.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:30 smithi165 ceph-mon[22357]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-08-20T03:00:30.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:30 smithi165 ceph-mon[22357]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-08-20T03:00:30.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:30 smithi165 ceph-mon[22357]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-08-20T03:00:30.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:30 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2399227316' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-20T03:00:30.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:30 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] up:active 2024-08-20T03:00:30.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:30 smithi099 ceph-mon[21984]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-08-20T03:00:30.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:30 smithi099 ceph-mon[21984]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-20T03:00:30.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:30 smithi099 ceph-mon[21984]: Cluster is now healthy 2024-08-20T03:00:30.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:30 smithi099 ceph-mon[21984]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-08-20T03:00:30.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:30 smithi099 ceph-mon[21984]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-08-20T03:00:30.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:30 smithi099 ceph-mon[21984]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-08-20T03:00:30.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:30 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2399227316' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-20T03:00:30.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:30 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] up:active 2024-08-20T03:00:30.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:30 smithi044 ceph-mon[18790]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-08-20T03:00:30.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:30 smithi044 ceph-mon[18790]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-20T03:00:30.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:30 smithi044 ceph-mon[18790]: Cluster is now healthy 2024-08-20T03:00:30.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:30 smithi044 ceph-mon[18790]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-08-20T03:00:30.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:30 smithi044 ceph-mon[18790]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-08-20T03:00:30.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:30 smithi044 ceph-mon[18790]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-08-20T03:00:30.760 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:30 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2399227316' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-20T03:00:31.129 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-08-20T03:00:31.129 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-08-20T03:00:31.390 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:31.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:31 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2399227316' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-20T03:00:31.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:31 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] up:active 2024-08-20T03:00:31.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:31 smithi165 ceph-mon[22357]: fsmap cephfs:5 7 up:standby 5 up:active 2024-08-20T03:00:31.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:31 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:00:31.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:31 smithi165 ceph-mon[22357]: pgmap v259: 97 pgs: 97 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 8 op/s 2024-08-20T03:00:31.637 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:31 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2399227316' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-20T03:00:31.637 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:31 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] up:active 2024-08-20T03:00:31.637 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:31 smithi044 ceph-mon[18790]: fsmap cephfs:5 7 up:standby 5 up:active 2024-08-20T03:00:31.637 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:31 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:00:31.637 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:31 smithi044 ceph-mon[18790]: pgmap v259: 97 pgs: 97 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 8 op/s 2024-08-20T03:00:31.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:31 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2399227316' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-20T03:00:31.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:31 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] up:active 2024-08-20T03:00:31.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:31 smithi099 ceph-mon[21984]: fsmap cephfs:5 7 up:standby 5 up:active 2024-08-20T03:00:31.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:31 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:00:31.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:31 smithi099 ceph-mon[21984]: pgmap v259: 97 pgs: 97 active+clean; 581 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 8 op/s 2024-08-20T03:00:31.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.830+0000 7f7e8f657640 1 -- 172.21.15.44:0/1323205541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e88107d60 msgr2=0x7f7e8810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:31.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.830+0000 7f7e8f657640 1 --2- 172.21.15.44:0/1323205541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e88107d60 0x7f7e8810e560 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009ca0 tx=0x7f7e8002a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:31.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.830+0000 7f7e8f657640 1 -- 172.21.15.44:0/1323205541 shutdown_connections 2024-08-20T03:00:31.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.830+0000 7f7e8f657640 1 --2- 172.21.15.44:0/1323205541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e88107d60 0x7f7e8810e560 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:31.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.830+0000 7f7e8f657640 1 --2- 172.21.15.44:0/1323205541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e881073e0 0x7f7e88107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:31.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.830+0000 7f7e8f657640 1 --2- 172.21.15.44:0/1323205541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e881019d0 0x7f7e88101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:31.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.830+0000 7f7e8f657640 1 -- 172.21.15.44:0/1323205541 >> 172.21.15.44:0/1323205541 conn(0x7f7e880fd440 msgr2=0x7f7e880ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:31.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.830+0000 7f7e8f657640 1 -- 172.21.15.44:0/1323205541 shutdown_connections 2024-08-20T03:00:31.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.830+0000 7f7e8f657640 1 -- 172.21.15.44:0/1323205541 wait complete. 2024-08-20T03:00:31.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.830+0000 7f7e8f657640 1 Processor -- start 2024-08-20T03:00:31.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.830+0000 7f7e8f657640 1 -- start start 2024-08-20T03:00:31.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8f657640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e881019d0 0x7f7e880fd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:31.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8f657640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e881073e0 0x7f7e880fde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:31.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8f657640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e88107d60 0x7f7e880fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:31.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8f657640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88111a20 con 0x7f7e881073e0 2024-08-20T03:00:31.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8f657640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e881118a0 con 0x7f7e881019d0 2024-08-20T03:00:31.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8f657640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7e88111ba0 con 0x7f7e88107d60 2024-08-20T03:00:31.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8cbf2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e881073e0 0x7f7e880fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:31.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8cbf2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e881073e0 0x7f7e880fde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57374/0 (socket says 172.21.15.44:57374) 2024-08-20T03:00:31.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8cbf2640 1 -- 172.21.15.44:0/1345071252 learned_addr learned my addr 172.21.15.44:0/1345071252 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:31.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8d3f3640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e881019d0 0x7f7e880fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:31.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8dbf4640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e88107d60 0x7f7e880fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:31.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8cbf2640 1 -- 172.21.15.44:0/1345071252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e88107d60 msgr2=0x7f7e880fe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:31.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8cbf2640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e88107d60 0x7f7e880fe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:31.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8cbf2640 1 -- 172.21.15.44:0/1345071252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e881019d0 msgr2=0x7f7e880fd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:31.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8cbf2640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e881019d0 0x7f7e880fd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:31.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8cbf2640 1 -- 172.21.15.44:0/1345071252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e881a7450 con 0x7f7e881073e0 2024-08-20T03:00:31.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8cbf2640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e881073e0 0x7f7e880fde80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800d950 tx=0x7f7e7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:31.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e7e7fc640 1 -- 172.21.15.44:0/1345071252 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78017070 con 0x7f7e881073e0 2024-08-20T03:00:31.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e7e7fc640 1 -- 172.21.15.44:0/1345071252 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e78009c30 con 0x7f7e881073e0 2024-08-20T03:00:31.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e8f657640 1 -- 172.21.15.44:0/1345071252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e881a7740 con 0x7f7e881073e0 2024-08-20T03:00:31.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.831+0000 7f7e7e7fc640 1 -- 172.21.15.44:0/1345071252 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800f620 con 0x7f7e881073e0 2024-08-20T03:00:31.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.832+0000 7f7e8f657640 1 -- 172.21.15.44:0/1345071252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e881a7c80 con 0x7f7e881073e0 2024-08-20T03:00:31.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.833+0000 7f7e8f657640 1 -- 172.21.15.44:0/1345071252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e88101db0 con 0x7f7e881073e0 2024-08-20T03:00:31.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.834+0000 7f7e7e7fc640 1 -- 172.21.15.44:0/1345071252 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) ==== 151076+0+0 (secure 0 0 0) 0x7f7e78010460 con 0x7f7e881073e0 2024-08-20T03:00:31.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.835+0000 7f7e7e7fc640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7e740b3c20 0x7f7e740b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:31.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.835+0000 7f7e8d3f3640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7e740b3c20 0x7f7e740b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:31.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.835+0000 7f7e7e7fc640 1 -- 172.21.15.44:0/1345071252 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f7e780e37c0 con 0x7f7e881073e0 2024-08-20T03:00:31.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.835+0000 7f7e8d3f3640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7e740b3c20 0x7f7e740b60e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e84005fd0 tx=0x7f7e8400b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:31.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:31.839+0000 7f7e7e7fc640 1 -- 172.21.15.44:0/1345071252 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7e78009de0 con 0x7f7e881073e0 2024-08-20T03:00:32.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.049+0000 7f7e8f657640 1 -- 172.21.15.44:0/1345071252 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7e8810abb0 con 0x7f7e740b3c20 2024-08-20T03:00:32.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.076+0000 7f7e7e7fc640 1 -- 172.21.15.44:0/1345071252 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e8810abb0 con 0x7f7e740b3c20 2024-08-20T03:00:32.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.077+0000 7f7e8f657640 1 -- 172.21.15.44:0/1345071252 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7e740b3c20 msgr2=0x7f7e740b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:32.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.077+0000 7f7e8f657640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7e740b3c20 0x7f7e740b60e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e84005fd0 tx=0x7f7e8400b390 comp rx=0 tx=0).stop 2024-08-20T03:00:32.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.078+0000 7f7e8f657640 1 -- 172.21.15.44:0/1345071252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e881073e0 msgr2=0x7f7e880fde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:32.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.078+0000 7f7e8f657640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e881073e0 0x7f7e880fde80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800d950 tx=0x7f7e7800de10 comp rx=0 tx=0).stop 2024-08-20T03:00:32.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.078+0000 7f7e8f657640 1 -- 172.21.15.44:0/1345071252 shutdown_connections 2024-08-20T03:00:32.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.078+0000 7f7e8f657640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7e740b3c20 0x7f7e740b60e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:32.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.078+0000 7f7e8f657640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e88107d60 0x7f7e880fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:32.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.078+0000 7f7e8f657640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e881073e0 0x7f7e880fde80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:32.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.078+0000 7f7e8f657640 1 --2- 172.21.15.44:0/1345071252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e881019d0 0x7f7e880fd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:32.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.078+0000 7f7e8f657640 1 -- 172.21.15.44:0/1345071252 >> 172.21.15.44:0/1345071252 conn(0x7f7e880fd440 msgr2=0x7f7e8806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:32.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.078+0000 7f7e8f657640 1 -- 172.21.15.44:0/1345071252 shutdown_connections 2024-08-20T03:00:32.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:32.078+0000 7f7e8f657640 1 -- 172.21.15.44:0/1345071252 wait complete. 2024-08-20T03:00:32.391 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:00:32 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:00:32.053+0000 7efc74f7a640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:00:32.473 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:00:32 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:00:32.053+0000 7efc74f7a640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:00:32.473 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:00:32 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:00:32.053+0000 7efc74f7a640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:00:32.473 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:00:32 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:00:32.053+0000 7efc74f7a640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:00:32.473 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:00:32 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:00:32.053+0000 7efc74f7a640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:00:32.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:32 smithi099 ceph-mon[21984]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:32.779 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:32 smithi044 ceph-mon[18790]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:32.811 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-08-20T03:00:32.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:32 smithi165 ceph-mon[22357]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:33.068 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:33.484 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:33 smithi044 ceph-mon[18790]: pgmap v260: 97 pgs: 97 active+clean; 584 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 5.0 KiB/s wr, 18 op/s 2024-08-20T03:00:33.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.568+0000 7f2f6f44f640 1 -- 172.21.15.44:0/1930637054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f68107e30 msgr2=0x7f2f6810e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:33.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.568+0000 7f2f6f44f640 1 --2- 172.21.15.44:0/1930637054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f68107e30 0x7f2f6810e580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009ca0 tx=0x7f2f6002a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:33.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 -- 172.21.15.44:0/1930637054 shutdown_connections 2024-08-20T03:00:33.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 --2- 172.21.15.44:0/1930637054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f68107e30 0x7f2f6810e580 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:33.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 --2- 172.21.15.44:0/1930637054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68107490 0x7f2f681078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:33.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 --2- 172.21.15.44:0/1930637054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f68102630 0x7f2f68102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:33.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 -- 172.21.15.44:0/1930637054 >> 172.21.15.44:0/1930637054 conn(0x7f2f68076c80 msgr2=0x7f2f68077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:33.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 -- 172.21.15.44:0/1930637054 shutdown_connections 2024-08-20T03:00:33.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 -- 172.21.15.44:0/1930637054 wait complete. 2024-08-20T03:00:33.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 Processor -- start 2024-08-20T03:00:33.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 -- start start 2024-08-20T03:00:33.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f68102630 0x7f2f68197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:33.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f68107490 0x7f2f681983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:33.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68107e30 0x7f2f6819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:33.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f681119b0 con 0x7f2f68107490 2024-08-20T03:00:33.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68111830 con 0x7f2f68107e30 2024-08-20T03:00:33.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.569+0000 7f2f6f44f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68111b30 con 0x7f2f68102630 2024-08-20T03:00:33.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6c9ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f68107490 0x7f2f681983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6c9ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f68107490 0x7f2f681983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57402/0 (socket says 172.21.15.44:57402) 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6c9ea640 1 -- 172.21.15.44:0/4070039714 learned_addr learned my addr 172.21.15.44:0/4070039714 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6d1eb640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f68102630 0x7f2f68197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6d9ec640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68107e30 0x7f2f6819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6c9ea640 1 -- 172.21.15.44:0/4070039714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f68102630 msgr2=0x7f2f68197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6c9ea640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f68102630 0x7f2f68197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6c9ea640 1 -- 172.21.15.44:0/4070039714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68107e30 msgr2=0x7f2f6819d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6c9ea640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68107e30 0x7f2f6819d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6c9ea640 1 -- 172.21.15.44:0/4070039714 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6819df10 con 0x7f2f68107490 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6d9ec640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68107e30 0x7f2f6819d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6c9ea640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f68107490 0x7f2f681983a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2f58007b50 tx=0x7f2f5800e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f5e7fc640 1 -- 172.21.15.44:0/4070039714 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58007c40 con 0x7f2f68107490 2024-08-20T03:00:33.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f5e7fc640 1 -- 172.21.15.44:0/4070039714 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f58007de0 con 0x7f2f68107490 2024-08-20T03:00:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f5e7fc640 1 -- 172.21.15.44:0/4070039714 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5800ace0 con 0x7f2f68107490 2024-08-20T03:00:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6f44f640 1 -- 172.21.15.44:0/4070039714 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6819e200 con 0x7f2f68107490 2024-08-20T03:00:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.570+0000 7f2f6f44f640 1 -- 172.21.15.44:0/4070039714 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f681012c0 con 0x7f2f68107490 2024-08-20T03:00:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.572+0000 7f2f6f44f640 1 -- 172.21.15.44:0/4070039714 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f68102a10 con 0x7f2f68107490 2024-08-20T03:00:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.573+0000 7f2f5e7fc640 1 -- 172.21.15.44:0/4070039714 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f2f5800eb30 con 0x7f2f68107490 2024-08-20T03:00:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.574+0000 7f2f5e7fc640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2f540b3cb0 0x7f2f540b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.574+0000 7f2f6d1eb640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2f540b3cb0 0x7f2f540b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.574+0000 7f2f5e7fc640 1 -- 172.21.15.44:0/4070039714 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f2f580e52f0 con 0x7f2f68107490 2024-08-20T03:00:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.574+0000 7f2f6d1eb640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2f540b3cb0 0x7f2f540b6170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f64005fd0 tx=0x7f2f6400b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:33.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.578+0000 7f2f5e7fc640 1 -- 172.21.15.44:0/4070039714 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2f58023060 con 0x7f2f68107490 2024-08-20T03:00:33.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.788+0000 7f2f6f44f640 1 -- 172.21.15.44:0/4070039714 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2f68101df0 con 0x7f2f540b3cb0 2024-08-20T03:00:33.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.814+0000 7f2f5e7fc640 1 -- 172.21.15.44:0/4070039714 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2f68101df0 con 0x7f2f540b3cb0 2024-08-20T03:00:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.816+0000 7f2f6f44f640 1 -- 172.21.15.44:0/4070039714 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2f540b3cb0 msgr2=0x7f2f540b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.816+0000 7f2f6f44f640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2f540b3cb0 0x7f2f540b6170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f64005fd0 tx=0x7f2f6400b390 comp rx=0 tx=0).stop 2024-08-20T03:00:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.816+0000 7f2f6f44f640 1 -- 172.21.15.44:0/4070039714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f68107490 msgr2=0x7f2f681983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.816+0000 7f2f6f44f640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f68107490 0x7f2f681983a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2f58007b50 tx=0x7f2f5800e420 comp rx=0 tx=0).stop 2024-08-20T03:00:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.816+0000 7f2f6f44f640 1 -- 172.21.15.44:0/4070039714 shutdown_connections 2024-08-20T03:00:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.816+0000 7f2f6f44f640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2f540b3cb0 0x7f2f540b6170 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.816+0000 7f2f6f44f640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f68107e30 0x7f2f6819d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.816+0000 7f2f6f44f640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f68107490 0x7f2f681983a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.816+0000 7f2f6f44f640 1 --2- 172.21.15.44:0/4070039714 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f68102630 0x7f2f68197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.816+0000 7f2f6f44f640 1 -- 172.21.15.44:0/4070039714 >> 172.21.15.44:0/4070039714 conn(0x7f2f68076c80 msgr2=0x7f2f68100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:33.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.816+0000 7f2f6f44f640 1 -- 172.21.15.44:0/4070039714 shutdown_connections 2024-08-20T03:00:33.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:33.816+0000 7f2f6f44f640 1 -- 172.21.15.44:0/4070039714 wait complete. 2024-08-20T03:00:33.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:33 smithi165 ceph-mon[22357]: pgmap v260: 97 pgs: 97 active+clean; 584 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 5.0 KiB/s wr, 18 op/s 2024-08-20T03:00:33.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:33 smithi099 ceph-mon[21984]: pgmap v260: 97 pgs: 97 active+clean; 584 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 5.0 KiB/s wr, 18 op/s 2024-08-20T03:00:34.498 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:34 smithi044 ceph-mon[18790]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-08-20T03:00:34.498 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:34 smithi044 ceph-mon[18790]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:34.529 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-08-20T03:00:34.784 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:34.856 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:34 smithi165 ceph-mon[22357]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-08-20T03:00:34.856 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:34 smithi165 ceph-mon[22357]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:34.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:34 smithi099 ceph-mon[21984]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-08-20T03:00:34.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:34 smithi099 ceph-mon[21984]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:35.136 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:00:34 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:00:34.854+0000 7f9a1477a640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-08-20T03:00:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.269+0000 7fbb09758640 1 -- 172.21.15.44:0/983421616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb041073b0 msgr2=0x7fbb041077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:35.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.269+0000 7fbb09758640 1 --2- 172.21.15.44:0/983421616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb041073b0 0x7fbb041077f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009900 tx=0x7fbaf802a410 comp rx=0 tx=0).stop 2024-08-20T03:00:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.269+0000 7fbb09758640 1 -- 172.21.15.44:0/983421616 shutdown_connections 2024-08-20T03:00:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.269+0000 7fbb09758640 1 --2- 172.21.15.44:0/983421616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb04107d30 0x7fbb0410e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:35.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.269+0000 7fbb09758640 1 --2- 172.21.15.44:0/983421616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb041073b0 0x7fbb041077f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.269+0000 7fbb09758640 1 --2- 172.21.15.44:0/983421616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041019a0 0x7fbb04101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:35.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.269+0000 7fbb09758640 1 -- 172.21.15.44:0/983421616 >> 172.21.15.44:0/983421616 conn(0x7fbb040fd3f0 msgr2=0x7fbb040ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.269+0000 7fbb09758640 1 -- 172.21.15.44:0/983421616 shutdown_connections 2024-08-20T03:00:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.269+0000 7fbb09758640 1 -- 172.21.15.44:0/983421616 wait complete. 2024-08-20T03:00:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.270+0000 7fbb09758640 1 Processor -- start 2024-08-20T03:00:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.270+0000 7fbb09758640 1 -- start start 2024-08-20T03:00:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.270+0000 7fbb09758640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041019a0 0x7fbb04198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.270+0000 7fbb09758640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb041073b0 0x7fbb041994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.270+0000 7fbb09758640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb04107d30 0x7fbb0419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.270+0000 7fbb09758640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04112b10 con 0x7fbb04107d30 2024-08-20T03:00:35.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.270+0000 7fbb09758640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04112990 con 0x7fbb041073b0 2024-08-20T03:00:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb09758640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04112c90 con 0x7fbb041019a0 2024-08-20T03:00:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb04107d30 0x7fbb0419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb04107d30 0x7fbb0419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57418/0 (socket says 172.21.15.44:57418) 2024-08-20T03:00:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb037fe640 1 -- 172.21.15.44:0/3601674571 learned_addr learned my addr 172.21.15.44:0/3601674571 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb02ffd640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041019a0 0x7fbb04198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb027fc640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb041073b0 0x7fbb041994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb037fe640 1 -- 172.21.15.44:0/3601674571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041019a0 msgr2=0x7fbb04198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb037fe640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041019a0 0x7fbb04198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb037fe640 1 -- 172.21.15.44:0/3601674571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb041073b0 msgr2=0x7fbb041994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:35.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb037fe640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb041073b0 0x7fbb041994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb037fe640 1 -- 172.21.15.44:0/3601674571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0419f010 con 0x7fbb04107d30 2024-08-20T03:00:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb027fc640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb041073b0 0x7fbb041994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:00:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb037fe640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb04107d30 0x7fbb0419e840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00d750 tx=0x7fbafc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbaebfff640 1 -- 172.21.15.44:0/3601674571 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc017070 con 0x7fbb04107d30 2024-08-20T03:00:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbaebfff640 1 -- 172.21.15.44:0/3601674571 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbafc0054d0 con 0x7fbb04107d30 2024-08-20T03:00:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb09758640 1 -- 172.21.15.44:0/3601674571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0419f300 con 0x7fbb04107d30 2024-08-20T03:00:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbaebfff640 1 -- 172.21.15.44:0/3601674571 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc004c60 con 0x7fbb04107d30 2024-08-20T03:00:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.271+0000 7fbb09758640 1 -- 172.21.15.44:0/3601674571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb041a59c0 con 0x7fbb04107d30 2024-08-20T03:00:35.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.272+0000 7fbb09758640 1 -- 172.21.15.44:0/3601674571 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb04101d80 con 0x7fbb04107d30 2024-08-20T03:00:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.274+0000 7fbaebfff640 1 -- 172.21.15.44:0/3601674571 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fbafc009bd0 con 0x7fbb04107d30 2024-08-20T03:00:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.274+0000 7fbaebfff640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbaec0b3d30 0x7fbaec0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.274+0000 7fbb02ffd640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbaec0b3d30 0x7fbaec0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.274+0000 7fbaebfff640 1 -- 172.21.15.44:0/3601674571 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fbafc0e3b60 con 0x7fbb04107d30 2024-08-20T03:00:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.275+0000 7fbb02ffd640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbaec0b3d30 0x7fbaec0b61f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005fd0 tx=0x7fbaf4009680 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:35.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.277+0000 7fbaebfff640 1 -- 172.21.15.44:0/3601674571 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fbafc0ab0f0 con 0x7fbb04107d30 2024-08-20T03:00:35.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.490+0000 7fbb09758640 1 -- 172.21.15.44:0/3601674571 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fbb040632c0 con 0x7fbaec0b3d30 2024-08-20T03:00:35.493 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:35 smithi044 ceph-mon[18790]: pgmap v261: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.3 KiB/s wr, 18 op/s 2024-08-20T03:00:35.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.495+0000 7fbaebfff640 1 -- 172.21.15.44:0/3601674571 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbb040632c0 con 0x7fbaec0b3d30 2024-08-20T03:00:35.498 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/qa/sv_0/9b854aa7-1d2c-4327-b6be-47aad46c8255 2024-08-20T03:00:35.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.497+0000 7fbb09758640 1 -- 172.21.15.44:0/3601674571 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbaec0b3d30 msgr2=0x7fbaec0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.497+0000 7fbb09758640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbaec0b3d30 0x7fbaec0b61f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005fd0 tx=0x7fbaf4009680 comp rx=0 tx=0).stop 2024-08-20T03:00:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.497+0000 7fbb09758640 1 -- 172.21.15.44:0/3601674571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb04107d30 msgr2=0x7fbb0419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.497+0000 7fbb09758640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb04107d30 0x7fbb0419e840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00d750 tx=0x7fbafc00dc10 comp rx=0 tx=0).stop 2024-08-20T03:00:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.497+0000 7fbb09758640 1 -- 172.21.15.44:0/3601674571 shutdown_connections 2024-08-20T03:00:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.497+0000 7fbb09758640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbaec0b3d30 0x7fbaec0b61f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.497+0000 7fbb09758640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb04107d30 0x7fbb0419e840 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.497+0000 7fbb09758640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb041073b0 0x7fbb041994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.497+0000 7fbb09758640 1 --2- 172.21.15.44:0/3601674571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb041019a0 0x7fbb04198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.497+0000 7fbb09758640 1 -- 172.21.15.44:0/3601674571 >> 172.21.15.44:0/3601674571 conn(0x7fbb040fd3f0 msgr2=0x7fbb040ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:35.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.498+0000 7fbb09758640 1 -- 172.21.15.44:0/3601674571 shutdown_connections 2024-08-20T03:00:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:35.498+0000 7fbb09758640 1 -- 172.21.15.44:0/3601674571 wait complete. 2024-08-20T03:00:35.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:35 smithi165 ceph-mon[22357]: pgmap v261: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.3 KiB/s wr, 18 op/s 2024-08-20T03:00:35.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:35 smithi099 ceph-mon[21984]: pgmap v261: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.3 KiB/s wr, 18 op/s 2024-08-20T03:00:36.188 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-08-20T03:00:36.444 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:36.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:36 smithi044 ceph-mon[18790]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:36.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:36 smithi165 ceph-mon[22357]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:36.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:36 smithi099 ceph-mon[21984]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:36.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.921+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/3153809030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9d4113830 msgr2=0x7fb9d4115c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:36.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.921+0000 7fb9dbbc4640 1 --2- 172.21.15.44:0/3153809030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9d4113830 0x7fb9d4115c20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009ca0 tx=0x7fb9cc02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:36.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.921+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/3153809030 shutdown_connections 2024-08-20T03:00:36.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.921+0000 7fb9dbbc4640 1 --2- 172.21.15.44:0/3153809030 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9d4113830 0x7fb9d4115c20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:36.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.921+0000 7fb9dbbc4640 1 --2- 172.21.15.44:0/3153809030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d4075b90 0x7fb9d41132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:36.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.921+0000 7fb9dbbc4640 1 --2- 172.21.15.44:0/3153809030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9d4075270 0x7fb9d4075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.921+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/3153809030 >> 172.21.15.44:0/3153809030 conn(0x7fb9d40fe540 msgr2=0x7fb9d4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.921+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/3153809030 shutdown_connections 2024-08-20T03:00:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.921+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/3153809030 wait complete. 2024-08-20T03:00:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9dbbc4640 1 Processor -- start 2024-08-20T03:00:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9dbbc4640 1 -- start start 2024-08-20T03:00:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9dbbc4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d4075270 0x7fb9d419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9dbbc4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9d4075b90 0x7fb9d419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9dbbc4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9d4113830 0x7fb9d41a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9dbbc4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d41190a0 con 0x7fb9d4075b90 2024-08-20T03:00:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9dbbc4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d4118f20 con 0x7fb9d4075270 2024-08-20T03:00:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9dbbc4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d4119220 con 0x7fb9d4113830 2024-08-20T03:00:36.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9d915f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9d4075b90 0x7fb9d419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9d915f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9d4075b90 0x7fb9d419fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57438/0 (socket says 172.21.15.44:57438) 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9d915f640 1 -- 172.21.15.44:0/620616152 learned_addr learned my addr 172.21.15.44:0/620616152 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9da161640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9d4113830 0x7fb9d41a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9d9960640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d4075270 0x7fb9d419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9d915f640 1 -- 172.21.15.44:0/620616152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9d4113830 msgr2=0x7fb9d41a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9d915f640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9d4113830 0x7fb9d41a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9d915f640 1 -- 172.21.15.44:0/620616152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d4075270 msgr2=0x7fb9d419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9d915f640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d4075270 0x7fb9d419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9d915f640 1 -- 172.21.15.44:0/620616152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d41a5760 con 0x7fb9d4075b90 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.922+0000 7fb9d9960640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d4075270 0x7fb9d419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.923+0000 7fb9d915f640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9d4075b90 0x7fb9d419fbf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400d950 tx=0x7fb9c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.923+0000 7fb9caffd640 1 -- 172.21.15.44:0/620616152 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4017070 con 0x7fb9d4075b90 2024-08-20T03:00:36.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.923+0000 7fb9caffd640 1 -- 172.21.15.44:0/620616152 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c4009c30 con 0x7fb9d4075b90 2024-08-20T03:00:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.923+0000 7fb9caffd640 1 -- 172.21.15.44:0/620616152 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400f620 con 0x7fb9d4075b90 2024-08-20T03:00:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.923+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/620616152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d41a5a50 con 0x7fb9d4075b90 2024-08-20T03:00:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.923+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/620616152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d41ac2e0 con 0x7fb9d4075b90 2024-08-20T03:00:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.924+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/620616152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9d4076780 con 0x7fb9d4075b90 2024-08-20T03:00:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.925+0000 7fb9caffd640 1 -- 172.21.15.44:0/620616152 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb9c4010460 con 0x7fb9d4075b90 2024-08-20T03:00:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.926+0000 7fb9caffd640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb9c00b3d30 0x7fb9c00b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.926+0000 7fb9d9960640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb9c00b3d30 0x7fb9c00b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.926+0000 7fb9caffd640 1 -- 172.21.15.44:0/620616152 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fb9c40e39a0 con 0x7fb9d4075b90 2024-08-20T03:00:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.926+0000 7fb9d9960640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb9c00b3d30 0x7fb9c00b61f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00058c0 tx=0x7fb9d0005d30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:36.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:36.929+0000 7fb9caffd640 1 -- 172.21.15.44:0/620616152 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb9c40aaf30 con 0x7fb9d4075b90 2024-08-20T03:00:37.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.140+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/620616152 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb9d40632c0 con 0x7fb9c00b3d30 2024-08-20T03:00:37.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.166+0000 7fb9caffd640 1 -- 172.21.15.44:0/620616152 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb9d40632c0 con 0x7fb9c00b3d30 2024-08-20T03:00:37.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.169+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/620616152 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb9c00b3d30 msgr2=0x7fb9c00b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:37.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.169+0000 7fb9dbbc4640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb9c00b3d30 0x7fb9c00b61f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00058c0 tx=0x7fb9d0005d30 comp rx=0 tx=0).stop 2024-08-20T03:00:37.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.169+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/620616152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9d4075b90 msgr2=0x7fb9d419fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:37.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.169+0000 7fb9dbbc4640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9d4075b90 0x7fb9d419fbf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400d950 tx=0x7fb9c400de10 comp rx=0 tx=0).stop 2024-08-20T03:00:37.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.169+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/620616152 shutdown_connections 2024-08-20T03:00:37.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.169+0000 7fb9dbbc4640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb9c00b3d30 0x7fb9c00b61f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:37.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.169+0000 7fb9dbbc4640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9d4113830 0x7fb9d41a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:37.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.169+0000 7fb9dbbc4640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9d4075b90 0x7fb9d419fbf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:37.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.169+0000 7fb9dbbc4640 1 --2- 172.21.15.44:0/620616152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9d4075270 0x7fb9d419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:37.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.169+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/620616152 >> 172.21.15.44:0/620616152 conn(0x7fb9d40fe540 msgr2=0x7fb9d40ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:37.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.169+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/620616152 shutdown_connections 2024-08-20T03:00:37.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:37.169+0000 7fb9dbbc4640 1 -- 172.21.15.44:0/620616152 wait complete. 2024-08-20T03:00:37.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:37 smithi044 ceph-mon[18790]: pgmap v262: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.6 KiB/s wr, 21 op/s 2024-08-20T03:00:37.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:37 smithi044 ceph-mon[18790]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:37.875 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-08-20T03:00:37.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:37 smithi165 ceph-mon[22357]: pgmap v262: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.6 KiB/s wr, 21 op/s 2024-08-20T03:00:37.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:37 smithi165 ceph-mon[22357]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:37.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:37 smithi099 ceph-mon[21984]: pgmap v262: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.6 KiB/s wr, 21 op/s 2024-08-20T03:00:37.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:37 smithi099 ceph-mon[21984]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:38.129 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.633+0000 7fe152550640 1 -- 172.21.15.44:0/1293437457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe14c107490 msgr2=0x7fe14c1078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.633+0000 7fe152550640 1 --2- 172.21.15.44:0/1293437457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe14c107490 0x7fe14c1078f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe140009900 tx=0x7fe14002a410 comp rx=0 tx=0).stop 2024-08-20T03:00:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.633+0000 7fe152550640 1 -- 172.21.15.44:0/1293437457 shutdown_connections 2024-08-20T03:00:38.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.633+0000 7fe152550640 1 --2- 172.21.15.44:0/1293437457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe14c107e30 0x7fe14c10e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.633+0000 7fe152550640 1 --2- 172.21.15.44:0/1293437457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe14c107490 0x7fe14c1078f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.633+0000 7fe152550640 1 --2- 172.21.15.44:0/1293437457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe14c102630 0x7fe14c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.633+0000 7fe152550640 1 -- 172.21.15.44:0/1293437457 >> 172.21.15.44:0/1293437457 conn(0x7fe14c076c80 msgr2=0x7fe14c077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.633+0000 7fe152550640 1 -- 172.21.15.44:0/1293437457 shutdown_connections 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.633+0000 7fe152550640 1 -- 172.21.15.44:0/1293437457 wait complete. 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.634+0000 7fe152550640 1 Processor -- start 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.634+0000 7fe152550640 1 -- start start 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.634+0000 7fe152550640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe14c102630 0x7fe14c103500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.634+0000 7fe152550640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe14c107490 0x7fe14c103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.634+0000 7fe152550640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe14c107e30 0x7fe14c103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.634+0000 7fe152550640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c111970 con 0x7fe14c107e30 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.634+0000 7fe152550640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1117f0 con 0x7fe14c107490 2024-08-20T03:00:38.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.634+0000 7fe152550640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c111af0 con 0x7fe14c102630 2024-08-20T03:00:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.634+0000 7fe150aed640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe14c107e30 0x7fe14c103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe150aed640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe14c107e30 0x7fe14c103f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54544/0 (socket says 172.21.15.44:54544) 2024-08-20T03:00:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe150aed640 1 -- 172.21.15.44:0/3988166884 learned_addr learned my addr 172.21.15.44:0/3988166884 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe14bfff640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe14c102630 0x7fe14c103500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe14b7fe640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe14c107490 0x7fe14c103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe150aed640 1 -- 172.21.15.44:0/3988166884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe14c102630 msgr2=0x7fe14c103500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe150aed640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe14c102630 0x7fe14c103500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe150aed640 1 -- 172.21.15.44:0/3988166884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe14c107490 msgr2=0x7fe14c103a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe150aed640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe14c107490 0x7fe14c103a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:38.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe150aed640 1 -- 172.21.15.44:0/3988166884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c1a73e0 con 0x7fe14c107e30 2024-08-20T03:00:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe14b7fe640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe14c107490 0x7fe14c103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:00:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe150aed640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe14c107e30 0x7fe14c103f80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe14400eaa0 tx=0x7fe14400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe1497fa640 1 -- 172.21.15.44:0/3988166884 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14400cdb0 con 0x7fe14c107e30 2024-08-20T03:00:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe1497fa640 1 -- 172.21.15.44:0/3988166884 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe144005020 con 0x7fe14c107e30 2024-08-20T03:00:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe152550640 1 -- 172.21.15.44:0/3988166884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14c1a76d0 con 0x7fe14c107e30 2024-08-20T03:00:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe1497fa640 1 -- 172.21.15.44:0/3988166884 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1440116d0 con 0x7fe14c107e30 2024-08-20T03:00:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.635+0000 7fe152550640 1 -- 172.21.15.44:0/3988166884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14c1a7bc0 con 0x7fe14c107e30 2024-08-20T03:00:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.636+0000 7fe152550640 1 -- 172.21.15.44:0/3988166884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe14c102a10 con 0x7fe14c107e30 2024-08-20T03:00:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.637+0000 7fe1497fa640 1 -- 172.21.15.44:0/3988166884 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fe1440051d0 con 0x7fe14c107e30 2024-08-20T03:00:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.638+0000 7fe1497fa640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe1340b3cb0 0x7fe1340b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.638+0000 7fe14bfff640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe1340b3cb0 0x7fe1340b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:38.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.638+0000 7fe1497fa640 1 -- 172.21.15.44:0/3988166884 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fe1440e3fe0 con 0x7fe14c107e30 2024-08-20T03:00:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.639+0000 7fe14bfff640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe1340b3cb0 0x7fe1340b6170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00bbe0 tx=0x7fe13c00b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:38.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.642+0000 7fe1497fa640 1 -- 172.21.15.44:0/3988166884 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe144016030 con 0x7fe14c107e30 2024-08-20T03:00:38.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.854+0000 7fe152550640 1 -- 172.21.15.44:0/3988166884 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fe14c0632c0 con 0x7fe1340b3cb0 2024-08-20T03:00:38.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.858+0000 7fe1497fa640 1 -- 172.21.15.44:0/3988166884 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe14c0632c0 con 0x7fe1340b3cb0 2024-08-20T03:00:38.860 INFO:teuthology.orchestra.run.smithi044.stdout:/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2 2024-08-20T03:00:38.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.860+0000 7fe152550640 1 -- 172.21.15.44:0/3988166884 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe1340b3cb0 msgr2=0x7fe1340b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.860+0000 7fe152550640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe1340b3cb0 0x7fe1340b6170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00bbe0 tx=0x7fe13c00b390 comp rx=0 tx=0).stop 2024-08-20T03:00:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.860+0000 7fe152550640 1 -- 172.21.15.44:0/3988166884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe14c107e30 msgr2=0x7fe14c103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.860+0000 7fe152550640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe14c107e30 0x7fe14c103f80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe14400eaa0 tx=0x7fe14400ef60 comp rx=0 tx=0).stop 2024-08-20T03:00:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.860+0000 7fe152550640 1 -- 172.21.15.44:0/3988166884 shutdown_connections 2024-08-20T03:00:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.860+0000 7fe152550640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe1340b3cb0 0x7fe1340b6170 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.860+0000 7fe152550640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe14c107e30 0x7fe14c103f80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.860+0000 7fe152550640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe14c107490 0x7fe14c103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.860+0000 7fe152550640 1 --2- 172.21.15.44:0/3988166884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe14c102630 0x7fe14c103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.860+0000 7fe152550640 1 -- 172.21.15.44:0/3988166884 >> 172.21.15.44:0/3988166884 conn(0x7fe14c076c80 msgr2=0x7fe14c105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.860+0000 7fe152550640 1 -- 172.21.15.44:0/3988166884 shutdown_connections 2024-08-20T03:00:38.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:38.860+0000 7fe152550640 1 -- 172.21.15.44:0/3988166884 wait complete. 2024-08-20T03:00:39.575 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:00:39.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:39 smithi044 ceph-mon[18790]: pgmap v263: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.9 KiB/s wr, 22 op/s 2024-08-20T03:00:39.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:39 smithi044 ceph-mon[18790]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:39.827 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:39.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:39 smithi165 ceph-mon[22357]: pgmap v263: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.9 KiB/s wr, 22 op/s 2024-08-20T03:00:39.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:39 smithi165 ceph-mon[22357]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:39.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:39 smithi099 ceph-mon[21984]: pgmap v263: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.9 KiB/s wr, 22 op/s 2024-08-20T03:00:39.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:39 smithi099 ceph-mon[21984]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:00:40.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.280+0000 7fb591b98640 1 -- 172.21.15.44:0/3331913413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb58c1039a0 msgr2=0x7fb58c103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:40.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.280+0000 7fb591b98640 1 --2- 172.21.15.44:0/3331913413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb58c1039a0 0x7fb58c103d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009900 tx=0x7fb57c02a410 comp rx=0 tx=0).stop 2024-08-20T03:00:40.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.280+0000 7fb591b98640 1 -- 172.21.15.44:0/3331913413 shutdown_connections 2024-08-20T03:00:40.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.280+0000 7fb591b98640 1 --2- 172.21.15.44:0/3331913413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb58c10ce00 0x7fb58c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:40.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.280+0000 7fb591b98640 1 --2- 172.21.15.44:0/3331913413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb58c1042c0 0x7fb58c10c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:40.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.280+0000 7fb591b98640 1 --2- 172.21.15.44:0/3331913413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb58c1039a0 0x7fb58c103d80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:40.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.280+0000 7fb591b98640 1 -- 172.21.15.44:0/3331913413 >> 172.21.15.44:0/3331913413 conn(0x7fb58c0fd400 msgr2=0x7fb58c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:40.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.281+0000 7fb591b98640 1 -- 172.21.15.44:0/3331913413 shutdown_connections 2024-08-20T03:00:40.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.281+0000 7fb591b98640 1 -- 172.21.15.44:0/3331913413 wait complete. 2024-08-20T03:00:40.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.281+0000 7fb591b98640 1 Processor -- start 2024-08-20T03:00:40.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.281+0000 7fb591b98640 1 -- start start 2024-08-20T03:00:40.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb591b98640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb58c1039a0 0x7fb58c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:40.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb591b98640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb58c1042c0 0x7fb58c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:40.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb591b98640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb58c10ce00 0x7fb58c19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:40.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb591b98640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c112ad0 con 0x7fb58c1039a0 2024-08-20T03:00:40.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb591b98640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c112950 con 0x7fb58c10ce00 2024-08-20T03:00:40.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb591b98640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb58c112c50 con 0x7fb58c1042c0 2024-08-20T03:00:40.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb58b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb58c1039a0 0x7fb58c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:40.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb58b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb58c1039a0 0x7fb58c199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54570/0 (socket says 172.21.15.44:54570) 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb58b7fe640 1 -- 172.21.15.44:0/1335172157 learned_addr learned my addr 172.21.15.44:0/1335172157 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb58bfff640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb58c10ce00 0x7fb58c19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb582ffd640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb58c1042c0 0x7fb58c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb58b7fe640 1 -- 172.21.15.44:0/1335172157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb58c1042c0 msgr2=0x7fb58c199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb58b7fe640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb58c1042c0 0x7fb58c199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb58b7fe640 1 -- 172.21.15.44:0/1335172157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb58c10ce00 msgr2=0x7fb58c19e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb58b7fe640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb58c10ce00 0x7fb58c19e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb58b7fe640 1 -- 172.21.15.44:0/1335172157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58c19f0d0 con 0x7fb58c1039a0 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.282+0000 7fb58b7fe640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb58c1039a0 0x7fb58c199020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb57c005940 tx=0x7fb57c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.283+0000 7fb5897fa640 1 -- 172.21.15.44:0/1335172157 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c03f030 con 0x7fb58c1039a0 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.283+0000 7fb5897fa640 1 -- 172.21.15.44:0/1335172157 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb57c003cb0 con 0x7fb58c1039a0 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.283+0000 7fb591b98640 1 -- 172.21.15.44:0/1335172157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58c19f360 con 0x7fb58c1039a0 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.283+0000 7fb5897fa640 1 -- 172.21.15.44:0/1335172157 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c035070 con 0x7fb58c1039a0 2024-08-20T03:00:40.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.283+0000 7fb591b98640 1 -- 172.21.15.44:0/1335172157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb58c1a5a80 con 0x7fb58c1039a0 2024-08-20T03:00:40.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.284+0000 7fb591b98640 1 -- 172.21.15.44:0/1335172157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb58c103d80 con 0x7fb58c1039a0 2024-08-20T03:00:40.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.285+0000 7fb5897fa640 1 -- 172.21.15.44:0/1335172157 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb57c042030 con 0x7fb58c1039a0 2024-08-20T03:00:40.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.286+0000 7fb5897fa640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb5780b3cb0 0x7fb5780b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:40.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.286+0000 7fb582ffd640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb5780b3cb0 0x7fb5780b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:40.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.286+0000 7fb5897fa640 1 -- 172.21.15.44:0/1335172157 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fb57c104940 con 0x7fb58c1039a0 2024-08-20T03:00:40.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.286+0000 7fb582ffd640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb5780b3cb0 0x7fb5780b6170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb58c10f4d0 tx=0x7fb574008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:40.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.289+0000 7fb5897fa640 1 -- 172.21.15.44:0/1335172157 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb57c0cbe50 con 0x7fb58c1039a0 2024-08-20T03:00:40.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.512+0000 7fb591b98640 1 -- 172.21.15.44:0/1335172157 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb58c19a150 con 0x7fb58c1039a0 2024-08-20T03:00:40.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.513+0000 7fb5897fa640 1 -- 172.21.15.44:0/1335172157 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11965 (secure 0 0 0) 0x7fb57c0cb750 con 0x7fb58c1039a0 2024-08-20T03:00:40.515 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:40.515 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"btime":"2024-08-20T03:00:30:384117+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:00:30.384102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24604,24589,24596,24583,24569]},"id":1}]} 2024-08-20T03:00:40.516 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-08-20T03:00:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.516+0000 7fb591b98640 1 -- 172.21.15.44:0/1335172157 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb5780b3cb0 msgr2=0x7fb5780b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:40.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.516+0000 7fb591b98640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb5780b3cb0 0x7fb5780b6170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb58c10f4d0 tx=0x7fb574008040 comp rx=0 tx=0).stop 2024-08-20T03:00:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.516+0000 7fb591b98640 1 -- 172.21.15.44:0/1335172157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb58c1039a0 msgr2=0x7fb58c199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.516+0000 7fb591b98640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb58c1039a0 0x7fb58c199020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb57c005940 tx=0x7fb57c0042f0 comp rx=0 tx=0).stop 2024-08-20T03:00:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.516+0000 7fb591b98640 1 -- 172.21.15.44:0/1335172157 shutdown_connections 2024-08-20T03:00:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.516+0000 7fb591b98640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb5780b3cb0 0x7fb5780b6170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.516+0000 7fb591b98640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb58c10ce00 0x7fb58c19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.516+0000 7fb591b98640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb58c1042c0 0x7fb58c199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.516+0000 7fb591b98640 1 --2- 172.21.15.44:0/1335172157 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb58c1039a0 0x7fb58c199020 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.516+0000 7fb591b98640 1 -- 172.21.15.44:0/1335172157 >> 172.21.15.44:0/1335172157 conn(0x7fb58c0fd400 msgr2=0x7fb58c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.516+0000 7fb591b98640 1 -- 172.21.15.44:0/1335172157 shutdown_connections 2024-08-20T03:00:40.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:40.516+0000 7fb591b98640 1 -- 172.21.15.44:0/1335172157 wait complete. 2024-08-20T03:00:41.228 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T03:00:41.483 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:41.730 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:41 smithi044 ceph-mon[18790]: pgmap v264: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-08-20T03:00:41.730 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:41 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1335172157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:00:41.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:41 smithi165 ceph-mon[22357]: pgmap v264: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-08-20T03:00:41.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:41 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1335172157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:00:41.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:41 smithi099 ceph-mon[21984]: pgmap v264: 97 pgs: 97 active+clean; 610 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-08-20T03:00:41.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:41 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1335172157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:00:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.956+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/557154967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0b8104300 msgr2=0x7fb0b810c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:41.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.956+0000 7fb0bf2c0640 1 --2- 172.21.15.44:0/557154967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0b8104300 0x7fb0b810c7f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009900 tx=0x7fb0ac02a410 comp rx=0 tx=0).stop 2024-08-20T03:00:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.956+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/557154967 shutdown_connections 2024-08-20T03:00:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.956+0000 7fb0bf2c0640 1 --2- 172.21.15.44:0/557154967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0b810ce40 0x7fb0b810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.956+0000 7fb0bf2c0640 1 --2- 172.21.15.44:0/557154967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0b8104300 0x7fb0b810c7f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.956+0000 7fb0bf2c0640 1 --2- 172.21.15.44:0/557154967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0b81039e0 0x7fb0b8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.956+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/557154967 >> 172.21.15.44:0/557154967 conn(0x7fb0b80fd440 msgr2=0x7fb0b80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.956+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/557154967 shutdown_connections 2024-08-20T03:00:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.956+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/557154967 wait complete. 2024-08-20T03:00:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.956+0000 7fb0bf2c0640 1 Processor -- start 2024-08-20T03:00:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.956+0000 7fb0bf2c0640 1 -- start start 2024-08-20T03:00:41.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bf2c0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0b81039e0 0x7fb0b819b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bf2c0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0b8104300 0x7fb0b819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bf2c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0b810ce40 0x7fb0b81a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bf2c0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8112ab0 con 0x7fb0b810ce40 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bf2c0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8112930 con 0x7fb0b81039e0 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bf2c0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8112c30 con 0x7fb0b8104300 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bd85d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0b810ce40 0x7fb0b81a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bd85d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0b810ce40 0x7fb0b81a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54594/0 (socket says 172.21.15.44:54594) 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bd85d640 1 -- 172.21.15.44:0/1683956122 learned_addr learned my addr 172.21.15.44:0/1683956122 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bc85b640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0b8104300 0x7fb0b819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bd05c640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0b81039e0 0x7fb0b819b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bd85d640 1 -- 172.21.15.44:0/1683956122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0b8104300 msgr2=0x7fb0b819b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bd85d640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0b8104300 0x7fb0b819b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:41.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bd85d640 1 -- 172.21.15.44:0/1683956122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0b81039e0 msgr2=0x7fb0b819b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bd85d640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0b81039e0 0x7fb0b819b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bd85d640 1 -- 172.21.15.44:0/1683956122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b81a1310 con 0x7fb0b810ce40 2024-08-20T03:00:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bd85d640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0b810ce40 0x7fb0b81a0b40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000d750 tx=0x7fb0b000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0b67fc640 1 -- 172.21.15.44:0/1683956122 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0017070 con 0x7fb0b810ce40 2024-08-20T03:00:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0b67fc640 1 -- 172.21.15.44:0/1683956122 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b00054d0 con 0x7fb0b810ce40 2024-08-20T03:00:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/1683956122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b81a1600 con 0x7fb0b810ce40 2024-08-20T03:00:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0b67fc640 1 -- 172.21.15.44:0/1683956122 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0004c60 con 0x7fb0b810ce40 2024-08-20T03:00:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.957+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/1683956122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b81a7cc0 con 0x7fb0b810ce40 2024-08-20T03:00:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.958+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/1683956122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b8103dc0 con 0x7fb0b810ce40 2024-08-20T03:00:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.960+0000 7fb0b67fc640 1 -- 172.21.15.44:0/1683956122 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb0b0009bd0 con 0x7fb0b810ce40 2024-08-20T03:00:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.960+0000 7fb0b67fc640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0a00b3cb0 0x7fb0a00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:41.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.960+0000 7fb0bd05c640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0a00b3cb0 0x7fb0a00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.961+0000 7fb0b67fc640 1 -- 172.21.15.44:0/1683956122 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fb0b00e37e0 con 0x7fb0b810ce40 2024-08-20T03:00:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.961+0000 7fb0bd05c640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0a00b3cb0 0x7fb0a00b6170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005fd0 tx=0x7fb0a8009870 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:41.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:41.964+0000 7fb0b67fc640 1 -- 172.21.15.44:0/1683956122 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb0b00aacf0 con 0x7fb0b810ce40 2024-08-20T03:00:42.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.151+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/1683956122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0b81a7f60 con 0x7fb0b810ce40 2024-08-20T03:00:42.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.152+0000 7fb0b67fc640 1 -- 172.21.15.44:0/1683956122 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22549 (secure 0 0 0) 0x7fb0b00aa5f0 con 0x7fb0b810ce40 2024-08-20T03:00:42.154 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:42.155 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":75,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","created":"2024-08-20T02:52:10.979264+0000","modified":"2024-08-20T03:00:18.935366+0000","last_up_change":"2024-08-20T02:59:26.404917+0000","last_in_change":"2024-08-20T02:59:03.707380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T02:55:46.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T03:00:14.354568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T03:00:16.869335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7708f8c-a799-4b8f-ab1e-78482d73ca1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6803","nonce":858898316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6805","nonce":858898316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6809","nonce":858898316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6807","nonce":858898316}]},"public_addr":"172.21.15.44:6803/858898316","cluster_addr":"172.21.15.44:6805/858898316","heartbeat_back_addr":"172.21.15.44:6809/858898316","heartbeat_front_addr":"172.21.15.44:6807/858898316","state":["exists","up"]},{"osd":1,"uuid":"12f07986-60bf-498f-a012-cd7ea607b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6801","nonce":4252911663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6803","nonce":4252911663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6807","nonce":4252911663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6805","nonce":4252911663}]},"public_addr":"172.21.15.99:6801/4252911663","cluster_addr":"172.21.15.99:6803/4252911663","heartbeat_back_addr":"172.21.15.99:6807/4252911663","heartbeat_front_addr":"172.21.15.99:6805/4252911663","state":["exists","up"]},{"osd":2,"uuid":"0f189000-ba7f-4412-b0d3-75e8265f6685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6801","nonce":1492740465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6803","nonce":1492740465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6807","nonce":1492740465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6805","nonce":1492740465}]},"public_addr":"172.21.15.165:6801/1492740465","cluster_addr":"172.21.15.165:6803/1492740465","heartbeat_back_addr":"172.21.15.165:6807/1492740465","heartbeat_front_addr":"172.21.15.165:6805/1492740465","state":["exists","up"]},{"osd":3,"uuid":"8f9b042c-d6b5-4eee-aa29-ffc41a48c20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6811","nonce":3423983702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6813","nonce":3423983702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6817","nonce":3423983702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6815","nonce":3423983702}]},"public_addr":"172.21.15.44:6811/3423983702","cluster_addr":"172.21.15.44:6813/3423983702","heartbeat_back_addr":"172.21.15.44:6817/3423983702","heartbeat_front_addr":"172.21.15.44:6815/3423983702","state":["exists","up"]},{"osd":4,"uuid":"4edebfdb-af10-49ee-b724-2a63b9219221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6809","nonce":3099617325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6811","nonce":3099617325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6815","nonce":3099617325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6813","nonce":3099617325}]},"public_addr":"172.21.15.99:6809/3099617325","cluster_addr":"172.21.15.99:6811/3099617325","heartbeat_back_addr":"172.21.15.99:6815/3099617325","heartbeat_front_addr":"172.21.15.99:6813/3099617325","state":["exists","up"]},{"osd":5,"uuid":"6d22c21f-cc43-4ef5-a75c-d4fc1a361c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6809","nonce":523245255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6811","nonce":523245255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6815","nonce":523245255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6813","nonce":523245255}]},"public_addr":"172.21.15.165:6809/523245255","cluster_addr":"172.21.15.165:6811/523245255","heartbeat_back_addr":"172.21.15.165:6815/523245255","heartbeat_front_addr":"172.21.15.165:6813/523245255","state":["exists","up"]},{"osd":6,"uuid":"03df0c2f-1b7c-4242-888c-e047322fd8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6819","nonce":1039098986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6821","nonce":1039098986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6825","nonce":1039098986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6823","nonce":1039098986}]},"public_addr":"172.21.15.44:6819/1039098986","cluster_addr":"172.21.15.44:6821/1039098986","heartbeat_back_addr":"172.21.15.44:6825/1039098986","heartbeat_front_addr":"172.21.15.44:6823/1039098986","state":["exists","up"]},{"osd":7,"uuid":"84bbc683-9b13-448e-897c-114458dae33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6817","nonce":1438613564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6819","nonce":1438613564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6823","nonce":1438613564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6821","nonce":1438613564}]},"public_addr":"172.21.15.99:6817/1438613564","cluster_addr":"172.21.15.99:6819/1438613564","heartbeat_back_addr":"172.21.15.99:6823/1438613564","heartbeat_front_addr":"172.21.15.99:6821/1438613564","state":["exists","up"]},{"osd":8,"uuid":"14d7ffd3-db05-4228-a12e-4c69ef65ef8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6817","nonce":632957688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6819","nonce":632957688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6823","nonce":632957688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6821","nonce":632957688}]},"public_addr":"172.21.15.165:6817/632957688","cluster_addr":"172.21.15.165:6819/632957688","heartbeat_back_addr":"172.21.15.165:6823/632957688","heartbeat_front_addr":"172.21.15.165:6821/632957688","state":["exists","up"]},{"osd":9,"uuid":"1ae468a5-3d72-4eb0-a1a6-cd87eae740b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6827","nonce":2426940093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6829","nonce":2426940093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6833","nonce":2426940093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6831","nonce":2426940093}]},"public_addr":"172.21.15.44:6827/2426940093","cluster_addr":"172.21.15.44:6829/2426940093","heartbeat_back_addr":"172.21.15.44:6833/2426940093","heartbeat_front_addr":"172.21.15.44:6831/2426940093","state":["exists","up"]},{"osd":10,"uuid":"fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6825","nonce":3275793038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6827","nonce":3275793038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6831","nonce":3275793038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6829","nonce":3275793038}]},"public_addr":"172.21.15.99:6825/3275793038","cluster_addr":"172.21.15.99:6827/3275793038","heartbeat_back_addr":"172.21.15.99:6831/3275793038","heartbeat_front_addr":"172.21.15.99:6829/3275793038","state":["exists","up"]},{"osd":11,"uuid":"50715410-1fc2-4818-b600-604a4b7cce9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6825","nonce":107688415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6827","nonce":107688415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6831","nonce":107688415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6829","nonce":107688415}]},"public_addr":"172.21.15.165:6825/107688415","cluster_addr":"172.21.15.165:6827/107688415","heartbeat_back_addr":"172.21.15.165:6831/107688415","heartbeat_front_addr":"172.21.15.165:6829/107688415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:54:56.697075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:20.486846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:43.353376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:08.333879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:32.186492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:55.891366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:19.681421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:45.475926+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:10.109795+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:34.444169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:59.147646+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:59:25.403071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2597827045":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/3582612619":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/1916809746":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2919033419":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:6800/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2331034359":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2572185146":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/4176802819":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/3654439062":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6801/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/2376733275":"2024-08-21T02:53:58.312659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T03:00:42.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.154+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/1683956122 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0a00b3cb0 msgr2=0x7fb0a00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:42.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.154+0000 7fb0bf2c0640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0a00b3cb0 0x7fb0a00b6170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005fd0 tx=0x7fb0a8009870 comp rx=0 tx=0).stop 2024-08-20T03:00:42.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.154+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/1683956122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0b810ce40 msgr2=0x7fb0b81a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:42.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.154+0000 7fb0bf2c0640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0b810ce40 0x7fb0b81a0b40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000d750 tx=0x7fb0b000dc10 comp rx=0 tx=0).stop 2024-08-20T03:00:42.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.154+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/1683956122 shutdown_connections 2024-08-20T03:00:42.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.154+0000 7fb0bf2c0640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0a00b3cb0 0x7fb0a00b6170 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:42.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.154+0000 7fb0bf2c0640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0b810ce40 0x7fb0b81a0b40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:42.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.154+0000 7fb0bf2c0640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0b8104300 0x7fb0b819b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:42.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.154+0000 7fb0bf2c0640 1 --2- 172.21.15.44:0/1683956122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0b81039e0 0x7fb0b819b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:42.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.154+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/1683956122 >> 172.21.15.44:0/1683956122 conn(0x7fb0b80fd440 msgr2=0x7fb0b810f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:42.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.154+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/1683956122 shutdown_connections 2024-08-20T03:00:42.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:42.154+0000 7fb0bf2c0640 1 -- 172.21.15.44:0/1683956122 wait complete. 2024-08-20T03:00:42.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:42 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1683956122' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:42.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:42 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1683956122' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:42.906 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-20T03:00:42.907 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T03:00:42.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:42 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1683956122' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:43.158 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:43.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:43 smithi044 ceph-mon[18790]: pgmap v265: 97 pgs: 97 active+clean; 633 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 23 op/s 2024-08-20T03:00:43.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.629+0000 7fb151ac1640 1 -- 172.21.15.44:0/1230749516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c102ab0 msgr2=0x7fb14c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:43.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.629+0000 7fb151ac1640 1 --2- 172.21.15.44:0/1230749516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c102ab0 0x7fb14c102e90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb13c009900 tx=0x7fb13c02a410 comp rx=0 tx=0).stop 2024-08-20T03:00:43.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.629+0000 7fb151ac1640 1 -- 172.21.15.44:0/1230749516 shutdown_connections 2024-08-20T03:00:43.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.629+0000 7fb151ac1640 1 --2- 172.21.15.44:0/1230749516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb14c108e40 0x7fb14c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.629+0000 7fb151ac1640 1 --2- 172.21.15.44:0/1230749516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb14c1084c0 0x7fb14c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.629+0000 7fb151ac1640 1 --2- 172.21.15.44:0/1230749516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c102ab0 0x7fb14c102e90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.629+0000 7fb151ac1640 1 -- 172.21.15.44:0/1230749516 >> 172.21.15.44:0/1230749516 conn(0x7fb14c0fe500 msgr2=0x7fb14c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.629+0000 7fb151ac1640 1 -- 172.21.15.44:0/1230749516 shutdown_connections 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.629+0000 7fb151ac1640 1 -- 172.21.15.44:0/1230749516 wait complete. 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.629+0000 7fb151ac1640 1 Processor -- start 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb151ac1640 1 -- start start 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb151ac1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb14c102ab0 0x7fb14c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb151ac1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c1084c0 0x7fb14c1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb151ac1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb14c108e40 0x7fb14c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb151ac1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c112b00 con 0x7fb14c1084c0 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb151ac1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c112980 con 0x7fb14c102ab0 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb151ac1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c112c80 con 0x7fb14c108e40 2024-08-20T03:00:43.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb14affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c1084c0 0x7fb14c1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb14affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c1084c0 0x7fb14c1994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54610/0 (socket says 172.21.15.44:54610) 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb14affd640 1 -- 172.21.15.44:0/2195819168 learned_addr learned my addr 172.21.15.44:0/2195819168 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb14b7fe640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb14c102ab0 0x7fb14c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb14bfff640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb14c108e40 0x7fb14c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb14affd640 1 -- 172.21.15.44:0/2195819168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb14c108e40 msgr2=0x7fb14c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb14affd640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb14c108e40 0x7fb14c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb14affd640 1 -- 172.21.15.44:0/2195819168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb14c102ab0 msgr2=0x7fb14c198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb14affd640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb14c102ab0 0x7fb14c198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb14affd640 1 -- 172.21.15.44:0/2195819168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c19f050 con 0x7fb14c1084c0 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.630+0000 7fb14bfff640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb14c108e40 0x7fb14c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.631+0000 7fb14affd640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c1084c0 0x7fb14c1994e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb14000d950 tx=0x7fb14000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.631+0000 7fb148ff9640 1 -- 172.21.15.44:0/2195819168 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140017070 con 0x7fb14c1084c0 2024-08-20T03:00:43.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.631+0000 7fb148ff9640 1 -- 172.21.15.44:0/2195819168 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb140009c30 con 0x7fb14c1084c0 2024-08-20T03:00:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.631+0000 7fb151ac1640 1 -- 172.21.15.44:0/2195819168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c19f340 con 0x7fb14c1084c0 2024-08-20T03:00:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.631+0000 7fb148ff9640 1 -- 172.21.15.44:0/2195819168 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14000f620 con 0x7fb14c1084c0 2024-08-20T03:00:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.631+0000 7fb151ac1640 1 -- 172.21.15.44:0/2195819168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c1a5a00 con 0x7fb14c1084c0 2024-08-20T03:00:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.632+0000 7fb151ac1640 1 -- 172.21.15.44:0/2195819168 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14c102e90 con 0x7fb14c1084c0 2024-08-20T03:00:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.633+0000 7fb148ff9640 1 -- 172.21.15.44:0/2195819168 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb140010460 con 0x7fb14c1084c0 2024-08-20T03:00:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.634+0000 7fb148ff9640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb1340b3cb0 0x7fb1340b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.634+0000 7fb14b7fe640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb1340b3cb0 0x7fb1340b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.634+0000 7fb148ff9640 1 -- 172.21.15.44:0/2195819168 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fb1400e3a50 con 0x7fb14c1084c0 2024-08-20T03:00:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.635+0000 7fb14b7fe640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb1340b3cb0 0x7fb1340b6170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb13c031440 tx=0x7fb13c005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:43.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.637+0000 7fb148ff9640 1 -- 172.21.15.44:0/2195819168 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb1400aaf00 con 0x7fb14c1084c0 2024-08-20T03:00:43.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.817+0000 7fb151ac1640 1 -- 172.21.15.44:0/2195819168 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb14c10ac80 con 0x7fb1340b3cb0 2024-08-20T03:00:43.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.822+0000 7fb148ff9640 1 -- 172.21.15.44:0/2195819168 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258906 (secure 0 0 0) 0x7fb14c10ac80 con 0x7fb1340b3cb0 2024-08-20T03:00:43.826 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:43.828 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-08-20T03:00:43.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.825+0000 7fb151ac1640 1 -- 172.21.15.44:0/2195819168 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb1340b3cb0 msgr2=0x7fb1340b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:43.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.825+0000 7fb151ac1640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb1340b3cb0 0x7fb1340b6170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb13c031440 tx=0x7fb13c005b50 comp rx=0 tx=0).stop 2024-08-20T03:00:43.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.825+0000 7fb151ac1640 1 -- 172.21.15.44:0/2195819168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c1084c0 msgr2=0x7fb14c1994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:43.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.825+0000 7fb151ac1640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c1084c0 0x7fb14c1994e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb14000d950 tx=0x7fb14000de10 comp rx=0 tx=0).stop 2024-08-20T03:00:43.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.825+0000 7fb151ac1640 1 -- 172.21.15.44:0/2195819168 shutdown_connections 2024-08-20T03:00:43.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.825+0000 7fb151ac1640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb1340b3cb0 0x7fb1340b6170 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:43.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.825+0000 7fb151ac1640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb14c108e40 0x7fb14c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:43.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.825+0000 7fb151ac1640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb14c1084c0 0x7fb14c1994e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:43.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.825+0000 7fb151ac1640 1 --2- 172.21.15.44:0/2195819168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb14c102ab0 0x7fb14c198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:43.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.825+0000 7fb151ac1640 1 -- 172.21.15.44:0/2195819168 >> 172.21.15.44:0/2195819168 conn(0x7fb14c0fe500 msgr2=0x7fb14c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:43.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.825+0000 7fb151ac1640 1 -- 172.21.15.44:0/2195819168 shutdown_connections 2024-08-20T03:00:43.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:43.825+0000 7fb151ac1640 1 -- 172.21.15.44:0/2195819168 wait complete. 2024-08-20T03:00:43.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:43 smithi165 ceph-mon[22357]: pgmap v265: 97 pgs: 97 active+clean; 633 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 23 op/s 2024-08-20T03:00:43.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:43 smithi099 ceph-mon[21984]: pgmap v265: 97 pgs: 97 active+clean; 633 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 23 op/s 2024-08-20T03:00:44.533 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-08-20T03:00:42.394151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647916,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":224,"num_read_kb":209,"num_write":363,"num_write_kb":2274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363548,"kb_used_data":10140,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124578276,"statfs":{"total":1151940427776,"available":1151568154624,"internally_reserved":0,"allocated":10383360,"data_stored":6724489,"data_compressed":107358,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":52506,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":143,"num_read_kb":140,"num_write":143,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002701"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.250895+0000","last_change":"2024-08-20T03:00:18.238595+0000","last_active":"2024-08-20T03:00:19.250895+0000","last_peered":"2024-08-20T03:00:19.250895+0000","last_clean":"2024-08-20T03:00:19.250895+0000","last_became_active":"2024-08-20T03:00:18.238306+0000","last_became_peered":"2024-08-20T03:00:18.238306+0000","last_unstale":"2024-08-20T03:00:19.250895+0000","last_undegraded":"2024-08-20T03:00:19.250895+0000","last_fullsized":"2024-08-20T03:00:19.250895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:33:04.681614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249682+0000","last_change":"2024-08-20T03:00:18.239722+0000","last_active":"2024-08-20T03:00:19.249682+0000","last_peered":"2024-08-20T03:00:19.249682+0000","last_clean":"2024-08-20T03:00:19.249682+0000","last_became_active":"2024-08-20T03:00:18.239391+0000","last_became_peered":"2024-08-20T03:00:18.239391+0000","last_unstale":"2024-08-20T03:00:19.249682+0000","last_undegraded":"2024-08-20T03:00:19.249682+0000","last_fullsized":"2024-08-20T03:00:19.249682+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:08:34.657173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249463+0000","last_change":"2024-08-20T03:00:18.242389+0000","last_active":"2024-08-20T03:00:19.249463+0000","last_peered":"2024-08-20T03:00:19.249463+0000","last_clean":"2024-08-20T03:00:19.249463+0000","last_became_active":"2024-08-20T03:00:18.242080+0000","last_became_peered":"2024-08-20T03:00:18.242080+0000","last_unstale":"2024-08-20T03:00:19.249463+0000","last_undegraded":"2024-08-20T03:00:19.249463+0000","last_fullsized":"2024-08-20T03:00:19.249463+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:37.042894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245403+0000","last_change":"2024-08-20T03:00:18.233709+0000","last_active":"2024-08-20T03:00:19.245403+0000","last_peered":"2024-08-20T03:00:19.245403+0000","last_clean":"2024-08-20T03:00:19.245403+0000","last_became_active":"2024-08-20T03:00:18.233402+0000","last_became_peered":"2024-08-20T03:00:18.233402+0000","last_unstale":"2024-08-20T03:00:19.245403+0000","last_undegraded":"2024-08-20T03:00:19.245403+0000","last_fullsized":"2024-08-20T03:00:19.245403+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:30:01.308181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.246592+0000","last_change":"2024-08-20T03:00:18.237770+0000","last_active":"2024-08-20T03:00:19.246592+0000","last_peered":"2024-08-20T03:00:19.246592+0000","last_clean":"2024-08-20T03:00:19.246592+0000","last_became_active":"2024-08-20T03:00:18.237475+0000","last_became_peered":"2024-08-20T03:00:18.237475+0000","last_unstale":"2024-08-20T03:00:19.246592+0000","last_undegraded":"2024-08-20T03:00:19.246592+0000","last_fullsized":"2024-08-20T03:00:19.246592+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:37:12.707935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245669+0000","last_change":"2024-08-20T03:00:18.245400+0000","last_active":"2024-08-20T03:00:19.245669+0000","last_peered":"2024-08-20T03:00:19.245669+0000","last_clean":"2024-08-20T03:00:19.245669+0000","last_became_active":"2024-08-20T03:00:18.244735+0000","last_became_peered":"2024-08-20T03:00:18.244735+0000","last_unstale":"2024-08-20T03:00:19.245669+0000","last_undegraded":"2024-08-20T03:00:19.245669+0000","last_fullsized":"2024-08-20T03:00:19.245669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:23:25.459441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234700+0000","last_change":"2024-08-20T03:00:18.247366+0000","last_active":"2024-08-20T03:00:19.234700+0000","last_peered":"2024-08-20T03:00:19.234700+0000","last_clean":"2024-08-20T03:00:19.234700+0000","last_became_active":"2024-08-20T03:00:18.247032+0000","last_became_peered":"2024-08-20T03:00:18.247032+0000","last_unstale":"2024-08-20T03:00:19.234700+0000","last_undegraded":"2024-08-20T03:00:19.234700+0000","last_fullsized":"2024-08-20T03:00:19.234700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:02:18.070393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245915+0000","last_change":"2024-08-20T03:00:18.240845+0000","last_active":"2024-08-20T03:00:19.245915+0000","last_peered":"2024-08-20T03:00:19.245915+0000","last_clean":"2024-08-20T03:00:19.245915+0000","last_became_active":"2024-08-20T03:00:18.240531+0000","last_became_peered":"2024-08-20T03:00:18.240531+0000","last_unstale":"2024-08-20T03:00:19.245915+0000","last_undegraded":"2024-08-20T03:00:19.245915+0000","last_fullsized":"2024-08-20T03:00:19.245915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:15:18.975804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249266+0000","last_change":"2024-08-20T03:00:18.240817+0000","last_active":"2024-08-20T03:00:19.249266+0000","last_peered":"2024-08-20T03:00:19.249266+0000","last_clean":"2024-08-20T03:00:19.249266+0000","last_became_active":"2024-08-20T03:00:18.240262+0000","last_became_peered":"2024-08-20T03:00:18.240262+0000","last_unstale":"2024-08-20T03:00:19.249266+0000","last_undegraded":"2024-08-20T03:00:19.249266+0000","last_fullsized":"2024-08-20T03:00:19.249266+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:30.512597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245601+0000","last_change":"2024-08-20T03:00:18.242139+0000","last_active":"2024-08-20T03:00:19.245601+0000","last_peered":"2024-08-20T03:00:19.245601+0000","last_clean":"2024-08-20T03:00:19.245601+0000","last_became_active":"2024-08-20T03:00:18.241817+0000","last_became_peered":"2024-08-20T03:00:18.241817+0000","last_unstale":"2024-08-20T03:00:19.245601+0000","last_undegraded":"2024-08-20T03:00:19.245601+0000","last_fullsized":"2024-08-20T03:00:19.245601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:25.121306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234785+0000","last_change":"2024-08-20T03:00:18.243205+0000","last_active":"2024-08-20T03:00:19.234785+0000","last_peered":"2024-08-20T03:00:19.234785+0000","last_clean":"2024-08-20T03:00:19.234785+0000","last_became_active":"2024-08-20T03:00:18.242296+0000","last_became_peered":"2024-08-20T03:00:18.242296+0000","last_unstale":"2024-08-20T03:00:19.234785+0000","last_undegraded":"2024-08-20T03:00:19.234785+0000","last_fullsized":"2024-08-20T03:00:19.234785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:50.490934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245585+0000","last_change":"2024-08-20T03:00:18.237205+0000","last_active":"2024-08-20T03:00:19.245585+0000","last_peered":"2024-08-20T03:00:19.245585+0000","last_clean":"2024-08-20T03:00:19.245585+0000","last_became_active":"2024-08-20T03:00:18.236631+0000","last_became_peered":"2024-08-20T03:00:18.236631+0000","last_unstale":"2024-08-20T03:00:19.245585+0000","last_undegraded":"2024-08-20T03:00:19.245585+0000","last_fullsized":"2024-08-20T03:00:19.245585+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:13.327598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.231337+0000","last_change":"2024-08-20T03:00:18.235768+0000","last_active":"2024-08-20T03:00:19.231337+0000","last_peered":"2024-08-20T03:00:19.231337+0000","last_clean":"2024-08-20T03:00:19.231337+0000","last_became_active":"2024-08-20T03:00:18.235461+0000","last_became_peered":"2024-08-20T03:00:18.235461+0000","last_unstale":"2024-08-20T03:00:19.231337+0000","last_undegraded":"2024-08-20T03:00:19.231337+0000","last_fullsized":"2024-08-20T03:00:19.231337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:08:22.238020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249572+0000","last_change":"2024-08-20T03:00:18.246560+0000","last_active":"2024-08-20T03:00:19.249572+0000","last_peered":"2024-08-20T03:00:19.249572+0000","last_clean":"2024-08-20T03:00:19.249572+0000","last_became_active":"2024-08-20T03:00:18.246270+0000","last_became_peered":"2024-08-20T03:00:18.246270+0000","last_unstale":"2024-08-20T03:00:19.249572+0000","last_undegraded":"2024-08-20T03:00:19.249572+0000","last_fullsized":"2024-08-20T03:00:19.249572+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:47:13.402635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249359+0000","last_change":"2024-08-20T03:00:18.245907+0000","last_active":"2024-08-20T03:00:19.249359+0000","last_peered":"2024-08-20T03:00:19.249359+0000","last_clean":"2024-08-20T03:00:19.249359+0000","last_became_active":"2024-08-20T03:00:18.245429+0000","last_became_peered":"2024-08-20T03:00:18.245429+0000","last_unstale":"2024-08-20T03:00:19.249359+0000","last_undegraded":"2024-08-20T03:00:19.249359+0000","last_fullsized":"2024-08-20T03:00:19.249359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:46.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.248555+0000","last_change":"2024-08-20T03:00:18.242561+0000","last_active":"2024-08-20T03:00:19.248555+0000","last_peered":"2024-08-20T03:00:19.248555+0000","last_clean":"2024-08-20T03:00:19.248555+0000","last_became_active":"2024-08-20T03:00:18.242048+0000","last_became_peered":"2024-08-20T03:00:18.242048+0000","last_unstale":"2024-08-20T03:00:19.248555+0000","last_undegraded":"2024-08-20T03:00:19.248555+0000","last_fullsized":"2024-08-20T03:00:19.248555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:27.784131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.250977+0000","last_change":"2024-08-20T03:00:18.238547+0000","last_active":"2024-08-20T03:00:19.250977+0000","last_peered":"2024-08-20T03:00:19.250977+0000","last_clean":"2024-08-20T03:00:19.250977+0000","last_became_active":"2024-08-20T03:00:18.238268+0000","last_became_peered":"2024-08-20T03:00:18.238268+0000","last_unstale":"2024-08-20T03:00:19.250977+0000","last_undegraded":"2024-08-20T03:00:19.250977+0000","last_fullsized":"2024-08-20T03:00:19.250977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:15:11.923674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.244953+0000","last_change":"2024-08-20T03:00:18.241995+0000","last_active":"2024-08-20T03:00:19.244953+0000","last_peered":"2024-08-20T03:00:19.244953+0000","last_clean":"2024-08-20T03:00:19.244953+0000","last_became_active":"2024-08-20T03:00:18.241705+0000","last_became_peered":"2024-08-20T03:00:18.241705+0000","last_unstale":"2024-08-20T03:00:19.244953+0000","last_undegraded":"2024-08-20T03:00:19.244953+0000","last_fullsized":"2024-08-20T03:00:19.244953+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:33:02.183712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.232295+0000","last_change":"2024-08-20T03:00:18.236800+0000","last_active":"2024-08-20T03:00:19.232295+0000","last_peered":"2024-08-20T03:00:19.232295+0000","last_clean":"2024-08-20T03:00:19.232295+0000","last_became_active":"2024-08-20T03:00:18.235909+0000","last_became_peered":"2024-08-20T03:00:18.235909+0000","last_unstale":"2024-08-20T03:00:19.232295+0000","last_undegraded":"2024-08-20T03:00:19.232295+0000","last_fullsized":"2024-08-20T03:00:19.232295+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:12.427760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249752+0000","last_change":"2024-08-20T03:00:18.237554+0000","last_active":"2024-08-20T03:00:19.249752+0000","last_peered":"2024-08-20T03:00:19.249752+0000","last_clean":"2024-08-20T03:00:19.249752+0000","last_became_active":"2024-08-20T03:00:18.237236+0000","last_became_peered":"2024-08-20T03:00:18.237236+0000","last_unstale":"2024-08-20T03:00:19.249752+0000","last_undegraded":"2024-08-20T03:00:19.249752+0000","last_fullsized":"2024-08-20T03:00:19.249752+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:13:42.997517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.231487+0000","last_change":"2024-08-20T03:00:18.243425+0000","last_active":"2024-08-20T03:00:19.231487+0000","last_peered":"2024-08-20T03:00:19.231487+0000","last_clean":"2024-08-20T03:00:19.231487+0000","last_became_active":"2024-08-20T03:00:18.243082+0000","last_became_peered":"2024-08-20T03:00:18.243082+0000","last_unstale":"2024-08-20T03:00:19.231487+0000","last_undegraded":"2024-08-20T03:00:19.231487+0000","last_fullsized":"2024-08-20T03:00:19.231487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:50.008655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249240+0000","last_change":"2024-08-20T03:00:18.242765+0000","last_active":"2024-08-20T03:00:19.249240+0000","last_peered":"2024-08-20T03:00:19.249240+0000","last_clean":"2024-08-20T03:00:19.249240+0000","last_became_active":"2024-08-20T03:00:18.242459+0000","last_became_peered":"2024-08-20T03:00:18.242459+0000","last_unstale":"2024-08-20T03:00:19.249240+0000","last_undegraded":"2024-08-20T03:00:19.249240+0000","last_fullsized":"2024-08-20T03:00:19.249240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:16.467068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245966+0000","last_change":"2024-08-20T03:00:18.236518+0000","last_active":"2024-08-20T03:00:19.245966+0000","last_peered":"2024-08-20T03:00:19.245966+0000","last_clean":"2024-08-20T03:00:19.245966+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:00:19.245966+0000","last_undegraded":"2024-08-20T03:00:19.245966+0000","last_fullsized":"2024-08-20T03:00:19.245966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:55:55.387134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245691+0000","last_change":"2024-08-20T03:00:18.236730+0000","last_active":"2024-08-20T03:00:19.245691+0000","last_peered":"2024-08-20T03:00:19.245691+0000","last_clean":"2024-08-20T03:00:19.245691+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:00:19.245691+0000","last_undegraded":"2024-08-20T03:00:19.245691+0000","last_fullsized":"2024-08-20T03:00:19.245691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:18.196161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.247220+0000","last_change":"2024-08-20T03:00:18.245390+0000","last_active":"2024-08-20T03:00:19.247220+0000","last_peered":"2024-08-20T03:00:19.247220+0000","last_clean":"2024-08-20T03:00:19.247220+0000","last_became_active":"2024-08-20T03:00:18.241175+0000","last_became_peered":"2024-08-20T03:00:18.241175+0000","last_unstale":"2024-08-20T03:00:19.247220+0000","last_undegraded":"2024-08-20T03:00:19.247220+0000","last_fullsized":"2024-08-20T03:00:19.247220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:51:45.792846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.232616+0000","last_change":"2024-08-20T03:00:18.242700+0000","last_active":"2024-08-20T03:00:19.232616+0000","last_peered":"2024-08-20T03:00:19.232616+0000","last_clean":"2024-08-20T03:00:19.232616+0000","last_became_active":"2024-08-20T03:00:18.242407+0000","last_became_peered":"2024-08-20T03:00:18.242407+0000","last_unstale":"2024-08-20T03:00:19.232616+0000","last_undegraded":"2024-08-20T03:00:19.232616+0000","last_fullsized":"2024-08-20T03:00:19.232616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:31:28.061514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.250462+0000","last_change":"2024-08-20T03:00:18.233894+0000","last_active":"2024-08-20T03:00:19.250462+0000","last_peered":"2024-08-20T03:00:19.250462+0000","last_clean":"2024-08-20T03:00:19.250462+0000","last_became_active":"2024-08-20T03:00:18.233584+0000","last_became_peered":"2024-08-20T03:00:18.233584+0000","last_unstale":"2024-08-20T03:00:19.250462+0000","last_undegraded":"2024-08-20T03:00:19.250462+0000","last_fullsized":"2024-08-20T03:00:19.250462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:36:53.156215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.231278+0000","last_change":"2024-08-20T03:00:18.239256+0000","last_active":"2024-08-20T03:00:19.231278+0000","last_peered":"2024-08-20T03:00:19.231278+0000","last_clean":"2024-08-20T03:00:19.231278+0000","last_became_active":"2024-08-20T03:00:18.238931+0000","last_became_peered":"2024-08-20T03:00:18.238931+0000","last_unstale":"2024-08-20T03:00:19.231278+0000","last_undegraded":"2024-08-20T03:00:19.231278+0000","last_fullsized":"2024-08-20T03:00:19.231278+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:38:42.536488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249465+0000","last_change":"2024-08-20T03:00:18.245187+0000","last_active":"2024-08-20T03:00:19.249465+0000","last_peered":"2024-08-20T03:00:19.249465+0000","last_clean":"2024-08-20T03:00:19.249465+0000","last_became_active":"2024-08-20T03:00:18.244894+0000","last_became_peered":"2024-08-20T03:00:18.244894+0000","last_unstale":"2024-08-20T03:00:19.249465+0000","last_undegraded":"2024-08-20T03:00:19.249465+0000","last_fullsized":"2024-08-20T03:00:19.249465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:16:58.002014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.232402+0000","last_change":"2024-08-20T03:00:18.241634+0000","last_active":"2024-08-20T03:00:19.232402+0000","last_peered":"2024-08-20T03:00:19.232402+0000","last_clean":"2024-08-20T03:00:19.232402+0000","last_became_active":"2024-08-20T03:00:18.241341+0000","last_became_peered":"2024-08-20T03:00:18.241341+0000","last_unstale":"2024-08-20T03:00:19.232402+0000","last_undegraded":"2024-08-20T03:00:19.232402+0000","last_fullsized":"2024-08-20T03:00:19.232402+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:37:16.185544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.232393+0000","last_change":"2024-08-20T03:00:18.239116+0000","last_active":"2024-08-20T03:00:19.232393+0000","last_peered":"2024-08-20T03:00:19.232393+0000","last_clean":"2024-08-20T03:00:19.232393+0000","last_became_active":"2024-08-20T03:00:18.236125+0000","last_became_peered":"2024-08-20T03:00:18.236125+0000","last_unstale":"2024-08-20T03:00:19.232393+0000","last_undegraded":"2024-08-20T03:00:19.232393+0000","last_fullsized":"2024-08-20T03:00:19.232393+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:57:00.332483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249712+0000","last_change":"2024-08-20T03:00:18.235996+0000","last_active":"2024-08-20T03:00:19.249712+0000","last_peered":"2024-08-20T03:00:19.249712+0000","last_clean":"2024-08-20T03:00:19.249712+0000","last_became_active":"2024-08-20T03:00:18.235670+0000","last_became_peered":"2024-08-20T03:00:18.235670+0000","last_unstale":"2024-08-20T03:00:19.249712+0000","last_undegraded":"2024-08-20T03:00:19.249712+0000","last_fullsized":"2024-08-20T03:00:19.249712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:59.992914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245508+0000","last_change":"2024-08-20T03:00:18.239679+0000","last_active":"2024-08-20T03:00:19.245508+0000","last_peered":"2024-08-20T03:00:19.245508+0000","last_clean":"2024-08-20T03:00:19.245508+0000","last_became_active":"2024-08-20T03:00:18.239361+0000","last_became_peered":"2024-08-20T03:00:18.239361+0000","last_unstale":"2024-08-20T03:00:19.245508+0000","last_undegraded":"2024-08-20T03:00:19.245508+0000","last_fullsized":"2024-08-20T03:00:19.245508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:28:09.430535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:37.407176+0000","last_change":"2024-08-20T03:00:16.221221+0000","last_active":"2024-08-20T03:00:37.407176+0000","last_peered":"2024-08-20T03:00:37.407176+0000","last_clean":"2024-08-20T03:00:37.407176+0000","last_became_active":"2024-08-20T03:00:16.220186+0000","last_became_peered":"2024-08-20T03:00:16.220186+0000","last_unstale":"2024-08-20T03:00:37.407176+0000","last_undegraded":"2024-08-20T03:00:37.407176+0000","last_fullsized":"2024-08-20T03:00:37.407176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:51.045235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.248996+0000","last_change":"2024-08-20T03:00:18.234809+0000","last_active":"2024-08-20T03:00:19.248996+0000","last_peered":"2024-08-20T03:00:19.248996+0000","last_clean":"2024-08-20T03:00:19.248996+0000","last_became_active":"2024-08-20T03:00:18.234486+0000","last_became_peered":"2024-08-20T03:00:18.234486+0000","last_unstale":"2024-08-20T03:00:19.248996+0000","last_undegraded":"2024-08-20T03:00:19.248996+0000","last_fullsized":"2024-08-20T03:00:19.248996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:11:52.527224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:28.379401+0000","last_change":"2024-08-20T03:00:16.216876+0000","last_active":"2024-08-20T03:00:28.379401+0000","last_peered":"2024-08-20T03:00:28.379401+0000","last_clean":"2024-08-20T03:00:28.379401+0000","last_became_active":"2024-08-20T03:00:16.216522+0000","last_became_peered":"2024-08-20T03:00:16.216522+0000","last_unstale":"2024-08-20T03:00:28.379401+0000","last_undegraded":"2024-08-20T03:00:28.379401+0000","last_fullsized":"2024-08-20T03:00:28.379401+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:22.363331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:27.371388+0000","last_change":"2024-08-20T03:00:16.213751+0000","last_active":"2024-08-20T03:00:27.371388+0000","last_peered":"2024-08-20T03:00:27.371388+0000","last_clean":"2024-08-20T03:00:27.371388+0000","last_became_active":"2024-08-20T03:00:16.213348+0000","last_became_peered":"2024-08-20T03:00:16.213348+0000","last_unstale":"2024-08-20T03:00:27.371388+0000","last_undegraded":"2024-08-20T03:00:27.371388+0000","last_fullsized":"2024-08-20T03:00:27.371388+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:01:08.637151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245981+0000","last_change":"2024-08-20T03:00:18.237972+0000","last_active":"2024-08-20T03:00:19.245981+0000","last_peered":"2024-08-20T03:00:19.245981+0000","last_clean":"2024-08-20T03:00:19.245981+0000","last_became_active":"2024-08-20T03:00:18.236769+0000","last_became_peered":"2024-08-20T03:00:18.236769+0000","last_unstale":"2024-08-20T03:00:19.245981+0000","last_undegraded":"2024-08-20T03:00:19.245981+0000","last_fullsized":"2024-08-20T03:00:19.245981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:35:21.001045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.390750+0000","last_change":"2024-08-20T03:00:16.221399+0000","last_active":"2024-08-20T03:00:29.390750+0000","last_peered":"2024-08-20T03:00:29.390750+0000","last_clean":"2024-08-20T03:00:29.390750+0000","last_became_active":"2024-08-20T03:00:16.220285+0000","last_became_peered":"2024-08-20T03:00:16.220285+0000","last_unstale":"2024-08-20T03:00:29.390750+0000","last_undegraded":"2024-08-20T03:00:29.390750+0000","last_fullsized":"2024-08-20T03:00:29.390750+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:17.034657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.244765+0000","last_change":"2024-08-20T03:00:18.236722+0000","last_active":"2024-08-20T03:00:19.244765+0000","last_peered":"2024-08-20T03:00:19.244765+0000","last_clean":"2024-08-20T03:00:19.244765+0000","last_became_active":"2024-08-20T03:00:18.236365+0000","last_became_peered":"2024-08-20T03:00:18.236365+0000","last_unstale":"2024-08-20T03:00:19.244765+0000","last_undegraded":"2024-08-20T03:00:19.244765+0000","last_fullsized":"2024-08-20T03:00:19.244765+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:17.344161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:40.410220+0000","last_change":"2024-08-20T03:00:16.222331+0000","last_active":"2024-08-20T03:00:40.410220+0000","last_peered":"2024-08-20T03:00:40.410220+0000","last_clean":"2024-08-20T03:00:40.410220+0000","last_became_active":"2024-08-20T03:00:16.222019+0000","last_became_peered":"2024-08-20T03:00:16.222019+0000","last_unstale":"2024-08-20T03:00:40.410220+0000","last_undegraded":"2024-08-20T03:00:40.410220+0000","last_fullsized":"2024-08-20T03:00:40.410220+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:46:55.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249388+0000","last_change":"2024-08-20T03:00:18.239020+0000","last_active":"2024-08-20T03:00:19.249388+0000","last_peered":"2024-08-20T03:00:19.249388+0000","last_clean":"2024-08-20T03:00:19.249388+0000","last_became_active":"2024-08-20T03:00:18.238620+0000","last_became_peered":"2024-08-20T03:00:18.238620+0000","last_unstale":"2024-08-20T03:00:19.249388+0000","last_undegraded":"2024-08-20T03:00:19.249388+0000","last_fullsized":"2024-08-20T03:00:19.249388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:17:12.632357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.390633+0000","last_change":"2024-08-20T03:00:16.214907+0000","last_active":"2024-08-20T03:00:29.390633+0000","last_peered":"2024-08-20T03:00:29.390633+0000","last_clean":"2024-08-20T03:00:29.390633+0000","last_became_active":"2024-08-20T03:00:16.214573+0000","last_became_peered":"2024-08-20T03:00:16.214573+0000","last_unstale":"2024-08-20T03:00:29.390633+0000","last_undegraded":"2024-08-20T03:00:29.390633+0000","last_fullsized":"2024-08-20T03:00:29.390633+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:21:34.084149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249851+0000","last_change":"2024-08-20T03:00:18.245524+0000","last_active":"2024-08-20T03:00:19.249851+0000","last_peered":"2024-08-20T03:00:19.249851+0000","last_clean":"2024-08-20T03:00:19.249851+0000","last_became_active":"2024-08-20T03:00:18.244966+0000","last_became_peered":"2024-08-20T03:00:18.244966+0000","last_unstale":"2024-08-20T03:00:19.249851+0000","last_undegraded":"2024-08-20T03:00:19.249851+0000","last_fullsized":"2024-08-20T03:00:19.249851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:03:21.584182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:28.377989+0000","last_change":"2024-08-20T03:00:16.470353+0000","last_active":"2024-08-20T03:00:28.377989+0000","last_peered":"2024-08-20T03:00:28.377989+0000","last_clean":"2024-08-20T03:00:28.377989+0000","last_became_active":"2024-08-20T03:00:16.469976+0000","last_became_peered":"2024-08-20T03:00:16.469976+0000","last_unstale":"2024-08-20T03:00:28.377989+0000","last_undegraded":"2024-08-20T03:00:28.377989+0000","last_fullsized":"2024-08-20T03:00:28.377989+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:25:03.595784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249905+0000","last_change":"2024-08-20T03:00:18.245244+0000","last_active":"2024-08-20T03:00:19.249905+0000","last_peered":"2024-08-20T03:00:19.249905+0000","last_clean":"2024-08-20T03:00:19.249905+0000","last_became_active":"2024-08-20T03:00:18.244934+0000","last_became_peered":"2024-08-20T03:00:18.244934+0000","last_unstale":"2024-08-20T03:00:19.249905+0000","last_undegraded":"2024-08-20T03:00:19.249905+0000","last_fullsized":"2024-08-20T03:00:19.249905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:33:35.035039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'27","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:40.411695+0000","last_change":"2024-08-20T03:00:16.218663+0000","last_active":"2024-08-20T03:00:40.411695+0000","last_peered":"2024-08-20T03:00:40.411695+0000","last_clean":"2024-08-20T03:00:40.411695+0000","last_became_active":"2024-08-20T03:00:16.218266+0000","last_became_peered":"2024-08-20T03:00:16.218266+0000","last_unstale":"2024-08-20T03:00:40.411695+0000","last_undegraded":"2024-08-20T03:00:40.411695+0000","last_fullsized":"2024-08-20T03:00:40.411695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:52.214528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":62,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.246914+0000","last_change":"2024-08-20T03:00:18.242931+0000","last_active":"2024-08-20T03:00:19.246914+0000","last_peered":"2024-08-20T03:00:19.246914+0000","last_clean":"2024-08-20T03:00:19.246914+0000","last_became_active":"2024-08-20T03:00:18.242619+0000","last_became_peered":"2024-08-20T03:00:18.242619+0000","last_unstale":"2024-08-20T03:00:19.246914+0000","last_undegraded":"2024-08-20T03:00:19.246914+0000","last_fullsized":"2024-08-20T03:00:19.246914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:43:02.296990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:30.393276+0000","last_change":"2024-08-20T03:00:16.214688+0000","last_active":"2024-08-20T03:00:30.393276+0000","last_peered":"2024-08-20T03:00:30.393276+0000","last_clean":"2024-08-20T03:00:30.393276+0000","last_became_active":"2024-08-20T03:00:16.214343+0000","last_became_peered":"2024-08-20T03:00:16.214343+0000","last_unstale":"2024-08-20T03:00:30.393276+0000","last_undegraded":"2024-08-20T03:00:30.393276+0000","last_fullsized":"2024-08-20T03:00:30.393276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:10:56.587164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.231157+0000","last_change":"2024-08-20T03:00:18.244158+0000","last_active":"2024-08-20T03:00:19.231157+0000","last_peered":"2024-08-20T03:00:19.231157+0000","last_clean":"2024-08-20T03:00:19.231157+0000","last_became_active":"2024-08-20T03:00:18.243864+0000","last_became_peered":"2024-08-20T03:00:18.243864+0000","last_unstale":"2024-08-20T03:00:19.231157+0000","last_undegraded":"2024-08-20T03:00:19.231157+0000","last_fullsized":"2024-08-20T03:00:19.231157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:10:31.373759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.390842+0000","last_change":"2024-08-20T03:00:16.220846+0000","last_active":"2024-08-20T03:00:29.390842+0000","last_peered":"2024-08-20T03:00:29.390842+0000","last_clean":"2024-08-20T03:00:29.390842+0000","last_became_active":"2024-08-20T03:00:16.220519+0000","last_became_peered":"2024-08-20T03:00:16.220519+0000","last_unstale":"2024-08-20T03:00:29.390842+0000","last_undegraded":"2024-08-20T03:00:29.390842+0000","last_fullsized":"2024-08-20T03:00:29.390842+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:43:41.721522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234671+0000","last_change":"2024-08-20T03:00:18.246555+0000","last_active":"2024-08-20T03:00:19.234671+0000","last_peered":"2024-08-20T03:00:19.234671+0000","last_clean":"2024-08-20T03:00:19.234671+0000","last_became_active":"2024-08-20T03:00:18.246182+0000","last_became_peered":"2024-08-20T03:00:18.246182+0000","last_unstale":"2024-08-20T03:00:19.234671+0000","last_undegraded":"2024-08-20T03:00:19.234671+0000","last_fullsized":"2024-08-20T03:00:19.234671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:55.827649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:37.406653+0000","last_change":"2024-08-20T03:00:16.216521+0000","last_active":"2024-08-20T03:00:37.406653+0000","last_peered":"2024-08-20T03:00:37.406653+0000","last_clean":"2024-08-20T03:00:37.406653+0000","last_became_active":"2024-08-20T03:00:16.216187+0000","last_became_peered":"2024-08-20T03:00:16.216187+0000","last_unstale":"2024-08-20T03:00:37.406653+0000","last_undegraded":"2024-08-20T03:00:37.406653+0000","last_fullsized":"2024-08-20T03:00:37.406653+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:41:13.639218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":12,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234426+0000","last_change":"2024-08-20T03:00:18.249030+0000","last_active":"2024-08-20T03:00:19.234426+0000","last_peered":"2024-08-20T03:00:19.234426+0000","last_clean":"2024-08-20T03:00:19.234426+0000","last_became_active":"2024-08-20T03:00:18.248701+0000","last_became_peered":"2024-08-20T03:00:18.248701+0000","last_unstale":"2024-08-20T03:00:19.234426+0000","last_undegraded":"2024-08-20T03:00:19.234426+0000","last_fullsized":"2024-08-20T03:00:19.234426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:17:11.829472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.389548+0000","last_change":"2024-08-20T03:00:16.470129+0000","last_active":"2024-08-20T03:00:29.389548+0000","last_peered":"2024-08-20T03:00:29.389548+0000","last_clean":"2024-08-20T03:00:29.389548+0000","last_became_active":"2024-08-20T03:00:16.469697+0000","last_became_peered":"2024-08-20T03:00:16.469697+0000","last_unstale":"2024-08-20T03:00:29.389548+0000","last_undegraded":"2024-08-20T03:00:29.389548+0000","last_fullsized":"2024-08-20T03:00:29.389548+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:42:22.623141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.247876+0000","last_change":"2024-08-20T03:00:18.245090+0000","last_active":"2024-08-20T03:00:19.247876+0000","last_peered":"2024-08-20T03:00:19.247876+0000","last_clean":"2024-08-20T03:00:19.247876+0000","last_became_active":"2024-08-20T03:00:18.244524+0000","last_became_peered":"2024-08-20T03:00:18.244524+0000","last_unstale":"2024-08-20T03:00:19.247876+0000","last_undegraded":"2024-08-20T03:00:19.247876+0000","last_fullsized":"2024-08-20T03:00:19.247876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:14.561183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:28.366159+0000","last_change":"2024-08-20T03:00:16.220042+0000","last_active":"2024-08-20T03:00:28.366159+0000","last_peered":"2024-08-20T03:00:28.366159+0000","last_clean":"2024-08-20T03:00:28.366159+0000","last_became_active":"2024-08-20T03:00:16.219567+0000","last_became_peered":"2024-08-20T03:00:16.219567+0000","last_unstale":"2024-08-20T03:00:28.366159+0000","last_undegraded":"2024-08-20T03:00:28.366159+0000","last_fullsized":"2024-08-20T03:00:28.366159+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:04.834039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.250137+0000","last_change":"2024-08-20T03:00:18.237497+0000","last_active":"2024-08-20T03:00:19.250137+0000","last_peered":"2024-08-20T03:00:19.250137+0000","last_clean":"2024-08-20T03:00:19.250137+0000","last_became_active":"2024-08-20T03:00:18.237170+0000","last_became_peered":"2024-08-20T03:00:18.237170+0000","last_unstale":"2024-08-20T03:00:19.250137+0000","last_undegraded":"2024-08-20T03:00:19.250137+0000","last_fullsized":"2024-08-20T03:00:19.250137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:52.387152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.387576+0000","last_change":"2024-08-20T03:00:16.214861+0000","last_active":"2024-08-20T03:00:29.387576+0000","last_peered":"2024-08-20T03:00:29.387576+0000","last_clean":"2024-08-20T03:00:29.387576+0000","last_became_active":"2024-08-20T03:00:16.214487+0000","last_became_peered":"2024-08-20T03:00:16.214487+0000","last_unstale":"2024-08-20T03:00:29.387576+0000","last_undegraded":"2024-08-20T03:00:29.387576+0000","last_fullsized":"2024-08-20T03:00:29.387576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:50:09.933304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.235076+0000","last_change":"2024-08-20T03:00:18.244685+0000","last_active":"2024-08-20T03:00:19.235076+0000","last_peered":"2024-08-20T03:00:19.235076+0000","last_clean":"2024-08-20T03:00:19.235076+0000","last_became_active":"2024-08-20T03:00:18.244372+0000","last_became_peered":"2024-08-20T03:00:18.244372+0000","last_unstale":"2024-08-20T03:00:19.235076+0000","last_undegraded":"2024-08-20T03:00:19.235076+0000","last_fullsized":"2024-08-20T03:00:19.235076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:21.821712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.390079+0000","last_change":"2024-08-20T03:00:16.222287+0000","last_active":"2024-08-20T03:00:29.390079+0000","last_peered":"2024-08-20T03:00:29.390079+0000","last_clean":"2024-08-20T03:00:29.390079+0000","last_became_active":"2024-08-20T03:00:16.221989+0000","last_became_peered":"2024-08-20T03:00:16.221989+0000","last_unstale":"2024-08-20T03:00:29.390079+0000","last_undegraded":"2024-08-20T03:00:29.390079+0000","last_fullsized":"2024-08-20T03:00:29.390079+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:32:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234901+0000","last_change":"2024-08-20T03:00:18.250488+0000","last_active":"2024-08-20T03:00:19.234901+0000","last_peered":"2024-08-20T03:00:19.234901+0000","last_clean":"2024-08-20T03:00:19.234901+0000","last_became_active":"2024-08-20T03:00:18.250094+0000","last_became_peered":"2024-08-20T03:00:18.250094+0000","last_unstale":"2024-08-20T03:00:19.234901+0000","last_undegraded":"2024-08-20T03:00:19.234901+0000","last_fullsized":"2024-08-20T03:00:19.234901+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:00:55.755920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:39.409652+0000","last_change":"2024-08-20T03:00:16.470223+0000","last_active":"2024-08-20T03:00:39.409652+0000","last_peered":"2024-08-20T03:00:39.409652+0000","last_clean":"2024-08-20T03:00:39.409652+0000","last_became_active":"2024-08-20T03:00:16.469871+0000","last_became_peered":"2024-08-20T03:00:16.469871+0000","last_unstale":"2024-08-20T03:00:39.409652+0000","last_undegraded":"2024-08-20T03:00:39.409652+0000","last_fullsized":"2024-08-20T03:00:39.409652+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:05:11.779164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234897+0000","last_change":"2024-08-20T03:00:18.242755+0000","last_active":"2024-08-20T03:00:19.234897+0000","last_peered":"2024-08-20T03:00:19.234897+0000","last_clean":"2024-08-20T03:00:19.234897+0000","last_became_active":"2024-08-20T03:00:18.242447+0000","last_became_peered":"2024-08-20T03:00:18.242447+0000","last_unstale":"2024-08-20T03:00:19.234897+0000","last_undegraded":"2024-08-20T03:00:19.234897+0000","last_fullsized":"2024-08-20T03:00:19.234897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:55:40.107933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":92,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234803+0000","last_change":"2024-08-20T02:59:02.774484+0000","last_active":"2024-08-20T03:00:19.234803+0000","last_peered":"2024-08-20T03:00:19.234803+0000","last_clean":"2024-08-20T03:00:19.234803+0000","last_became_active":"2024-08-20T02:59:02.769208+0000","last_became_peered":"2024-08-20T02:59:02.769208+0000","last_unstale":"2024-08-20T03:00:19.234803+0000","last_undegraded":"2024-08-20T03:00:19.234803+0000","last_fullsized":"2024-08-20T03:00:19.234803+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_clean_scrub_stamp":"2024-08-20T02:55:46.827249+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:36:27.931786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:30.393399+0000","last_change":"2024-08-20T03:00:16.213869+0000","last_active":"2024-08-20T03:00:30.393399+0000","last_peered":"2024-08-20T03:00:30.393399+0000","last_clean":"2024-08-20T03:00:30.393399+0000","last_became_active":"2024-08-20T03:00:16.213478+0000","last_became_peered":"2024-08-20T03:00:16.213478+0000","last_unstale":"2024-08-20T03:00:30.393399+0000","last_undegraded":"2024-08-20T03:00:30.393399+0000","last_fullsized":"2024-08-20T03:00:30.393399+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:27.830497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.246079+0000","last_change":"2024-08-20T03:00:18.242283+0000","last_active":"2024-08-20T03:00:19.246079+0000","last_peered":"2024-08-20T03:00:19.246079+0000","last_clean":"2024-08-20T03:00:19.246079+0000","last_became_active":"2024-08-20T03:00:18.241943+0000","last_became_peered":"2024-08-20T03:00:18.241943+0000","last_unstale":"2024-08-20T03:00:19.246079+0000","last_undegraded":"2024-08-20T03:00:19.246079+0000","last_fullsized":"2024-08-20T03:00:19.246079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:22:25.776964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:27.371512+0000","last_change":"2024-08-20T03:00:16.220087+0000","last_active":"2024-08-20T03:00:27.371512+0000","last_peered":"2024-08-20T03:00:27.371512+0000","last_clean":"2024-08-20T03:00:27.371512+0000","last_became_active":"2024-08-20T03:00:16.219542+0000","last_became_peered":"2024-08-20T03:00:16.219542+0000","last_unstale":"2024-08-20T03:00:27.371512+0000","last_undegraded":"2024-08-20T03:00:27.371512+0000","last_fullsized":"2024-08-20T03:00:27.371512+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:49:20.985529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.231114+0000","last_change":"2024-08-20T03:00:18.230706+0000","last_active":"2024-08-20T03:00:19.231114+0000","last_peered":"2024-08-20T03:00:19.231114+0000","last_clean":"2024-08-20T03:00:19.231114+0000","last_became_active":"2024-08-20T03:00:18.230413+0000","last_became_peered":"2024-08-20T03:00:18.230413+0000","last_unstale":"2024-08-20T03:00:19.231114+0000","last_undegraded":"2024-08-20T03:00:19.231114+0000","last_fullsized":"2024-08-20T03:00:19.231114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:52:18.772867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:40.410057+0000","last_change":"2024-08-20T03:00:16.468073+0000","last_active":"2024-08-20T03:00:40.410057+0000","last_peered":"2024-08-20T03:00:40.410057+0000","last_clean":"2024-08-20T03:00:40.410057+0000","last_became_active":"2024-08-20T03:00:16.467560+0000","last_became_peered":"2024-08-20T03:00:16.467560+0000","last_unstale":"2024-08-20T03:00:40.410057+0000","last_undegraded":"2024-08-20T03:00:40.410057+0000","last_fullsized":"2024-08-20T03:00:40.410057+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:42:24.499777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245883+0000","last_change":"2024-08-20T03:00:18.232537+0000","last_active":"2024-08-20T03:00:19.245883+0000","last_peered":"2024-08-20T03:00:19.245883+0000","last_clean":"2024-08-20T03:00:19.245883+0000","last_became_active":"2024-08-20T03:00:18.232273+0000","last_became_peered":"2024-08-20T03:00:18.232273+0000","last_unstale":"2024-08-20T03:00:19.245883+0000","last_undegraded":"2024-08-20T03:00:19.245883+0000","last_fullsized":"2024-08-20T03:00:19.245883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:36:04.981869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'6","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:36.405014+0000","last_change":"2024-08-20T03:00:16.214807+0000","last_active":"2024-08-20T03:00:36.405014+0000","last_peered":"2024-08-20T03:00:36.405014+0000","last_clean":"2024-08-20T03:00:36.405014+0000","last_became_active":"2024-08-20T03:00:16.214439+0000","last_became_peered":"2024-08-20T03:00:16.214439+0000","last_unstale":"2024-08-20T03:00:36.405014+0000","last_undegraded":"2024-08-20T03:00:36.405014+0000","last_fullsized":"2024-08-20T03:00:36.405014+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:07.519151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.247741+0000","last_change":"2024-08-20T03:00:18.247220+0000","last_active":"2024-08-20T03:00:19.247741+0000","last_peered":"2024-08-20T03:00:19.247741+0000","last_clean":"2024-08-20T03:00:19.247741+0000","last_became_active":"2024-08-20T03:00:18.246896+0000","last_became_peered":"2024-08-20T03:00:18.246896+0000","last_unstale":"2024-08-20T03:00:19.247741+0000","last_undegraded":"2024-08-20T03:00:19.247741+0000","last_fullsized":"2024-08-20T03:00:19.247741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:43:16.690590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.244978+0000","last_change":"2024-08-20T03:00:16.468280+0000","last_active":"2024-08-20T03:00:19.244978+0000","last_peered":"2024-08-20T03:00:19.244978+0000","last_clean":"2024-08-20T03:00:19.244978+0000","last_became_active":"2024-08-20T03:00:16.467930+0000","last_became_peered":"2024-08-20T03:00:16.467930+0000","last_unstale":"2024-08-20T03:00:19.244978+0000","last_undegraded":"2024-08-20T03:00:19.244978+0000","last_fullsized":"2024-08-20T03:00:19.244978+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:30.423563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:20.985562+0000","last_change":"2024-08-20T03:00:18.240322+0000","last_active":"2024-08-20T03:00:20.985562+0000","last_peered":"2024-08-20T03:00:20.985562+0000","last_clean":"2024-08-20T03:00:20.985562+0000","last_became_active":"2024-08-20T03:00:18.239973+0000","last_became_peered":"2024-08-20T03:00:18.239973+0000","last_unstale":"2024-08-20T03:00:20.985562+0000","last_undegraded":"2024-08-20T03:00:20.985562+0000","last_fullsized":"2024-08-20T03:00:20.985562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:41:28.259762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.389954+0000","last_change":"2024-08-20T03:00:16.468307+0000","last_active":"2024-08-20T03:00:29.389954+0000","last_peered":"2024-08-20T03:00:29.389954+0000","last_clean":"2024-08-20T03:00:29.389954+0000","last_became_active":"2024-08-20T03:00:16.467993+0000","last_became_peered":"2024-08-20T03:00:16.467993+0000","last_unstale":"2024-08-20T03:00:29.389954+0000","last_undegraded":"2024-08-20T03:00:29.389954+0000","last_fullsized":"2024-08-20T03:00:29.389954+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:34.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.231452+0000","last_change":"2024-08-20T03:00:18.234009+0000","last_active":"2024-08-20T03:00:19.231452+0000","last_peered":"2024-08-20T03:00:19.231452+0000","last_clean":"2024-08-20T03:00:19.231452+0000","last_became_active":"2024-08-20T03:00:18.233311+0000","last_became_peered":"2024-08-20T03:00:18.233311+0000","last_unstale":"2024-08-20T03:00:19.231452+0000","last_undegraded":"2024-08-20T03:00:19.231452+0000","last_fullsized":"2024-08-20T03:00:19.231452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:53:10.759339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'14","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:40.246072+0000","last_change":"2024-08-20T03:00:16.214932+0000","last_active":"2024-08-20T03:00:40.246072+0000","last_peered":"2024-08-20T03:00:40.246072+0000","last_clean":"2024-08-20T03:00:40.246072+0000","last_became_active":"2024-08-20T03:00:16.214584+0000","last_became_peered":"2024-08-20T03:00:16.214584+0000","last_unstale":"2024-08-20T03:00:40.246072+0000","last_undegraded":"2024-08-20T03:00:40.246072+0000","last_fullsized":"2024-08-20T03:00:40.246072+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:47.270685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.247324+0000","last_change":"2024-08-20T03:00:18.247892+0000","last_active":"2024-08-20T03:00:19.247324+0000","last_peered":"2024-08-20T03:00:19.247324+0000","last_clean":"2024-08-20T03:00:19.247324+0000","last_became_active":"2024-08-20T03:00:18.245903+0000","last_became_peered":"2024-08-20T03:00:18.245903+0000","last_unstale":"2024-08-20T03:00:19.247324+0000","last_undegraded":"2024-08-20T03:00:19.247324+0000","last_fullsized":"2024-08-20T03:00:19.247324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:08:50.909638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.390328+0000","last_change":"2024-08-20T03:00:16.372503+0000","last_active":"2024-08-20T03:00:29.390328+0000","last_peered":"2024-08-20T03:00:29.390328+0000","last_clean":"2024-08-20T03:00:29.390328+0000","last_became_active":"2024-08-20T03:00:16.372218+0000","last_became_peered":"2024-08-20T03:00:16.372218+0000","last_unstale":"2024-08-20T03:00:29.390328+0000","last_undegraded":"2024-08-20T03:00:29.390328+0000","last_fullsized":"2024-08-20T03:00:29.390328+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:13.341900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249685+0000","last_change":"2024-08-20T03:00:18.244738+0000","last_active":"2024-08-20T03:00:19.249685+0000","last_peered":"2024-08-20T03:00:19.249685+0000","last_clean":"2024-08-20T03:00:19.249685+0000","last_became_active":"2024-08-20T03:00:18.244398+0000","last_became_peered":"2024-08-20T03:00:18.244398+0000","last_unstale":"2024-08-20T03:00:19.249685+0000","last_undegraded":"2024-08-20T03:00:19.249685+0000","last_fullsized":"2024-08-20T03:00:19.249685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:45:14.540125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:26.360681+0000","last_change":"2024-08-20T03:00:16.468251+0000","last_active":"2024-08-20T03:00:26.360681+0000","last_peered":"2024-08-20T03:00:26.360681+0000","last_clean":"2024-08-20T03:00:26.360681+0000","last_became_active":"2024-08-20T03:00:16.467912+0000","last_became_peered":"2024-08-20T03:00:16.467912+0000","last_unstale":"2024-08-20T03:00:26.360681+0000","last_undegraded":"2024-08-20T03:00:26.360681+0000","last_fullsized":"2024-08-20T03:00:26.360681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:43.799747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245518+0000","last_change":"2024-08-20T03:00:18.234624+0000","last_active":"2024-08-20T03:00:19.245518+0000","last_peered":"2024-08-20T03:00:19.245518+0000","last_clean":"2024-08-20T03:00:19.245518+0000","last_became_active":"2024-08-20T03:00:18.234331+0000","last_became_peered":"2024-08-20T03:00:18.234331+0000","last_unstale":"2024-08-20T03:00:19.245518+0000","last_undegraded":"2024-08-20T03:00:19.245518+0000","last_fullsized":"2024-08-20T03:00:19.245518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:05:09.665645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.389400+0000","last_change":"2024-08-20T03:00:16.216658+0000","last_active":"2024-08-20T03:00:29.389400+0000","last_peered":"2024-08-20T03:00:29.389400+0000","last_clean":"2024-08-20T03:00:29.389400+0000","last_became_active":"2024-08-20T03:00:16.216352+0000","last_became_peered":"2024-08-20T03:00:16.216352+0000","last_unstale":"2024-08-20T03:00:29.389400+0000","last_undegraded":"2024-08-20T03:00:29.389400+0000","last_fullsized":"2024-08-20T03:00:29.389400+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:20.763280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249912+0000","last_change":"2024-08-20T03:00:18.240984+0000","last_active":"2024-08-20T03:00:19.249912+0000","last_peered":"2024-08-20T03:00:19.249912+0000","last_clean":"2024-08-20T03:00:19.249912+0000","last_became_active":"2024-08-20T03:00:18.240640+0000","last_became_peered":"2024-08-20T03:00:18.240640+0000","last_unstale":"2024-08-20T03:00:19.249912+0000","last_undegraded":"2024-08-20T03:00:19.249912+0000","last_fullsized":"2024-08-20T03:00:19.249912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:05.589186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:28.378602+0000","last_change":"2024-08-20T03:00:16.214818+0000","last_active":"2024-08-20T03:00:28.378602+0000","last_peered":"2024-08-20T03:00:28.378602+0000","last_clean":"2024-08-20T03:00:28.378602+0000","last_became_active":"2024-08-20T03:00:16.214493+0000","last_became_peered":"2024-08-20T03:00:16.214493+0000","last_unstale":"2024-08-20T03:00:28.378602+0000","last_undegraded":"2024-08-20T03:00:28.378602+0000","last_fullsized":"2024-08-20T03:00:28.378602+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:56:12.895142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.246612+0000","last_change":"2024-08-20T03:00:18.240350+0000","last_active":"2024-08-20T03:00:19.246612+0000","last_peered":"2024-08-20T03:00:19.246612+0000","last_clean":"2024-08-20T03:00:19.246612+0000","last_became_active":"2024-08-20T03:00:18.239464+0000","last_became_peered":"2024-08-20T03:00:18.239464+0000","last_unstale":"2024-08-20T03:00:19.246612+0000","last_undegraded":"2024-08-20T03:00:19.246612+0000","last_fullsized":"2024-08-20T03:00:19.246612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:48.715130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:26.361292+0000","last_change":"2024-08-20T03:00:16.216144+0000","last_active":"2024-08-20T03:00:26.361292+0000","last_peered":"2024-08-20T03:00:26.361292+0000","last_clean":"2024-08-20T03:00:26.361292+0000","last_became_active":"2024-08-20T03:00:16.215792+0000","last_became_peered":"2024-08-20T03:00:16.215792+0000","last_unstale":"2024-08-20T03:00:26.361292+0000","last_undegraded":"2024-08-20T03:00:26.361292+0000","last_fullsized":"2024-08-20T03:00:26.361292+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:25:01.500878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.235056+0000","last_change":"2024-08-20T03:00:18.245286+0000","last_active":"2024-08-20T03:00:19.235056+0000","last_peered":"2024-08-20T03:00:19.235056+0000","last_clean":"2024-08-20T03:00:19.235056+0000","last_became_active":"2024-08-20T03:00:18.244760+0000","last_became_peered":"2024-08-20T03:00:18.244760+0000","last_unstale":"2024-08-20T03:00:19.235056+0000","last_undegraded":"2024-08-20T03:00:19.235056+0000","last_fullsized":"2024-08-20T03:00:19.235056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:09.415794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.248848+0000","last_change":"2024-08-20T03:00:18.244713+0000","last_active":"2024-08-20T03:00:19.248848+0000","last_peered":"2024-08-20T03:00:19.248848+0000","last_clean":"2024-08-20T03:00:19.248848+0000","last_became_active":"2024-08-20T03:00:18.244368+0000","last_became_peered":"2024-08-20T03:00:18.244368+0000","last_unstale":"2024-08-20T03:00:19.248848+0000","last_undegraded":"2024-08-20T03:00:19.248848+0000","last_fullsized":"2024-08-20T03:00:19.248848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:17:21.407541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.389710+0000","last_change":"2024-08-20T03:00:16.220019+0000","last_active":"2024-08-20T03:00:29.389710+0000","last_peered":"2024-08-20T03:00:29.389710+0000","last_clean":"2024-08-20T03:00:29.389710+0000","last_became_active":"2024-08-20T03:00:16.219624+0000","last_became_peered":"2024-08-20T03:00:16.219624+0000","last_unstale":"2024-08-20T03:00:29.389710+0000","last_undegraded":"2024-08-20T03:00:29.389710+0000","last_fullsized":"2024-08-20T03:00:29.389710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:33:24.926695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.250279+0000","last_change":"2024-08-20T03:00:18.242156+0000","last_active":"2024-08-20T03:00:19.250279+0000","last_peered":"2024-08-20T03:00:19.250279+0000","last_clean":"2024-08-20T03:00:19.250279+0000","last_became_active":"2024-08-20T03:00:18.241811+0000","last_became_peered":"2024-08-20T03:00:18.241811+0000","last_unstale":"2024-08-20T03:00:19.250279+0000","last_undegraded":"2024-08-20T03:00:19.250279+0000","last_fullsized":"2024-08-20T03:00:19.250279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:58.230658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:30.385077+0000","last_change":"2024-08-20T03:00:16.215625+0000","last_active":"2024-08-20T03:00:30.385077+0000","last_peered":"2024-08-20T03:00:30.385077+0000","last_clean":"2024-08-20T03:00:30.385077+0000","last_became_active":"2024-08-20T03:00:16.215087+0000","last_became_peered":"2024-08-20T03:00:16.215087+0000","last_unstale":"2024-08-20T03:00:30.385077+0000","last_undegraded":"2024-08-20T03:00:30.385077+0000","last_fullsized":"2024-08-20T03:00:30.385077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:35.433875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249869+0000","last_change":"2024-08-20T03:00:18.245256+0000","last_active":"2024-08-20T03:00:19.249869+0000","last_peered":"2024-08-20T03:00:19.249869+0000","last_clean":"2024-08-20T03:00:19.249869+0000","last_became_active":"2024-08-20T03:00:18.244339+0000","last_became_peered":"2024-08-20T03:00:18.244339+0000","last_unstale":"2024-08-20T03:00:19.249869+0000","last_undegraded":"2024-08-20T03:00:19.249869+0000","last_fullsized":"2024-08-20T03:00:19.249869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:00.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:20.200015+0000","last_change":"2024-08-20T03:00:16.470004+0000","last_active":"2024-08-20T03:00:20.200015+0000","last_peered":"2024-08-20T03:00:20.200015+0000","last_clean":"2024-08-20T03:00:20.200015+0000","last_became_active":"2024-08-20T03:00:16.469646+0000","last_became_peered":"2024-08-20T03:00:16.469646+0000","last_unstale":"2024-08-20T03:00:20.200015+0000","last_undegraded":"2024-08-20T03:00:20.200015+0000","last_fullsized":"2024-08-20T03:00:20.200015+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:35.244683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.247346+0000","last_change":"2024-08-20T03:00:18.238053+0000","last_active":"2024-08-20T03:00:19.247346+0000","last_peered":"2024-08-20T03:00:19.247346+0000","last_clean":"2024-08-20T03:00:19.247346+0000","last_became_active":"2024-08-20T03:00:18.237186+0000","last_became_peered":"2024-08-20T03:00:18.237186+0000","last_unstale":"2024-08-20T03:00:19.247346+0000","last_undegraded":"2024-08-20T03:00:19.247346+0000","last_fullsized":"2024-08-20T03:00:19.247346+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:14.425688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:40.409338+0000","last_change":"2024-08-20T03:00:16.214584+0000","last_active":"2024-08-20T03:00:40.409338+0000","last_peered":"2024-08-20T03:00:40.409338+0000","last_clean":"2024-08-20T03:00:40.409338+0000","last_became_active":"2024-08-20T03:00:16.214255+0000","last_became_peered":"2024-08-20T03:00:16.214255+0000","last_unstale":"2024-08-20T03:00:40.409338+0000","last_undegraded":"2024-08-20T03:00:40.409338+0000","last_fullsized":"2024-08-20T03:00:40.409338+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:16:30.983745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57314,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":143,"num_read_kb":140,"num_write":238,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":162895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281428,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743442,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":819200,"data_stored":400341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939670,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1041480,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103196,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":400271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299424,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":400649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":993558,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":827392,"data_stored":400379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":992845,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":827392,"data_stored":399849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":399813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378686,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":451622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542210,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":442355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705734,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":401327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":43206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":51846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T03:00:44.538 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T03:00:44.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:44 smithi044 ceph-mon[18790]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:00:44.790 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:44.816 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:44 smithi099 ceph-mon[21984]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:00:44.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:44 smithi165 ceph-mon[22357]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:00:45.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.303+0000 7fedef34d640 1 -- 172.21.15.44:0/1806487896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede810ce40 msgr2=0x7fede810f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.303+0000 7fedef34d640 1 --2- 172.21.15.44:0/1806487896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede810ce40 0x7fede810f2a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fede0009ca0 tx=0x7fede002a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.303+0000 7fedef34d640 1 -- 172.21.15.44:0/1806487896 shutdown_connections 2024-08-20T03:00:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.303+0000 7fedef34d640 1 --2- 172.21.15.44:0/1806487896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede810ce40 0x7fede810f2a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.303+0000 7fedef34d640 1 --2- 172.21.15.44:0/1806487896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede8104300 0x7fede810c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.303+0000 7fedef34d640 1 --2- 172.21.15.44:0/1806487896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede81039e0 0x7fede8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.303+0000 7fedef34d640 1 -- 172.21.15.44:0/1806487896 >> 172.21.15.44:0/1806487896 conn(0x7fede80fd440 msgr2=0x7fede80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.303+0000 7fedef34d640 1 -- 172.21.15.44:0/1806487896 shutdown_connections 2024-08-20T03:00:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.303+0000 7fedef34d640 1 -- 172.21.15.44:0/1806487896 wait complete. 2024-08-20T03:00:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7fedef34d640 1 Processor -- start 2024-08-20T03:00:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7fedef34d640 1 -- start start 2024-08-20T03:00:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7fedef34d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede81039e0 0x7fede8101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:45.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7fedef34d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede8104300 0x7fede8101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7fedef34d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede810ce40 0x7fede8102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7fedef34d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fede8112af0 con 0x7fede81039e0 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7fedef34d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fede8112970 con 0x7fede8104300 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7fedef34d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fede8112c70 con 0x7fede810ce40 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7feded0e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede81039e0 0x7fede8101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7feded0e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede81039e0 0x7fede8101690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54630/0 (socket says 172.21.15.44:54630) 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7feded0e9640 1 -- 172.21.15.44:0/2978235835 learned_addr learned my addr 172.21.15.44:0/2978235835 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7fedec8e8640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede8104300 0x7fede8101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7feded8ea640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede810ce40 0x7fede8102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7feded0e9640 1 -- 172.21.15.44:0/2978235835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede810ce40 msgr2=0x7fede8102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7feded0e9640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede810ce40 0x7fede8102110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7feded0e9640 1 -- 172.21.15.44:0/2978235835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede8104300 msgr2=0x7fede8101bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:45.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7feded0e9640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede8104300 0x7fede8101bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:45.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.304+0000 7feded0e9640 1 -- 172.21.15.44:0/2978235835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede81a73e0 con 0x7fede81039e0 2024-08-20T03:00:45.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.305+0000 7feded0e9640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede81039e0 0x7fede8101690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fede400b4a0 tx=0x7fede400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:45.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.305+0000 7fedde7fc640 1 -- 172.21.15.44:0/2978235835 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4014070 con 0x7fede81039e0 2024-08-20T03:00:45.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.305+0000 7fedde7fc640 1 -- 172.21.15.44:0/2978235835 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede4010ac0 con 0x7fede81039e0 2024-08-20T03:00:45.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.305+0000 7fedde7fc640 1 -- 172.21.15.44:0/2978235835 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4017c30 con 0x7fede81039e0 2024-08-20T03:00:45.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.305+0000 7fedef34d640 1 -- 172.21.15.44:0/2978235835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede81a76d0 con 0x7fede81039e0 2024-08-20T03:00:45.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.305+0000 7fedef34d640 1 -- 172.21.15.44:0/2978235835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede81a7c40 con 0x7fede81039e0 2024-08-20T03:00:45.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.306+0000 7fedef34d640 1 -- 172.21.15.44:0/2978235835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede8103dc0 con 0x7fede81039e0 2024-08-20T03:00:45.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.308+0000 7fedde7fc640 1 -- 172.21.15.44:0/2978235835 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fede4017070 con 0x7fede81039e0 2024-08-20T03:00:45.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.309+0000 7fedde7fc640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fedd40b3d30 0x7fedd40b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:45.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.309+0000 7fedec8e8640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fedd40b3d30 0x7fedd40b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:45.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.309+0000 7fedde7fc640 1 -- 172.21.15.44:0/2978235835 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fede40e3480 con 0x7fede81039e0 2024-08-20T03:00:45.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.309+0000 7fedec8e8640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fedd40b3d30 0x7fedd40b61f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fede8103180 tx=0x7fedd800b2e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:45.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.312+0000 7fedde7fc640 1 -- 172.21.15.44:0/2978235835 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fede40aaa40 con 0x7fede81039e0 2024-08-20T03:00:45.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.496+0000 7fedef34d640 1 -- 172.21.15.44:0/2978235835 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fede80ff760 con 0x7fedd40b3d30 2024-08-20T03:00:45.499 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:45 smithi044 ceph-mon[18790]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-08-20T03:00:45.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.500+0000 7fedde7fc640 1 -- 172.21.15.44:0/2978235835 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258901 (secure 0 0 0) 0x7fede80ff760 con 0x7fedd40b3d30 2024-08-20T03:00:45.505 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:45.505 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-08-20T03:00:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.503+0000 7fedef34d640 1 -- 172.21.15.44:0/2978235835 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fedd40b3d30 msgr2=0x7fedd40b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.503+0000 7fedef34d640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fedd40b3d30 0x7fedd40b61f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fede8103180 tx=0x7fedd800b2e0 comp rx=0 tx=0).stop 2024-08-20T03:00:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.503+0000 7fedef34d640 1 -- 172.21.15.44:0/2978235835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede81039e0 msgr2=0x7fede8101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.503+0000 7fedef34d640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede81039e0 0x7fede8101690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fede400b4a0 tx=0x7fede400b960 comp rx=0 tx=0).stop 2024-08-20T03:00:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.503+0000 7fedef34d640 1 -- 172.21.15.44:0/2978235835 shutdown_connections 2024-08-20T03:00:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.503+0000 7fedef34d640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fedd40b3d30 0x7fedd40b61f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.503+0000 7fedef34d640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede810ce40 0x7fede8102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.503+0000 7fedef34d640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fede8104300 0x7fede8101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:45.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.503+0000 7fedef34d640 1 --2- 172.21.15.44:0/2978235835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fede81039e0 0x7fede8101690 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.503+0000 7fedef34d640 1 -- 172.21.15.44:0/2978235835 >> 172.21.15.44:0/2978235835 conn(0x7fede80fd440 msgr2=0x7fede80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.503+0000 7fedef34d640 1 -- 172.21.15.44:0/2978235835 shutdown_connections 2024-08-20T03:00:45.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:45.503+0000 7fedef34d640 1 -- 172.21.15.44:0/2978235835 wait complete. 2024-08-20T03:00:45.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:45 smithi165 ceph-mon[22357]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-08-20T03:00:45.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:45 smithi099 ceph-mon[21984]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-08-20T03:00:46.276 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-08-20T03:00:44.394670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647916,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":239,"num_read_kb":209,"num_write":363,"num_write_kb":2274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363560,"kb_used_data":10152,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124578264,"statfs":{"total":1151940427776,"available":1151568142336,"internally_reserved":0,"allocated":10395648,"data_stored":6733536,"data_compressed":107358,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50136,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":133,"num_read_kb":117,"num_write":47,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002684"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.250895+0000","last_change":"2024-08-20T03:00:18.238595+0000","last_active":"2024-08-20T03:00:19.250895+0000","last_peered":"2024-08-20T03:00:19.250895+0000","last_clean":"2024-08-20T03:00:19.250895+0000","last_became_active":"2024-08-20T03:00:18.238306+0000","last_became_peered":"2024-08-20T03:00:18.238306+0000","last_unstale":"2024-08-20T03:00:19.250895+0000","last_undegraded":"2024-08-20T03:00:19.250895+0000","last_fullsized":"2024-08-20T03:00:19.250895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:33:04.681614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249682+0000","last_change":"2024-08-20T03:00:18.239722+0000","last_active":"2024-08-20T03:00:19.249682+0000","last_peered":"2024-08-20T03:00:19.249682+0000","last_clean":"2024-08-20T03:00:19.249682+0000","last_became_active":"2024-08-20T03:00:18.239391+0000","last_became_peered":"2024-08-20T03:00:18.239391+0000","last_unstale":"2024-08-20T03:00:19.249682+0000","last_undegraded":"2024-08-20T03:00:19.249682+0000","last_fullsized":"2024-08-20T03:00:19.249682+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:08:34.657173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249463+0000","last_change":"2024-08-20T03:00:18.242389+0000","last_active":"2024-08-20T03:00:19.249463+0000","last_peered":"2024-08-20T03:00:19.249463+0000","last_clean":"2024-08-20T03:00:19.249463+0000","last_became_active":"2024-08-20T03:00:18.242080+0000","last_became_peered":"2024-08-20T03:00:18.242080+0000","last_unstale":"2024-08-20T03:00:19.249463+0000","last_undegraded":"2024-08-20T03:00:19.249463+0000","last_fullsized":"2024-08-20T03:00:19.249463+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:37.042894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245403+0000","last_change":"2024-08-20T03:00:18.233709+0000","last_active":"2024-08-20T03:00:19.245403+0000","last_peered":"2024-08-20T03:00:19.245403+0000","last_clean":"2024-08-20T03:00:19.245403+0000","last_became_active":"2024-08-20T03:00:18.233402+0000","last_became_peered":"2024-08-20T03:00:18.233402+0000","last_unstale":"2024-08-20T03:00:19.245403+0000","last_undegraded":"2024-08-20T03:00:19.245403+0000","last_fullsized":"2024-08-20T03:00:19.245403+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:30:01.308181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.246592+0000","last_change":"2024-08-20T03:00:18.237770+0000","last_active":"2024-08-20T03:00:19.246592+0000","last_peered":"2024-08-20T03:00:19.246592+0000","last_clean":"2024-08-20T03:00:19.246592+0000","last_became_active":"2024-08-20T03:00:18.237475+0000","last_became_peered":"2024-08-20T03:00:18.237475+0000","last_unstale":"2024-08-20T03:00:19.246592+0000","last_undegraded":"2024-08-20T03:00:19.246592+0000","last_fullsized":"2024-08-20T03:00:19.246592+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:37:12.707935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245669+0000","last_change":"2024-08-20T03:00:18.245400+0000","last_active":"2024-08-20T03:00:19.245669+0000","last_peered":"2024-08-20T03:00:19.245669+0000","last_clean":"2024-08-20T03:00:19.245669+0000","last_became_active":"2024-08-20T03:00:18.244735+0000","last_became_peered":"2024-08-20T03:00:18.244735+0000","last_unstale":"2024-08-20T03:00:19.245669+0000","last_undegraded":"2024-08-20T03:00:19.245669+0000","last_fullsized":"2024-08-20T03:00:19.245669+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:23:25.459441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234700+0000","last_change":"2024-08-20T03:00:18.247366+0000","last_active":"2024-08-20T03:00:19.234700+0000","last_peered":"2024-08-20T03:00:19.234700+0000","last_clean":"2024-08-20T03:00:19.234700+0000","last_became_active":"2024-08-20T03:00:18.247032+0000","last_became_peered":"2024-08-20T03:00:18.247032+0000","last_unstale":"2024-08-20T03:00:19.234700+0000","last_undegraded":"2024-08-20T03:00:19.234700+0000","last_fullsized":"2024-08-20T03:00:19.234700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:02:18.070393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245915+0000","last_change":"2024-08-20T03:00:18.240845+0000","last_active":"2024-08-20T03:00:19.245915+0000","last_peered":"2024-08-20T03:00:19.245915+0000","last_clean":"2024-08-20T03:00:19.245915+0000","last_became_active":"2024-08-20T03:00:18.240531+0000","last_became_peered":"2024-08-20T03:00:18.240531+0000","last_unstale":"2024-08-20T03:00:19.245915+0000","last_undegraded":"2024-08-20T03:00:19.245915+0000","last_fullsized":"2024-08-20T03:00:19.245915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:15:18.975804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249266+0000","last_change":"2024-08-20T03:00:18.240817+0000","last_active":"2024-08-20T03:00:19.249266+0000","last_peered":"2024-08-20T03:00:19.249266+0000","last_clean":"2024-08-20T03:00:19.249266+0000","last_became_active":"2024-08-20T03:00:18.240262+0000","last_became_peered":"2024-08-20T03:00:18.240262+0000","last_unstale":"2024-08-20T03:00:19.249266+0000","last_undegraded":"2024-08-20T03:00:19.249266+0000","last_fullsized":"2024-08-20T03:00:19.249266+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:30.512597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245601+0000","last_change":"2024-08-20T03:00:18.242139+0000","last_active":"2024-08-20T03:00:19.245601+0000","last_peered":"2024-08-20T03:00:19.245601+0000","last_clean":"2024-08-20T03:00:19.245601+0000","last_became_active":"2024-08-20T03:00:18.241817+0000","last_became_peered":"2024-08-20T03:00:18.241817+0000","last_unstale":"2024-08-20T03:00:19.245601+0000","last_undegraded":"2024-08-20T03:00:19.245601+0000","last_fullsized":"2024-08-20T03:00:19.245601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:25.121306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234785+0000","last_change":"2024-08-20T03:00:18.243205+0000","last_active":"2024-08-20T03:00:19.234785+0000","last_peered":"2024-08-20T03:00:19.234785+0000","last_clean":"2024-08-20T03:00:19.234785+0000","last_became_active":"2024-08-20T03:00:18.242296+0000","last_became_peered":"2024-08-20T03:00:18.242296+0000","last_unstale":"2024-08-20T03:00:19.234785+0000","last_undegraded":"2024-08-20T03:00:19.234785+0000","last_fullsized":"2024-08-20T03:00:19.234785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:50.490934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245585+0000","last_change":"2024-08-20T03:00:18.237205+0000","last_active":"2024-08-20T03:00:19.245585+0000","last_peered":"2024-08-20T03:00:19.245585+0000","last_clean":"2024-08-20T03:00:19.245585+0000","last_became_active":"2024-08-20T03:00:18.236631+0000","last_became_peered":"2024-08-20T03:00:18.236631+0000","last_unstale":"2024-08-20T03:00:19.245585+0000","last_undegraded":"2024-08-20T03:00:19.245585+0000","last_fullsized":"2024-08-20T03:00:19.245585+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:13.327598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.231337+0000","last_change":"2024-08-20T03:00:18.235768+0000","last_active":"2024-08-20T03:00:19.231337+0000","last_peered":"2024-08-20T03:00:19.231337+0000","last_clean":"2024-08-20T03:00:19.231337+0000","last_became_active":"2024-08-20T03:00:18.235461+0000","last_became_peered":"2024-08-20T03:00:18.235461+0000","last_unstale":"2024-08-20T03:00:19.231337+0000","last_undegraded":"2024-08-20T03:00:19.231337+0000","last_fullsized":"2024-08-20T03:00:19.231337+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:08:22.238020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249572+0000","last_change":"2024-08-20T03:00:18.246560+0000","last_active":"2024-08-20T03:00:19.249572+0000","last_peered":"2024-08-20T03:00:19.249572+0000","last_clean":"2024-08-20T03:00:19.249572+0000","last_became_active":"2024-08-20T03:00:18.246270+0000","last_became_peered":"2024-08-20T03:00:18.246270+0000","last_unstale":"2024-08-20T03:00:19.249572+0000","last_undegraded":"2024-08-20T03:00:19.249572+0000","last_fullsized":"2024-08-20T03:00:19.249572+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:47:13.402635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249359+0000","last_change":"2024-08-20T03:00:18.245907+0000","last_active":"2024-08-20T03:00:19.249359+0000","last_peered":"2024-08-20T03:00:19.249359+0000","last_clean":"2024-08-20T03:00:19.249359+0000","last_became_active":"2024-08-20T03:00:18.245429+0000","last_became_peered":"2024-08-20T03:00:18.245429+0000","last_unstale":"2024-08-20T03:00:19.249359+0000","last_undegraded":"2024-08-20T03:00:19.249359+0000","last_fullsized":"2024-08-20T03:00:19.249359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:46.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.248555+0000","last_change":"2024-08-20T03:00:18.242561+0000","last_active":"2024-08-20T03:00:19.248555+0000","last_peered":"2024-08-20T03:00:19.248555+0000","last_clean":"2024-08-20T03:00:19.248555+0000","last_became_active":"2024-08-20T03:00:18.242048+0000","last_became_peered":"2024-08-20T03:00:18.242048+0000","last_unstale":"2024-08-20T03:00:19.248555+0000","last_undegraded":"2024-08-20T03:00:19.248555+0000","last_fullsized":"2024-08-20T03:00:19.248555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:27.784131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.250977+0000","last_change":"2024-08-20T03:00:18.238547+0000","last_active":"2024-08-20T03:00:19.250977+0000","last_peered":"2024-08-20T03:00:19.250977+0000","last_clean":"2024-08-20T03:00:19.250977+0000","last_became_active":"2024-08-20T03:00:18.238268+0000","last_became_peered":"2024-08-20T03:00:18.238268+0000","last_unstale":"2024-08-20T03:00:19.250977+0000","last_undegraded":"2024-08-20T03:00:19.250977+0000","last_fullsized":"2024-08-20T03:00:19.250977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:15:11.923674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.244953+0000","last_change":"2024-08-20T03:00:18.241995+0000","last_active":"2024-08-20T03:00:19.244953+0000","last_peered":"2024-08-20T03:00:19.244953+0000","last_clean":"2024-08-20T03:00:19.244953+0000","last_became_active":"2024-08-20T03:00:18.241705+0000","last_became_peered":"2024-08-20T03:00:18.241705+0000","last_unstale":"2024-08-20T03:00:19.244953+0000","last_undegraded":"2024-08-20T03:00:19.244953+0000","last_fullsized":"2024-08-20T03:00:19.244953+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:33:02.183712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.232295+0000","last_change":"2024-08-20T03:00:18.236800+0000","last_active":"2024-08-20T03:00:19.232295+0000","last_peered":"2024-08-20T03:00:19.232295+0000","last_clean":"2024-08-20T03:00:19.232295+0000","last_became_active":"2024-08-20T03:00:18.235909+0000","last_became_peered":"2024-08-20T03:00:18.235909+0000","last_unstale":"2024-08-20T03:00:19.232295+0000","last_undegraded":"2024-08-20T03:00:19.232295+0000","last_fullsized":"2024-08-20T03:00:19.232295+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:12.427760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249752+0000","last_change":"2024-08-20T03:00:18.237554+0000","last_active":"2024-08-20T03:00:19.249752+0000","last_peered":"2024-08-20T03:00:19.249752+0000","last_clean":"2024-08-20T03:00:19.249752+0000","last_became_active":"2024-08-20T03:00:18.237236+0000","last_became_peered":"2024-08-20T03:00:18.237236+0000","last_unstale":"2024-08-20T03:00:19.249752+0000","last_undegraded":"2024-08-20T03:00:19.249752+0000","last_fullsized":"2024-08-20T03:00:19.249752+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:13:42.997517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.231487+0000","last_change":"2024-08-20T03:00:18.243425+0000","last_active":"2024-08-20T03:00:19.231487+0000","last_peered":"2024-08-20T03:00:19.231487+0000","last_clean":"2024-08-20T03:00:19.231487+0000","last_became_active":"2024-08-20T03:00:18.243082+0000","last_became_peered":"2024-08-20T03:00:18.243082+0000","last_unstale":"2024-08-20T03:00:19.231487+0000","last_undegraded":"2024-08-20T03:00:19.231487+0000","last_fullsized":"2024-08-20T03:00:19.231487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:50.008655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249240+0000","last_change":"2024-08-20T03:00:18.242765+0000","last_active":"2024-08-20T03:00:19.249240+0000","last_peered":"2024-08-20T03:00:19.249240+0000","last_clean":"2024-08-20T03:00:19.249240+0000","last_became_active":"2024-08-20T03:00:18.242459+0000","last_became_peered":"2024-08-20T03:00:18.242459+0000","last_unstale":"2024-08-20T03:00:19.249240+0000","last_undegraded":"2024-08-20T03:00:19.249240+0000","last_fullsized":"2024-08-20T03:00:19.249240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:16.467068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245966+0000","last_change":"2024-08-20T03:00:18.236518+0000","last_active":"2024-08-20T03:00:19.245966+0000","last_peered":"2024-08-20T03:00:19.245966+0000","last_clean":"2024-08-20T03:00:19.245966+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:00:19.245966+0000","last_undegraded":"2024-08-20T03:00:19.245966+0000","last_fullsized":"2024-08-20T03:00:19.245966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:55:55.387134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245691+0000","last_change":"2024-08-20T03:00:18.236730+0000","last_active":"2024-08-20T03:00:19.245691+0000","last_peered":"2024-08-20T03:00:19.245691+0000","last_clean":"2024-08-20T03:00:19.245691+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:00:19.245691+0000","last_undegraded":"2024-08-20T03:00:19.245691+0000","last_fullsized":"2024-08-20T03:00:19.245691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:18.196161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.247220+0000","last_change":"2024-08-20T03:00:18.245390+0000","last_active":"2024-08-20T03:00:19.247220+0000","last_peered":"2024-08-20T03:00:19.247220+0000","last_clean":"2024-08-20T03:00:19.247220+0000","last_became_active":"2024-08-20T03:00:18.241175+0000","last_became_peered":"2024-08-20T03:00:18.241175+0000","last_unstale":"2024-08-20T03:00:19.247220+0000","last_undegraded":"2024-08-20T03:00:19.247220+0000","last_fullsized":"2024-08-20T03:00:19.247220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:51:45.792846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.232616+0000","last_change":"2024-08-20T03:00:18.242700+0000","last_active":"2024-08-20T03:00:19.232616+0000","last_peered":"2024-08-20T03:00:19.232616+0000","last_clean":"2024-08-20T03:00:19.232616+0000","last_became_active":"2024-08-20T03:00:18.242407+0000","last_became_peered":"2024-08-20T03:00:18.242407+0000","last_unstale":"2024-08-20T03:00:19.232616+0000","last_undegraded":"2024-08-20T03:00:19.232616+0000","last_fullsized":"2024-08-20T03:00:19.232616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:31:28.061514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.250462+0000","last_change":"2024-08-20T03:00:18.233894+0000","last_active":"2024-08-20T03:00:19.250462+0000","last_peered":"2024-08-20T03:00:19.250462+0000","last_clean":"2024-08-20T03:00:19.250462+0000","last_became_active":"2024-08-20T03:00:18.233584+0000","last_became_peered":"2024-08-20T03:00:18.233584+0000","last_unstale":"2024-08-20T03:00:19.250462+0000","last_undegraded":"2024-08-20T03:00:19.250462+0000","last_fullsized":"2024-08-20T03:00:19.250462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:36:53.156215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.231278+0000","last_change":"2024-08-20T03:00:18.239256+0000","last_active":"2024-08-20T03:00:19.231278+0000","last_peered":"2024-08-20T03:00:19.231278+0000","last_clean":"2024-08-20T03:00:19.231278+0000","last_became_active":"2024-08-20T03:00:18.238931+0000","last_became_peered":"2024-08-20T03:00:18.238931+0000","last_unstale":"2024-08-20T03:00:19.231278+0000","last_undegraded":"2024-08-20T03:00:19.231278+0000","last_fullsized":"2024-08-20T03:00:19.231278+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:38:42.536488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249465+0000","last_change":"2024-08-20T03:00:18.245187+0000","last_active":"2024-08-20T03:00:19.249465+0000","last_peered":"2024-08-20T03:00:19.249465+0000","last_clean":"2024-08-20T03:00:19.249465+0000","last_became_active":"2024-08-20T03:00:18.244894+0000","last_became_peered":"2024-08-20T03:00:18.244894+0000","last_unstale":"2024-08-20T03:00:19.249465+0000","last_undegraded":"2024-08-20T03:00:19.249465+0000","last_fullsized":"2024-08-20T03:00:19.249465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:16:58.002014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.232402+0000","last_change":"2024-08-20T03:00:18.241634+0000","last_active":"2024-08-20T03:00:19.232402+0000","last_peered":"2024-08-20T03:00:19.232402+0000","last_clean":"2024-08-20T03:00:19.232402+0000","last_became_active":"2024-08-20T03:00:18.241341+0000","last_became_peered":"2024-08-20T03:00:18.241341+0000","last_unstale":"2024-08-20T03:00:19.232402+0000","last_undegraded":"2024-08-20T03:00:19.232402+0000","last_fullsized":"2024-08-20T03:00:19.232402+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:37:16.185544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.232393+0000","last_change":"2024-08-20T03:00:18.239116+0000","last_active":"2024-08-20T03:00:19.232393+0000","last_peered":"2024-08-20T03:00:19.232393+0000","last_clean":"2024-08-20T03:00:19.232393+0000","last_became_active":"2024-08-20T03:00:18.236125+0000","last_became_peered":"2024-08-20T03:00:18.236125+0000","last_unstale":"2024-08-20T03:00:19.232393+0000","last_undegraded":"2024-08-20T03:00:19.232393+0000","last_fullsized":"2024-08-20T03:00:19.232393+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:57:00.332483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249712+0000","last_change":"2024-08-20T03:00:18.235996+0000","last_active":"2024-08-20T03:00:19.249712+0000","last_peered":"2024-08-20T03:00:19.249712+0000","last_clean":"2024-08-20T03:00:19.249712+0000","last_became_active":"2024-08-20T03:00:18.235670+0000","last_became_peered":"2024-08-20T03:00:18.235670+0000","last_unstale":"2024-08-20T03:00:19.249712+0000","last_undegraded":"2024-08-20T03:00:19.249712+0000","last_fullsized":"2024-08-20T03:00:19.249712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:59.992914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245508+0000","last_change":"2024-08-20T03:00:18.239679+0000","last_active":"2024-08-20T03:00:19.245508+0000","last_peered":"2024-08-20T03:00:19.245508+0000","last_clean":"2024-08-20T03:00:19.245508+0000","last_became_active":"2024-08-20T03:00:18.239361+0000","last_became_peered":"2024-08-20T03:00:18.239361+0000","last_unstale":"2024-08-20T03:00:19.245508+0000","last_undegraded":"2024-08-20T03:00:19.245508+0000","last_fullsized":"2024-08-20T03:00:19.245508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:28:09.430535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:42.413403+0000","last_change":"2024-08-20T03:00:16.221221+0000","last_active":"2024-08-20T03:00:42.413403+0000","last_peered":"2024-08-20T03:00:42.413403+0000","last_clean":"2024-08-20T03:00:42.413403+0000","last_became_active":"2024-08-20T03:00:16.220186+0000","last_became_peered":"2024-08-20T03:00:16.220186+0000","last_unstale":"2024-08-20T03:00:42.413403+0000","last_undegraded":"2024-08-20T03:00:42.413403+0000","last_fullsized":"2024-08-20T03:00:42.413403+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:51.045235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.248996+0000","last_change":"2024-08-20T03:00:18.234809+0000","last_active":"2024-08-20T03:00:19.248996+0000","last_peered":"2024-08-20T03:00:19.248996+0000","last_clean":"2024-08-20T03:00:19.248996+0000","last_became_active":"2024-08-20T03:00:18.234486+0000","last_became_peered":"2024-08-20T03:00:18.234486+0000","last_unstale":"2024-08-20T03:00:19.248996+0000","last_undegraded":"2024-08-20T03:00:19.248996+0000","last_fullsized":"2024-08-20T03:00:19.248996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:11:52.527224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:28.379401+0000","last_change":"2024-08-20T03:00:16.216876+0000","last_active":"2024-08-20T03:00:28.379401+0000","last_peered":"2024-08-20T03:00:28.379401+0000","last_clean":"2024-08-20T03:00:28.379401+0000","last_became_active":"2024-08-20T03:00:16.216522+0000","last_became_peered":"2024-08-20T03:00:16.216522+0000","last_unstale":"2024-08-20T03:00:28.379401+0000","last_undegraded":"2024-08-20T03:00:28.379401+0000","last_fullsized":"2024-08-20T03:00:28.379401+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:22.363331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:27.371388+0000","last_change":"2024-08-20T03:00:16.213751+0000","last_active":"2024-08-20T03:00:27.371388+0000","last_peered":"2024-08-20T03:00:27.371388+0000","last_clean":"2024-08-20T03:00:27.371388+0000","last_became_active":"2024-08-20T03:00:16.213348+0000","last_became_peered":"2024-08-20T03:00:16.213348+0000","last_unstale":"2024-08-20T03:00:27.371388+0000","last_undegraded":"2024-08-20T03:00:27.371388+0000","last_fullsized":"2024-08-20T03:00:27.371388+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:01:08.637151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245981+0000","last_change":"2024-08-20T03:00:18.237972+0000","last_active":"2024-08-20T03:00:19.245981+0000","last_peered":"2024-08-20T03:00:19.245981+0000","last_clean":"2024-08-20T03:00:19.245981+0000","last_became_active":"2024-08-20T03:00:18.236769+0000","last_became_peered":"2024-08-20T03:00:18.236769+0000","last_unstale":"2024-08-20T03:00:19.245981+0000","last_undegraded":"2024-08-20T03:00:19.245981+0000","last_fullsized":"2024-08-20T03:00:19.245981+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:35:21.001045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.390750+0000","last_change":"2024-08-20T03:00:16.221399+0000","last_active":"2024-08-20T03:00:29.390750+0000","last_peered":"2024-08-20T03:00:29.390750+0000","last_clean":"2024-08-20T03:00:29.390750+0000","last_became_active":"2024-08-20T03:00:16.220285+0000","last_became_peered":"2024-08-20T03:00:16.220285+0000","last_unstale":"2024-08-20T03:00:29.390750+0000","last_undegraded":"2024-08-20T03:00:29.390750+0000","last_fullsized":"2024-08-20T03:00:29.390750+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:17.034657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.244765+0000","last_change":"2024-08-20T03:00:18.236722+0000","last_active":"2024-08-20T03:00:19.244765+0000","last_peered":"2024-08-20T03:00:19.244765+0000","last_clean":"2024-08-20T03:00:19.244765+0000","last_became_active":"2024-08-20T03:00:18.236365+0000","last_became_peered":"2024-08-20T03:00:18.236365+0000","last_unstale":"2024-08-20T03:00:19.244765+0000","last_undegraded":"2024-08-20T03:00:19.244765+0000","last_fullsized":"2024-08-20T03:00:19.244765+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:17.344161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:40.410220+0000","last_change":"2024-08-20T03:00:16.222331+0000","last_active":"2024-08-20T03:00:40.410220+0000","last_peered":"2024-08-20T03:00:40.410220+0000","last_clean":"2024-08-20T03:00:40.410220+0000","last_became_active":"2024-08-20T03:00:16.222019+0000","last_became_peered":"2024-08-20T03:00:16.222019+0000","last_unstale":"2024-08-20T03:00:40.410220+0000","last_undegraded":"2024-08-20T03:00:40.410220+0000","last_fullsized":"2024-08-20T03:00:40.410220+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:46:55.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249388+0000","last_change":"2024-08-20T03:00:18.239020+0000","last_active":"2024-08-20T03:00:19.249388+0000","last_peered":"2024-08-20T03:00:19.249388+0000","last_clean":"2024-08-20T03:00:19.249388+0000","last_became_active":"2024-08-20T03:00:18.238620+0000","last_became_peered":"2024-08-20T03:00:18.238620+0000","last_unstale":"2024-08-20T03:00:19.249388+0000","last_undegraded":"2024-08-20T03:00:19.249388+0000","last_fullsized":"2024-08-20T03:00:19.249388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:17:12.632357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.390633+0000","last_change":"2024-08-20T03:00:16.214907+0000","last_active":"2024-08-20T03:00:29.390633+0000","last_peered":"2024-08-20T03:00:29.390633+0000","last_clean":"2024-08-20T03:00:29.390633+0000","last_became_active":"2024-08-20T03:00:16.214573+0000","last_became_peered":"2024-08-20T03:00:16.214573+0000","last_unstale":"2024-08-20T03:00:29.390633+0000","last_undegraded":"2024-08-20T03:00:29.390633+0000","last_fullsized":"2024-08-20T03:00:29.390633+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:21:34.084149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249851+0000","last_change":"2024-08-20T03:00:18.245524+0000","last_active":"2024-08-20T03:00:19.249851+0000","last_peered":"2024-08-20T03:00:19.249851+0000","last_clean":"2024-08-20T03:00:19.249851+0000","last_became_active":"2024-08-20T03:00:18.244966+0000","last_became_peered":"2024-08-20T03:00:18.244966+0000","last_unstale":"2024-08-20T03:00:19.249851+0000","last_undegraded":"2024-08-20T03:00:19.249851+0000","last_fullsized":"2024-08-20T03:00:19.249851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:03:21.584182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:28.377989+0000","last_change":"2024-08-20T03:00:16.470353+0000","last_active":"2024-08-20T03:00:28.377989+0000","last_peered":"2024-08-20T03:00:28.377989+0000","last_clean":"2024-08-20T03:00:28.377989+0000","last_became_active":"2024-08-20T03:00:16.469976+0000","last_became_peered":"2024-08-20T03:00:16.469976+0000","last_unstale":"2024-08-20T03:00:28.377989+0000","last_undegraded":"2024-08-20T03:00:28.377989+0000","last_fullsized":"2024-08-20T03:00:28.377989+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:25:03.595784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249905+0000","last_change":"2024-08-20T03:00:18.245244+0000","last_active":"2024-08-20T03:00:19.249905+0000","last_peered":"2024-08-20T03:00:19.249905+0000","last_clean":"2024-08-20T03:00:19.249905+0000","last_became_active":"2024-08-20T03:00:18.244934+0000","last_became_peered":"2024-08-20T03:00:18.244934+0000","last_unstale":"2024-08-20T03:00:19.249905+0000","last_undegraded":"2024-08-20T03:00:19.249905+0000","last_fullsized":"2024-08-20T03:00:19.249905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:33:35.035039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'27","reported_seq":74,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:40.411695+0000","last_change":"2024-08-20T03:00:16.218663+0000","last_active":"2024-08-20T03:00:40.411695+0000","last_peered":"2024-08-20T03:00:40.411695+0000","last_clean":"2024-08-20T03:00:40.411695+0000","last_became_active":"2024-08-20T03:00:16.218266+0000","last_became_peered":"2024-08-20T03:00:16.218266+0000","last_unstale":"2024-08-20T03:00:40.411695+0000","last_undegraded":"2024-08-20T03:00:40.411695+0000","last_fullsized":"2024-08-20T03:00:40.411695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:52.214528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":62,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.246914+0000","last_change":"2024-08-20T03:00:18.242931+0000","last_active":"2024-08-20T03:00:19.246914+0000","last_peered":"2024-08-20T03:00:19.246914+0000","last_clean":"2024-08-20T03:00:19.246914+0000","last_became_active":"2024-08-20T03:00:18.242619+0000","last_became_peered":"2024-08-20T03:00:18.242619+0000","last_unstale":"2024-08-20T03:00:19.246914+0000","last_undegraded":"2024-08-20T03:00:19.246914+0000","last_fullsized":"2024-08-20T03:00:19.246914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:43:02.296990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:30.393276+0000","last_change":"2024-08-20T03:00:16.214688+0000","last_active":"2024-08-20T03:00:30.393276+0000","last_peered":"2024-08-20T03:00:30.393276+0000","last_clean":"2024-08-20T03:00:30.393276+0000","last_became_active":"2024-08-20T03:00:16.214343+0000","last_became_peered":"2024-08-20T03:00:16.214343+0000","last_unstale":"2024-08-20T03:00:30.393276+0000","last_undegraded":"2024-08-20T03:00:30.393276+0000","last_fullsized":"2024-08-20T03:00:30.393276+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:10:56.587164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.231157+0000","last_change":"2024-08-20T03:00:18.244158+0000","last_active":"2024-08-20T03:00:19.231157+0000","last_peered":"2024-08-20T03:00:19.231157+0000","last_clean":"2024-08-20T03:00:19.231157+0000","last_became_active":"2024-08-20T03:00:18.243864+0000","last_became_peered":"2024-08-20T03:00:18.243864+0000","last_unstale":"2024-08-20T03:00:19.231157+0000","last_undegraded":"2024-08-20T03:00:19.231157+0000","last_fullsized":"2024-08-20T03:00:19.231157+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:10:31.373759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.390842+0000","last_change":"2024-08-20T03:00:16.220846+0000","last_active":"2024-08-20T03:00:29.390842+0000","last_peered":"2024-08-20T03:00:29.390842+0000","last_clean":"2024-08-20T03:00:29.390842+0000","last_became_active":"2024-08-20T03:00:16.220519+0000","last_became_peered":"2024-08-20T03:00:16.220519+0000","last_unstale":"2024-08-20T03:00:29.390842+0000","last_undegraded":"2024-08-20T03:00:29.390842+0000","last_fullsized":"2024-08-20T03:00:29.390842+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:43:41.721522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234671+0000","last_change":"2024-08-20T03:00:18.246555+0000","last_active":"2024-08-20T03:00:19.234671+0000","last_peered":"2024-08-20T03:00:19.234671+0000","last_clean":"2024-08-20T03:00:19.234671+0000","last_became_active":"2024-08-20T03:00:18.246182+0000","last_became_peered":"2024-08-20T03:00:18.246182+0000","last_unstale":"2024-08-20T03:00:19.234671+0000","last_undegraded":"2024-08-20T03:00:19.234671+0000","last_fullsized":"2024-08-20T03:00:19.234671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:55.827649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:42.413018+0000","last_change":"2024-08-20T03:00:16.216521+0000","last_active":"2024-08-20T03:00:42.413018+0000","last_peered":"2024-08-20T03:00:42.413018+0000","last_clean":"2024-08-20T03:00:42.413018+0000","last_became_active":"2024-08-20T03:00:16.216187+0000","last_became_peered":"2024-08-20T03:00:16.216187+0000","last_unstale":"2024-08-20T03:00:42.413018+0000","last_undegraded":"2024-08-20T03:00:42.413018+0000","last_fullsized":"2024-08-20T03:00:42.413018+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:41:13.639218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234426+0000","last_change":"2024-08-20T03:00:18.249030+0000","last_active":"2024-08-20T03:00:19.234426+0000","last_peered":"2024-08-20T03:00:19.234426+0000","last_clean":"2024-08-20T03:00:19.234426+0000","last_became_active":"2024-08-20T03:00:18.248701+0000","last_became_peered":"2024-08-20T03:00:18.248701+0000","last_unstale":"2024-08-20T03:00:19.234426+0000","last_undegraded":"2024-08-20T03:00:19.234426+0000","last_fullsized":"2024-08-20T03:00:19.234426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:17:11.829472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.389548+0000","last_change":"2024-08-20T03:00:16.470129+0000","last_active":"2024-08-20T03:00:29.389548+0000","last_peered":"2024-08-20T03:00:29.389548+0000","last_clean":"2024-08-20T03:00:29.389548+0000","last_became_active":"2024-08-20T03:00:16.469697+0000","last_became_peered":"2024-08-20T03:00:16.469697+0000","last_unstale":"2024-08-20T03:00:29.389548+0000","last_undegraded":"2024-08-20T03:00:29.389548+0000","last_fullsized":"2024-08-20T03:00:29.389548+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:42:22.623141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.247876+0000","last_change":"2024-08-20T03:00:18.245090+0000","last_active":"2024-08-20T03:00:19.247876+0000","last_peered":"2024-08-20T03:00:19.247876+0000","last_clean":"2024-08-20T03:00:19.247876+0000","last_became_active":"2024-08-20T03:00:18.244524+0000","last_became_peered":"2024-08-20T03:00:18.244524+0000","last_unstale":"2024-08-20T03:00:19.247876+0000","last_undegraded":"2024-08-20T03:00:19.247876+0000","last_fullsized":"2024-08-20T03:00:19.247876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:14.561183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:28.366159+0000","last_change":"2024-08-20T03:00:16.220042+0000","last_active":"2024-08-20T03:00:28.366159+0000","last_peered":"2024-08-20T03:00:28.366159+0000","last_clean":"2024-08-20T03:00:28.366159+0000","last_became_active":"2024-08-20T03:00:16.219567+0000","last_became_peered":"2024-08-20T03:00:16.219567+0000","last_unstale":"2024-08-20T03:00:28.366159+0000","last_undegraded":"2024-08-20T03:00:28.366159+0000","last_fullsized":"2024-08-20T03:00:28.366159+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:04.834039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.250137+0000","last_change":"2024-08-20T03:00:18.237497+0000","last_active":"2024-08-20T03:00:19.250137+0000","last_peered":"2024-08-20T03:00:19.250137+0000","last_clean":"2024-08-20T03:00:19.250137+0000","last_became_active":"2024-08-20T03:00:18.237170+0000","last_became_peered":"2024-08-20T03:00:18.237170+0000","last_unstale":"2024-08-20T03:00:19.250137+0000","last_undegraded":"2024-08-20T03:00:19.250137+0000","last_fullsized":"2024-08-20T03:00:19.250137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:52.387152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.387576+0000","last_change":"2024-08-20T03:00:16.214861+0000","last_active":"2024-08-20T03:00:29.387576+0000","last_peered":"2024-08-20T03:00:29.387576+0000","last_clean":"2024-08-20T03:00:29.387576+0000","last_became_active":"2024-08-20T03:00:16.214487+0000","last_became_peered":"2024-08-20T03:00:16.214487+0000","last_unstale":"2024-08-20T03:00:29.387576+0000","last_undegraded":"2024-08-20T03:00:29.387576+0000","last_fullsized":"2024-08-20T03:00:29.387576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:50:09.933304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.235076+0000","last_change":"2024-08-20T03:00:18.244685+0000","last_active":"2024-08-20T03:00:19.235076+0000","last_peered":"2024-08-20T03:00:19.235076+0000","last_clean":"2024-08-20T03:00:19.235076+0000","last_became_active":"2024-08-20T03:00:18.244372+0000","last_became_peered":"2024-08-20T03:00:18.244372+0000","last_unstale":"2024-08-20T03:00:19.235076+0000","last_undegraded":"2024-08-20T03:00:19.235076+0000","last_fullsized":"2024-08-20T03:00:19.235076+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:21.821712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.390079+0000","last_change":"2024-08-20T03:00:16.222287+0000","last_active":"2024-08-20T03:00:29.390079+0000","last_peered":"2024-08-20T03:00:29.390079+0000","last_clean":"2024-08-20T03:00:29.390079+0000","last_became_active":"2024-08-20T03:00:16.221989+0000","last_became_peered":"2024-08-20T03:00:16.221989+0000","last_unstale":"2024-08-20T03:00:29.390079+0000","last_undegraded":"2024-08-20T03:00:29.390079+0000","last_fullsized":"2024-08-20T03:00:29.390079+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:32:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234901+0000","last_change":"2024-08-20T03:00:18.250488+0000","last_active":"2024-08-20T03:00:19.234901+0000","last_peered":"2024-08-20T03:00:19.234901+0000","last_clean":"2024-08-20T03:00:19.234901+0000","last_became_active":"2024-08-20T03:00:18.250094+0000","last_became_peered":"2024-08-20T03:00:18.250094+0000","last_unstale":"2024-08-20T03:00:19.234901+0000","last_undegraded":"2024-08-20T03:00:19.234901+0000","last_fullsized":"2024-08-20T03:00:19.234901+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:00:55.755920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:39.409652+0000","last_change":"2024-08-20T03:00:16.470223+0000","last_active":"2024-08-20T03:00:39.409652+0000","last_peered":"2024-08-20T03:00:39.409652+0000","last_clean":"2024-08-20T03:00:39.409652+0000","last_became_active":"2024-08-20T03:00:16.469871+0000","last_became_peered":"2024-08-20T03:00:16.469871+0000","last_unstale":"2024-08-20T03:00:39.409652+0000","last_undegraded":"2024-08-20T03:00:39.409652+0000","last_fullsized":"2024-08-20T03:00:39.409652+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:05:11.779164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234897+0000","last_change":"2024-08-20T03:00:18.242755+0000","last_active":"2024-08-20T03:00:19.234897+0000","last_peered":"2024-08-20T03:00:19.234897+0000","last_clean":"2024-08-20T03:00:19.234897+0000","last_became_active":"2024-08-20T03:00:18.242447+0000","last_became_peered":"2024-08-20T03:00:18.242447+0000","last_unstale":"2024-08-20T03:00:19.234897+0000","last_undegraded":"2024-08-20T03:00:19.234897+0000","last_fullsized":"2024-08-20T03:00:19.234897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:55:40.107933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":92,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.234803+0000","last_change":"2024-08-20T02:59:02.774484+0000","last_active":"2024-08-20T03:00:19.234803+0000","last_peered":"2024-08-20T03:00:19.234803+0000","last_clean":"2024-08-20T03:00:19.234803+0000","last_became_active":"2024-08-20T02:59:02.769208+0000","last_became_peered":"2024-08-20T02:59:02.769208+0000","last_unstale":"2024-08-20T03:00:19.234803+0000","last_undegraded":"2024-08-20T03:00:19.234803+0000","last_fullsized":"2024-08-20T03:00:19.234803+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_clean_scrub_stamp":"2024-08-20T02:55:46.827249+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:36:27.931786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:30.393399+0000","last_change":"2024-08-20T03:00:16.213869+0000","last_active":"2024-08-20T03:00:30.393399+0000","last_peered":"2024-08-20T03:00:30.393399+0000","last_clean":"2024-08-20T03:00:30.393399+0000","last_became_active":"2024-08-20T03:00:16.213478+0000","last_became_peered":"2024-08-20T03:00:16.213478+0000","last_unstale":"2024-08-20T03:00:30.393399+0000","last_undegraded":"2024-08-20T03:00:30.393399+0000","last_fullsized":"2024-08-20T03:00:30.393399+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:27.830497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.246079+0000","last_change":"2024-08-20T03:00:18.242283+0000","last_active":"2024-08-20T03:00:19.246079+0000","last_peered":"2024-08-20T03:00:19.246079+0000","last_clean":"2024-08-20T03:00:19.246079+0000","last_became_active":"2024-08-20T03:00:18.241943+0000","last_became_peered":"2024-08-20T03:00:18.241943+0000","last_unstale":"2024-08-20T03:00:19.246079+0000","last_undegraded":"2024-08-20T03:00:19.246079+0000","last_fullsized":"2024-08-20T03:00:19.246079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:22:25.776964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:27.371512+0000","last_change":"2024-08-20T03:00:16.220087+0000","last_active":"2024-08-20T03:00:27.371512+0000","last_peered":"2024-08-20T03:00:27.371512+0000","last_clean":"2024-08-20T03:00:27.371512+0000","last_became_active":"2024-08-20T03:00:16.219542+0000","last_became_peered":"2024-08-20T03:00:16.219542+0000","last_unstale":"2024-08-20T03:00:27.371512+0000","last_undegraded":"2024-08-20T03:00:27.371512+0000","last_fullsized":"2024-08-20T03:00:27.371512+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:49:20.985529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.231114+0000","last_change":"2024-08-20T03:00:18.230706+0000","last_active":"2024-08-20T03:00:19.231114+0000","last_peered":"2024-08-20T03:00:19.231114+0000","last_clean":"2024-08-20T03:00:19.231114+0000","last_became_active":"2024-08-20T03:00:18.230413+0000","last_became_peered":"2024-08-20T03:00:18.230413+0000","last_unstale":"2024-08-20T03:00:19.231114+0000","last_undegraded":"2024-08-20T03:00:19.231114+0000","last_fullsized":"2024-08-20T03:00:19.231114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:52:18.772867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:40.410057+0000","last_change":"2024-08-20T03:00:16.468073+0000","last_active":"2024-08-20T03:00:40.410057+0000","last_peered":"2024-08-20T03:00:40.410057+0000","last_clean":"2024-08-20T03:00:40.410057+0000","last_became_active":"2024-08-20T03:00:16.467560+0000","last_became_peered":"2024-08-20T03:00:16.467560+0000","last_unstale":"2024-08-20T03:00:40.410057+0000","last_undegraded":"2024-08-20T03:00:40.410057+0000","last_fullsized":"2024-08-20T03:00:40.410057+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:42:24.499777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245883+0000","last_change":"2024-08-20T03:00:18.232537+0000","last_active":"2024-08-20T03:00:19.245883+0000","last_peered":"2024-08-20T03:00:19.245883+0000","last_clean":"2024-08-20T03:00:19.245883+0000","last_became_active":"2024-08-20T03:00:18.232273+0000","last_became_peered":"2024-08-20T03:00:18.232273+0000","last_unstale":"2024-08-20T03:00:19.245883+0000","last_undegraded":"2024-08-20T03:00:19.245883+0000","last_fullsized":"2024-08-20T03:00:19.245883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:36:04.981869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'6","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:41.411351+0000","last_change":"2024-08-20T03:00:16.214807+0000","last_active":"2024-08-20T03:00:41.411351+0000","last_peered":"2024-08-20T03:00:41.411351+0000","last_clean":"2024-08-20T03:00:41.411351+0000","last_became_active":"2024-08-20T03:00:16.214439+0000","last_became_peered":"2024-08-20T03:00:16.214439+0000","last_unstale":"2024-08-20T03:00:41.411351+0000","last_undegraded":"2024-08-20T03:00:41.411351+0000","last_fullsized":"2024-08-20T03:00:41.411351+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:07.519151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.247741+0000","last_change":"2024-08-20T03:00:18.247220+0000","last_active":"2024-08-20T03:00:19.247741+0000","last_peered":"2024-08-20T03:00:19.247741+0000","last_clean":"2024-08-20T03:00:19.247741+0000","last_became_active":"2024-08-20T03:00:18.246896+0000","last_became_peered":"2024-08-20T03:00:18.246896+0000","last_unstale":"2024-08-20T03:00:19.247741+0000","last_undegraded":"2024-08-20T03:00:19.247741+0000","last_fullsized":"2024-08-20T03:00:19.247741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:43:16.690590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.244978+0000","last_change":"2024-08-20T03:00:16.468280+0000","last_active":"2024-08-20T03:00:19.244978+0000","last_peered":"2024-08-20T03:00:19.244978+0000","last_clean":"2024-08-20T03:00:19.244978+0000","last_became_active":"2024-08-20T03:00:16.467930+0000","last_became_peered":"2024-08-20T03:00:16.467930+0000","last_unstale":"2024-08-20T03:00:19.244978+0000","last_undegraded":"2024-08-20T03:00:19.244978+0000","last_fullsized":"2024-08-20T03:00:19.244978+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:30.423563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:20.985562+0000","last_change":"2024-08-20T03:00:18.240322+0000","last_active":"2024-08-20T03:00:20.985562+0000","last_peered":"2024-08-20T03:00:20.985562+0000","last_clean":"2024-08-20T03:00:20.985562+0000","last_became_active":"2024-08-20T03:00:18.239973+0000","last_became_peered":"2024-08-20T03:00:18.239973+0000","last_unstale":"2024-08-20T03:00:20.985562+0000","last_undegraded":"2024-08-20T03:00:20.985562+0000","last_fullsized":"2024-08-20T03:00:20.985562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:41:28.259762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.389954+0000","last_change":"2024-08-20T03:00:16.468307+0000","last_active":"2024-08-20T03:00:29.389954+0000","last_peered":"2024-08-20T03:00:29.389954+0000","last_clean":"2024-08-20T03:00:29.389954+0000","last_became_active":"2024-08-20T03:00:16.467993+0000","last_became_peered":"2024-08-20T03:00:16.467993+0000","last_unstale":"2024-08-20T03:00:29.389954+0000","last_undegraded":"2024-08-20T03:00:29.389954+0000","last_fullsized":"2024-08-20T03:00:29.389954+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:34.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.231452+0000","last_change":"2024-08-20T03:00:18.234009+0000","last_active":"2024-08-20T03:00:19.231452+0000","last_peered":"2024-08-20T03:00:19.231452+0000","last_clean":"2024-08-20T03:00:19.231452+0000","last_became_active":"2024-08-20T03:00:18.233311+0000","last_became_peered":"2024-08-20T03:00:18.233311+0000","last_unstale":"2024-08-20T03:00:19.231452+0000","last_undegraded":"2024-08-20T03:00:19.231452+0000","last_fullsized":"2024-08-20T03:00:19.231452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:53:10.759339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'14","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:40.246072+0000","last_change":"2024-08-20T03:00:16.214932+0000","last_active":"2024-08-20T03:00:40.246072+0000","last_peered":"2024-08-20T03:00:40.246072+0000","last_clean":"2024-08-20T03:00:40.246072+0000","last_became_active":"2024-08-20T03:00:16.214584+0000","last_became_peered":"2024-08-20T03:00:16.214584+0000","last_unstale":"2024-08-20T03:00:40.246072+0000","last_undegraded":"2024-08-20T03:00:40.246072+0000","last_fullsized":"2024-08-20T03:00:40.246072+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:47.270685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.247324+0000","last_change":"2024-08-20T03:00:18.247892+0000","last_active":"2024-08-20T03:00:19.247324+0000","last_peered":"2024-08-20T03:00:19.247324+0000","last_clean":"2024-08-20T03:00:19.247324+0000","last_became_active":"2024-08-20T03:00:18.245903+0000","last_became_peered":"2024-08-20T03:00:18.245903+0000","last_unstale":"2024-08-20T03:00:19.247324+0000","last_undegraded":"2024-08-20T03:00:19.247324+0000","last_fullsized":"2024-08-20T03:00:19.247324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:08:50.909638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.390328+0000","last_change":"2024-08-20T03:00:16.372503+0000","last_active":"2024-08-20T03:00:29.390328+0000","last_peered":"2024-08-20T03:00:29.390328+0000","last_clean":"2024-08-20T03:00:29.390328+0000","last_became_active":"2024-08-20T03:00:16.372218+0000","last_became_peered":"2024-08-20T03:00:16.372218+0000","last_unstale":"2024-08-20T03:00:29.390328+0000","last_undegraded":"2024-08-20T03:00:29.390328+0000","last_fullsized":"2024-08-20T03:00:29.390328+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:13.341900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249685+0000","last_change":"2024-08-20T03:00:18.244738+0000","last_active":"2024-08-20T03:00:19.249685+0000","last_peered":"2024-08-20T03:00:19.249685+0000","last_clean":"2024-08-20T03:00:19.249685+0000","last_became_active":"2024-08-20T03:00:18.244398+0000","last_became_peered":"2024-08-20T03:00:18.244398+0000","last_unstale":"2024-08-20T03:00:19.249685+0000","last_undegraded":"2024-08-20T03:00:19.249685+0000","last_fullsized":"2024-08-20T03:00:19.249685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:45:14.540125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:26.360681+0000","last_change":"2024-08-20T03:00:16.468251+0000","last_active":"2024-08-20T03:00:26.360681+0000","last_peered":"2024-08-20T03:00:26.360681+0000","last_clean":"2024-08-20T03:00:26.360681+0000","last_became_active":"2024-08-20T03:00:16.467912+0000","last_became_peered":"2024-08-20T03:00:16.467912+0000","last_unstale":"2024-08-20T03:00:26.360681+0000","last_undegraded":"2024-08-20T03:00:26.360681+0000","last_fullsized":"2024-08-20T03:00:26.360681+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:43.799747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.245518+0000","last_change":"2024-08-20T03:00:18.234624+0000","last_active":"2024-08-20T03:00:19.245518+0000","last_peered":"2024-08-20T03:00:19.245518+0000","last_clean":"2024-08-20T03:00:19.245518+0000","last_became_active":"2024-08-20T03:00:18.234331+0000","last_became_peered":"2024-08-20T03:00:18.234331+0000","last_unstale":"2024-08-20T03:00:19.245518+0000","last_undegraded":"2024-08-20T03:00:19.245518+0000","last_fullsized":"2024-08-20T03:00:19.245518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:05:09.665645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.389400+0000","last_change":"2024-08-20T03:00:16.216658+0000","last_active":"2024-08-20T03:00:29.389400+0000","last_peered":"2024-08-20T03:00:29.389400+0000","last_clean":"2024-08-20T03:00:29.389400+0000","last_became_active":"2024-08-20T03:00:16.216352+0000","last_became_peered":"2024-08-20T03:00:16.216352+0000","last_unstale":"2024-08-20T03:00:29.389400+0000","last_undegraded":"2024-08-20T03:00:29.389400+0000","last_fullsized":"2024-08-20T03:00:29.389400+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:20.763280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249912+0000","last_change":"2024-08-20T03:00:18.240984+0000","last_active":"2024-08-20T03:00:19.249912+0000","last_peered":"2024-08-20T03:00:19.249912+0000","last_clean":"2024-08-20T03:00:19.249912+0000","last_became_active":"2024-08-20T03:00:18.240640+0000","last_became_peered":"2024-08-20T03:00:18.240640+0000","last_unstale":"2024-08-20T03:00:19.249912+0000","last_undegraded":"2024-08-20T03:00:19.249912+0000","last_fullsized":"2024-08-20T03:00:19.249912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:05.589186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:28.378602+0000","last_change":"2024-08-20T03:00:16.214818+0000","last_active":"2024-08-20T03:00:28.378602+0000","last_peered":"2024-08-20T03:00:28.378602+0000","last_clean":"2024-08-20T03:00:28.378602+0000","last_became_active":"2024-08-20T03:00:16.214493+0000","last_became_peered":"2024-08-20T03:00:16.214493+0000","last_unstale":"2024-08-20T03:00:28.378602+0000","last_undegraded":"2024-08-20T03:00:28.378602+0000","last_fullsized":"2024-08-20T03:00:28.378602+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:56:12.895142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.246612+0000","last_change":"2024-08-20T03:00:18.240350+0000","last_active":"2024-08-20T03:00:19.246612+0000","last_peered":"2024-08-20T03:00:19.246612+0000","last_clean":"2024-08-20T03:00:19.246612+0000","last_became_active":"2024-08-20T03:00:18.239464+0000","last_became_peered":"2024-08-20T03:00:18.239464+0000","last_unstale":"2024-08-20T03:00:19.246612+0000","last_undegraded":"2024-08-20T03:00:19.246612+0000","last_fullsized":"2024-08-20T03:00:19.246612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:48.715130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:26.361292+0000","last_change":"2024-08-20T03:00:16.216144+0000","last_active":"2024-08-20T03:00:26.361292+0000","last_peered":"2024-08-20T03:00:26.361292+0000","last_clean":"2024-08-20T03:00:26.361292+0000","last_became_active":"2024-08-20T03:00:16.215792+0000","last_became_peered":"2024-08-20T03:00:16.215792+0000","last_unstale":"2024-08-20T03:00:26.361292+0000","last_undegraded":"2024-08-20T03:00:26.361292+0000","last_fullsized":"2024-08-20T03:00:26.361292+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:25:01.500878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.235056+0000","last_change":"2024-08-20T03:00:18.245286+0000","last_active":"2024-08-20T03:00:19.235056+0000","last_peered":"2024-08-20T03:00:19.235056+0000","last_clean":"2024-08-20T03:00:19.235056+0000","last_became_active":"2024-08-20T03:00:18.244760+0000","last_became_peered":"2024-08-20T03:00:18.244760+0000","last_unstale":"2024-08-20T03:00:19.235056+0000","last_undegraded":"2024-08-20T03:00:19.235056+0000","last_fullsized":"2024-08-20T03:00:19.235056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:09.415794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.248848+0000","last_change":"2024-08-20T03:00:18.244713+0000","last_active":"2024-08-20T03:00:19.248848+0000","last_peered":"2024-08-20T03:00:19.248848+0000","last_clean":"2024-08-20T03:00:19.248848+0000","last_became_active":"2024-08-20T03:00:18.244368+0000","last_became_peered":"2024-08-20T03:00:18.244368+0000","last_unstale":"2024-08-20T03:00:19.248848+0000","last_undegraded":"2024-08-20T03:00:19.248848+0000","last_fullsized":"2024-08-20T03:00:19.248848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:17:21.407541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:29.389710+0000","last_change":"2024-08-20T03:00:16.220019+0000","last_active":"2024-08-20T03:00:29.389710+0000","last_peered":"2024-08-20T03:00:29.389710+0000","last_clean":"2024-08-20T03:00:29.389710+0000","last_became_active":"2024-08-20T03:00:16.219624+0000","last_became_peered":"2024-08-20T03:00:16.219624+0000","last_unstale":"2024-08-20T03:00:29.389710+0000","last_undegraded":"2024-08-20T03:00:29.389710+0000","last_fullsized":"2024-08-20T03:00:29.389710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:33:24.926695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.250279+0000","last_change":"2024-08-20T03:00:18.242156+0000","last_active":"2024-08-20T03:00:19.250279+0000","last_peered":"2024-08-20T03:00:19.250279+0000","last_clean":"2024-08-20T03:00:19.250279+0000","last_became_active":"2024-08-20T03:00:18.241811+0000","last_became_peered":"2024-08-20T03:00:18.241811+0000","last_unstale":"2024-08-20T03:00:19.250279+0000","last_undegraded":"2024-08-20T03:00:19.250279+0000","last_fullsized":"2024-08-20T03:00:19.250279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:58.230658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:30.385077+0000","last_change":"2024-08-20T03:00:16.215625+0000","last_active":"2024-08-20T03:00:30.385077+0000","last_peered":"2024-08-20T03:00:30.385077+0000","last_clean":"2024-08-20T03:00:30.385077+0000","last_became_active":"2024-08-20T03:00:16.215087+0000","last_became_peered":"2024-08-20T03:00:16.215087+0000","last_unstale":"2024-08-20T03:00:30.385077+0000","last_undegraded":"2024-08-20T03:00:30.385077+0000","last_fullsized":"2024-08-20T03:00:30.385077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:35.433875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.249869+0000","last_change":"2024-08-20T03:00:18.245256+0000","last_active":"2024-08-20T03:00:19.249869+0000","last_peered":"2024-08-20T03:00:19.249869+0000","last_clean":"2024-08-20T03:00:19.249869+0000","last_became_active":"2024-08-20T03:00:18.244339+0000","last_became_peered":"2024-08-20T03:00:18.244339+0000","last_unstale":"2024-08-20T03:00:19.249869+0000","last_undegraded":"2024-08-20T03:00:19.249869+0000","last_fullsized":"2024-08-20T03:00:19.249869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:00.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:20.200015+0000","last_change":"2024-08-20T03:00:16.470004+0000","last_active":"2024-08-20T03:00:20.200015+0000","last_peered":"2024-08-20T03:00:20.200015+0000","last_clean":"2024-08-20T03:00:20.200015+0000","last_became_active":"2024-08-20T03:00:16.469646+0000","last_became_peered":"2024-08-20T03:00:16.469646+0000","last_unstale":"2024-08-20T03:00:20.200015+0000","last_undegraded":"2024-08-20T03:00:20.200015+0000","last_fullsized":"2024-08-20T03:00:20.200015+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:35.244683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:19.247346+0000","last_change":"2024-08-20T03:00:18.238053+0000","last_active":"2024-08-20T03:00:19.247346+0000","last_peered":"2024-08-20T03:00:19.247346+0000","last_clean":"2024-08-20T03:00:19.247346+0000","last_became_active":"2024-08-20T03:00:18.237186+0000","last_became_peered":"2024-08-20T03:00:18.237186+0000","last_unstale":"2024-08-20T03:00:19.247346+0000","last_undegraded":"2024-08-20T03:00:19.247346+0000","last_fullsized":"2024-08-20T03:00:19.247346+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:14.425688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-08-20T03:00:40.409338+0000","last_change":"2024-08-20T03:00:16.214584+0000","last_active":"2024-08-20T03:00:40.409338+0000","last_peered":"2024-08-20T03:00:40.409338+0000","last_clean":"2024-08-20T03:00:40.409338+0000","last_became_active":"2024-08-20T03:00:16.214255+0000","last_became_peered":"2024-08-20T03:00:16.214255+0000","last_unstale":"2024-08-20T03:00:40.409338+0000","last_undegraded":"2024-08-20T03:00:40.409338+0000","last_fullsized":"2024-08-20T03:00:40.409338+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:16:30.983745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57314,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":158,"num_read_kb":140,"num_write":238,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":171942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281428,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743442,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":819200,"data_stored":400341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939670,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":1041480,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103196,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":400271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299424,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":839680,"data_stored":400649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":993558,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":827392,"data_stored":400379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":992845,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":827392,"data_stored":399849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":399813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378686,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":451622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542211,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":451402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705735,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":401327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":52253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":51846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T03:00:46.280 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-20T03:00:46.281 INFO:tasks.cephadm:Setting up client nodes... 2024-08-20T03:00:46.281 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-08-20T03:00:46.534 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:46.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:46 smithi044 ceph-mon[18790]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:00:46.787 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:46 smithi165 ceph-mon[22357]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:00:46.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:46 smithi099 ceph-mon[21984]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:00:47.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.023+0000 7f5d76cdb640 1 -- 172.21.15.44:0/780172540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d701084f0 msgr2=0x7f5d70108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.023+0000 7f5d76cdb640 1 --2- 172.21.15.44:0/780172540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d701084f0 0x7f5d70108930 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009900 tx=0x7f5d5c02a410 comp rx=0 tx=0).stop 2024-08-20T03:00:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.023+0000 7f5d76cdb640 1 -- 172.21.15.44:0/780172540 shutdown_connections 2024-08-20T03:00:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.023+0000 7f5d76cdb640 1 --2- 172.21.15.44:0/780172540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d70108e70 0x7f5d7010f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.023+0000 7f5d76cdb640 1 --2- 172.21.15.44:0/780172540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d701084f0 0x7f5d70108930 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.023+0000 7f5d76cdb640 1 --2- 172.21.15.44:0/780172540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d70102ae0 0x7f5d70102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.023+0000 7f5d76cdb640 1 -- 172.21.15.44:0/780172540 >> 172.21.15.44:0/780172540 conn(0x7f5d700fe570 msgr2=0x7f5d70100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.023+0000 7f5d76cdb640 1 -- 172.21.15.44:0/780172540 shutdown_connections 2024-08-20T03:00:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.023+0000 7f5d76cdb640 1 -- 172.21.15.44:0/780172540 wait complete. 2024-08-20T03:00:47.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d76cdb640 1 Processor -- start 2024-08-20T03:00:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d76cdb640 1 -- start start 2024-08-20T03:00:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d76cdb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d70102ae0 0x7f5d70198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d76cdb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d701084f0 0x7f5d70199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d76cdb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d70108e70 0x7f5d7019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d76cdb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5d70112b30 con 0x7f5d70108e70 2024-08-20T03:00:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d76cdb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d701129b0 con 0x7f5d70102ae0 2024-08-20T03:00:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d76cdb640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5d70112cb0 con 0x7f5d701084f0 2024-08-20T03:00:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d75278640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d70108e70 0x7f5d7019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d75278640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d70108e70 0x7f5d7019e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54646/0 (socket says 172.21.15.44:54646) 2024-08-20T03:00:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d75278640 1 -- 172.21.15.44:0/1500687627 learned_addr learned my addr 172.21.15.44:0/1500687627 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d74a77640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d70102ae0 0x7f5d70198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:47.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d6ffff640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d701084f0 0x7f5d70199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d75278640 1 -- 172.21.15.44:0/1500687627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d701084f0 msgr2=0x7f5d70199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d75278640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d701084f0 0x7f5d70199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d75278640 1 -- 172.21.15.44:0/1500687627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d70102ae0 msgr2=0x7f5d70198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d75278640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d70102ae0 0x7f5d70198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d75278640 1 -- 172.21.15.44:0/1500687627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7019f090 con 0x7f5d70108e70 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.024+0000 7f5d75278640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d70108e70 0x7f5d7019e8c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009fd0 tx=0x7f5d6800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.025+0000 7f5d6dffb640 1 -- 172.21.15.44:0/1500687627 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68017070 con 0x7f5d70108e70 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.025+0000 7f5d6dffb640 1 -- 172.21.15.44:0/1500687627 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d68003e20 con 0x7f5d70108e70 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.025+0000 7f5d76cdb640 1 -- 172.21.15.44:0/1500687627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d7019f380 con 0x7f5d70108e70 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.025+0000 7f5d6dffb640 1 -- 172.21.15.44:0/1500687627 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68010460 con 0x7f5d70108e70 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.025+0000 7f5d76cdb640 1 -- 172.21.15.44:0/1500687627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d701a5a40 con 0x7f5d70108e70 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.026+0000 7f5d76cdb640 1 -- 172.21.15.44:0/1500687627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d70102ec0 con 0x7f5d70108e70 2024-08-20T03:00:47.038 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.027+0000 7f5d6dffb640 1 -- 172.21.15.44:0/1500687627 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f5d68003970 con 0x7f5d70108e70 2024-08-20T03:00:47.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.028+0000 7f5d6dffb640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d600b3cb0 0x7f5d600b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:47.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.028+0000 7f5d74a77640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d600b3cb0 0x7f5d600b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:47.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.028+0000 7f5d6dffb640 1 -- 172.21.15.44:0/1500687627 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f5d680e3b90 con 0x7f5d70108e70 2024-08-20T03:00:47.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.028+0000 7f5d74a77640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d600b3cb0 0x7f5d600b6170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d64005fd0 tx=0x7f5d6400b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:47.039 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.032+0000 7f5d6dffb640 1 -- 172.21.15.44:0/1500687627 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5d680ab0a0 con 0x7f5d70108e70 2024-08-20T03:00:47.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.288+0000 7f5d76cdb640 1 -- 172.21.15.44:0/1500687627 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f5d7002e340 con 0x7f5d70108e70 2024-08-20T03:00:47.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.290+0000 7f5d6dffb640 1 -- 172.21.15.44:0/1500687627 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f5d680aa9a0 con 0x7f5d70108e70 2024-08-20T03:00:47.292 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-08-20T03:00:47.292 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDfBsRm0Sg/ERAAEGe7vByKnrUrd0rQQe5fHw== 2024-08-20T03:00:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.292+0000 7f5d76cdb640 1 -- 172.21.15.44:0/1500687627 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d600b3cb0 msgr2=0x7f5d600b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.292+0000 7f5d76cdb640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d600b3cb0 0x7f5d600b6170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d64005fd0 tx=0x7f5d6400b390 comp rx=0 tx=0).stop 2024-08-20T03:00:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.292+0000 7f5d76cdb640 1 -- 172.21.15.44:0/1500687627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d70108e70 msgr2=0x7f5d7019e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.292+0000 7f5d76cdb640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d70108e70 0x7f5d7019e8c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009fd0 tx=0x7f5d6800b260 comp rx=0 tx=0).stop 2024-08-20T03:00:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.292+0000 7f5d76cdb640 1 -- 172.21.15.44:0/1500687627 shutdown_connections 2024-08-20T03:00:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.292+0000 7f5d76cdb640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d600b3cb0 0x7f5d600b6170 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.292+0000 7f5d76cdb640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d70108e70 0x7f5d7019e8c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.292+0000 7f5d76cdb640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d701084f0 0x7f5d70199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.292+0000 7f5d76cdb640 1 --2- 172.21.15.44:0/1500687627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d70102ae0 0x7f5d70198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:47.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.292+0000 7f5d76cdb640 1 -- 172.21.15.44:0/1500687627 >> 172.21.15.44:0/1500687627 conn(0x7f5d700fe570 msgr2=0x7f5d70100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.292+0000 7f5d76cdb640 1 -- 172.21.15.44:0/1500687627 shutdown_connections 2024-08-20T03:00:47.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:47.292+0000 7f5d76cdb640 1 -- 172.21.15.44:0/1500687627 wait complete. 2024-08-20T03:00:47.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:47 smithi044 ceph-mon[18790]: pgmap v267: 97 pgs: 97 active+clean; 641 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-08-20T03:00:47.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:47 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1500687627' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-20T03:00:47.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:47 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1500687627' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-20T03:00:47.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:47 smithi165 ceph-mon[22357]: pgmap v267: 97 pgs: 97 active+clean; 641 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-08-20T03:00:47.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:47 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1500687627' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-20T03:00:47.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:47 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1500687627' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-20T03:00:47.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:47 smithi099 ceph-mon[21984]: pgmap v267: 97 pgs: 97 active+clean; 641 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-08-20T03:00:47.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:47 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1500687627' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-20T03:00:47.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:47 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1500687627' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-20T03:00:47.971 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T03:00:47.971 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-08-20T03:00:47.971 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-20T03:00:48.009 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph config log 1 --format=json 2024-08-20T03:00:48.302 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:48.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.791+0000 7f942b1aa640 1 -- 172.21.15.44:0/1718967881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94241073a0 msgr2=0x7f94241077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:48.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.791+0000 7f942b1aa640 1 --2- 172.21.15.44:0/1718967881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94241073a0 0x7f94241077e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9418009ca0 tx=0x7f941802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:48.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.792+0000 7f942b1aa640 1 -- 172.21.15.44:0/1718967881 shutdown_connections 2024-08-20T03:00:48.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.792+0000 7f942b1aa640 1 --2- 172.21.15.44:0/1718967881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9424107d20 0x7f942410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:48.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.792+0000 7f942b1aa640 1 --2- 172.21.15.44:0/1718967881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94241073a0 0x7f94241077e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:48.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.792+0000 7f942b1aa640 1 --2- 172.21.15.44:0/1718967881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9424101990 0x7f9424101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:48.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.792+0000 7f942b1aa640 1 -- 172.21.15.44:0/1718967881 >> 172.21.15.44:0/1718967881 conn(0x7f94240fd400 msgr2=0x7f94240ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:48.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.792+0000 7f942b1aa640 1 -- 172.21.15.44:0/1718967881 shutdown_connections 2024-08-20T03:00:48.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.792+0000 7f942b1aa640 1 -- 172.21.15.44:0/1718967881 wait complete. 2024-08-20T03:00:48.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.792+0000 7f942b1aa640 1 Processor -- start 2024-08-20T03:00:48.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.792+0000 7f942b1aa640 1 -- start start 2024-08-20T03:00:48.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f942b1aa640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9424101990 0x7f9424198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:48.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f942b1aa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94241073a0 0x7f9424199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:48.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f942b1aa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d20 0x7f942419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:48.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f942b1aa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9424112b00 con 0x7f9424107d20 2024-08-20T03:00:48.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f942b1aa640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9424112980 con 0x7f94241073a0 2024-08-20T03:00:48.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f942b1aa640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9424112c80 con 0x7f9424101990 2024-08-20T03:00:48.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f9429747640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d20 0x7f942419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:48.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f9429747640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d20 0x7f942419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39248/0 (socket says 172.21.15.44:39248) 2024-08-20T03:00:48.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f9429747640 1 -- 172.21.15.44:0/1507273957 learned_addr learned my addr 172.21.15.44:0/1507273957 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:48.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f9423fff640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94241073a0 0x7f9424199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:48.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f9428f46640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9424101990 0x7f9424198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:48.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f9429747640 1 -- 172.21.15.44:0/1507273957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9424101990 msgr2=0x7f9424198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:48.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f9429747640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9424101990 0x7f9424198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:48.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f9429747640 1 -- 172.21.15.44:0/1507273957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94241073a0 msgr2=0x7f9424199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:48.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f9429747640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94241073a0 0x7f9424199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:48.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f9429747640 1 -- 172.21.15.44:0/1507273957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942419f080 con 0x7f9424107d20 2024-08-20T03:00:48.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f9423fff640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94241073a0 0x7f9424199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:00:48.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.793+0000 7f9429747640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d20 0x7f942419e8b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f941c009fd0 tx=0x7f941c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:48.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.794+0000 7f9421ffb640 1 -- 172.21.15.44:0/1507273957 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c011830 con 0x7f9424107d20 2024-08-20T03:00:48.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.794+0000 7f9421ffb640 1 -- 172.21.15.44:0/1507273957 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f941c016070 con 0x7f9424107d20 2024-08-20T03:00:48.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.794+0000 7f942b1aa640 1 -- 172.21.15.44:0/1507273957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942419f370 con 0x7f9424107d20 2024-08-20T03:00:48.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.794+0000 7f9421ffb640 1 -- 172.21.15.44:0/1507273957 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c010de0 con 0x7f9424107d20 2024-08-20T03:00:48.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.794+0000 7f942b1aa640 1 -- 172.21.15.44:0/1507273957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94241a5ab0 con 0x7f9424107d20 2024-08-20T03:00:48.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.795+0000 7f942b1aa640 1 -- 172.21.15.44:0/1507273957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9424101d70 con 0x7f9424107d20 2024-08-20T03:00:48.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.796+0000 7f9421ffb640 1 -- 172.21.15.44:0/1507273957 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f941c0119d0 con 0x7f9424107d20 2024-08-20T03:00:48.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.796+0000 7f9421ffb640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f940c0b3cb0 0x7f940c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:48.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.797+0000 7f9428f46640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f940c0b3cb0 0x7f940c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:48.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.797+0000 7f9421ffb640 1 -- 172.21.15.44:0/1507273957 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f941c0e3430 con 0x7f9424107d20 2024-08-20T03:00:48.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.797+0000 7f9428f46640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f940c0b3cb0 0x7f940c0b6170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9414009b60 tx=0x7f9414005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:48.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.801+0000 7f9421ffb640 1 -- 172.21.15.44:0/1507273957 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f941c0aaa50 con 0x7f9424107d20 2024-08-20T03:00:48.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.984+0000 7f942b1aa640 1 -- 172.21.15.44:0/1507273957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f9424109570 con 0x7f9424107d20 2024-08-20T03:00:48.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.985+0000 7f9421ffb640 1 -- 172.21.15.44:0/1507273957 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v55) ==== 86+0+133 (secure 0 0 0) 0x7f941c0aa350 con 0x7f9424107d20 2024-08-20T03:00:48.988 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:48.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.987+0000 7f942b1aa640 1 -- 172.21.15.44:0/1507273957 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f940c0b3cb0 msgr2=0x7f940c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:48.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.987+0000 7f942b1aa640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f940c0b3cb0 0x7f940c0b6170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9414009b60 tx=0x7f9414005f90 comp rx=0 tx=0).stop 2024-08-20T03:00:48.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.987+0000 7f942b1aa640 1 -- 172.21.15.44:0/1507273957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d20 msgr2=0x7f942419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:48.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.987+0000 7f942b1aa640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d20 0x7f942419e8b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f941c009fd0 tx=0x7f941c00b8e0 comp rx=0 tx=0).stop 2024-08-20T03:00:48.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.987+0000 7f942b1aa640 1 -- 172.21.15.44:0/1507273957 shutdown_connections 2024-08-20T03:00:48.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.987+0000 7f942b1aa640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f940c0b3cb0 0x7f940c0b6170 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:48.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.987+0000 7f942b1aa640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d20 0x7f942419e8b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:48.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.987+0000 7f942b1aa640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94241073a0 0x7f9424199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:48.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.987+0000 7f942b1aa640 1 --2- 172.21.15.44:0/1507273957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9424101990 0x7f9424198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:48.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.987+0000 7f942b1aa640 1 -- 172.21.15.44:0/1507273957 >> 172.21.15.44:0/1507273957 conn(0x7f94240fd400 msgr2=0x7f94240ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:48.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.987+0000 7f942b1aa640 1 -- 172.21.15.44:0/1507273957 shutdown_connections 2024-08-20T03:00:48.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:48.988+0000 7f942b1aa640 1 -- 172.21.15.44:0/1507273957 wait complete. 2024-08-20T03:00:49.618 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:49 smithi044 ceph-mon[18790]: pgmap v268: 97 pgs: 97 active+clean; 641 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-08-20T03:00:49.619 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:49 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1507273957' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-20T03:00:49.647 INFO:teuthology.orchestra.run.smithi044.stdout:[{"version":55,"timestamp":"2024-08-20T03:00:08.253841+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-08-20T03:00:49.648 INFO:tasks.ceph_manager:config epoch is 55 2024-08-20T03:00:49.648 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-20T03:00:49.648 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-08-20T03:00:49.649 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph mgr dump --format=json 2024-08-20T03:00:49.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:49 smithi165 ceph-mon[22357]: pgmap v268: 97 pgs: 97 active+clean; 641 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-08-20T03:00:49.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:49 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1507273957' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-20T03:00:49.904 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:49.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:49 smithi099 ceph-mon[21984]: pgmap v268: 97 pgs: 97 active+clean; 641 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-08-20T03:00:49.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:49 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1507273957' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-20T03:00:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.405+0000 7fa9dca57640 1 -- 172.21.15.44:0/568814293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9d8108e70 msgr2=0x7fa9d810f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.405+0000 7fa9dca57640 1 --2- 172.21.15.44:0/568814293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9d8108e70 0x7fa9d810f670 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009ca0 tx=0x7fa9d002a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.405+0000 7fa9dca57640 1 -- 172.21.15.44:0/568814293 shutdown_connections 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.405+0000 7fa9dca57640 1 --2- 172.21.15.44:0/568814293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9d8108e70 0x7fa9d810f670 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.405+0000 7fa9dca57640 1 --2- 172.21.15.44:0/568814293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9d81084f0 0x7fa9d8108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.405+0000 7fa9dca57640 1 --2- 172.21.15.44:0/568814293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9d8102ae0 0x7fa9d8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.405+0000 7fa9dca57640 1 -- 172.21.15.44:0/568814293 >> 172.21.15.44:0/568814293 conn(0x7fa9d80fe570 msgr2=0x7fa9d8100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.405+0000 7fa9dca57640 1 -- 172.21.15.44:0/568814293 shutdown_connections 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.405+0000 7fa9dca57640 1 -- 172.21.15.44:0/568814293 wait complete. 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9dca57640 1 Processor -- start 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9dca57640 1 -- start start 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9dca57640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9d8102ae0 0x7fa9d80751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9dca57640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9d81084f0 0x7fa9d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9dca57640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9d8108e70 0x7fa9d8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9dca57640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8112b30 con 0x7fa9d8108e70 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9dca57640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d81129b0 con 0x7fa9d8102ae0 2024-08-20T03:00:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9dca57640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d8112cb0 con 0x7fa9d81084f0 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9d6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9d8108e70 0x7fa9d8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9d6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9d8108e70 0x7fa9d8075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39266/0 (socket says 172.21.15.44:39266) 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9d6d9d640 1 -- 172.21.15.44:0/1235558612 learned_addr learned my addr 172.21.15.44:0/1235558612 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9d659c640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9d8102ae0 0x7fa9d80751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9d5d9b640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9d81084f0 0x7fa9d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9d6d9d640 1 -- 172.21.15.44:0/1235558612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9d81084f0 msgr2=0x7fa9d8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9d6d9d640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9d81084f0 0x7fa9d8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9d6d9d640 1 -- 172.21.15.44:0/1235558612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9d8102ae0 msgr2=0x7fa9d80751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9d6d9d640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9d8102ae0 0x7fa9d80751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9d6d9d640 1 -- 172.21.15.44:0/1235558612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d81a73e0 con 0x7fa9d8108e70 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.406+0000 7fa9d5d9b640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9d81084f0 0x7fa9d8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.407+0000 7fa9d6d9d640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9d8108e70 0x7fa9d8075c50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0005cc0 tx=0x7fa9d002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.407+0000 7fa9c77fe640 1 -- 172.21.15.44:0/1235558612 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0003cf0 con 0x7fa9d8108e70 2024-08-20T03:00:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.407+0000 7fa9c77fe640 1 -- 172.21.15.44:0/1235558612 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d00044f0 con 0x7fa9d8108e70 2024-08-20T03:00:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.407+0000 7fa9dca57640 1 -- 172.21.15.44:0/1235558612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d81a7670 con 0x7fa9d8108e70 2024-08-20T03:00:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.407+0000 7fa9c77fe640 1 -- 172.21.15.44:0/1235558612 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d002fd10 con 0x7fa9d8108e70 2024-08-20T03:00:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.407+0000 7fa9dca57640 1 -- 172.21.15.44:0/1235558612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d81a7c30 con 0x7fa9d8108e70 2024-08-20T03:00:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.408+0000 7fa9dca57640 1 -- 172.21.15.44:0/1235558612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d8102ec0 con 0x7fa9d8108e70 2024-08-20T03:00:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.409+0000 7fa9c77fe640 1 -- 172.21.15.44:0/1235558612 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fa9d0004040 con 0x7fa9d8108e70 2024-08-20T03:00:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.410+0000 7fa9c77fe640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa9c00b3cb0 0x7fa9c00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.410+0000 7fa9d659c640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa9c00b3cb0 0x7fa9c00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.410+0000 7fa9c77fe640 1 -- 172.21.15.44:0/1235558612 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fa9d0105690 con 0x7fa9d8108e70 2024-08-20T03:00:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.410+0000 7fa9d659c640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa9c00b3cb0 0x7fa9c00b6170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0058c0 tx=0x7fa9cc00a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:50.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.413+0000 7fa9c77fe640 1 -- 172.21.15.44:0/1235558612 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa9d00cccb0 con 0x7fa9d8108e70 2024-08-20T03:00:50.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.649+0000 7fa9dca57640 1 -- 172.21.15.44:0/1235558612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa9d802e340 con 0x7fa9d8108e70 2024-08-20T03:00:50.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.654+0000 7fa9c77fe640 1 -- 172.21.15.44:0/1235558612 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+291033 (secure 0 0 0) 0x7fa9d00cc5b0 con 0x7fa9d8108e70 2024-08-20T03:00:50.660 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:50.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.657+0000 7fa9dca57640 1 -- 172.21.15.44:0/1235558612 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa9c00b3cb0 msgr2=0x7fa9c00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:50.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.657+0000 7fa9dca57640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa9c00b3cb0 0x7fa9c00b6170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0058c0 tx=0x7fa9cc00a2d0 comp rx=0 tx=0).stop 2024-08-20T03:00:50.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.657+0000 7fa9dca57640 1 -- 172.21.15.44:0/1235558612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9d8108e70 msgr2=0x7fa9d8075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:50.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.657+0000 7fa9dca57640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9d8108e70 0x7fa9d8075c50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0005cc0 tx=0x7fa9d002f470 comp rx=0 tx=0).stop 2024-08-20T03:00:50.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.657+0000 7fa9dca57640 1 -- 172.21.15.44:0/1235558612 shutdown_connections 2024-08-20T03:00:50.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.657+0000 7fa9dca57640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa9c00b3cb0 0x7fa9c00b6170 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:50.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.657+0000 7fa9dca57640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa9d8108e70 0x7fa9d8075c50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:50.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.657+0000 7fa9dca57640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9d81084f0 0x7fa9d8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:50.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.657+0000 7fa9dca57640 1 --2- 172.21.15.44:0/1235558612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9d8102ae0 0x7fa9d80751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:50.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.657+0000 7fa9dca57640 1 -- 172.21.15.44:0/1235558612 >> 172.21.15.44:0/1235558612 conn(0x7fa9d80fe570 msgr2=0x7fa9d8100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:50.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.657+0000 7fa9dca57640 1 -- 172.21.15.44:0/1235558612 shutdown_connections 2024-08-20T03:00:50.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:50.657+0000 7fa9dca57640 1 -- 172.21.15.44:0/1235558612 wait complete. 2024-08-20T03:00:51.350 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":21,"flags":0,"active_gid":14220,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":587429394},{"type":"v1","addr":"172.21.15.44:6801","nonce":587429394}]},"active_addr":"172.21.15.44:6801/587429394","active_change":"2024-08-20T02:53:58.312813+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24142,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24146,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.44:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":2273808426}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":4115618083}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":2487088235}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":1832265932}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":4234139439}]}]} 2024-08-20T03:00:51.357 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-08-20T03:00:51.357 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-08-20T03:00:51.358 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T03:00:51.613 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:51.733 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:51 smithi044 ceph-mon[18790]: pgmap v269: 97 pgs: 97 active+clean; 641 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-08-20T03:00:51.734 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:51 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1235558612' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-20T03:00:51.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:51 smithi165 ceph-mon[22357]: pgmap v269: 97 pgs: 97 active+clean; 641 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-08-20T03:00:51.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:51 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1235558612' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-20T03:00:51.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:51 smithi099 ceph-mon[21984]: pgmap v269: 97 pgs: 97 active+clean; 641 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-08-20T03:00:51.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:51 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1235558612' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-20T03:00:52.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.108+0000 7f8f4262d640 1 -- 172.21.15.44:0/670678879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3c10ce00 msgr2=0x7f8f3c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:52.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.108+0000 7f8f4262d640 1 --2- 172.21.15.44:0/670678879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3c10ce00 0x7f8f3c10f250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009ca0 tx=0x7f8f3402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:52.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.108+0000 7f8f4262d640 1 -- 172.21.15.44:0/670678879 shutdown_connections 2024-08-20T03:00:52.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.108+0000 7f8f4262d640 1 --2- 172.21.15.44:0/670678879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3c10ce00 0x7f8f3c10f250 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.108+0000 7f8f4262d640 1 --2- 172.21.15.44:0/670678879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f3c104250 0x7f8f3c10c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.108+0000 7f8f4262d640 1 --2- 172.21.15.44:0/670678879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f3c103850 0x7f8f3c103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.108+0000 7f8f4262d640 1 -- 172.21.15.44:0/670678879 >> 172.21.15.44:0/670678879 conn(0x7f8f3c0fd400 msgr2=0x7f8f3c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.108+0000 7f8f4262d640 1 -- 172.21.15.44:0/670678879 shutdown_connections 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.108+0000 7f8f4262d640 1 -- 172.21.15.44:0/670678879 wait complete. 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.109+0000 7f8f4262d640 1 Processor -- start 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.109+0000 7f8f4262d640 1 -- start start 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.109+0000 7f8f4262d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3c103850 0x7f8f3c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.109+0000 7f8f4262d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f3c104250 0x7f8f3c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.109+0000 7f8f4262d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f3c10ce00 0x7f8f3c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.109+0000 7f8f4262d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c112a60 con 0x7f8f3c103850 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.109+0000 7f8f4262d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1128e0 con 0x7f8f3c104250 2024-08-20T03:00:52.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.109+0000 7f8f4262d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c112be0 con 0x7f8f3c10ce00 2024-08-20T03:00:52.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f3bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3c103850 0x7f8f3c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:52.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f3bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3c103850 0x7f8f3c198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39278/0 (socket says 172.21.15.44:39278) 2024-08-20T03:00:52.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f3bfff640 1 -- 172.21.15.44:0/2506030638 learned_addr learned my addr 172.21.15.44:0/2506030638 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:52.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f3b7fe640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f3c104250 0x7f8f3c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:52.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f3bfff640 1 -- 172.21.15.44:0/2506030638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f3c10ce00 msgr2=0x7f8f3c19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:52.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f40bca640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f3c10ce00 0x7f8f3c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:52.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f3bfff640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f3c10ce00 0x7f8f3c19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:52.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f3bfff640 1 -- 172.21.15.44:0/2506030638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f3c104250 msgr2=0x7f8f3c199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:52.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f3bfff640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f3c104250 0x7f8f3c199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:52.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f3bfff640 1 -- 172.21.15.44:0/2506030638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c19ef80 con 0x7f8f3c103850 2024-08-20T03:00:52.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f3b7fe640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f3c104250 0x7f8f3c199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:00:52.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f3bfff640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3c103850 0x7f8f3c198ed0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00d950 tx=0x7f8f2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:52.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f397fa640 1 -- 172.21.15.44:0/2506030638 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c017070 con 0x7f8f3c103850 2024-08-20T03:00:52.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f4262d640 1 -- 172.21.15.44:0/2506030638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c19f270 con 0x7f8f3c103850 2024-08-20T03:00:52.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f397fa640 1 -- 172.21.15.44:0/2506030638 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f2c009c30 con 0x7f8f3c103850 2024-08-20T03:00:52.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f397fa640 1 -- 172.21.15.44:0/2506030638 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00f620 con 0x7f8f3c103850 2024-08-20T03:00:52.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.110+0000 7f8f4262d640 1 -- 172.21.15.44:0/2506030638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c1a5b00 con 0x7f8f3c103850 2024-08-20T03:00:52.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.111+0000 7f8f4262d640 1 -- 172.21.15.44:0/2506030638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3c103c30 con 0x7f8f3c103850 2024-08-20T03:00:52.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.113+0000 7f8f397fa640 1 -- 172.21.15.44:0/2506030638 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f8f2c010460 con 0x7f8f3c103850 2024-08-20T03:00:52.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.113+0000 7f8f397fa640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8f240b3cb0 0x7f8f240b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:52.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.113+0000 7f8f3b7fe640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8f240b3cb0 0x7f8f240b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:52.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.113+0000 7f8f397fa640 1 -- 172.21.15.44:0/2506030638 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f8f2c0e3ab0 con 0x7f8f3c103850 2024-08-20T03:00:52.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.114+0000 7f8f3b7fe640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8f240b3cb0 0x7f8f240b6170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0ffc70 tx=0x7f8f30005d90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:52.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.117+0000 7f8f397fa640 1 -- 172.21.15.44:0/2506030638 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8f2c0aafc0 con 0x7f8f3c103850 2024-08-20T03:00:52.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.302+0000 7f8f4262d640 1 -- 172.21.15.44:0/2506030638 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f3c1a5cd0 con 0x7f8f3c103850 2024-08-20T03:00:52.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.303+0000 7f8f397fa640 1 -- 172.21.15.44:0/2506030638 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22549 (secure 0 0 0) 0x7f8f2c009de0 con 0x7f8f3c103850 2024-08-20T03:00:52.305 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:52.306 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":75,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","created":"2024-08-20T02:52:10.979264+0000","modified":"2024-08-20T03:00:18.935366+0000","last_up_change":"2024-08-20T02:59:26.404917+0000","last_in_change":"2024-08-20T02:59:03.707380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T02:55:46.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T03:00:14.354568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T03:00:16.869335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7708f8c-a799-4b8f-ab1e-78482d73ca1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6803","nonce":858898316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6805","nonce":858898316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6809","nonce":858898316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6807","nonce":858898316}]},"public_addr":"172.21.15.44:6803/858898316","cluster_addr":"172.21.15.44:6805/858898316","heartbeat_back_addr":"172.21.15.44:6809/858898316","heartbeat_front_addr":"172.21.15.44:6807/858898316","state":["exists","up"]},{"osd":1,"uuid":"12f07986-60bf-498f-a012-cd7ea607b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6801","nonce":4252911663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6803","nonce":4252911663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6807","nonce":4252911663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6805","nonce":4252911663}]},"public_addr":"172.21.15.99:6801/4252911663","cluster_addr":"172.21.15.99:6803/4252911663","heartbeat_back_addr":"172.21.15.99:6807/4252911663","heartbeat_front_addr":"172.21.15.99:6805/4252911663","state":["exists","up"]},{"osd":2,"uuid":"0f189000-ba7f-4412-b0d3-75e8265f6685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6801","nonce":1492740465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6803","nonce":1492740465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6807","nonce":1492740465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6805","nonce":1492740465}]},"public_addr":"172.21.15.165:6801/1492740465","cluster_addr":"172.21.15.165:6803/1492740465","heartbeat_back_addr":"172.21.15.165:6807/1492740465","heartbeat_front_addr":"172.21.15.165:6805/1492740465","state":["exists","up"]},{"osd":3,"uuid":"8f9b042c-d6b5-4eee-aa29-ffc41a48c20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6811","nonce":3423983702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6813","nonce":3423983702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6817","nonce":3423983702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6815","nonce":3423983702}]},"public_addr":"172.21.15.44:6811/3423983702","cluster_addr":"172.21.15.44:6813/3423983702","heartbeat_back_addr":"172.21.15.44:6817/3423983702","heartbeat_front_addr":"172.21.15.44:6815/3423983702","state":["exists","up"]},{"osd":4,"uuid":"4edebfdb-af10-49ee-b724-2a63b9219221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6809","nonce":3099617325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6811","nonce":3099617325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6815","nonce":3099617325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6813","nonce":3099617325}]},"public_addr":"172.21.15.99:6809/3099617325","cluster_addr":"172.21.15.99:6811/3099617325","heartbeat_back_addr":"172.21.15.99:6815/3099617325","heartbeat_front_addr":"172.21.15.99:6813/3099617325","state":["exists","up"]},{"osd":5,"uuid":"6d22c21f-cc43-4ef5-a75c-d4fc1a361c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6809","nonce":523245255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6811","nonce":523245255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6815","nonce":523245255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6813","nonce":523245255}]},"public_addr":"172.21.15.165:6809/523245255","cluster_addr":"172.21.15.165:6811/523245255","heartbeat_back_addr":"172.21.15.165:6815/523245255","heartbeat_front_addr":"172.21.15.165:6813/523245255","state":["exists","up"]},{"osd":6,"uuid":"03df0c2f-1b7c-4242-888c-e047322fd8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6819","nonce":1039098986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6821","nonce":1039098986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6825","nonce":1039098986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6823","nonce":1039098986}]},"public_addr":"172.21.15.44:6819/1039098986","cluster_addr":"172.21.15.44:6821/1039098986","heartbeat_back_addr":"172.21.15.44:6825/1039098986","heartbeat_front_addr":"172.21.15.44:6823/1039098986","state":["exists","up"]},{"osd":7,"uuid":"84bbc683-9b13-448e-897c-114458dae33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6817","nonce":1438613564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6819","nonce":1438613564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6823","nonce":1438613564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6821","nonce":1438613564}]},"public_addr":"172.21.15.99:6817/1438613564","cluster_addr":"172.21.15.99:6819/1438613564","heartbeat_back_addr":"172.21.15.99:6823/1438613564","heartbeat_front_addr":"172.21.15.99:6821/1438613564","state":["exists","up"]},{"osd":8,"uuid":"14d7ffd3-db05-4228-a12e-4c69ef65ef8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6817","nonce":632957688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6819","nonce":632957688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6823","nonce":632957688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6821","nonce":632957688}]},"public_addr":"172.21.15.165:6817/632957688","cluster_addr":"172.21.15.165:6819/632957688","heartbeat_back_addr":"172.21.15.165:6823/632957688","heartbeat_front_addr":"172.21.15.165:6821/632957688","state":["exists","up"]},{"osd":9,"uuid":"1ae468a5-3d72-4eb0-a1a6-cd87eae740b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6827","nonce":2426940093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6829","nonce":2426940093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6833","nonce":2426940093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6831","nonce":2426940093}]},"public_addr":"172.21.15.44:6827/2426940093","cluster_addr":"172.21.15.44:6829/2426940093","heartbeat_back_addr":"172.21.15.44:6833/2426940093","heartbeat_front_addr":"172.21.15.44:6831/2426940093","state":["exists","up"]},{"osd":10,"uuid":"fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6825","nonce":3275793038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6827","nonce":3275793038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6831","nonce":3275793038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6829","nonce":3275793038}]},"public_addr":"172.21.15.99:6825/3275793038","cluster_addr":"172.21.15.99:6827/3275793038","heartbeat_back_addr":"172.21.15.99:6831/3275793038","heartbeat_front_addr":"172.21.15.99:6829/3275793038","state":["exists","up"]},{"osd":11,"uuid":"50715410-1fc2-4818-b600-604a4b7cce9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6825","nonce":107688415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6827","nonce":107688415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6831","nonce":107688415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6829","nonce":107688415}]},"public_addr":"172.21.15.165:6825/107688415","cluster_addr":"172.21.15.165:6827/107688415","heartbeat_back_addr":"172.21.15.165:6831/107688415","heartbeat_front_addr":"172.21.15.165:6829/107688415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:54:56.697075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:20.486846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:43.353376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:08.333879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:32.186492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:55.891366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:19.681421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:45.475926+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:10.109795+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:34.444169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:59.147646+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:59:25.403071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2597827045":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/3582612619":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/1916809746":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2919033419":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:6800/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2331034359":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2572185146":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/4176802819":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/3654439062":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6801/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/2376733275":"2024-08-21T02:53:58.312659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T03:00:52.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.305+0000 7f8f4262d640 1 -- 172.21.15.44:0/2506030638 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8f240b3cb0 msgr2=0x7f8f240b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:52.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.305+0000 7f8f4262d640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8f240b3cb0 0x7f8f240b6170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0ffc70 tx=0x7f8f30005d90 comp rx=0 tx=0).stop 2024-08-20T03:00:52.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.305+0000 7f8f4262d640 1 -- 172.21.15.44:0/2506030638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3c103850 msgr2=0x7f8f3c198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:52.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.305+0000 7f8f4262d640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3c103850 0x7f8f3c198ed0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00d950 tx=0x7f8f2c00de10 comp rx=0 tx=0).stop 2024-08-20T03:00:52.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.305+0000 7f8f4262d640 1 -- 172.21.15.44:0/2506030638 shutdown_connections 2024-08-20T03:00:52.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.305+0000 7f8f4262d640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8f240b3cb0 0x7f8f240b6170 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:52.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.305+0000 7f8f4262d640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8f3c10ce00 0x7f8f3c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:52.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.305+0000 7f8f4262d640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f3c104250 0x7f8f3c199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:52.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.305+0000 7f8f4262d640 1 --2- 172.21.15.44:0/2506030638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f3c103850 0x7f8f3c198ed0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:52.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.305+0000 7f8f4262d640 1 -- 172.21.15.44:0/2506030638 >> 172.21.15.44:0/2506030638 conn(0x7f8f3c0fd400 msgr2=0x7f8f3c10f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:52.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.305+0000 7f8f4262d640 1 -- 172.21.15.44:0/2506030638 shutdown_connections 2024-08-20T03:00:52.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:52.305+0000 7f8f4262d640 1 -- 172.21.15.44:0/2506030638 wait complete. 2024-08-20T03:00:52.635 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:52 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2506030638' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:52.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:52 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2506030638' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:52.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:52 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2506030638' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:53.034 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-08-20T03:00:53.034 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T03:00:53.288 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:53.564 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:53 smithi044 ceph-mon[18790]: pgmap v270: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-08-20T03:00:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.812+0000 7fb7718a2640 1 -- 172.21.15.44:0/3453325453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb76c101990 msgr2=0x7fb76c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.812+0000 7fb7718a2640 1 --2- 172.21.15.44:0/3453325453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb76c101990 0x7fb76c101d70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009900 tx=0x7fb75c02a410 comp rx=0 tx=0).stop 2024-08-20T03:00:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.812+0000 7fb7718a2640 1 -- 172.21.15.44:0/3453325453 shutdown_connections 2024-08-20T03:00:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.812+0000 7fb7718a2640 1 --2- 172.21.15.44:0/3453325453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb76c107d20 0x7fb76c10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:53.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.812+0000 7fb7718a2640 1 --2- 172.21.15.44:0/3453325453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c1073a0 0x7fb76c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.812+0000 7fb7718a2640 1 --2- 172.21.15.44:0/3453325453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb76c101990 0x7fb76c101d70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.812+0000 7fb7718a2640 1 -- 172.21.15.44:0/3453325453 >> 172.21.15.44:0/3453325453 conn(0x7fb76c0fd400 msgr2=0x7fb76c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.812+0000 7fb7718a2640 1 -- 172.21.15.44:0/3453325453 shutdown_connections 2024-08-20T03:00:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.812+0000 7fb7718a2640 1 -- 172.21.15.44:0/3453325453 wait complete. 2024-08-20T03:00:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.812+0000 7fb7718a2640 1 Processor -- start 2024-08-20T03:00:53.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb7718a2640 1 -- start start 2024-08-20T03:00:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb7718a2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb76c101990 0x7fb76c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb7718a2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c1073a0 0x7fb76c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb7718a2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb76c107d20 0x7fb76c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb7718a2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c1119e0 con 0x7fb76c101990 2024-08-20T03:00:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb7718a2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c111860 con 0x7fb76c1073a0 2024-08-20T03:00:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb7718a2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c111b60 con 0x7fb76c107d20 2024-08-20T03:00:53.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb76affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb76c101990 0x7fb76c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb76affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb76c101990 0x7fb76c199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39294/0 (socket says 172.21.15.44:39294) 2024-08-20T03:00:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb76affd640 1 -- 172.21.15.44:0/2773599930 learned_addr learned my addr 172.21.15.44:0/2773599930 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb76b7fe640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb76c107d20 0x7fb76c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb76a7fc640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c1073a0 0x7fb76c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb76affd640 1 -- 172.21.15.44:0/2773599930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb76c107d20 msgr2=0x7fb76c19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb76affd640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb76c107d20 0x7fb76c19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:53.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb76affd640 1 -- 172.21.15.44:0/2773599930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c1073a0 msgr2=0x7fb76c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb76affd640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c1073a0 0x7fb76c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb76affd640 1 -- 172.21.15.44:0/2773599930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c1a5210 con 0x7fb76c101990 2024-08-20T03:00:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb76a7fc640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c1073a0 0x7fb76c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:00:53.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.813+0000 7fb76affd640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb76c101990 0x7fb76c199e30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb75c005940 tx=0x7fb75c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:53.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.814+0000 7fb7708a0640 1 -- 172.21.15.44:0/2773599930 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c03f030 con 0x7fb76c101990 2024-08-20T03:00:53.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.814+0000 7fb7718a2640 1 -- 172.21.15.44:0/2773599930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76c1a54a0 con 0x7fb76c101990 2024-08-20T03:00:53.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.814+0000 7fb7708a0640 1 -- 172.21.15.44:0/2773599930 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb75c003cb0 con 0x7fb76c101990 2024-08-20T03:00:53.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.814+0000 7fb7708a0640 1 -- 172.21.15.44:0/2773599930 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c035070 con 0x7fb76c101990 2024-08-20T03:00:53.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.814+0000 7fb7718a2640 1 -- 172.21.15.44:0/2773599930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76c1a5a50 con 0x7fb76c101990 2024-08-20T03:00:53.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.815+0000 7fb7718a2640 1 -- 172.21.15.44:0/2773599930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb76c101d70 con 0x7fb76c101990 2024-08-20T03:00:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.816+0000 7fb7708a0640 1 -- 172.21.15.44:0/2773599930 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb75c042030 con 0x7fb76c101990 2024-08-20T03:00:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.817+0000 7fb7708a0640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb7540b3cb0 0x7fb7540b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.817+0000 7fb76a7fc640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb7540b3cb0 0x7fb7540b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.817+0000 7fb7708a0640 1 -- 172.21.15.44:0/2773599930 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fb75c104630 con 0x7fb76c101990 2024-08-20T03:00:53.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.817+0000 7fb76a7fc640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb7540b3cb0 0x7fb7540b6170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb760006fd0 tx=0x7fb760008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:53.821+0000 7fb7708a0640 1 -- 172.21.15.44:0/2773599930 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb75c0cbb20 con 0x7fb76c101990 2024-08-20T03:00:53.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:53 smithi165 ceph-mon[22357]: pgmap v270: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-08-20T03:00:53.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:53 smithi099 ceph-mon[21984]: pgmap v270: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-08-20T03:00:54.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.008+0000 7fb7718a2640 1 -- 172.21.15.44:0/2773599930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb76c109570 con 0x7fb76c101990 2024-08-20T03:00:54.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.009+0000 7fb7708a0640 1 -- 172.21.15.44:0/2773599930 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22549 (secure 0 0 0) 0x7fb75c0cb420 con 0x7fb76c101990 2024-08-20T03:00:54.011 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:00:54.012 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":75,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","created":"2024-08-20T02:52:10.979264+0000","modified":"2024-08-20T03:00:18.935366+0000","last_up_change":"2024-08-20T02:59:26.404917+0000","last_in_change":"2024-08-20T02:59:03.707380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T02:55:46.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T03:00:14.354568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T03:00:16.869335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7708f8c-a799-4b8f-ab1e-78482d73ca1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6803","nonce":858898316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6805","nonce":858898316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6809","nonce":858898316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6807","nonce":858898316}]},"public_addr":"172.21.15.44:6803/858898316","cluster_addr":"172.21.15.44:6805/858898316","heartbeat_back_addr":"172.21.15.44:6809/858898316","heartbeat_front_addr":"172.21.15.44:6807/858898316","state":["exists","up"]},{"osd":1,"uuid":"12f07986-60bf-498f-a012-cd7ea607b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6801","nonce":4252911663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6803","nonce":4252911663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6807","nonce":4252911663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6805","nonce":4252911663}]},"public_addr":"172.21.15.99:6801/4252911663","cluster_addr":"172.21.15.99:6803/4252911663","heartbeat_back_addr":"172.21.15.99:6807/4252911663","heartbeat_front_addr":"172.21.15.99:6805/4252911663","state":["exists","up"]},{"osd":2,"uuid":"0f189000-ba7f-4412-b0d3-75e8265f6685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6801","nonce":1492740465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6803","nonce":1492740465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6807","nonce":1492740465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6805","nonce":1492740465}]},"public_addr":"172.21.15.165:6801/1492740465","cluster_addr":"172.21.15.165:6803/1492740465","heartbeat_back_addr":"172.21.15.165:6807/1492740465","heartbeat_front_addr":"172.21.15.165:6805/1492740465","state":["exists","up"]},{"osd":3,"uuid":"8f9b042c-d6b5-4eee-aa29-ffc41a48c20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6811","nonce":3423983702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6813","nonce":3423983702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6817","nonce":3423983702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6815","nonce":3423983702}]},"public_addr":"172.21.15.44:6811/3423983702","cluster_addr":"172.21.15.44:6813/3423983702","heartbeat_back_addr":"172.21.15.44:6817/3423983702","heartbeat_front_addr":"172.21.15.44:6815/3423983702","state":["exists","up"]},{"osd":4,"uuid":"4edebfdb-af10-49ee-b724-2a63b9219221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6809","nonce":3099617325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6811","nonce":3099617325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6815","nonce":3099617325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6813","nonce":3099617325}]},"public_addr":"172.21.15.99:6809/3099617325","cluster_addr":"172.21.15.99:6811/3099617325","heartbeat_back_addr":"172.21.15.99:6815/3099617325","heartbeat_front_addr":"172.21.15.99:6813/3099617325","state":["exists","up"]},{"osd":5,"uuid":"6d22c21f-cc43-4ef5-a75c-d4fc1a361c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6809","nonce":523245255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6811","nonce":523245255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6815","nonce":523245255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6813","nonce":523245255}]},"public_addr":"172.21.15.165:6809/523245255","cluster_addr":"172.21.15.165:6811/523245255","heartbeat_back_addr":"172.21.15.165:6815/523245255","heartbeat_front_addr":"172.21.15.165:6813/523245255","state":["exists","up"]},{"osd":6,"uuid":"03df0c2f-1b7c-4242-888c-e047322fd8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6819","nonce":1039098986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6821","nonce":1039098986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6825","nonce":1039098986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6823","nonce":1039098986}]},"public_addr":"172.21.15.44:6819/1039098986","cluster_addr":"172.21.15.44:6821/1039098986","heartbeat_back_addr":"172.21.15.44:6825/1039098986","heartbeat_front_addr":"172.21.15.44:6823/1039098986","state":["exists","up"]},{"osd":7,"uuid":"84bbc683-9b13-448e-897c-114458dae33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6817","nonce":1438613564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6819","nonce":1438613564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6823","nonce":1438613564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6821","nonce":1438613564}]},"public_addr":"172.21.15.99:6817/1438613564","cluster_addr":"172.21.15.99:6819/1438613564","heartbeat_back_addr":"172.21.15.99:6823/1438613564","heartbeat_front_addr":"172.21.15.99:6821/1438613564","state":["exists","up"]},{"osd":8,"uuid":"14d7ffd3-db05-4228-a12e-4c69ef65ef8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6817","nonce":632957688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6819","nonce":632957688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6823","nonce":632957688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6821","nonce":632957688}]},"public_addr":"172.21.15.165:6817/632957688","cluster_addr":"172.21.15.165:6819/632957688","heartbeat_back_addr":"172.21.15.165:6823/632957688","heartbeat_front_addr":"172.21.15.165:6821/632957688","state":["exists","up"]},{"osd":9,"uuid":"1ae468a5-3d72-4eb0-a1a6-cd87eae740b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6827","nonce":2426940093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6829","nonce":2426940093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6833","nonce":2426940093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6831","nonce":2426940093}]},"public_addr":"172.21.15.44:6827/2426940093","cluster_addr":"172.21.15.44:6829/2426940093","heartbeat_back_addr":"172.21.15.44:6833/2426940093","heartbeat_front_addr":"172.21.15.44:6831/2426940093","state":["exists","up"]},{"osd":10,"uuid":"fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6825","nonce":3275793038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6827","nonce":3275793038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6831","nonce":3275793038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6829","nonce":3275793038}]},"public_addr":"172.21.15.99:6825/3275793038","cluster_addr":"172.21.15.99:6827/3275793038","heartbeat_back_addr":"172.21.15.99:6831/3275793038","heartbeat_front_addr":"172.21.15.99:6829/3275793038","state":["exists","up"]},{"osd":11,"uuid":"50715410-1fc2-4818-b600-604a4b7cce9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6825","nonce":107688415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6827","nonce":107688415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6831","nonce":107688415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6829","nonce":107688415}]},"public_addr":"172.21.15.165:6825/107688415","cluster_addr":"172.21.15.165:6827/107688415","heartbeat_back_addr":"172.21.15.165:6831/107688415","heartbeat_front_addr":"172.21.15.165:6829/107688415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:54:56.697075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:20.486846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:43.353376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:08.333879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:32.186492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:55.891366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:19.681421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:45.475926+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:10.109795+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:34.444169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:59.147646+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:59:25.403071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2597827045":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/3582612619":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/1916809746":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2919033419":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:6800/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2331034359":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2572185146":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/4176802819":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/3654439062":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6801/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/2376733275":"2024-08-21T02:53:58.312659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T03:00:54.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.011+0000 7fb7718a2640 1 -- 172.21.15.44:0/2773599930 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb7540b3cb0 msgr2=0x7fb7540b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:54.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.011+0000 7fb7718a2640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb7540b3cb0 0x7fb7540b6170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb760006fd0 tx=0x7fb760008040 comp rx=0 tx=0).stop 2024-08-20T03:00:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.011+0000 7fb7718a2640 1 -- 172.21.15.44:0/2773599930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb76c101990 msgr2=0x7fb76c199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.011+0000 7fb7718a2640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb76c101990 0x7fb76c199e30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb75c005940 tx=0x7fb75c0042f0 comp rx=0 tx=0).stop 2024-08-20T03:00:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.011+0000 7fb7718a2640 1 -- 172.21.15.44:0/2773599930 shutdown_connections 2024-08-20T03:00:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.011+0000 7fb7718a2640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb7540b3cb0 0x7fb7540b6170 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.011+0000 7fb7718a2640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb76c107d20 0x7fb76c19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.011+0000 7fb7718a2640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb76c1073a0 0x7fb76c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.011+0000 7fb7718a2640 1 --2- 172.21.15.44:0/2773599930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb76c101990 0x7fb76c199e30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.011+0000 7fb7718a2640 1 -- 172.21.15.44:0/2773599930 >> 172.21.15.44:0/2773599930 conn(0x7fb76c0fd400 msgr2=0x7fb76c0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.011+0000 7fb7718a2640 1 -- 172.21.15.44:0/2773599930 shutdown_connections 2024-08-20T03:00:54.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:54.011+0000 7fb7718a2640 1 -- 172.21.15.44:0/2773599930 wait complete. 2024-08-20T03:00:54.723 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:54 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2773599930' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:54.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-08-20T03:00:54.752 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-08-20T03:00:54.752 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-08-20T03:00:54.753 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-08-20T03:00:54.753 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-08-20T03:00:54.753 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-08-20T03:00:54.754 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-08-20T03:00:54.754 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-08-20T03:00:54.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-08-20T03:00:54.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-08-20T03:00:54.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-08-20T03:00:54.756 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-08-20T03:00:54.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:54 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2773599930' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:54.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:54 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2773599930' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:00:55.469 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:55.537 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:55.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:55 smithi044 ceph-mon[18790]: pgmap v271: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-08-20T03:00:55.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:55 smithi165 ceph-mon[22357]: pgmap v271: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-08-20T03:00:55.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:55 smithi099 ceph-mon[21984]: pgmap v271: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-08-20T03:00:56.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.090+0000 7fd47c997640 1 -- 172.21.15.44:0/3068554315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4781084e0 msgr2=0x7fd478108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:56.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.090+0000 7fd47c997640 1 --2- 172.21.15.44:0/3068554315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4781084e0 0x7fd478108920 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009900 tx=0x7fd46c02a410 comp rx=0 tx=0).stop 2024-08-20T03:00:56.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.091+0000 7fd47c997640 1 -- 172.21.15.44:0/3068554315 shutdown_connections 2024-08-20T03:00:56.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.091+0000 7fd47c997640 1 --2- 172.21.15.44:0/3068554315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd478108e60 0x7fd47810f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:56.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.091+0000 7fd47c997640 1 --2- 172.21.15.44:0/3068554315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4781084e0 0x7fd478108920 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:56.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.091+0000 7fd47c997640 1 --2- 172.21.15.44:0/3068554315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd478102ad0 0x7fd478102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:56.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.091+0000 7fd47c997640 1 -- 172.21.15.44:0/3068554315 >> 172.21.15.44:0/3068554315 conn(0x7fd4780fe540 msgr2=0x7fd478100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:56.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.091+0000 7fd47c997640 1 -- 172.21.15.44:0/3068554315 shutdown_connections 2024-08-20T03:00:56.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.091+0000 7fd47c997640 1 -- 172.21.15.44:0/3068554315 wait complete. 2024-08-20T03:00:56.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.091+0000 7fd47c997640 1 Processor -- start 2024-08-20T03:00:56.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd47c997640 1 -- start start 2024-08-20T03:00:56.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd47c997640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd478102ad0 0x7fd478198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd47c997640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4781084e0 0x7fd4781994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd47c997640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd478108e60 0x7fd47819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd47c997640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd478112b00 con 0x7fd478108e60 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd47c997640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd478112980 con 0x7fd4781084e0 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd47c997640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd478112c80 con 0x7fd478102ad0 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd476d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd478108e60 0x7fd47819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd476d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd478108e60 0x7fd47819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39326/0 (socket says 172.21.15.44:39326) 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd476d9d640 1 -- 172.21.15.44:0/2406625542 learned_addr learned my addr 172.21.15.44:0/2406625542 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd47659c640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd478102ad0 0x7fd478198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd475d9b640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4781084e0 0x7fd4781994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd476d9d640 1 -- 172.21.15.44:0/2406625542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd478102ad0 msgr2=0x7fd478198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd476d9d640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd478102ad0 0x7fd478198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:56.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd476d9d640 1 -- 172.21.15.44:0/2406625542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4781084e0 msgr2=0x7fd4781994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd476d9d640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4781084e0 0x7fd4781994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd476d9d640 1 -- 172.21.15.44:0/2406625542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47819f040 con 0x7fd478108e60 2024-08-20T03:00:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd475d9b640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4781084e0 0x7fd4781994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:00:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.092+0000 7fd476d9d640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd478108e60 0x7fd47819e870 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd47000eaa0 tx=0x7fd47000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.093+0000 7fd4677fe640 1 -- 172.21.15.44:0/2406625542 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47000cdb0 con 0x7fd478108e60 2024-08-20T03:00:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.093+0000 7fd4677fe640 1 -- 172.21.15.44:0/2406625542 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd470005020 con 0x7fd478108e60 2024-08-20T03:00:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.093+0000 7fd47c997640 1 -- 172.21.15.44:0/2406625542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47819f330 con 0x7fd478108e60 2024-08-20T03:00:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.093+0000 7fd4677fe640 1 -- 172.21.15.44:0/2406625542 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4700116d0 con 0x7fd478108e60 2024-08-20T03:00:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.093+0000 7fd47c997640 1 -- 172.21.15.44:0/2406625542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4781a59f0 con 0x7fd478108e60 2024-08-20T03:00:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.093+0000 7fd47c997640 1 -- 172.21.15.44:0/2406625542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd478063610 con 0x7fd478108e60 2024-08-20T03:00:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.095+0000 7fd4677fe640 1 -- 172.21.15.44:0/2406625542 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fd4700051d0 con 0x7fd478108e60 2024-08-20T03:00:56.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.095+0000 7fd4677fe640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4600b3d30 0x7fd4600b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.096+0000 7fd47659c640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4600b3d30 0x7fd4600b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.096+0000 7fd4677fe640 1 -- 172.21.15.44:0/2406625542 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fd4700e4370 con 0x7fd478108e60 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.096+0000 7fd4677fe640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316] conn(0x7fd4600bd800 0x7fd4600bfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.096+0000 7fd4677fe640 1 -- 172.21.15.44:0/2406625542 --> [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd470014ec0 con 0x7fd4600bd800 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.096+0000 7fd4677fe640 1 -- 172.21.15.44:0/2406625542 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fd4700e47c0 con 0x7fd478108e60 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.096+0000 7fd475d9b640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316] conn(0x7fd4600bd800 0x7fd4600bfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.096+0000 7fd47659c640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4600b3d30 0x7fd4600b61f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd4680058c0 tx=0x7fd46800a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.096+0000 7fd475d9b640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316] conn(0x7fd4600bd800 0x7fd4600bfc20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.097+0000 7fd4677fe640 1 -- 172.21.15.44:0/2406625542 <== osd.0 v2:172.21.15.44:6802/858898316 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd470014ec0 con 0x7fd4600bd800 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.111+0000 7fd47c997640 1 -- 172.21.15.44:0/2406625542 --> [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd478063a90 con 0x7fd4600bd800 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.111+0000 7fd4677fe640 1 -- 172.21.15.44:0/2406625542 <== osd.0 v2:172.21.15.44:6802/858898316 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd478063a90 con 0x7fd4600bd800 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.111+0000 7fd47c997640 1 -- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316] conn(0x7fd4600bd800 msgr2=0x7fd4600bfc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.111+0000 7fd47c997640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316] conn(0x7fd4600bd800 0x7fd4600bfc20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:56.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.111+0000 7fd47c997640 1 -- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4600b3d30 msgr2=0x7fd4600b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:56.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.111+0000 7fd47c997640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4600b3d30 0x7fd4600b61f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd4680058c0 tx=0x7fd46800a2d0 comp rx=0 tx=0).stop 2024-08-20T03:00:56.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.111+0000 7fd47c997640 1 -- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd478108e60 msgr2=0x7fd47819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:56.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.112+0000 7fd47c997640 1 --2- 172.21.15.44:0/2406625542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd478108e60 0x7fd47819e870 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd47000eaa0 tx=0x7fd47000ef60 comp rx=0 tx=0).stop 2024-08-20T03:00:56.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.112+0000 7fd476d9d640 1 -- 172.21.15.44:0/2406625542 reap_dead start 2024-08-20T03:00:56.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.112+0000 7fd47c997640 1 -- 172.21.15.44:0/2406625542 shutdown_connections 2024-08-20T03:00:56.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.112+0000 7fd47c997640 1 -- 172.21.15.44:0/2406625542 >> 172.21.15.44:0/2406625542 conn(0x7fd4780fe540 msgr2=0x7fd478100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:56.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.112+0000 7fd47c997640 1 -- 172.21.15.44:0/2406625542 shutdown_connections 2024-08-20T03:00:56.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:56.112+0000 7fd47c997640 1 -- 172.21.15.44:0/2406625542 wait complete. 2024-08-20T03:00:56.827 INFO:teuthology.orchestra.run.smithi044.stdout:38654705738 2024-08-20T03:00:56.828 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-08-20T03:00:57.132 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:57.132 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:57.132 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:57.133 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:57.133 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:57.133 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:57.136 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:00:57.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.377+0000 7fbb78278640 1 -- 172.21.15.44:0/456276054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb70104270 msgr2=0x7fbb7010c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:57.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.377+0000 7fbb78278640 1 --2- 172.21.15.44:0/456276054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb70104270 0x7fbb7010c850 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009900 tx=0x7fbb6002a410 comp rx=0 tx=0).stop 2024-08-20T03:00:57.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.377+0000 7fbb78278640 1 -- 172.21.15.44:0/456276054 shutdown_connections 2024-08-20T03:00:57.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.377+0000 7fbb78278640 1 --2- 172.21.15.44:0/456276054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7010ce20 0x7fbb7010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:57.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.377+0000 7fbb78278640 1 --2- 172.21.15.44:0/456276054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb70104270 0x7fbb7010c850 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:57.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.377+0000 7fbb78278640 1 --2- 172.21.15.44:0/456276054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb70103870 0x7fbb70103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:57.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.377+0000 7fbb78278640 1 -- 172.21.15.44:0/456276054 >> 172.21.15.44:0/456276054 conn(0x7fbb700fd400 msgr2=0x7fbb700ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:57.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.377+0000 7fbb78278640 1 -- 172.21.15.44:0/456276054 shutdown_connections 2024-08-20T03:00:57.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.377+0000 7fbb78278640 1 -- 172.21.15.44:0/456276054 wait complete. 2024-08-20T03:00:57.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb78278640 1 Processor -- start 2024-08-20T03:00:57.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb78278640 1 -- start start 2024-08-20T03:00:57.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb78278640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb70103870 0x7fbb701016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb78278640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb70104270 0x7fbb70101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb78278640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7010ce20 0x7fbb70102170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb78278640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbb70112a60 con 0x7fbb70103870 2024-08-20T03:00:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb78278640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb701128e0 con 0x7fbb70104270 2024-08-20T03:00:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb78278640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbb70112be0 con 0x7fbb7010ce20 2024-08-20T03:00:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb76014640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb70103870 0x7fbb701016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb76014640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb70103870 0x7fbb701016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39356/0 (socket says 172.21.15.44:39356) 2024-08-20T03:00:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb76014640 1 -- 172.21.15.44:0/556018836 learned_addr learned my addr 172.21.15.44:0/556018836 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb75813640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb70104270 0x7fbb70101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb76014640 1 -- 172.21.15.44:0/556018836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7010ce20 msgr2=0x7fbb70102170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb76815640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7010ce20 0x7fbb70102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:57.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb76014640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb7010ce20 0x7fbb70102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb76014640 1 -- 172.21.15.44:0/556018836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb70104270 msgr2=0x7fbb70101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.378+0000 7fbb76014640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb70104270 0x7fbb70101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.379+0000 7fbb76014640 1 -- 172.21.15.44:0/556018836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb701a7370 con 0x7fbb70103870 2024-08-20T03:00:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.379+0000 7fbb76014640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb70103870 0x7fbb701016f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00d950 tx=0x7fbb6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.379+0000 7fbb677fe640 1 -- 172.21.15.44:0/556018836 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c017070 con 0x7fbb70103870 2024-08-20T03:00:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.379+0000 7fbb78278640 1 -- 172.21.15.44:0/556018836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb701a7660 con 0x7fbb70103870 2024-08-20T03:00:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.379+0000 7fbb677fe640 1 -- 172.21.15.44:0/556018836 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb6c009c30 con 0x7fbb70103870 2024-08-20T03:00:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.379+0000 7fbb677fe640 1 -- 172.21.15.44:0/556018836 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c00f620 con 0x7fbb70103870 2024-08-20T03:00:57.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.379+0000 7fbb78278640 1 -- 172.21.15.44:0/556018836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb701a7b70 con 0x7fbb70103870 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.380+0000 7fbb78278640 1 -- 172.21.15.44:0/556018836 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb70063610 con 0x7fbb70103870 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.381+0000 7fbb677fe640 1 -- 172.21.15.44:0/556018836 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fbb6c010460 con 0x7fbb70103870 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.382+0000 7fbb677fe640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbb5c0b3cb0 0x7fbb5c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.382+0000 7fbb75813640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbb5c0b3cb0 0x7fbb5c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.382+0000 7fbb677fe640 1 -- 172.21.15.44:0/556018836 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fbb6c0e3e40 con 0x7fbb70103870 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.382+0000 7fbb677fe640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325] conn(0x7fbb5c0bd780 0x7fbb5c0bfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.382+0000 7fbb677fe640 1 -- 172.21.15.44:0/556018836 --> [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb6c00d130 con 0x7fbb5c0bd780 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.382+0000 7fbb677fe640 1 -- 172.21.15.44:0/556018836 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fbb6c0e4290 con 0x7fbb70103870 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.382+0000 7fbb75813640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbb5c0b3cb0 0x7fbb5c0b6170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbb60031440 tx=0x7fbb60005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.382+0000 7fbb76815640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325] conn(0x7fbb5c0bd780 0x7fbb5c0bfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.383+0000 7fbb76815640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325] conn(0x7fbb5c0bd780 0x7fbb5c0bfba0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.384+0000 7fbb677fe640 1 -- 172.21.15.44:0/556018836 <== osd.4 v2:172.21.15.99:6808/3099617325 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb6c00d130 con 0x7fbb5c0bd780 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.397+0000 7fbb78278640 1 -- 172.21.15.44:0/556018836 --> [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb70063a10 con 0x7fbb5c0bd780 2024-08-20T03:00:57.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.398+0000 7fbb677fe640 1 -- 172.21.15.44:0/556018836 <== osd.4 v2:172.21.15.99:6808/3099617325 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbb70063a10 con 0x7fbb5c0bd780 2024-08-20T03:00:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.398+0000 7fbb78278640 1 -- 172.21.15.44:0/556018836 >> [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325] conn(0x7fbb5c0bd780 msgr2=0x7fbb5c0bfba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.398+0000 7fbb78278640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325] conn(0x7fbb5c0bd780 0x7fbb5c0bfba0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.398+0000 7fbb78278640 1 -- 172.21.15.44:0/556018836 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbb5c0b3cb0 msgr2=0x7fbb5c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.398+0000 7fbb78278640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbb5c0b3cb0 0x7fbb5c0b6170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbb60031440 tx=0x7fbb60005b50 comp rx=0 tx=0).stop 2024-08-20T03:00:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.398+0000 7fbb78278640 1 -- 172.21.15.44:0/556018836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb70103870 msgr2=0x7fbb701016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.398+0000 7fbb78278640 1 --2- 172.21.15.44:0/556018836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb70103870 0x7fbb701016f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00d950 tx=0x7fbb6c00de10 comp rx=0 tx=0).stop 2024-08-20T03:00:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.398+0000 7fbb76815640 1 -- 172.21.15.44:0/556018836 reap_dead start 2024-08-20T03:00:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.398+0000 7fbb78278640 1 -- 172.21.15.44:0/556018836 shutdown_connections 2024-08-20T03:00:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.398+0000 7fbb78278640 1 -- 172.21.15.44:0/556018836 >> 172.21.15.44:0/556018836 conn(0x7fbb700fd400 msgr2=0x7fbb700fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.398+0000 7fbb78278640 1 -- 172.21.15.44:0/556018836 shutdown_connections 2024-08-20T03:00:57.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:57.398+0000 7fbb78278640 1 -- 172.21.15.44:0/556018836 wait complete. 2024-08-20T03:00:57.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:57 smithi044 ceph-mon[18790]: pgmap v272: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1023 B/s wr, 11 op/s 2024-08-20T03:00:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:57 smithi165 ceph-mon[22357]: pgmap v272: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1023 B/s wr, 11 op/s 2024-08-20T03:00:57.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:57 smithi099 ceph-mon[21984]: pgmap v272: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1023 B/s wr, 11 op/s 2024-08-20T03:00:58.437 INFO:teuthology.orchestra.run.smithi044.stdout:133143986230 2024-08-20T03:00:58.438 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-08-20T03:00:58.658 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]} : dispatch 2024-08-20T03:00:58.658 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-08-20T03:00:58.658 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]} : dispatch 2024-08-20T03:00:58.658 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]} : dispatch 2024-08-20T03:00:58.658 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-08-20T03:00:58.658 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-08-20T03:00:58.658 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:58 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-08-20T03:00:58.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]} : dispatch 2024-08-20T03:00:58.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-08-20T03:00:58.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]} : dispatch 2024-08-20T03:00:58.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]} : dispatch 2024-08-20T03:00:58.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-08-20T03:00:58.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-08-20T03:00:58.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:58 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-08-20T03:00:58.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]} : dispatch 2024-08-20T03:00:58.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-08-20T03:00:58.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]} : dispatch 2024-08-20T03:00:58.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]} : dispatch 2024-08-20T03:00:58.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-08-20T03:00:58.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-08-20T03:00:58.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:58 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-08-20T03:00:59.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.051+0000 7f4ab4e89640 1 -- 172.21.15.44:0/4163937206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ab01039e0 msgr2=0x7f4ab0103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:59.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.051+0000 7f4ab4e89640 1 --2- 172.21.15.44:0/4163937206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ab01039e0 0x7f4ab0103dc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009ca0 tx=0x7f4aa002a8c0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.051+0000 7f4ab4e89640 1 -- 172.21.15.44:0/4163937206 shutdown_connections 2024-08-20T03:00:59.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.051+0000 7f4ab4e89640 1 --2- 172.21.15.44:0/4163937206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab010ce40 0x7f4ab010f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.051+0000 7f4ab4e89640 1 --2- 172.21.15.44:0/4163937206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab0104300 0x7f4ab010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.051+0000 7f4ab4e89640 1 --2- 172.21.15.44:0/4163937206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ab01039e0 0x7f4ab0103dc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.051+0000 7f4ab4e89640 1 -- 172.21.15.44:0/4163937206 >> 172.21.15.44:0/4163937206 conn(0x7f4ab00fd440 msgr2=0x7f4ab00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.051+0000 7f4ab4e89640 1 -- 172.21.15.44:0/4163937206 shutdown_connections 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.051+0000 7f4ab4e89640 1 -- 172.21.15.44:0/4163937206 wait complete. 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4ab4e89640 1 Processor -- start 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4ab4e89640 1 -- start start 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4ab4e89640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ab01039e0 0x7f4ab0199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4ab4e89640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab0104300 0x7f4ab01995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4ab4e89640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab010ce40 0x7f4ab019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4ab4e89640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0112af0 con 0x7f4ab01039e0 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4ab4e89640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0112970 con 0x7f4ab0104300 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4ab4e89640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0112c70 con 0x7f4ab010ce40 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4aae59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ab01039e0 0x7f4ab0199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:59.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4aae59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ab01039e0 0x7f4ab0199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49480/0 (socket says 172.21.15.44:49480) 2024-08-20T03:00:59.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4aae59c640 1 -- 172.21.15.44:0/1204828384 learned_addr learned my addr 172.21.15.44:0/1204828384 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:59.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4aadd9b640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab0104300 0x7f4ab01995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:59.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4aae59c640 1 -- 172.21.15.44:0/1204828384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab010ce40 msgr2=0x7f4ab019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:59.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4aae59c640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ab010ce40 0x7f4ab019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4aae59c640 1 -- 172.21.15.44:0/1204828384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab0104300 msgr2=0x7f4ab01995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:59.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4aae59c640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ab0104300 0x7f4ab01995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.052+0000 7f4aae59c640 1 -- 172.21.15.44:0/1204828384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab019f130 con 0x7f4ab01039e0 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.053+0000 7f4aae59c640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ab01039e0 0x7f4ab0199080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0006010 tx=0x7f4aa00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.053+0000 7f4aa77fe640 1 -- 172.21.15.44:0/1204828384 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0003bf0 con 0x7f4ab01039e0 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.053+0000 7f4aa77fe640 1 -- 172.21.15.44:0/1204828384 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa0003d90 con 0x7f4ab01039e0 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.053+0000 7f4ab4e89640 1 -- 172.21.15.44:0/1204828384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab019f3c0 con 0x7f4ab01039e0 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.053+0000 7f4aa77fe640 1 -- 172.21.15.44:0/1204828384 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa002fc20 con 0x7f4ab01039e0 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.053+0000 7f4ab4e89640 1 -- 172.21.15.44:0/1204828384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab01a5ae0 con 0x7f4ab01039e0 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.054+0000 7f4ab4e89640 1 -- 172.21.15.44:0/1204828384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ab0063610 con 0x7f4ab01039e0 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.055+0000 7f4aa77fe640 1 -- 172.21.15.44:0/1204828384 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f4aa00046a0 con 0x7f4ab01039e0 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.056+0000 7f4aa77fe640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4a9c0b3d30 0x7f4a9c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.056+0000 7f4aadd9b640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4a9c0b3d30 0x7f4a9c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.056+0000 7f4aa77fe640 1 -- 172.21.15.44:0/1204828384 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8405+0+0 (secure 0 0 0) 0x7f4aa01054d0 con 0x7f4ab01039e0 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.056+0000 7f4aa77fe640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255] conn(0x7f4a9c0bdae0 0x7f4a9c0bff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:59.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.056+0000 7f4aa77fe640 1 -- 172.21.15.44:0/1204828384 --> [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4aa003fec0 con 0x7f4a9c0bdae0 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.056+0000 7f4aa77fe640 1 -- 172.21.15.44:0/1204828384 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f4aa0105920 con 0x7f4ab01039e0 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.056+0000 7f4aadd9b640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4a9c0b3d30 0x7f4a9c0b61f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4ab010f500 tx=0x7f4a9800b2e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.057+0000 7f4aaed9d640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255] conn(0x7f4a9c0bdae0 0x7f4a9c0bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.057+0000 7f4aaed9d640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255] conn(0x7f4a9c0bdae0 0x7f4a9c0bff80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.058+0000 7f4aa77fe640 1 -- 172.21.15.44:0/1204828384 <== osd.5 v2:172.21.15.165:6808/523245255 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4aa003fec0 con 0x7f4a9c0bdae0 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.072+0000 7f4ab4e89640 1 -- 172.21.15.44:0/1204828384 --> [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ab0063a10 con 0x7f4a9c0bdae0 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.072+0000 7f4aa77fe640 1 -- 172.21.15.44:0/1204828384 <== osd.5 v2:172.21.15.165:6808/523245255 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4ab0063a10 con 0x7f4a9c0bdae0 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.072+0000 7f4ab4e89640 1 -- 172.21.15.44:0/1204828384 >> [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255] conn(0x7f4a9c0bdae0 msgr2=0x7f4a9c0bff80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.072+0000 7f4ab4e89640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255] conn(0x7f4a9c0bdae0 0x7f4a9c0bff80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.072+0000 7f4ab4e89640 1 -- 172.21.15.44:0/1204828384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4a9c0b3d30 msgr2=0x7f4a9c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.072+0000 7f4ab4e89640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4a9c0b3d30 0x7f4a9c0b61f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4ab010f500 tx=0x7f4a9800b2e0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.072+0000 7f4ab4e89640 1 -- 172.21.15.44:0/1204828384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ab01039e0 msgr2=0x7f4ab0199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:59.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.072+0000 7f4ab4e89640 1 --2- 172.21.15.44:0/1204828384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ab01039e0 0x7f4ab0199080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0006010 tx=0x7f4aa00319f0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.072+0000 7f4aaed9d640 1 -- 172.21.15.44:0/1204828384 reap_dead start 2024-08-20T03:00:59.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.072+0000 7f4ab4e89640 1 -- 172.21.15.44:0/1204828384 shutdown_connections 2024-08-20T03:00:59.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.072+0000 7f4ab4e89640 1 -- 172.21.15.44:0/1204828384 >> 172.21.15.44:0/1204828384 conn(0x7f4ab00fd440 msgr2=0x7f4ab00fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:59.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.072+0000 7f4ab4e89640 1 -- 172.21.15.44:0/1204828384 shutdown_connections 2024-08-20T03:00:59.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.073+0000 7f4ab4e89640 1 -- 172.21.15.44:0/1204828384 wait complete. 2024-08-20T03:00:59.731 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:59 smithi044 ceph-mon[18790]: pgmap v273: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-08-20T03:00:59.732 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-08-20T03:00:59.732 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-08-20T03:00:59.732 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-08-20T03:00:59.732 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-08-20T03:00:59.732 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-08-20T03:00:59.732 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-08-20T03:00:59.732 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:59 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-08-20T03:00:59.732 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:00:59 smithi044 ceph-mon[18790]: osdmap e76: 12 total, 12 up, 12 in 2024-08-20T03:00:59.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.811+0000 7f4ea25c6640 1 -- 172.21.15.44:0/3142928361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c1019b0 msgr2=0x7f4e9c101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:59.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.811+0000 7f4ea25c6640 1 --2- 172.21.15.44:0/3142928361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c1019b0 0x7f4e9c101d90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009900 tx=0x7f4e8c02a410 comp rx=0 tx=0).stop 2024-08-20T03:00:59.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.811+0000 7f4ea25c6640 1 -- 172.21.15.44:0/3142928361 shutdown_connections 2024-08-20T03:00:59.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.811+0000 7f4ea25c6640 1 --2- 172.21.15.44:0/3142928361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c107d40 0x7f4e9c10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.811+0000 7f4ea25c6640 1 --2- 172.21.15.44:0/3142928361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e9c1073c0 0x7f4e9c107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.811+0000 7f4ea25c6640 1 --2- 172.21.15.44:0/3142928361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c1019b0 0x7f4e9c101d90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.811+0000 7f4ea25c6640 1 -- 172.21.15.44:0/3142928361 >> 172.21.15.44:0/3142928361 conn(0x7f4e9c0fd420 msgr2=0x7f4e9c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.811+0000 7f4ea25c6640 1 -- 172.21.15.44:0/3142928361 shutdown_connections 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.811+0000 7f4ea25c6640 1 -- 172.21.15.44:0/3142928361 wait complete. 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea25c6640 1 Processor -- start 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea25c6640 1 -- start start 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea25c6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e9c1019b0 0x7f4e9c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea25c6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c1073c0 0x7f4e9c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea25c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c107d40 0x7f4e9c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea25c6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c111a00 con 0x7f4e9c107d40 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea25c6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c111880 con 0x7f4e9c1073c0 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea25c6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c111b80 con 0x7f4e9c1019b0 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea0b63640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c107d40 0x7f4e9c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:59.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea0b63640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c107d40 0x7f4e9c198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49490/0 (socket says 172.21.15.44:49490) 2024-08-20T03:00:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea0b63640 1 -- 172.21.15.44:0/726068652 learned_addr learned my addr 172.21.15.44:0/726068652 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:00:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4e9bfff640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e9c1019b0 0x7f4e9c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4e9b7fe640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c1073c0 0x7f4e9c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea0b63640 1 -- 172.21.15.44:0/726068652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e9c1019b0 msgr2=0x7f4e9c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea0b63640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e9c1019b0 0x7f4e9c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea0b63640 1 -- 172.21.15.44:0/726068652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c1073c0 msgr2=0x7f4e9c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea0b63640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c1073c0 0x7f4e9c198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4ea0b63640 1 -- 172.21.15.44:0/726068652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c1a73a0 con 0x7f4e9c107d40 2024-08-20T03:00:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.812+0000 7f4e9b7fe640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e9c1073c0 0x7f4e9c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:00:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.813+0000 7f4ea0b63640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c107d40 0x7f4e9c198640 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400eaa0 tx=0x7f4e9400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.813+0000 7f4e997fa640 1 -- 172.21.15.44:0/726068652 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400cdb0 con 0x7f4e9c107d40 2024-08-20T03:00:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.813+0000 7f4e997fa640 1 -- 172.21.15.44:0/726068652 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e94005020 con 0x7f4e9c107d40 2024-08-20T03:00:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.813+0000 7f4ea25c6640 1 -- 172.21.15.44:0/726068652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9c1a7690 con 0x7f4e9c107d40 2024-08-20T03:00:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.813+0000 7f4e997fa640 1 -- 172.21.15.44:0/726068652 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e940116d0 con 0x7f4e9c107d40 2024-08-20T03:00:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.813+0000 7f4ea25c6640 1 -- 172.21.15.44:0/726068652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9c1a7b80 con 0x7f4e9c107d40 2024-08-20T03:00:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.814+0000 7f4ea25c6640 1 -- 172.21.15.44:0/726068652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e9c063610 con 0x7f4e9c107d40 2024-08-20T03:00:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.816+0000 7f4e997fa640 1 -- 172.21.15.44:0/726068652 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f4e940051d0 con 0x7f4e9c107d40 2024-08-20T03:00:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.816+0000 7f4e997fa640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4e840b3cb0 0x7f4e840b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.816+0000 7f4e9bfff640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4e840b3cb0 0x7f4e840b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:59.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.816+0000 7f4e997fa640 1 -- 172.21.15.44:0/726068652 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4e940e44c0 con 0x7f4e9c107d40 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.816+0000 7f4e997fa640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688] conn(0x7f4e840bda60 0x7f4e840bff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.816+0000 7f4e997fa640 1 -- 172.21.15.44:0/726068652 --> [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e94014ec0 con 0x7f4e840bda60 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.816+0000 7f4e997fa640 1 -- 172.21.15.44:0/726068652 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f4e940e4910 con 0x7f4e9c107d40 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.817+0000 7f4e9bfff640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4e840b3cb0 0x7f4e840b6170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c031440 tx=0x7f4e8c005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.817+0000 7f4e9b7fe640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688] conn(0x7f4e840bda60 0x7f4e840bff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.817+0000 7f4e9b7fe640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688] conn(0x7f4e840bda60 0x7f4e840bff00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.818+0000 7f4e997fa640 1 -- 172.21.15.44:0/726068652 <== osd.8 v2:172.21.15.165:6816/632957688 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e94014ec0 con 0x7f4e840bda60 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.832+0000 7f4ea25c6640 1 -- 172.21.15.44:0/726068652 --> [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4e9c063a90 con 0x7f4e840bda60 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.832+0000 7f4e997fa640 1 -- 172.21.15.44:0/726068652 <== osd.8 v2:172.21.15.165:6816/632957688 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4e9c063a90 con 0x7f4e840bda60 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.833+0000 7f4ea25c6640 1 -- 172.21.15.44:0/726068652 >> [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688] conn(0x7f4e840bda60 msgr2=0x7f4e840bff00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.833+0000 7f4ea25c6640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688] conn(0x7f4e840bda60 0x7f4e840bff00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.833+0000 7f4ea25c6640 1 -- 172.21.15.44:0/726068652 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4e840b3cb0 msgr2=0x7f4e840b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:59.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.833+0000 7f4ea25c6640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4e840b3cb0 0x7f4e840b6170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c031440 tx=0x7f4e8c005b50 comp rx=0 tx=0).stop 2024-08-20T03:00:59.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.833+0000 7f4ea25c6640 1 -- 172.21.15.44:0/726068652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c107d40 msgr2=0x7f4e9c198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:00:59.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.833+0000 7f4ea25c6640 1 --2- 172.21.15.44:0/726068652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4e9c107d40 0x7f4e9c198640 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400eaa0 tx=0x7f4e9400ef60 comp rx=0 tx=0).stop 2024-08-20T03:00:59.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.833+0000 7f4ea0b63640 1 -- 172.21.15.44:0/726068652 reap_dead start 2024-08-20T03:00:59.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.833+0000 7f4ea25c6640 1 -- 172.21.15.44:0/726068652 shutdown_connections 2024-08-20T03:00:59.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.833+0000 7f4ea25c6640 1 -- 172.21.15.44:0/726068652 >> 172.21.15.44:0/726068652 conn(0x7f4e9c0fd420 msgr2=0x7f4e9c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:00:59.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.833+0000 7f4ea25c6640 1 -- 172.21.15.44:0/726068652 shutdown_connections 2024-08-20T03:00:59.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:00:59.833+0000 7f4ea25c6640 1 -- 172.21.15.44:0/726068652 wait complete. 2024-08-20T03:00:59.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:59 smithi165 ceph-mon[22357]: pgmap v273: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-08-20T03:00:59.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-08-20T03:00:59.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-08-20T03:00:59.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-08-20T03:00:59.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-08-20T03:00:59.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-08-20T03:00:59.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-08-20T03:00:59.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:59 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-08-20T03:00:59.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:00:59 smithi165 ceph-mon[22357]: osdmap e76: 12 total, 12 up, 12 in 2024-08-20T03:00:59.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:59 smithi099 ceph-mon[21984]: pgmap v273: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-08-20T03:00:59.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-08-20T03:00:59.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-08-20T03:00:59.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-08-20T03:00:59.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-08-20T03:00:59.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-08-20T03:00:59.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-08-20T03:00:59.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:59 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-08-20T03:00:59.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:00:59 smithi099 ceph-mon[21984]: osdmap e76: 12 total, 12 up, 12 in 2024-08-20T03:01:00.584 INFO:teuthology.orchestra.run.smithi044.stdout:223338299428 2024-08-20T03:01:00.585 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-08-20T03:01:00.777 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:00 smithi044 ceph-mon[18790]: osdmap e77: 12 total, 12 up, 12 in 2024-08-20T03:01:00.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:00 smithi165 ceph-mon[22357]: osdmap e77: 12 total, 12 up, 12 in 2024-08-20T03:01:00.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:00 smithi099 ceph-mon[21984]: osdmap e77: 12 total, 12 up, 12 in 2024-08-20T03:01:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.277+0000 7fe668dae640 1 -- 172.21.15.44:0/3262276192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe66410ce20 msgr2=0x7fe66410f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.277+0000 7fe668dae640 1 --2- 172.21.15.44:0/3262276192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe66410ce20 0x7fe66410f290 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009ca0 tx=0x7fe65c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.277+0000 7fe668dae640 1 -- 172.21.15.44:0/3262276192 shutdown_connections 2024-08-20T03:01:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.277+0000 7fe668dae640 1 --2- 172.21.15.44:0/3262276192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe66410ce20 0x7fe66410f290 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.277+0000 7fe668dae640 1 --2- 172.21.15.44:0/3262276192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe664104270 0x7fe66410c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.277+0000 7fe668dae640 1 --2- 172.21.15.44:0/3262276192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe664103870 0x7fe664103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.277+0000 7fe668dae640 1 -- 172.21.15.44:0/3262276192 >> 172.21.15.44:0/3262276192 conn(0x7fe6640fd440 msgr2=0x7fe6640ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.277+0000 7fe668dae640 1 -- 172.21.15.44:0/3262276192 shutdown_connections 2024-08-20T03:01:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.277+0000 7fe668dae640 1 -- 172.21.15.44:0/3262276192 wait complete. 2024-08-20T03:01:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe668dae640 1 Processor -- start 2024-08-20T03:01:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe668dae640 1 -- start start 2024-08-20T03:01:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe668dae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe664103870 0x7fe664198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe668dae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe664104270 0x7fe6641994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe668dae640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe66410ce20 0x7fe66419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe668dae640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe664112aa0 con 0x7fe664103870 2024-08-20T03:01:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe668dae640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe664112920 con 0x7fe664104270 2024-08-20T03:01:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe668dae640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe664112c20 con 0x7fe66410ce20 2024-08-20T03:01:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe66259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe664103870 0x7fe664198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe66259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe664103870 0x7fe664198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49508/0 (socket says 172.21.15.44:49508) 2024-08-20T03:01:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe66259c640 1 -- 172.21.15.44:0/3937814007 learned_addr learned my addr 172.21.15.44:0/3937814007 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe662d9d640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe66410ce20 0x7fe66419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe65bfff640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe664104270 0x7fe6641994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe66259c640 1 -- 172.21.15.44:0/3937814007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe66410ce20 msgr2=0x7fe66419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe66259c640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe66410ce20 0x7fe66419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe66259c640 1 -- 172.21.15.44:0/3937814007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe664104270 msgr2=0x7fe6641994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe66259c640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe664104270 0x7fe6641994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.278+0000 7fe66259c640 1 -- 172.21.15.44:0/3937814007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66419f020 con 0x7fe664103870 2024-08-20T03:01:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.279+0000 7fe65bfff640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe664104270 0x7fe6641994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.279+0000 7fe66259c640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe664103870 0x7fe664198f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe65400e9f0 tx=0x7fe65400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.279+0000 7fe65b7fe640 1 -- 172.21.15.44:0/3937814007 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65400cd80 con 0x7fe664103870 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.279+0000 7fe65b7fe640 1 -- 172.21.15.44:0/3937814007 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6540054d0 con 0x7fe664103870 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.279+0000 7fe668dae640 1 -- 172.21.15.44:0/3937814007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66419f310 con 0x7fe664103870 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.279+0000 7fe65b7fe640 1 -- 172.21.15.44:0/3937814007 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654010690 con 0x7fe664103870 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.279+0000 7fe668dae640 1 -- 172.21.15.44:0/3937814007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6641a59d0 con 0x7fe664103870 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.280+0000 7fe668dae640 1 -- 172.21.15.44:0/3937814007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe664063610 con 0x7fe664103870 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.281+0000 7fe65b7fe640 1 -- 172.21.15.44:0/3937814007 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fe654004e30 con 0x7fe664103870 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.282+0000 7fe65b7fe640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe6500b3cb0 0x7fe6500b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.282+0000 7fe65b7fe640 1 -- 172.21.15.44:0/3937814007 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe6540e4820 con 0x7fe664103870 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.282+0000 7fe65bfff640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe6500b3cb0 0x7fe6500b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.282+0000 7fe65b7fe640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564] conn(0x7fe6500bda60 0x7fe6500bff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.282+0000 7fe65b7fe640 1 -- 172.21.15.44:0/3937814007 --> [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe654015e80 con 0x7fe6500bda60 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.282+0000 7fe65b7fe640 1 -- 172.21.15.44:0/3937814007 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe6540e4c70 con 0x7fe664103870 2024-08-20T03:01:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.282+0000 7fe65bfff640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe6500b3cb0 0x7fe6500b6170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6640ffcf0 tx=0x7fe64c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.283+0000 7fe662d9d640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564] conn(0x7fe6500bda60 0x7fe6500bff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.283+0000 7fe662d9d640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564] conn(0x7fe6500bda60 0x7fe6500bff00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.284+0000 7fe65b7fe640 1 -- 172.21.15.44:0/3937814007 <== osd.7 v2:172.21.15.99:6816/1438613564 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe654015e80 con 0x7fe6500bda60 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.298+0000 7fe668dae640 1 -- 172.21.15.44:0/3937814007 --> [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe664063a10 con 0x7fe6500bda60 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.298+0000 7fe65b7fe640 1 -- 172.21.15.44:0/3937814007 <== osd.7 v2:172.21.15.99:6816/1438613564 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe664063a10 con 0x7fe6500bda60 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.299+0000 7fe668dae640 1 -- 172.21.15.44:0/3937814007 >> [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564] conn(0x7fe6500bda60 msgr2=0x7fe6500bff00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.299+0000 7fe668dae640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564] conn(0x7fe6500bda60 0x7fe6500bff00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.299+0000 7fe668dae640 1 -- 172.21.15.44:0/3937814007 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe6500b3cb0 msgr2=0x7fe6500b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.299+0000 7fe668dae640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe6500b3cb0 0x7fe6500b6170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6640ffcf0 tx=0x7fe64c005f90 comp rx=0 tx=0).stop 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.299+0000 7fe668dae640 1 -- 172.21.15.44:0/3937814007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe664103870 msgr2=0x7fe664198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.299+0000 7fe668dae640 1 --2- 172.21.15.44:0/3937814007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe664103870 0x7fe664198f70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe65400e9f0 tx=0x7fe65400edf0 comp rx=0 tx=0).stop 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.299+0000 7fe662d9d640 1 -- 172.21.15.44:0/3937814007 reap_dead start 2024-08-20T03:01:01.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.299+0000 7fe668dae640 1 -- 172.21.15.44:0/3937814007 shutdown_connections 2024-08-20T03:01:01.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.299+0000 7fe668dae640 1 -- 172.21.15.44:0/3937814007 >> 172.21.15.44:0/3937814007 conn(0x7fe6640fd440 msgr2=0x7fe6640fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:01.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.299+0000 7fe668dae640 1 -- 172.21.15.44:0/3937814007 shutdown_connections 2024-08-20T03:01:01.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.299+0000 7fe668dae640 1 -- 172.21.15.44:0/3937814007 wait complete. 2024-08-20T03:01:01.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:01 smithi044 ceph-mon[18790]: pgmap v276: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s 2024-08-20T03:01:01.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:01 smithi165 ceph-mon[22357]: pgmap v276: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s 2024-08-20T03:01:01.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:01 smithi099 ceph-mon[21984]: pgmap v276: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 8 op/s 2024-08-20T03:01:02.044 INFO:teuthology.orchestra.run.smithi044.stdout:201863462954 2024-08-20T03:01:02.044 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-08-20T03:01:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.194+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/2418267185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc1073a0 msgr2=0x7fe3bc1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.194+0000 7fe3c1c5a640 1 --2- 172.21.15.44:0/2418267185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc1073a0 0x7fe3bc1077e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009900 tx=0x7fe3b002a410 comp rx=0 tx=0).stop 2024-08-20T03:01:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.194+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/2418267185 shutdown_connections 2024-08-20T03:01:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.194+0000 7fe3c1c5a640 1 --2- 172.21.15.44:0/2418267185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3bc107d20 0x7fe3bc10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.194+0000 7fe3c1c5a640 1 --2- 172.21.15.44:0/2418267185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc1073a0 0x7fe3bc1077e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.194+0000 7fe3c1c5a640 1 --2- 172.21.15.44:0/2418267185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3bc101990 0x7fe3bc101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.194+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/2418267185 >> 172.21.15.44:0/2418267185 conn(0x7fe3bc0fd400 msgr2=0x7fe3bc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:02.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.194+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/2418267185 shutdown_connections 2024-08-20T03:01:02.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.194+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/2418267185 wait complete. 2024-08-20T03:01:02.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.195+0000 7fe3c1c5a640 1 Processor -- start 2024-08-20T03:01:02.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.195+0000 7fe3c1c5a640 1 -- start start 2024-08-20T03:01:02.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.195+0000 7fe3c1c5a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc101990 0x7fe3bc197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:02.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.195+0000 7fe3c1c5a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3bc1073a0 0x7fe3bc198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:02.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.195+0000 7fe3c1c5a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3bc107d20 0x7fe3bc19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:02.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.195+0000 7fe3c1c5a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc1119e0 con 0x7fe3bc101990 2024-08-20T03:01:02.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.195+0000 7fe3c1c5a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc111860 con 0x7fe3bc1073a0 2024-08-20T03:01:02.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.195+0000 7fe3c1c5a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe3bc111b60 con 0x7fe3bc107d20 2024-08-20T03:01:02.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3bb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc101990 0x7fe3bc197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:02.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3bb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc101990 0x7fe3bc197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49532/0 (socket says 172.21.15.44:49532) 2024-08-20T03:01:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3bb7fe640 1 -- 172.21.15.44:0/3189967965 learned_addr learned my addr 172.21.15.44:0/3189967965 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3bbfff640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3bc107d20 0x7fe3bc19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3baffd640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3bc1073a0 0x7fe3bc198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3bb7fe640 1 -- 172.21.15.44:0/3189967965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3bc107d20 msgr2=0x7fe3bc19d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3bb7fe640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3bc107d20 0x7fe3bc19d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3bb7fe640 1 -- 172.21.15.44:0/3189967965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3bc1073a0 msgr2=0x7fe3bc198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3bb7fe640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3bc1073a0 0x7fe3bc198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3bb7fe640 1 -- 172.21.15.44:0/3189967965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3bc19df00 con 0x7fe3bc101990 2024-08-20T03:01:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3bb7fe640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc101990 0x7fe3bc197e50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac007910 tx=0x7fe3ac007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3b8ff9640 1 -- 172.21.15.44:0/3189967965 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac019070 con 0x7fe3bc101990 2024-08-20T03:01:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/3189967965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3bc19e1f0 con 0x7fe3bc101990 2024-08-20T03:01:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3b8ff9640 1 -- 172.21.15.44:0/3189967965 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3ac005020 con 0x7fe3bc101990 2024-08-20T03:01:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3b8ff9640 1 -- 172.21.15.44:0/3189967965 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac004a00 con 0x7fe3bc101990 2024-08-20T03:01:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.196+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/3189967965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3bc06a980 con 0x7fe3bc101990 2024-08-20T03:01:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.197+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/3189967965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe3bc06af30 con 0x7fe3bc101990 2024-08-20T03:01:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.199+0000 7fe3b8ff9640 1 -- 172.21.15.44:0/3189967965 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fe3ac01c070 con 0x7fe3bc101990 2024-08-20T03:01:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.200+0000 7fe3b8ff9640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe3a40b3cb0 0x7fe3a40b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.200+0000 7fe3baffd640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe3a40b3cb0 0x7fe3a40b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.200+0000 7fe3b8ff9640 1 -- 172.21.15.44:0/3189967965 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe3ac0e5290 con 0x7fe3bc101990 2024-08-20T03:01:02.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.200+0000 7fe3b8ff9640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702] conn(0x7fe3a40bda60 0x7fe3a40bff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.200+0000 7fe3b8ff9640 1 -- 172.21.15.44:0/3189967965 --> [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3a40c05d0 con 0x7fe3a40bda60 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.200+0000 7fe3b8ff9640 1 -- 172.21.15.44:0/3189967965 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe3ac0e56e0 con 0x7fe3bc101990 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.200+0000 7fe3bbfff640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702] conn(0x7fe3a40bda60 0x7fe3a40bff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.200+0000 7fe3baffd640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe3a40b3cb0 0x7fe3a40b6170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0031440 tx=0x7fe3b00095f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.201+0000 7fe3bbfff640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702] conn(0x7fe3a40bda60 0x7fe3a40bff00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.201+0000 7fe3b8ff9640 1 -- 172.21.15.44:0/3189967965 <== osd.3 v2:172.21.15.44:6810/3423983702 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3a40c05d0 con 0x7fe3a40bda60 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.215+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/3189967965 --> [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe3bc06b3b0 con 0x7fe3a40bda60 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.215+0000 7fe3b8ff9640 1 -- 172.21.15.44:0/3189967965 <== osd.3 v2:172.21.15.44:6810/3423983702 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe3bc06b3b0 con 0x7fe3a40bda60 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.216+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702] conn(0x7fe3a40bda60 msgr2=0x7fe3a40bff00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.216+0000 7fe3c1c5a640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702] conn(0x7fe3a40bda60 0x7fe3a40bff00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.216+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe3a40b3cb0 msgr2=0x7fe3a40b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.216+0000 7fe3c1c5a640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe3a40b3cb0 0x7fe3a40b6170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0031440 tx=0x7fe3b00095f0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.216+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc101990 msgr2=0x7fe3bc197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:02.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.216+0000 7fe3c1c5a640 1 --2- 172.21.15.44:0/3189967965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc101990 0x7fe3bc197e50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac007910 tx=0x7fe3ac007dd0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.216+0000 7fe3bbfff640 1 -- 172.21.15.44:0/3189967965 reap_dead start 2024-08-20T03:01:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.216+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/3189967965 shutdown_connections 2024-08-20T03:01:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.216+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/3189967965 >> 172.21.15.44:0/3189967965 conn(0x7fe3bc0fd400 msgr2=0x7fe3bc0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.216+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/3189967965 shutdown_connections 2024-08-20T03:01:02.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.216+0000 7fe3c1c5a640 1 -- 172.21.15.44:0/3189967965 wait complete. 2024-08-20T03:01:02.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.182+0000 7f57f8b9f640 1 -- 172.21.15.44:0/2757024567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f4101990 msgr2=0x7f57f4101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:02.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.182+0000 7f57f8b9f640 1 --2- 172.21.15.44:0/2757024567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f4101990 0x7f57f4101d70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f57e4009900 tx=0x7f57e402a410 comp rx=0 tx=0).stop 2024-08-20T03:01:02.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.182+0000 7f57f8b9f640 1 -- 172.21.15.44:0/2757024567 shutdown_connections 2024-08-20T03:01:02.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.182+0000 7f57f8b9f640 1 --2- 172.21.15.44:0/2757024567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57f4107d20 0x7f57f410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.182+0000 7f57f8b9f640 1 --2- 172.21.15.44:0/2757024567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57f41073a0 0x7f57f41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.182+0000 7f57f8b9f640 1 --2- 172.21.15.44:0/2757024567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f4101990 0x7f57f4101d70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.182+0000 7f57f8b9f640 1 -- 172.21.15.44:0/2757024567 >> 172.21.15.44:0/2757024567 conn(0x7f57f40fd400 msgr2=0x7f57f40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.182+0000 7f57f8b9f640 1 -- 172.21.15.44:0/2757024567 shutdown_connections 2024-08-20T03:01:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.182+0000 7f57f8b9f640 1 -- 172.21.15.44:0/2757024567 wait complete. 2024-08-20T03:01:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f8b9f640 1 Processor -- start 2024-08-20T03:01:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f8b9f640 1 -- start start 2024-08-20T03:01:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f8b9f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f4101990 0x7f57f40fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f8b9f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57f41073a0 0x7f57f40fde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f8b9f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57f4107d20 0x7f57f40fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f8b9f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f57f41119e0 con 0x7f57f4101990 2024-08-20T03:01:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f8b9f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f57f4111860 con 0x7f57f41073a0 2024-08-20T03:01:02.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f8b9f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f57f4111b60 con 0x7f57f4107d20 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f4101990 0x7f57f40fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f4101990 0x7f57f40fd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49494/0 (socket says 172.21.15.44:49494) 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f259c640 1 -- 172.21.15.44:0/1961891971 learned_addr learned my addr 172.21.15.44:0/1961891971 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f2d9d640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57f4107d20 0x7f57f40fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f1d9b640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57f41073a0 0x7f57f40fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f259c640 1 -- 172.21.15.44:0/1961891971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57f4107d20 msgr2=0x7f57f40fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f259c640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57f4107d20 0x7f57f40fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f259c640 1 -- 172.21.15.44:0/1961891971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57f41073a0 msgr2=0x7f57f40fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f259c640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57f41073a0 0x7f57f40fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f259c640 1 -- 172.21.15.44:0/1961891971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f41a7410 con 0x7f57f4101990 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.183+0000 7f57f1d9b640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57f41073a0 0x7f57f40fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.184+0000 7f57f259c640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f4101990 0x7f57f40fd900 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f57e4005940 tx=0x7f57e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:02.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.184+0000 7f57e37fe640 1 -- 172.21.15.44:0/1961891971 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e403f030 con 0x7f57f4101990 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.184+0000 7f57e37fe640 1 -- 172.21.15.44:0/1961891971 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57e4003cb0 con 0x7f57f4101990 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.184+0000 7f57f8b9f640 1 -- 172.21.15.44:0/1961891971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f41a76a0 con 0x7f57f4101990 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.184+0000 7f57e37fe640 1 -- 172.21.15.44:0/1961891971 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4035070 con 0x7f57f4101990 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.184+0000 7f57f8b9f640 1 -- 172.21.15.44:0/1961891971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f41a7b80 con 0x7f57f4101990 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.185+0000 7f57f8b9f640 1 -- 172.21.15.44:0/1961891971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57f40635d0 con 0x7f57f4101990 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.186+0000 7f57e37fe640 1 -- 172.21.15.44:0/1961891971 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f57e4042030 con 0x7f57f4101990 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.187+0000 7f57e37fe640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f57dc0b3cb0 0x7f57dc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.187+0000 7f57f1d9b640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f57dc0b3cb0 0x7f57dc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.187+0000 7f57e37fe640 1 -- 172.21.15.44:0/1961891971 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f57e4104a00 con 0x7f57f4101990 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.187+0000 7f57e37fe640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465] conn(0x7f57dc0bda60 0x7f57dc0bff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.187+0000 7f57e37fe640 1 -- 172.21.15.44:0/1961891971 --> [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57e4046e50 con 0x7f57dc0bda60 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.187+0000 7f57e37fe640 1 -- 172.21.15.44:0/1961891971 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f57e4104e50 con 0x7f57f4101990 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.187+0000 7f57f1d9b640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f57dc0b3cb0 0x7f57dc0b6170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f57e80058c0 tx=0x7f57e8005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:02.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.187+0000 7f57f2d9d640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465] conn(0x7f57dc0bda60 0x7f57dc0bff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.188+0000 7f57f2d9d640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465] conn(0x7f57dc0bda60 0x7f57dc0bff00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.188+0000 7f57e37fe640 1 -- 172.21.15.44:0/1961891971 <== osd.2 v2:172.21.15.165:6800/1492740465 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57e4046e50 con 0x7f57dc0bda60 2024-08-20T03:01:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.202+0000 7f57f8b9f640 1 -- 172.21.15.44:0/1961891971 --> [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f57f4063a50 con 0x7f57dc0bda60 2024-08-20T03:01:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.203+0000 7f57e37fe640 1 -- 172.21.15.44:0/1961891971 <== osd.2 v2:172.21.15.165:6800/1492740465 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f57f4063a50 con 0x7f57dc0bda60 2024-08-20T03:01:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.203+0000 7f57f8b9f640 1 -- 172.21.15.44:0/1961891971 >> [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465] conn(0x7f57dc0bda60 msgr2=0x7f57dc0bff00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.203+0000 7f57f8b9f640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465] conn(0x7f57dc0bda60 0x7f57dc0bff00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.204+0000 7f57f8b9f640 1 -- 172.21.15.44:0/1961891971 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f57dc0b3cb0 msgr2=0x7f57dc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.204+0000 7f57f8b9f640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f57dc0b3cb0 0x7f57dc0b6170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f57e80058c0 tx=0x7f57e8005850 comp rx=0 tx=0).stop 2024-08-20T03:01:02.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.204+0000 7f57f8b9f640 1 -- 172.21.15.44:0/1961891971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f4101990 msgr2=0x7f57f40fd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.204+0000 7f57f8b9f640 1 --2- 172.21.15.44:0/1961891971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57f4101990 0x7f57f40fd900 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f57e4005940 tx=0x7f57e40042f0 comp rx=0 tx=0).stop 2024-08-20T03:01:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.204+0000 7f57f2d9d640 1 -- 172.21.15.44:0/1961891971 reap_dead start 2024-08-20T03:01:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.204+0000 7f57f8b9f640 1 -- 172.21.15.44:0/1961891971 shutdown_connections 2024-08-20T03:01:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.204+0000 7f57f8b9f640 1 -- 172.21.15.44:0/1961891971 >> 172.21.15.44:0/1961891971 conn(0x7f57f40fd400 msgr2=0x7f57f406aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.204+0000 7f57f8b9f640 1 -- 172.21.15.44:0/1961891971 shutdown_connections 2024-08-20T03:01:02.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:01.204+0000 7f57f8b9f640 1 -- 172.21.15.44:0/1961891971 wait complete. 2024-08-20T03:01:03.268 INFO:teuthology.orchestra.run.smithi044.stdout:81604378691 2024-08-20T03:01:03.269 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-08-20T03:01:03.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.301+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/1986114418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a68104300 msgr2=0x7f8a6810c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:03.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.301+0000 7f8a6e8a7640 1 --2- 172.21.15.44:0/1986114418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a68104300 0x7f8a6810c7f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c009900 tx=0x7f8a5c02a410 comp rx=0 tx=0).stop 2024-08-20T03:01:03.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.301+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/1986114418 shutdown_connections 2024-08-20T03:01:03.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.302+0000 7f8a6e8a7640 1 --2- 172.21.15.44:0/1986114418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a6810ce40 0x7f8a6810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.302+0000 7f8a6e8a7640 1 --2- 172.21.15.44:0/1986114418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a68104300 0x7f8a6810c7f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.302+0000 7f8a6e8a7640 1 --2- 172.21.15.44:0/1986114418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a681039e0 0x7f8a68103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.302+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/1986114418 >> 172.21.15.44:0/1986114418 conn(0x7f8a680fd440 msgr2=0x7f8a680ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.302+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/1986114418 shutdown_connections 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.302+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/1986114418 wait complete. 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.302+0000 7f8a6e8a7640 1 Processor -- start 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.302+0000 7f8a6e8a7640 1 -- start start 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.302+0000 7f8a6e8a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a681039e0 0x7f8a68101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.302+0000 7f8a6e8a7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a68104300 0x7f8a68101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a6e8a7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a6810ce40 0x7f8a68102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a6e8a7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68112ab0 con 0x7f8a681039e0 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a6e8a7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68112930 con 0x7f8a6810ce40 2024-08-20T03:01:03.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a6e8a7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68112c30 con 0x7f8a68104300 2024-08-20T03:01:03.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a67fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a681039e0 0x7f8a68101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:03.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a67fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a681039e0 0x7f8a68101690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49548/0 (socket says 172.21.15.44:49548) 2024-08-20T03:01:03.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a67fff640 1 -- 172.21.15.44:0/3043912813 learned_addr learned my addr 172.21.15.44:0/3043912813 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:03.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a67fff640 1 -- 172.21.15.44:0/3043912813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a68104300 msgr2=0x7f8a68101bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:03.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a67fff640 1 --2- 172.21.15.44:0/3043912813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a68104300 0x7f8a68101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:03.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a67fff640 1 -- 172.21.15.44:0/3043912813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a6810ce40 msgr2=0x7f8a68102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a67fff640 1 --2- 172.21.15.44:0/3043912813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a6810ce40 0x7f8a68102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a67fff640 1 -- 172.21.15.44:0/3043912813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a681a73c0 con 0x7f8a681039e0 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a67fff640 1 --2- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a681039e0 0x7f8a68101690 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800d770 tx=0x7f8a5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a657fa640 1 -- 172.21.15.44:0/3043912813 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58017070 con 0x7f8a681039e0 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/3043912813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a681a76b0 con 0x7f8a681039e0 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a657fa640 1 -- 172.21.15.44:0/3043912813 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a580054d0 con 0x7f8a681039e0 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a657fa640 1 -- 172.21.15.44:0/3043912813 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58004c30 con 0x7f8a681039e0 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.303+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/3043912813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a681a7bf0 con 0x7f8a681039e0 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.304+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/3043912813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8a680ff3a0 con 0x7f8a681039e0 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.306+0000 7f8a657fa640 1 -- 172.21.15.44:0/3043912813 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f8a58004dd0 con 0x7f8a681039e0 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.306+0000 7f8a657fa640 1 --2- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8a500b3cb0 0x7f8a500b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.306+0000 7f8a677fe640 1 --2- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8a500b3cb0 0x7f8a500b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.306+0000 7f8a657fa640 1 -- 172.21.15.44:0/3043912813 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f8a580e3e20 con 0x7f8a681039e0 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.307+0000 7f8a657fa640 1 --2- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093] conn(0x7f8a500bda60 0x7f8a500bff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.307+0000 7f8a657fa640 1 -- 172.21.15.44:0/3043912813 --> [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a58010e50 con 0x7f8a500bda60 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.307+0000 7f8a657fa640 1 -- 172.21.15.44:0/3043912813 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f8a580e4270 con 0x7f8a681039e0 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.307+0000 7f8a6ce44640 1 --2- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093] conn(0x7f8a500bda60 0x7f8a500bff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.307+0000 7f8a677fe640 1 --2- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8a500b3cb0 0x7f8a500b6170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c02a930 tx=0x7f8a5c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.307+0000 7f8a6ce44640 1 --2- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093] conn(0x7f8a500bda60 0x7f8a500bff00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.308+0000 7f8a657fa640 1 -- 172.21.15.44:0/3043912813 <== osd.9 v2:172.21.15.44:6826/2426940093 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a58010e50 con 0x7f8a500bda60 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.322+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/3043912813 --> [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8a6806b890 con 0x7f8a500bda60 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.322+0000 7f8a657fa640 1 -- 172.21.15.44:0/3043912813 <== osd.9 v2:172.21.15.44:6826/2426940093 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8a6806b890 con 0x7f8a500bda60 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.322+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093] conn(0x7f8a500bda60 msgr2=0x7f8a500bff00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.322+0000 7f8a6e8a7640 1 --2- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093] conn(0x7f8a500bda60 0x7f8a500bff00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.323+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8a500b3cb0 msgr2=0x7f8a500b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.323+0000 7f8a6e8a7640 1 --2- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8a500b3cb0 0x7f8a500b6170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c02a930 tx=0x7f8a5c031400 comp rx=0 tx=0).stop 2024-08-20T03:01:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.323+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a681039e0 msgr2=0x7f8a68101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.323+0000 7f8a6e8a7640 1 --2- 172.21.15.44:0/3043912813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a681039e0 0x7f8a68101690 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800d770 tx=0x7f8a5800dc30 comp rx=0 tx=0).stop 2024-08-20T03:01:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.323+0000 7f8a6ce44640 1 -- 172.21.15.44:0/3043912813 reap_dead start 2024-08-20T03:01:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.323+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/3043912813 shutdown_connections 2024-08-20T03:01:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.323+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/3043912813 >> 172.21.15.44:0/3043912813 conn(0x7f8a680fd440 msgr2=0x7f8a6810f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.323+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/3043912813 shutdown_connections 2024-08-20T03:01:03.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:02.323+0000 7f8a6e8a7640 1 -- 172.21.15.44:0/3043912813 wait complete. 2024-08-20T03:01:03.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:03 smithi044 ceph-mon[18790]: pgmap v277: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s; 7.3 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T03:01:03.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:03 smithi165 ceph-mon[22357]: pgmap v277: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s; 7.3 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T03:01:03.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:03 smithi099 ceph-mon[21984]: pgmap v277: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s; 7.3 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T03:01:04.055 INFO:teuthology.orchestra.run.smithi044.stdout:154618822707 2024-08-20T03:01:04.055 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-08-20T03:01:04.103 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:04.207 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:04.680 INFO:teuthology.orchestra.run.smithi044.stdout:111669149757 2024-08-20T03:01:04.681 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-08-20T03:01:05.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.553+0000 7faf862d7640 1 -- 172.21.15.44:0/1860010866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8010ce40 msgr2=0x7faf8010f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.553+0000 7faf862d7640 1 --2- 172.21.15.44:0/1860010866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8010ce40 0x7faf8010f2a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faf78009ca0 tx=0x7faf7802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.553+0000 7faf862d7640 1 -- 172.21.15.44:0/1860010866 shutdown_connections 2024-08-20T03:01:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.553+0000 7faf862d7640 1 --2- 172.21.15.44:0/1860010866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8010ce40 0x7faf8010f2a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.553+0000 7faf862d7640 1 --2- 172.21.15.44:0/1860010866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf80104300 0x7faf8010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.553+0000 7faf862d7640 1 --2- 172.21.15.44:0/1860010866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf801039e0 0x7faf80103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.553+0000 7faf862d7640 1 -- 172.21.15.44:0/1860010866 >> 172.21.15.44:0/1860010866 conn(0x7faf800fd440 msgr2=0x7faf800ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.553+0000 7faf862d7640 1 -- 172.21.15.44:0/1860010866 shutdown_connections 2024-08-20T03:01:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.553+0000 7faf862d7640 1 -- 172.21.15.44:0/1860010866 wait complete. 2024-08-20T03:01:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.553+0000 7faf862d7640 1 Processor -- start 2024-08-20T03:01:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.553+0000 7faf862d7640 1 -- start start 2024-08-20T03:01:05.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf862d7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf801039e0 0x7faf80198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf862d7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf80104300 0x7faf80199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf862d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8010ce40 0x7faf8019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf862d7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf80112ab0 con 0x7faf8010ce40 2024-08-20T03:01:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf862d7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf80112930 con 0x7faf80104300 2024-08-20T03:01:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf862d7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7faf80112c30 con 0x7faf801039e0 2024-08-20T03:01:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf84874640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8010ce40 0x7faf8019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf84874640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8010ce40 0x7faf8019e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49568/0 (socket says 172.21.15.44:49568) 2024-08-20T03:01:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf84874640 1 -- 172.21.15.44:0/2531844850 learned_addr learned my addr 172.21.15.44:0/2531844850 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf7ffff640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf801039e0 0x7faf80198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:05.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf7f7fe640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf80104300 0x7faf80199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:05.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf84874640 1 -- 172.21.15.44:0/2531844850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf801039e0 msgr2=0x7faf80198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:05.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf84874640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf801039e0 0x7faf80198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:05.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf84874640 1 -- 172.21.15.44:0/2531844850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf80104300 msgr2=0x7faf80199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:05.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf84874640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf80104300 0x7faf80199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:05.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf84874640 1 -- 172.21.15.44:0/2531844850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf8019ef70 con 0x7faf8010ce40 2024-08-20T03:01:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf7f7fe640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf80104300 0x7faf80199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf7ffff640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf801039e0 0x7faf80198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.554+0000 7faf84874640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8010ce40 0x7faf8019e7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf78005cc0 tx=0x7faf7802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.555+0000 7faf7d7fa640 1 -- 172.21.15.44:0/2531844850 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf78003bf0 con 0x7faf8010ce40 2024-08-20T03:01:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.555+0000 7faf862d7640 1 -- 172.21.15.44:0/2531844850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf8019f200 con 0x7faf8010ce40 2024-08-20T03:01:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.555+0000 7faf7d7fa640 1 -- 172.21.15.44:0/2531844850 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf78003d90 con 0x7faf8010ce40 2024-08-20T03:01:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.555+0000 7faf7d7fa640 1 -- 172.21.15.44:0/2531844850 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7802fbc0 con 0x7faf8010ce40 2024-08-20T03:01:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.555+0000 7faf862d7640 1 -- 172.21.15.44:0/2531844850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf801a5920 con 0x7faf8010ce40 2024-08-20T03:01:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.555+0000 7faf862d7640 1 -- 172.21.15.44:0/2531844850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf800ff3a0 con 0x7faf8010ce40 2024-08-20T03:01:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.557+0000 7faf7d7fa640 1 -- 172.21.15.44:0/2531844850 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7faf7802fd60 con 0x7faf8010ce40 2024-08-20T03:01:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.558+0000 7faf7d7fa640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faf680b3cb0 0x7faf680b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:05.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.558+0000 7faf7ffff640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faf680b3cb0 0x7faf680b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.558+0000 7faf7d7fa640 1 -- 172.21.15.44:0/2531844850 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7faf78105760 con 0x7faf8010ce40 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.558+0000 7faf7d7fa640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415] conn(0x7faf680bda60 0x7faf680bff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.558+0000 7faf7d7fa640 1 -- 172.21.15.44:0/2531844850 --> [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf680c05d0 con 0x7faf680bda60 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.558+0000 7faf7d7fa640 1 -- 172.21.15.44:0/2531844850 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7faf78105b10 con 0x7faf8010ce40 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.558+0000 7faf7ffff640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faf680b3cb0 0x7faf680b6170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faf7000abe0 tx=0x7faf7000a5c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.558+0000 7faf7f7fe640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415] conn(0x7faf680bda60 0x7faf680bff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.559+0000 7faf7f7fe640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415] conn(0x7faf680bda60 0x7faf680bff00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.560+0000 7faf7d7fa640 1 -- 172.21.15.44:0/2531844850 <== osd.11 v2:172.21.15.165:6824/107688415 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faf680c05d0 con 0x7faf680bda60 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.573+0000 7faf862d7640 1 -- 172.21.15.44:0/2531844850 --> [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf8006b890 con 0x7faf680bda60 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.574+0000 7faf7d7fa640 1 -- 172.21.15.44:0/2531844850 <== osd.11 v2:172.21.15.165:6824/107688415 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faf8006b890 con 0x7faf680bda60 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.574+0000 7faf862d7640 1 -- 172.21.15.44:0/2531844850 >> [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415] conn(0x7faf680bda60 msgr2=0x7faf680bff00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.574+0000 7faf862d7640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415] conn(0x7faf680bda60 0x7faf680bff00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:05.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.574+0000 7faf862d7640 1 -- 172.21.15.44:0/2531844850 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faf680b3cb0 msgr2=0x7faf680b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:05.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.574+0000 7faf862d7640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faf680b3cb0 0x7faf680b6170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faf7000abe0 tx=0x7faf7000a5c0 comp rx=0 tx=0).stop 2024-08-20T03:01:05.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.574+0000 7faf862d7640 1 -- 172.21.15.44:0/2531844850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8010ce40 msgr2=0x7faf8019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:05.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.574+0000 7faf862d7640 1 --2- 172.21.15.44:0/2531844850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf8010ce40 0x7faf8019e7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf78005cc0 tx=0x7faf7802f380 comp rx=0 tx=0).stop 2024-08-20T03:01:05.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.574+0000 7faf84874640 1 -- 172.21.15.44:0/2531844850 reap_dead start 2024-08-20T03:01:05.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.574+0000 7faf862d7640 1 -- 172.21.15.44:0/2531844850 shutdown_connections 2024-08-20T03:01:05.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.574+0000 7faf862d7640 1 -- 172.21.15.44:0/2531844850 >> 172.21.15.44:0/2531844850 conn(0x7faf800fd440 msgr2=0x7faf8010f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:05.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.574+0000 7faf862d7640 1 -- 172.21.15.44:0/2531844850 shutdown_connections 2024-08-20T03:01:05.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:05.574+0000 7faf862d7640 1 -- 172.21.15.44:0/2531844850 wait complete. 2024-08-20T03:01:05.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:05 smithi044 ceph-mon[18790]: pgmap v278: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 7.3 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T03:01:05.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:05 smithi165 ceph-mon[22357]: pgmap v278: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 7.3 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T03:01:05.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:05 smithi099 ceph-mon[21984]: pgmap v278: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s; 7.3 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T03:01:06.366 INFO:teuthology.orchestra.run.smithi044.stdout:296352743447 2024-08-20T03:01:06.366 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-08-20T03:01:06.869 INFO:teuthology.orchestra.run.smithi044.stdout:249108103201 2024-08-20T03:01:06.869 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-08-20T03:01:07.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.182+0000 7f946e0fa640 1 -- 172.21.15.44:0/1431963024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94681084e0 msgr2=0x7f9468108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:07.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.182+0000 7f946e0fa640 1 --2- 172.21.15.44:0/1431963024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94681084e0 0x7f9468108920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9450009900 tx=0x7f945002a410 comp rx=0 tx=0).stop 2024-08-20T03:01:07.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.182+0000 7f946e0fa640 1 -- 172.21.15.44:0/1431963024 shutdown_connections 2024-08-20T03:01:07.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.182+0000 7f946e0fa640 1 --2- 172.21.15.44:0/1431963024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9468108e60 0x7f946810f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.182+0000 7f946e0fa640 1 --2- 172.21.15.44:0/1431963024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94681084e0 0x7f9468108920 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.182+0000 7f946e0fa640 1 --2- 172.21.15.44:0/1431963024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9468102ad0 0x7f9468102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.182+0000 7f946e0fa640 1 -- 172.21.15.44:0/1431963024 >> 172.21.15.44:0/1431963024 conn(0x7f94680fe540 msgr2=0x7f9468100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.182+0000 7f946e0fa640 1 -- 172.21.15.44:0/1431963024 shutdown_connections 2024-08-20T03:01:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.182+0000 7f946e0fa640 1 -- 172.21.15.44:0/1431963024 wait complete. 2024-08-20T03:01:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.183+0000 7f946e0fa640 1 Processor -- start 2024-08-20T03:01:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.183+0000 7f946e0fa640 1 -- start start 2024-08-20T03:01:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.183+0000 7f946e0fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9468102ad0 0x7f9468198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.183+0000 7f946e0fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94681084e0 0x7f9468199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:07.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.183+0000 7f946e0fa640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9468108e60 0x7f946819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.183+0000 7f946e0fa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9468112b70 con 0x7f9468102ad0 2024-08-20T03:01:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.183+0000 7f946e0fa640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f94681129f0 con 0x7f94681084e0 2024-08-20T03:01:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.183+0000 7f946e0fa640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9468112cf0 con 0x7f9468108e60 2024-08-20T03:01:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.183+0000 7f94677fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9468102ad0 0x7f9468198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.183+0000 7f94677fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9468102ad0 0x7f9468198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49588/0 (socket says 172.21.15.44:49588) 2024-08-20T03:01:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.183+0000 7f94677fe640 1 -- 172.21.15.44:0/1689670769 learned_addr learned my addr 172.21.15.44:0/1689670769 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f945effd640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94681084e0 0x7f9468199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:07.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f9467fff640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9468108e60 0x7f946819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f94677fe640 1 -- 172.21.15.44:0/1689670769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9468108e60 msgr2=0x7f946819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f94677fe640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9468108e60 0x7f946819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f94677fe640 1 -- 172.21.15.44:0/1689670769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94681084e0 msgr2=0x7f9468199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f94677fe640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94681084e0 0x7f9468199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f94677fe640 1 -- 172.21.15.44:0/1689670769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946819f070 con 0x7f9468102ad0 2024-08-20T03:01:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f9467fff640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9468108e60 0x7f946819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f94677fe640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9468102ad0 0x7f9468198fc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f945800b4a0 tx=0x7f945800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f94657fa640 1 -- 172.21.15.44:0/1689670769 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458014070 con 0x7f9468102ad0 2024-08-20T03:01:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f946e0fa640 1 -- 172.21.15.44:0/1689670769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946819f360 con 0x7f9468102ad0 2024-08-20T03:01:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f94657fa640 1 -- 172.21.15.44:0/1689670769 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9458010ac0 con 0x7f9468102ad0 2024-08-20T03:01:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f94657fa640 1 -- 172.21.15.44:0/1689670769 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458017c30 con 0x7f9468102ad0 2024-08-20T03:01:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.184+0000 7f946e0fa640 1 -- 172.21.15.44:0/1689670769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94681a5bf0 con 0x7f9468102ad0 2024-08-20T03:01:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.185+0000 7f946e0fa640 1 -- 172.21.15.44:0/1689670769 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9468063610 con 0x7f9468102ad0 2024-08-20T03:01:07.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.186+0000 7f94657fa640 1 -- 172.21.15.44:0/1689670769 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f9458017070 con 0x7f9468102ad0 2024-08-20T03:01:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.187+0000 7f94657fa640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f94540b3cb0 0x7f94540b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.187+0000 7f945effd640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f94540b3cb0 0x7f94540b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.187+0000 7f94657fa640 1 -- 172.21.15.44:0/1689670769 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f94580e35f0 con 0x7f9468102ad0 2024-08-20T03:01:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.187+0000 7f94657fa640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986] conn(0x7f94540bda60 0x7f94540bff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.187+0000 7f94657fa640 1 -- 172.21.15.44:0/1689670769 --> [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9458011e50 con 0x7f94540bda60 2024-08-20T03:01:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.187+0000 7f94657fa640 1 -- 172.21.15.44:0/1689670769 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f94580e3a40 con 0x7f9468102ad0 2024-08-20T03:01:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.187+0000 7f9467fff640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986] conn(0x7f94540bda60 0x7f94540bff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.188+0000 7f945effd640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f94540b3cb0 0x7f94540b6170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9450000c00 tx=0x7f9450031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.188+0000 7f9467fff640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986] conn(0x7f94540bda60 0x7f94540bff00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.188+0000 7f94657fa640 1 -- 172.21.15.44:0/1689670769 <== osd.6 v2:172.21.15.44:6818/1039098986 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9458011e50 con 0x7f94540bda60 2024-08-20T03:01:07.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.202+0000 7f946e0fa640 1 -- 172.21.15.44:0/1689670769 --> [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9468063a90 con 0x7f94540bda60 2024-08-20T03:01:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.202+0000 7f94657fa640 1 -- 172.21.15.44:0/1689670769 <== osd.6 v2:172.21.15.44:6818/1039098986 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9468063a90 con 0x7f94540bda60 2024-08-20T03:01:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.203+0000 7f946e0fa640 1 -- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986] conn(0x7f94540bda60 msgr2=0x7f94540bff00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.203+0000 7f946e0fa640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986] conn(0x7f94540bda60 0x7f94540bff00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.203+0000 7f946e0fa640 1 -- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f94540b3cb0 msgr2=0x7f94540b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.203+0000 7f946e0fa640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f94540b3cb0 0x7f94540b6170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9450000c00 tx=0x7f9450031400 comp rx=0 tx=0).stop 2024-08-20T03:01:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.203+0000 7f946e0fa640 1 -- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9468102ad0 msgr2=0x7f9468198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.203+0000 7f946e0fa640 1 --2- 172.21.15.44:0/1689670769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9468102ad0 0x7f9468198fc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f945800b4a0 tx=0x7f945800b960 comp rx=0 tx=0).stop 2024-08-20T03:01:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.203+0000 7f9467fff640 1 -- 172.21.15.44:0/1689670769 reap_dead start 2024-08-20T03:01:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.203+0000 7f946e0fa640 1 -- 172.21.15.44:0/1689670769 shutdown_connections 2024-08-20T03:01:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.203+0000 7f946e0fa640 1 -- 172.21.15.44:0/1689670769 >> 172.21.15.44:0/1689670769 conn(0x7f94680fe540 msgr2=0x7f9468100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.203+0000 7f946e0fa640 1 -- 172.21.15.44:0/1689670769 shutdown_connections 2024-08-20T03:01:07.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.203+0000 7f946e0fa640 1 -- 172.21.15.44:0/1689670769 wait complete. 2024-08-20T03:01:07.774 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:07 smithi044 ceph-mon[18790]: pgmap v279: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 7.3 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T03:01:07.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.852+0000 7f0ec4f16640 1 -- 172.21.15.44:0/2824178519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec0104230 msgr2=0x7f0ec010c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:07.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.852+0000 7f0ec4f16640 1 --2- 172.21.15.44:0/2824178519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec0104230 0x7f0ec010c810 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009900 tx=0x7f0eb402a410 comp rx=0 tx=0).stop 2024-08-20T03:01:07.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.852+0000 7f0ec4f16640 1 -- 172.21.15.44:0/2824178519 shutdown_connections 2024-08-20T03:01:07.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.852+0000 7f0ec4f16640 1 --2- 172.21.15.44:0/2824178519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ec010cde0 0x7f0ec010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.852+0000 7f0ec4f16640 1 --2- 172.21.15.44:0/2824178519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec0104230 0x7f0ec010c810 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.852+0000 7f0ec4f16640 1 --2- 172.21.15.44:0/2824178519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec0103830 0x7f0ec0103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.852+0000 7f0ec4f16640 1 -- 172.21.15.44:0/2824178519 >> 172.21.15.44:0/2824178519 conn(0x7f0ec00fd400 msgr2=0x7f0ec00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:07.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.852+0000 7f0ec4f16640 1 -- 172.21.15.44:0/2824178519 shutdown_connections 2024-08-20T03:01:07.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.852+0000 7f0ec4f16640 1 -- 172.21.15.44:0/2824178519 wait complete. 2024-08-20T03:01:07.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.852+0000 7f0ec4f16640 1 Processor -- start 2024-08-20T03:01:07.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ec4f16640 1 -- start start 2024-08-20T03:01:07.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ec4f16640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ec0103830 0x7f0ec0198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:07.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ec4f16640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec0104230 0x7f0ec0199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:07.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ec4f16640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec010cde0 0x7f0ec019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:07.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ec4f16640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0112aa0 con 0x7f0ec0104230 2024-08-20T03:01:07.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ec4f16640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0112920 con 0x7f0ec0103830 2024-08-20T03:01:07.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ec4f16640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0112c20 con 0x7f0ec010cde0 2024-08-20T03:01:07.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ebdd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec0104230 0x7f0ec0199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:07.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ebdd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec0104230 0x7f0ec0199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40288/0 (socket says 172.21.15.44:40288) 2024-08-20T03:01:07.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ebdd9b640 1 -- 172.21.15.44:0/3260891226 learned_addr learned my addr 172.21.15.44:0/3260891226 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:07.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ebed9d640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec010cde0 0x7f0ec019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:07.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ebe59c640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ec0103830 0x7f0ec0198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:07.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ebdd9b640 1 -- 172.21.15.44:0/3260891226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec010cde0 msgr2=0x7f0ec019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:07.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ebdd9b640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec010cde0 0x7f0ec019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ebdd9b640 1 -- 172.21.15.44:0/3260891226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ec0103830 msgr2=0x7f0ec0198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:07.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ebdd9b640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ec0103830 0x7f0ec0198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ebdd9b640 1 -- 172.21.15.44:0/3260891226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec019efe0 con 0x7f0ec0104230 2024-08-20T03:01:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ebe59c640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ec0103830 0x7f0ec0198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.853+0000 7f0ebdd9b640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec0104230 0x7f0ec0199470 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4006010 tx=0x7f0eb40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.854+0000 7f0eaf7fe640 1 -- 172.21.15.44:0/3260891226 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb403f030 con 0x7f0ec0104230 2024-08-20T03:01:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.854+0000 7f0ec4f16640 1 -- 172.21.15.44:0/3260891226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec019f270 con 0x7f0ec0104230 2024-08-20T03:01:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.854+0000 7f0eaf7fe640 1 -- 172.21.15.44:0/3260891226 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb4003800 con 0x7f0ec0104230 2024-08-20T03:01:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.854+0000 7f0eaf7fe640 1 -- 172.21.15.44:0/3260891226 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4035070 con 0x7f0ec0104230 2024-08-20T03:01:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.854+0000 7f0ec4f16640 1 -- 172.21.15.44:0/3260891226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec01a5990 con 0x7f0ec0104230 2024-08-20T03:01:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.854+0000 7f0ec4f16640 1 -- 172.21.15.44:0/3260891226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ec00ff360 con 0x7f0ec0104230 2024-08-20T03:01:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.856+0000 7f0eaf7fe640 1 -- 172.21.15.44:0/3260891226 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f0eb4003cb0 con 0x7f0ec0104230 2024-08-20T03:01:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.857+0000 7f0eaf7fe640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0ea80b3cb0 0x7f0ea80b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:07.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.857+0000 7f0ebe59c640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0ea80b3cb0 0x7f0ea80b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:07.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.857+0000 7f0eaf7fe640 1 -- 172.21.15.44:0/3260891226 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0eb4104f90 con 0x7f0ec0104230 2024-08-20T03:01:07.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.857+0000 7f0eaf7fe640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663] conn(0x7f0ea80bda60 0x7f0ea80bff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:07.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.857+0000 7f0eaf7fe640 1 -- 172.21.15.44:0/3260891226 --> [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0eb4044e50 con 0x7f0ea80bda60 2024-08-20T03:01:07.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.857+0000 7f0eaf7fe640 1 -- 172.21.15.44:0/3260891226 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f0eb41053e0 con 0x7f0ec0104230 2024-08-20T03:01:07.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.858+0000 7f0ebe59c640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0ea80b3cb0 0x7f0ea80b6170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0008e00 tx=0x7f0eb0008350 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:07.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.858+0000 7f0ebed9d640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663] conn(0x7f0ea80bda60 0x7f0ea80bff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.858+0000 7f0ebed9d640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663] conn(0x7f0ea80bda60 0x7f0ea80bff00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.859+0000 7f0eaf7fe640 1 -- 172.21.15.44:0/3260891226 <== osd.1 v2:172.21.15.99:6800/4252911663 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0eb4044e50 con 0x7f0ea80bda60 2024-08-20T03:01:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.872+0000 7f0ec4f16640 1 -- 172.21.15.44:0/3260891226 --> [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ec006ae50 con 0x7f0ea80bda60 2024-08-20T03:01:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.873+0000 7f0eaf7fe640 1 -- 172.21.15.44:0/3260891226 <== osd.1 v2:172.21.15.99:6800/4252911663 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0ec006ae50 con 0x7f0ea80bda60 2024-08-20T03:01:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.873+0000 7f0ec4f16640 1 -- 172.21.15.44:0/3260891226 >> [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663] conn(0x7f0ea80bda60 msgr2=0x7f0ea80bff00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.873+0000 7f0ec4f16640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663] conn(0x7f0ea80bda60 0x7f0ea80bff00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.874+0000 7f0ec4f16640 1 -- 172.21.15.44:0/3260891226 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0ea80b3cb0 msgr2=0x7f0ea80b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.874+0000 7f0ec4f16640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0ea80b3cb0 0x7f0ea80b6170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0008e00 tx=0x7f0eb0008350 comp rx=0 tx=0).stop 2024-08-20T03:01:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.874+0000 7f0ec4f16640 1 -- 172.21.15.44:0/3260891226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec0104230 msgr2=0x7f0ec0199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.874+0000 7f0ec4f16640 1 --2- 172.21.15.44:0/3260891226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec0104230 0x7f0ec0199470 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4006010 tx=0x7f0eb40042f0 comp rx=0 tx=0).stop 2024-08-20T03:01:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.874+0000 7f0ebed9d640 1 -- 172.21.15.44:0/3260891226 reap_dead start 2024-08-20T03:01:07.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.874+0000 7f0ec4f16640 1 -- 172.21.15.44:0/3260891226 shutdown_connections 2024-08-20T03:01:07.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.874+0000 7f0ec4f16640 1 -- 172.21.15.44:0/3260891226 >> 172.21.15.44:0/3260891226 conn(0x7f0ec00fd400 msgr2=0x7f0ec010f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:07.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.874+0000 7f0ec4f16640 1 -- 172.21.15.44:0/3260891226 shutdown_connections 2024-08-20T03:01:07.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:07.874+0000 7f0ec4f16640 1 -- 172.21.15.44:0/3260891226 wait complete. 2024-08-20T03:01:07.888 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:07 smithi165 ceph-mon[22357]: pgmap v279: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 7.3 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T03:01:07.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:07 smithi099 ceph-mon[21984]: pgmap v279: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 7.3 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T03:01:08.101 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:08.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.280+0000 7f50055f7640 1 -- 172.21.15.44:0/674898783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50001084c0 msgr2=0x7f5000108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:08.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.280+0000 7f50055f7640 1 --2- 172.21.15.44:0/674898783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50001084c0 0x7f5000108900 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009900 tx=0x7f4ff402a410 comp rx=0 tx=0).stop 2024-08-20T03:01:08.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.280+0000 7f50055f7640 1 -- 172.21.15.44:0/674898783 shutdown_connections 2024-08-20T03:01:08.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.280+0000 7f50055f7640 1 --2- 172.21.15.44:0/674898783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5000108e40 0x7f500010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:08.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.280+0000 7f50055f7640 1 --2- 172.21.15.44:0/674898783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50001084c0 0x7f5000108900 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:08.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.280+0000 7f50055f7640 1 --2- 172.21.15.44:0/674898783 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5000100f10 0x7f50001012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:08.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.280+0000 7f50055f7640 1 -- 172.21.15.44:0/674898783 >> 172.21.15.44:0/674898783 conn(0x7f5000077e30 msgr2=0x7f50000ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:08.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.280+0000 7f50055f7640 1 -- 172.21.15.44:0/674898783 shutdown_connections 2024-08-20T03:01:08.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.280+0000 7f50055f7640 1 -- 172.21.15.44:0/674898783 wait complete. 2024-08-20T03:01:08.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.281+0000 7f50055f7640 1 Processor -- start 2024-08-20T03:01:08.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.281+0000 7f50055f7640 1 -- start start 2024-08-20T03:01:08.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.281+0000 7f50055f7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5000100f10 0x7f5000198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:08.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.281+0000 7f50055f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001084c0 0x7f50001994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:08.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.281+0000 7f50055f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000108e40 0x7f500019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.281+0000 7f50055f7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5000112b00 con 0x7f5000108e40 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.281+0000 7f50055f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5000112980 con 0x7f50001084c0 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.281+0000 7f50055f7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5000112c80 con 0x7f5000100f10 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.281+0000 7f4fff7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000108e40 0x7f500019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.281+0000 7f4fff7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000108e40 0x7f500019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40306/0 (socket says 172.21.15.44:40306) 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.281+0000 7f4fff7fe640 1 -- 172.21.15.44:0/3879337323 learned_addr learned my addr 172.21.15.44:0/3879337323 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f4ffeffd640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5000100f10 0x7f5000198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f4ffe7fc640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001084c0 0x7f50001994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f4fff7fe640 1 -- 172.21.15.44:0/3879337323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5000100f10 msgr2=0x7f5000198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f4fff7fe640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5000100f10 0x7f5000198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f4fff7fe640 1 -- 172.21.15.44:0/3879337323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001084c0 msgr2=0x7f50001994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f4fff7fe640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001084c0 0x7f50001994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f4fff7fe640 1 -- 172.21.15.44:0/3879337323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500019f040 con 0x7f5000108e40 2024-08-20T03:01:08.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f4ffe7fc640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50001084c0 0x7f50001994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f4fff7fe640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000108e40 0x7f500019e870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800eaa0 tx=0x7f4ff800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f4fe7fff640 1 -- 172.21.15.44:0/3879337323 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff800cdb0 con 0x7f5000108e40 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f50055f7640 1 -- 172.21.15.44:0/3879337323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500019f330 con 0x7f5000108e40 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f4fe7fff640 1 -- 172.21.15.44:0/3879337323 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff8005020 con 0x7f5000108e40 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f4fe7fff640 1 -- 172.21.15.44:0/3879337323 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80116d0 con 0x7f5000108e40 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.282+0000 7f50055f7640 1 -- 172.21.15.44:0/3879337323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50001a59f0 con 0x7f5000108e40 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.283+0000 7f50055f7640 1 -- 172.21.15.44:0/3879337323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5000063610 con 0x7f5000108e40 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.284+0000 7f4fe7fff640 1 -- 172.21.15.44:0/3879337323 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f4ff80051d0 con 0x7f5000108e40 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.285+0000 7f4fe7fff640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4fe80b3cb0 0x7f4fe80b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.285+0000 7f4ffeffd640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4fe80b3cb0 0x7f4fe80b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.285+0000 7f4fe7fff640 1 -- 172.21.15.44:0/3879337323 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4ff80e45b0 con 0x7f5000108e40 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.285+0000 7f4fe7fff640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038] conn(0x7f4fe80bda60 0x7f4fe80bff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.285+0000 7f4fe7fff640 1 -- 172.21.15.44:0/3879337323 --> [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ff8014ec0 con 0x7f4fe80bda60 2024-08-20T03:01:08.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.285+0000 7f4fe7fff640 1 -- 172.21.15.44:0/3879337323 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f4ff80e4960 con 0x7f5000108e40 2024-08-20T03:01:08.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.286+0000 7f4ffeffd640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4fe80b3cb0 0x7f4fe80b6170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0005fd0 tx=0x7f4ff0005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:08.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.286+0000 7f4ffe7fc640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038] conn(0x7f4fe80bda60 0x7f4fe80bff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:08.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.286+0000 7f4ffe7fc640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038] conn(0x7f4fe80bda60 0x7f4fe80bff00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:08.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.287+0000 7f4fe7fff640 1 -- 172.21.15.44:0/3879337323 <== osd.10 v2:172.21.15.99:6824/3275793038 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ff8014ec0 con 0x7f4fe80bda60 2024-08-20T03:01:08.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.300+0000 7f50055f7640 1 -- 172.21.15.44:0/3879337323 --> [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5000063a90 con 0x7f4fe80bda60 2024-08-20T03:01:08.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.301+0000 7f4fe7fff640 1 -- 172.21.15.44:0/3879337323 <== osd.10 v2:172.21.15.99:6824/3275793038 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5000063a90 con 0x7f4fe80bda60 2024-08-20T03:01:08.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.301+0000 7f50055f7640 1 -- 172.21.15.44:0/3879337323 >> [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038] conn(0x7f4fe80bda60 msgr2=0x7f4fe80bff00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:08.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.301+0000 7f50055f7640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038] conn(0x7f4fe80bda60 0x7f4fe80bff00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:08.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.301+0000 7f50055f7640 1 -- 172.21.15.44:0/3879337323 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4fe80b3cb0 msgr2=0x7f4fe80b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:08.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.301+0000 7f50055f7640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4fe80b3cb0 0x7f4fe80b6170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0005fd0 tx=0x7f4ff0005850 comp rx=0 tx=0).stop 2024-08-20T03:01:08.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.301+0000 7f50055f7640 1 -- 172.21.15.44:0/3879337323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000108e40 msgr2=0x7f500019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:08.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.301+0000 7f50055f7640 1 --2- 172.21.15.44:0/3879337323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5000108e40 0x7f500019e870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800eaa0 tx=0x7f4ff800ef60 comp rx=0 tx=0).stop 2024-08-20T03:01:08.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.301+0000 7f4fff7fe640 1 -- 172.21.15.44:0/3879337323 reap_dead start 2024-08-20T03:01:08.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.302+0000 7f50055f7640 1 -- 172.21.15.44:0/3879337323 shutdown_connections 2024-08-20T03:01:08.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.302+0000 7f50055f7640 1 -- 172.21.15.44:0/3879337323 >> 172.21.15.44:0/3879337323 conn(0x7f5000077e30 msgr2=0x7f5000104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:08.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.302+0000 7f50055f7640 1 -- 172.21.15.44:0/3879337323 shutdown_connections 2024-08-20T03:01:08.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:08.302+0000 7f50055f7640 1 -- 172.21.15.44:0/3879337323 wait complete. 2024-08-20T03:01:09.088 INFO:teuthology.orchestra.run.smithi044.stdout:270582939676 2024-08-20T03:01:09.088 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-08-20T03:01:09.597 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:09 smithi044 ceph-mon[18790]: pgmap v280: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 5.9 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T03:01:09.598 INFO:teuthology.orchestra.run.smithi044.stdout:176093659184 2024-08-20T03:01:09.599 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-08-20T03:01:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:09 smithi165 ceph-mon[22357]: pgmap v280: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 5.9 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T03:01:09.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:09 smithi099 ceph-mon[21984]: pgmap v280: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 5.9 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T03:01:10.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.019+0000 7efc2a77c640 1 -- 172.21.15.44:0/2903134894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc241019d0 msgr2=0x7efc24101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:10.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.019+0000 7efc2a77c640 1 --2- 172.21.15.44:0/2903134894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc241019d0 0x7efc24101db0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efc14009900 tx=0x7efc1402a410 comp rx=0 tx=0).stop 2024-08-20T03:01:10.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.019+0000 7efc2a77c640 1 -- 172.21.15.44:0/2903134894 shutdown_connections 2024-08-20T03:01:10.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.019+0000 7efc2a77c640 1 --2- 172.21.15.44:0/2903134894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc24107d60 0x7efc2410e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:10.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.019+0000 7efc2a77c640 1 --2- 172.21.15.44:0/2903134894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc241073e0 0x7efc24107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:10.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.019+0000 7efc2a77c640 1 --2- 172.21.15.44:0/2903134894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc241019d0 0x7efc24101db0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:10.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.019+0000 7efc2a77c640 1 -- 172.21.15.44:0/2903134894 >> 172.21.15.44:0/2903134894 conn(0x7efc240fd440 msgr2=0x7efc240ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:10.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.019+0000 7efc2a77c640 1 -- 172.21.15.44:0/2903134894 shutdown_connections 2024-08-20T03:01:10.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.019+0000 7efc2a77c640 1 -- 172.21.15.44:0/2903134894 wait complete. 2024-08-20T03:01:10.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.019+0000 7efc2a77c640 1 Processor -- start 2024-08-20T03:01:10.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc2a77c640 1 -- start start 2024-08-20T03:01:10.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc2a77c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc241019d0 0x7efc24198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc2a77c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc241073e0 0x7efc24199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc2a77c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc24107d60 0x7efc2419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc2a77c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efc24112b40 con 0x7efc24107d60 2024-08-20T03:01:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc2a77c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efc241129c0 con 0x7efc241019d0 2024-08-20T03:01:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc2a77c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efc24112cc0 con 0x7efc241073e0 2024-08-20T03:01:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc28d19640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc24107d60 0x7efc2419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc28d19640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc24107d60 0x7efc2419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40326/0 (socket says 172.21.15.44:40326) 2024-08-20T03:01:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc28d19640 1 -- 172.21.15.44:0/3678530006 learned_addr learned my addr 172.21.15.44:0/3678530006 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc23fff640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc241019d0 0x7efc24198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc237fe640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc241073e0 0x7efc24199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc28d19640 1 -- 172.21.15.44:0/3678530006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc241073e0 msgr2=0x7efc24199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:10.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc28d19640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc241073e0 0x7efc24199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc28d19640 1 -- 172.21.15.44:0/3678530006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc241019d0 msgr2=0x7efc24198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.020+0000 7efc28d19640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc241019d0 0x7efc24198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.021+0000 7efc28d19640 1 -- 172.21.15.44:0/3678530006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2419f000 con 0x7efc24107d60 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.021+0000 7efc23fff640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc241019d0 0x7efc24198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.021+0000 7efc237fe640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc241073e0 0x7efc24199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.021+0000 7efc28d19640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc24107d60 0x7efc2419e830 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00b970 tx=0x7efc1c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.021+0000 7efc217fa640 1 -- 172.21.15.44:0/3678530006 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c019070 con 0x7efc24107d60 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.021+0000 7efc2a77c640 1 -- 172.21.15.44:0/3678530006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2419f2f0 con 0x7efc24107d60 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.021+0000 7efc217fa640 1 -- 172.21.15.44:0/3678530006 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc1c014440 con 0x7efc24107d60 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.021+0000 7efc217fa640 1 -- 172.21.15.44:0/3678530006 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0075d0 con 0x7efc24107d60 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.021+0000 7efc2a77c640 1 -- 172.21.15.44:0/3678530006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc241a5a30 con 0x7efc24107d60 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.022+0000 7efc2a77c640 1 -- 172.21.15.44:0/3678530006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc24101db0 con 0x7efc24107d60 2024-08-20T03:01:10.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.023+0000 7efc217fa640 1 -- 172.21.15.44:0/3678530006 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7efc1c01c070 con 0x7efc24107d60 2024-08-20T03:01:10.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.024+0000 7efc217fa640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efc0c0b3cb0 0x7efc0c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:10.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.024+0000 7efc23fff640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efc0c0b3cb0 0x7efc0c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:10.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.024+0000 7efc217fa640 1 -- 172.21.15.44:0/3678530006 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efc1c0e3880 con 0x7efc24107d60 2024-08-20T03:01:10.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.025+0000 7efc23fff640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efc0c0b3cb0 0x7efc0c0b6170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efc14000c00 tx=0x7efc14031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:10.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.027+0000 7efc217fa640 1 -- 172.21.15.44:0/3678530006 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7efc1c0aadd0 con 0x7efc24107d60 2024-08-20T03:01:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:10.152 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:10.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.230+0000 7efc2a77c640 1 -- 172.21.15.44:0/3678530006 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7efc2402e340 con 0x7efc24107d60 2024-08-20T03:01:10.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.231+0000 7efc217fa640 1 -- 172.21.15.44:0/3678530006 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc1c0aa6d0 con 0x7efc24107d60 2024-08-20T03:01:10.234 INFO:teuthology.orchestra.run.smithi044.stdout:38654705740 2024-08-20T03:01:10.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.233+0000 7efc2a77c640 1 -- 172.21.15.44:0/3678530006 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efc0c0b3cb0 msgr2=0x7efc0c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:10.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.233+0000 7efc2a77c640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efc0c0b3cb0 0x7efc0c0b6170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efc14000c00 tx=0x7efc14031400 comp rx=0 tx=0).stop 2024-08-20T03:01:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.233+0000 7efc2a77c640 1 -- 172.21.15.44:0/3678530006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc24107d60 msgr2=0x7efc2419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.233+0000 7efc2a77c640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc24107d60 0x7efc2419e830 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00b970 tx=0x7efc1c00be30 comp rx=0 tx=0).stop 2024-08-20T03:01:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.233+0000 7efc2a77c640 1 -- 172.21.15.44:0/3678530006 shutdown_connections 2024-08-20T03:01:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.233+0000 7efc2a77c640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efc0c0b3cb0 0x7efc0c0b6170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.233+0000 7efc2a77c640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc24107d60 0x7efc2419e830 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.233+0000 7efc2a77c640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc241073e0 0x7efc24199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.233+0000 7efc2a77c640 1 --2- 172.21.15.44:0/3678530006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efc241019d0 0x7efc24198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:10.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.233+0000 7efc2a77c640 1 -- 172.21.15.44:0/3678530006 >> 172.21.15.44:0/3678530006 conn(0x7efc240fd440 msgr2=0x7efc240ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.233+0000 7efc2a77c640 1 -- 172.21.15.44:0/3678530006 shutdown_connections 2024-08-20T03:01:10.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:10.233+0000 7efc2a77c640 1 -- 172.21.15.44:0/3678530006 wait complete. 2024-08-20T03:01:10.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:10 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3678530006' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-20T03:01:10.812 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:10.815 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:10.815 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:10.815 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:10.858 INFO:teuthology.orchestra.run.smithi044.stdout:60129542216 2024-08-20T03:01:10.858 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-08-20T03:01:10.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:10 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3678530006' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-20T03:01:10.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:10 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3678530006' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-20T03:01:11.218 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:11.767 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:11 smithi044 ceph-mon[18790]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 5.3 KiB/s, 0 keys/s, 1 objects/s recovering 2024-08-20T03:01:11.767 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:11 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1748301089' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-20T03:01:11.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:11 smithi165 ceph-mon[22357]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 5.3 KiB/s, 0 keys/s, 1 objects/s recovering 2024-08-20T03:01:11.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:11 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1748301089' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-20T03:01:11.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:11 smithi099 ceph-mon[21984]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 5.3 KiB/s, 0 keys/s, 1 objects/s recovering 2024-08-20T03:01:11.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:11 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1748301089' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-20T03:01:11.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.298+0000 7f78a6dcf640 1 -- 172.21.15.44:0/4048775440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a0102ad0 msgr2=0x7f78a0102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.298+0000 7f78a6dcf640 1 --2- 172.21.15.44:0/4048775440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a0102ad0 0x7f78a0102eb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7890009900 tx=0x7f789002a410 comp rx=0 tx=0).stop 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.298+0000 7f78a6dcf640 1 -- 172.21.15.44:0/4048775440 shutdown_connections 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.298+0000 7f78a6dcf640 1 --2- 172.21.15.44:0/4048775440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a0108e60 0x7f78a010f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.298+0000 7f78a6dcf640 1 --2- 172.21.15.44:0/4048775440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a01084e0 0x7f78a0108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.298+0000 7f78a6dcf640 1 --2- 172.21.15.44:0/4048775440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a0102ad0 0x7f78a0102eb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.298+0000 7f78a6dcf640 1 -- 172.21.15.44:0/4048775440 >> 172.21.15.44:0/4048775440 conn(0x7f78a00fe540 msgr2=0x7f78a0100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.298+0000 7f78a6dcf640 1 -- 172.21.15.44:0/4048775440 shutdown_connections 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.298+0000 7f78a6dcf640 1 -- 172.21.15.44:0/4048775440 wait complete. 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.298+0000 7f78a6dcf640 1 Processor -- start 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a6dcf640 1 -- start start 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a6dcf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a0102ad0 0x7f78a00751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a6dcf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a01084e0 0x7f78a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:11.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a6dcf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a0108e60 0x7f78a0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a6dcf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0112b20 con 0x7f78a0108e60 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a6dcf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78a01129a0 con 0x7f78a01084e0 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a6dcf640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0112ca0 con 0x7f78a0102ad0 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a536c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a0108e60 0x7f78a0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a536c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a0108e60 0x7f78a0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40332/0 (socket says 172.21.15.44:40332) 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a536c640 1 -- 172.21.15.44:0/1748301089 learned_addr learned my addr 172.21.15.44:0/1748301089 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f789ffff640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a01084e0 0x7f78a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a4b6b640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a0102ad0 0x7f78a00751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a536c640 1 -- 172.21.15.44:0/1748301089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a0102ad0 msgr2=0x7f78a00751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a536c640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a0102ad0 0x7f78a00751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a536c640 1 -- 172.21.15.44:0/1748301089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a01084e0 msgr2=0x7f78a0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a536c640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a01084e0 0x7f78a0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:11.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a536c640 1 -- 172.21.15.44:0/1748301089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a01a7430 con 0x7f78a0108e60 2024-08-20T03:01:11.944 INFO:teuthology.orchestra.run.smithi044.stdout:133143986233 2024-08-20T03:01:11.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.299+0000 7f78a536c640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a0108e60 0x7f78a0075c50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f789800cce0 tx=0x7f789800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:11.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.300+0000 7f789dffb640 1 -- 172.21.15.44:0/1748301089 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789800f450 con 0x7f78a0108e60 2024-08-20T03:01:11.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.300+0000 7f789dffb640 1 -- 172.21.15.44:0/1748301089 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f789800faa0 con 0x7f78a0108e60 2024-08-20T03:01:11.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.300+0000 7f78a6dcf640 1 -- 172.21.15.44:0/1748301089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a01a7720 con 0x7f78a0108e60 2024-08-20T03:01:11.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.300+0000 7f789dffb640 1 -- 172.21.15.44:0/1748301089 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898018a80 con 0x7f78a0108e60 2024-08-20T03:01:11.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.300+0000 7f78a6dcf640 1 -- 172.21.15.44:0/1748301089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a01a7c10 con 0x7f78a0108e60 2024-08-20T03:01:11.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.301+0000 7f78a6dcf640 1 -- 172.21.15.44:0/1748301089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a0102eb0 con 0x7f78a0108e60 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.302+0000 7f789dffb640 1 -- 172.21.15.44:0/1748301089 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f789800fc50 con 0x7f78a0108e60 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.303+0000 7f789dffb640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f788c0b3cb0 0x7f788c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.303+0000 7f78a4b6b640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f788c0b3cb0 0x7f788c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.303+0000 7f789dffb640 1 -- 172.21.15.44:0/1748301089 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f78980e4290 con 0x7f78a0108e60 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.303+0000 7f78a4b6b640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f788c0b3cb0 0x7f788c0b6170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7890031440 tx=0x7f7890005e80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.307+0000 7f789dffb640 1 -- 172.21.15.44:0/1748301089 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7898014030 con 0x7f78a0108e60 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.499+0000 7f78a6dcf640 1 -- 172.21.15.44:0/1748301089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f78a010a6b0 con 0x7f78a0108e60 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.499+0000 7f789dffb640 1 -- 172.21.15.44:0/1748301089 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f78980ab6d0 con 0x7f78a0108e60 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.501+0000 7f78a6dcf640 1 -- 172.21.15.44:0/1748301089 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f788c0b3cb0 msgr2=0x7f788c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.501+0000 7f78a6dcf640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f788c0b3cb0 0x7f788c0b6170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7890031440 tx=0x7f7890005e80 comp rx=0 tx=0).stop 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.501+0000 7f78a6dcf640 1 -- 172.21.15.44:0/1748301089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a0108e60 msgr2=0x7f78a0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.501+0000 7f78a6dcf640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a0108e60 0x7f78a0075c50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f789800cce0 tx=0x7f789800a620 comp rx=0 tx=0).stop 2024-08-20T03:01:11.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.501+0000 7f78a6dcf640 1 -- 172.21.15.44:0/1748301089 shutdown_connections 2024-08-20T03:01:11.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.501+0000 7f78a6dcf640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f788c0b3cb0 0x7f788c0b6170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:11.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.501+0000 7f78a6dcf640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a0108e60 0x7f78a0075c50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:11.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.501+0000 7f78a6dcf640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78a01084e0 0x7f78a0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:11.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.501+0000 7f78a6dcf640 1 --2- 172.21.15.44:0/1748301089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a0102ad0 0x7f78a00751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:11.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.501+0000 7f78a6dcf640 1 -- 172.21.15.44:0/1748301089 >> 172.21.15.44:0/1748301089 conn(0x7f78a00fe540 msgr2=0x7f78a0100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:11.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.502+0000 7f78a6dcf640 1 -- 172.21.15.44:0/1748301089 shutdown_connections 2024-08-20T03:01:11.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.502+0000 7f78a6dcf640 1 -- 172.21.15.44:0/1748301089 wait complete. 2024-08-20T03:01:12.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.920+0000 7fe87229a640 1 -- 172.21.15.44:0/870202668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe86c102ad0 msgr2=0x7fe86c102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:12.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.920+0000 7fe87229a640 1 --2- 172.21.15.44:0/870202668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe86c102ad0 0x7fe86c102eb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009900 tx=0x7fe85c02a410 comp rx=0 tx=0).stop 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.920+0000 7fe87229a640 1 -- 172.21.15.44:0/870202668 shutdown_connections 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.920+0000 7fe87229a640 1 --2- 172.21.15.44:0/870202668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe86c108e60 0x7fe86c10f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.920+0000 7fe87229a640 1 --2- 172.21.15.44:0/870202668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe86c1084e0 0x7fe86c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.920+0000 7fe87229a640 1 --2- 172.21.15.44:0/870202668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe86c102ad0 0x7fe86c102eb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.920+0000 7fe87229a640 1 -- 172.21.15.44:0/870202668 >> 172.21.15.44:0/870202668 conn(0x7fe86c0fe540 msgr2=0x7fe86c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.920+0000 7fe87229a640 1 -- 172.21.15.44:0/870202668 shutdown_connections 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.920+0000 7fe87229a640 1 -- 172.21.15.44:0/870202668 wait complete. 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe87229a640 1 Processor -- start 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe87229a640 1 -- start start 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe87229a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe86c102ad0 0x7fe86c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe87229a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe86c1084e0 0x7fe86c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe87229a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe86c108e60 0x7fe86c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe87229a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c112b20 con 0x7fe86c102ad0 2024-08-20T03:01:12.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe87229a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c1129a0 con 0x7fe86c1084e0 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe87229a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c112ca0 con 0x7fe86c108e60 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe86bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe86c102ad0 0x7fe86c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe86bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe86c102ad0 0x7fe86c198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40346/0 (socket says 172.21.15.44:40346) 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe86bfff640 1 -- 172.21.15.44:0/3759477268 learned_addr learned my addr 172.21.15.44:0/3759477268 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe863fff640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe86c1084e0 0x7fe86c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe86bfff640 1 -- 172.21.15.44:0/3759477268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe86c108e60 msgr2=0x7fe86c19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe86bfff640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe86c108e60 0x7fe86c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe86bfff640 1 -- 172.21.15.44:0/3759477268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe86c1084e0 msgr2=0x7fe86c199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe86bfff640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe86c1084e0 0x7fe86c199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe86bfff640 1 -- 172.21.15.44:0/3759477268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c19efc0 con 0x7fe86c102ad0 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.921+0000 7fe863fff640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe86c1084e0 0x7fe86c199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.922+0000 7fe86bfff640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe86c102ad0 0x7fe86c198f10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe85c006010 tx=0x7fe85c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.922+0000 7fe869ffb640 1 -- 172.21.15.44:0/3759477268 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c03f030 con 0x7fe86c102ad0 2024-08-20T03:01:12.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.922+0000 7fe87229a640 1 -- 172.21.15.44:0/3759477268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86c19f250 con 0x7fe86c102ad0 2024-08-20T03:01:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.922+0000 7fe869ffb640 1 -- 172.21.15.44:0/3759477268 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe85c003800 con 0x7fe86c102ad0 2024-08-20T03:01:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.922+0000 7fe869ffb640 1 -- 172.21.15.44:0/3759477268 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe85c035070 con 0x7fe86c102ad0 2024-08-20T03:01:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.922+0000 7fe87229a640 1 -- 172.21.15.44:0/3759477268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86c1a5970 con 0x7fe86c102ad0 2024-08-20T03:01:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.923+0000 7fe87229a640 1 -- 172.21.15.44:0/3759477268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe86c102eb0 con 0x7fe86c102ad0 2024-08-20T03:01:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.924+0000 7fe869ffb640 1 -- 172.21.15.44:0/3759477268 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fe85c003cb0 con 0x7fe86c102ad0 2024-08-20T03:01:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.925+0000 7fe869ffb640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe8580b3cb0 0x7fe8580b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.925+0000 7fe863fff640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe8580b3cb0 0x7fe8580b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.925+0000 7fe869ffb640 1 -- 172.21.15.44:0/3759477268 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe85c104b50 con 0x7fe86c102ad0 2024-08-20T03:01:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.925+0000 7fe863fff640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe8580b3cb0 0x7fe8580b6170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe86c101f30 tx=0x7fe854005d90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:11.928+0000 7fe869ffb640 1 -- 172.21.15.44:0/3759477268 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe85c0cbf90 con 0x7fe86c102ad0 2024-08-20T03:01:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.113+0000 7fe87229a640 1 -- 172.21.15.44:0/3759477268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe86c1a5cf0 con 0x7fe86c102ad0 2024-08-20T03:01:12.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.113+0000 7fe869ffb640 1 -- 172.21.15.44:0/3759477268 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe85c0cb890 con 0x7fe86c102ad0 2024-08-20T03:01:12.299 INFO:teuthology.orchestra.run.smithi044.stdout:81604378692 2024-08-20T03:01:12.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.115+0000 7fe87229a640 1 -- 172.21.15.44:0/3759477268 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe8580b3cb0 msgr2=0x7fe8580b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:12.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.115+0000 7fe87229a640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe8580b3cb0 0x7fe8580b6170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe86c101f30 tx=0x7fe854005d90 comp rx=0 tx=0).stop 2024-08-20T03:01:12.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.115+0000 7fe87229a640 1 -- 172.21.15.44:0/3759477268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe86c102ad0 msgr2=0x7fe86c198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:12.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.115+0000 7fe87229a640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe86c102ad0 0x7fe86c198f10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe85c006010 tx=0x7fe85c0042f0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.115+0000 7fe87229a640 1 -- 172.21.15.44:0/3759477268 shutdown_connections 2024-08-20T03:01:12.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.115+0000 7fe87229a640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe8580b3cb0 0x7fe8580b6170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.115+0000 7fe87229a640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe86c108e60 0x7fe86c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.115+0000 7fe87229a640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe86c1084e0 0x7fe86c199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.115+0000 7fe87229a640 1 --2- 172.21.15.44:0/3759477268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe86c102ad0 0x7fe86c198f10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.115+0000 7fe87229a640 1 -- 172.21.15.44:0/3759477268 >> 172.21.15.44:0/3759477268 conn(0x7fe86c0fe540 msgr2=0x7fe86c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:12.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.115+0000 7fe87229a640 1 -- 172.21.15.44:0/3759477268 shutdown_connections 2024-08-20T03:01:12.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.115+0000 7fe87229a640 1 -- 172.21.15.44:0/3759477268 wait complete. 2024-08-20T03:01:12.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.467+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/487479754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ecc1019b0 msgr2=0x7f0ecc101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.467+0000 7f0ed3dfd640 1 --2- 172.21.15.44:0/487479754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ecc1019b0 0x7f0ecc101d90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009900 tx=0x7f0ec802a410 comp rx=0 tx=0).stop 2024-08-20T03:01:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.467+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/487479754 shutdown_connections 2024-08-20T03:01:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.467+0000 7f0ed3dfd640 1 --2- 172.21.15.44:0/487479754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ecc107d40 0x7f0ecc10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.467+0000 7f0ed3dfd640 1 --2- 172.21.15.44:0/487479754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ecc1073c0 0x7f0ecc107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.467+0000 7f0ed3dfd640 1 --2- 172.21.15.44:0/487479754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ecc1019b0 0x7f0ecc101d90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.467+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/487479754 >> 172.21.15.44:0/487479754 conn(0x7f0ecc0fd440 msgr2=0x7f0ecc0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.467+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/487479754 shutdown_connections 2024-08-20T03:01:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.467+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/487479754 wait complete. 2024-08-20T03:01:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed3dfd640 1 Processor -- start 2024-08-20T03:01:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed3dfd640 1 -- start start 2024-08-20T03:01:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed3dfd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ecc1019b0 0x7f0ecc1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:12.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed3dfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ecc1073c0 0x7f0ecc198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:12.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed3dfd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ecc107d40 0x7f0ecc19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:12.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed3dfd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc111a00 con 0x7f0ecc1019b0 2024-08-20T03:01:12.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed3dfd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc111880 con 0x7f0ecc1073c0 2024-08-20T03:01:12.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed3dfd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0ecc111b80 con 0x7f0ecc107d40 2024-08-20T03:01:12.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed1b99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ecc1019b0 0x7f0ecc1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:12.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed1b99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ecc1019b0 0x7f0ecc1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40370/0 (socket says 172.21.15.44:40370) 2024-08-20T03:01:12.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed1b99640 1 -- 172.21.15.44:0/109722693 learned_addr learned my addr 172.21.15.44:0/109722693 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:12.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed1398640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ecc1073c0 0x7f0ecc198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:12.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed239a640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ecc107d40 0x7f0ecc19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:12.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed1b99640 1 -- 172.21.15.44:0/109722693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ecc107d40 msgr2=0x7f0ecc19d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:12.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed1b99640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ecc107d40 0x7f0ecc19d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed1b99640 1 -- 172.21.15.44:0/109722693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ecc1073c0 msgr2=0x7f0ecc198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:12.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed1b99640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ecc1073c0 0x7f0ecc198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.468+0000 7f0ed1b99640 1 -- 172.21.15.44:0/109722693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ecc19e1a0 con 0x7f0ecc1019b0 2024-08-20T03:01:12.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.469+0000 7f0ed1b99640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ecc1019b0 0x7f0ecc1980f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8006010 tx=0x7f0ec8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:12.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.469+0000 7f0ec2ffd640 1 -- 172.21.15.44:0/109722693 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8035050 con 0x7f0ecc1019b0 2024-08-20T03:01:12.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.469+0000 7f0ec2ffd640 1 -- 172.21.15.44:0/109722693 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ec80035f0 con 0x7f0ecc1019b0 2024-08-20T03:01:12.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.469+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/109722693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ecc19e430 con 0x7f0ecc1019b0 2024-08-20T03:01:12.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.469+0000 7f0ec2ffd640 1 -- 172.21.15.44:0/109722693 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8032260 con 0x7f0ecc1019b0 2024-08-20T03:01:12.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.469+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/109722693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ecc06b190 con 0x7f0ecc1019b0 2024-08-20T03:01:12.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.470+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/109722693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ecc102c60 con 0x7f0ecc1019b0 2024-08-20T03:01:12.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.471+0000 7f0ec2ffd640 1 -- 172.21.15.44:0/109722693 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f0ec8048060 con 0x7f0ecc1019b0 2024-08-20T03:01:12.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.472+0000 7f0ec2ffd640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0eb80b3d30 0x7f0eb80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:12.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.472+0000 7f0ed1398640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0eb80b3d30 0x7f0eb80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:12.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.472+0000 7f0ec2ffd640 1 -- 172.21.15.44:0/109722693 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0ec81059c0 con 0x7f0ecc1019b0 2024-08-20T03:01:12.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.472+0000 7f0ed1398640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0eb80b3d30 0x7f0eb80b61f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc106eb0 tx=0x7f0ebc00a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:12.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.476+0000 7f0ec2ffd640 1 -- 172.21.15.44:0/109722693 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0ec80cce00 con 0x7f0ecc1019b0 2024-08-20T03:01:12.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.666+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/109722693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0ecc06af50 con 0x7f0ecc1019b0 2024-08-20T03:01:12.670 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:12 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3759477268' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-20T03:01:12.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.666+0000 7f0ec2ffd640 1 -- 172.21.15.44:0/109722693 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0ec80c6e20 con 0x7f0ecc1019b0 2024-08-20T03:01:12.670 INFO:teuthology.orchestra.run.smithi044.stdout:111669149759 2024-08-20T03:01:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.668+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/109722693 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0eb80b3d30 msgr2=0x7f0eb80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.668+0000 7f0ed3dfd640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0eb80b3d30 0x7f0eb80b61f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc106eb0 tx=0x7f0ebc00a2d0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.668+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/109722693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ecc1019b0 msgr2=0x7f0ecc1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.668+0000 7f0ed3dfd640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ecc1019b0 0x7f0ecc1980f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8006010 tx=0x7f0ec8004270 comp rx=0 tx=0).stop 2024-08-20T03:01:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.668+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/109722693 shutdown_connections 2024-08-20T03:01:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.668+0000 7f0ed3dfd640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0eb80b3d30 0x7f0eb80b61f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.668+0000 7f0ed3dfd640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ecc107d40 0x7f0ecc19d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.668+0000 7f0ed3dfd640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ecc1073c0 0x7f0ecc198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.669+0000 7f0ed3dfd640 1 --2- 172.21.15.44:0/109722693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ecc1019b0 0x7f0ecc1980f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.669+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/109722693 >> 172.21.15.44:0/109722693 conn(0x7f0ecc0fd440 msgr2=0x7f0ecc0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.669+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/109722693 shutdown_connections 2024-08-20T03:01:12.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:12.669+0000 7f0ed3dfd640 1 -- 172.21.15.44:0/109722693 wait complete. 2024-08-20T03:01:12.785 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:12.786 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:12.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:12 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3759477268' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-20T03:01:12.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:12 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3759477268' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-20T03:01:13.423 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149757 got 111669149759 for osd.3 2024-08-20T03:01:13.423 DEBUG:teuthology.parallel:result is None 2024-08-20T03:01:13.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:13 smithi044 ceph-mon[18790]: pgmap v282: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 4.9 KiB/s, 0 keys/s, 1 objects/s recovering 2024-08-20T03:01:13.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:13 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/109722693' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-20T03:01:13.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:13 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:01:13.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:13 smithi165 ceph-mon[22357]: pgmap v282: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 4.9 KiB/s, 0 keys/s, 1 objects/s recovering 2024-08-20T03:01:13.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:13 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/109722693' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-20T03:01:13.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:13 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:01:13.900 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:13 smithi099 ceph-mon[21984]: pgmap v282: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 4.9 KiB/s, 0 keys/s, 1 objects/s recovering 2024-08-20T03:01:13.900 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:13 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/109722693' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-20T03:01:13.900 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:13 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:01:14.378 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378691 got 81604378692 for osd.2 2024-08-20T03:01:14.378 DEBUG:teuthology.parallel:result is None 2024-08-20T03:01:14.466 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:14.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.866+0000 7f1d79e3f640 1 -- 172.21.15.44:0/3954699325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d74104240 msgr2=0x7f1d7410c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:14.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.866+0000 7f1d79e3f640 1 --2- 172.21.15.44:0/3954699325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d74104240 0x7f1d7410c820 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009900 tx=0x7f1d6802a410 comp rx=0 tx=0).stop 2024-08-20T03:01:14.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.866+0000 7f1d79e3f640 1 -- 172.21.15.44:0/3954699325 shutdown_connections 2024-08-20T03:01:14.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.866+0000 7f1d79e3f640 1 --2- 172.21.15.44:0/3954699325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7410cdf0 0x7f1d7410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:14.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.866+0000 7f1d79e3f640 1 --2- 172.21.15.44:0/3954699325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d74104240 0x7f1d7410c820 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:14.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.866+0000 7f1d79e3f640 1 --2- 172.21.15.44:0/3954699325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d74103840 0x7f1d74103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:14.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.866+0000 7f1d79e3f640 1 -- 172.21.15.44:0/3954699325 >> 172.21.15.44:0/3954699325 conn(0x7f1d740fd430 msgr2=0x7f1d740ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:14.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.866+0000 7f1d79e3f640 1 -- 172.21.15.44:0/3954699325 shutdown_connections 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.866+0000 7f1d79e3f640 1 -- 172.21.15.44:0/3954699325 wait complete. 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d79e3f640 1 Processor -- start 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d79e3f640 1 -- start start 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d79e3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d74103840 0x7f1d74198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d79e3f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d74104240 0x7f1d741994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d79e3f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7410cdf0 0x7f1d7419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d79e3f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74112a70 con 0x7f1d74103840 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d79e3f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741128f0 con 0x7f1d74104240 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d79e3f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74112bf0 con 0x7f1d7410cdf0 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d737fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d74103840 0x7f1d74198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d737fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d74103840 0x7f1d74198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40386/0 (socket says 172.21.15.44:40386) 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d737fe640 1 -- 172.21.15.44:0/4250765515 learned_addr learned my addr 172.21.15.44:0/4250765515 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d72ffd640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d74104240 0x7f1d741994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:14.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d73fff640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7410cdf0 0x7f1d7419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d737fe640 1 -- 172.21.15.44:0/4250765515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7410cdf0 msgr2=0x7f1d7419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d737fe640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7410cdf0 0x7f1d7419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d737fe640 1 -- 172.21.15.44:0/4250765515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d74104240 msgr2=0x7f1d741994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.867+0000 7f1d737fe640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d74104240 0x7f1d741994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.868+0000 7f1d737fe640 1 -- 172.21.15.44:0/4250765515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7419f030 con 0x7f1d74103840 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.868+0000 7f1d73fff640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7410cdf0 0x7f1d7419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.868+0000 7f1d737fe640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d74103840 0x7f1d74198f80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400ebc0 tx=0x7f1d6400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.868+0000 7f1d70ff9640 1 -- 172.21.15.44:0/4250765515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64010630 con 0x7f1d74103840 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.868+0000 7f1d70ff9640 1 -- 172.21.15.44:0/4250765515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d64010c80 con 0x7f1d74103840 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.868+0000 7f1d79e3f640 1 -- 172.21.15.44:0/4250765515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7419f320 con 0x7f1d74103840 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.868+0000 7f1d70ff9640 1 -- 172.21.15.44:0/4250765515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64018b60 con 0x7f1d74103840 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.868+0000 7f1d79e3f640 1 -- 172.21.15.44:0/4250765515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d741a59e0 con 0x7f1d74103840 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.869+0000 7f1d79e3f640 1 -- 172.21.15.44:0/4250765515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d74103c20 con 0x7f1d74103840 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.870+0000 7f1d70ff9640 1 -- 172.21.15.44:0/4250765515 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f1d64018d00 con 0x7f1d74103840 2024-08-20T03:01:14.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.871+0000 7f1d70ff9640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1d5c0b3cb0 0x7f1d5c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:14.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.871+0000 7f1d72ffd640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1d5c0b3cb0 0x7f1d5c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:14.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.871+0000 7f1d70ff9640 1 -- 172.21.15.44:0/4250765515 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1d640e49a0 con 0x7f1d74103840 2024-08-20T03:01:14.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.872+0000 7f1d72ffd640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1d5c0b3cb0 0x7f1d5c0b6170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d68031440 tx=0x7f1d68005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:14.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:14.874+0000 7f1d70ff9640 1 -- 172.21.15.44:0/4250765515 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f1d640abde0 con 0x7f1d74103840 2024-08-20T03:01:15.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.061+0000 7f1d79e3f640 1 -- 172.21.15.44:0/4250765515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1d7419a0b0 con 0x7f1d74103840 2024-08-20T03:01:15.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.061+0000 7f1d70ff9640 1 -- 172.21.15.44:0/4250765515 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1d640ab6e0 con 0x7f1d74103840 2024-08-20T03:01:15.064 INFO:teuthology.orchestra.run.smithi044.stdout:154618822710 2024-08-20T03:01:15.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.063+0000 7f1d79e3f640 1 -- 172.21.15.44:0/4250765515 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1d5c0b3cb0 msgr2=0x7f1d5c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:15.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.063+0000 7f1d79e3f640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1d5c0b3cb0 0x7f1d5c0b6170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1d68031440 tx=0x7f1d68005b50 comp rx=0 tx=0).stop 2024-08-20T03:01:15.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.063+0000 7f1d79e3f640 1 -- 172.21.15.44:0/4250765515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d74103840 msgr2=0x7f1d74198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:15.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.063+0000 7f1d79e3f640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d74103840 0x7f1d74198f80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400ebc0 tx=0x7f1d6400c6a0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.063+0000 7f1d79e3f640 1 -- 172.21.15.44:0/4250765515 shutdown_connections 2024-08-20T03:01:15.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.063+0000 7f1d79e3f640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1d5c0b3cb0 0x7f1d5c0b6170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.063+0000 7f1d79e3f640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7410cdf0 0x7f1d7419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.063+0000 7f1d79e3f640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d74104240 0x7f1d741994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.063+0000 7f1d79e3f640 1 --2- 172.21.15.44:0/4250765515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d74103840 0x7f1d74198f80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.063+0000 7f1d79e3f640 1 -- 172.21.15.44:0/4250765515 >> 172.21.15.44:0/4250765515 conn(0x7f1d740fd430 msgr2=0x7f1d740fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:15.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.063+0000 7f1d79e3f640 1 -- 172.21.15.44:0/4250765515 shutdown_connections 2024-08-20T03:01:15.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.063+0000 7f1d79e3f640 1 -- 172.21.15.44:0/4250765515 wait complete. 2024-08-20T03:01:15.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.077+0000 7f95b17f3640 1 -- 172.21.15.44:0/734332429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac1073a0 msgr2=0x7f95ac1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:15.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.077+0000 7f95b17f3640 1 --2- 172.21.15.44:0/734332429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac1073a0 0x7f95ac1077e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9594009900 tx=0x7f959402a410 comp rx=0 tx=0).stop 2024-08-20T03:01:15.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.077+0000 7f95b17f3640 1 -- 172.21.15.44:0/734332429 shutdown_connections 2024-08-20T03:01:15.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.077+0000 7f95b17f3640 1 --2- 172.21.15.44:0/734332429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95ac107d20 0x7f95ac10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.077+0000 7f95b17f3640 1 --2- 172.21.15.44:0/734332429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac1073a0 0x7f95ac1077e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.077+0000 7f95b17f3640 1 --2- 172.21.15.44:0/734332429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95ac101990 0x7f95ac101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.077+0000 7f95b17f3640 1 -- 172.21.15.44:0/734332429 >> 172.21.15.44:0/734332429 conn(0x7f95ac0fd400 msgr2=0x7f95ac0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.077+0000 7f95b17f3640 1 -- 172.21.15.44:0/734332429 shutdown_connections 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.077+0000 7f95b17f3640 1 -- 172.21.15.44:0/734332429 wait complete. 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.078+0000 7f95b17f3640 1 Processor -- start 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.078+0000 7f95b17f3640 1 -- start start 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95b17f3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac101990 0x7f95ac198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95b17f3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95ac1073a0 0x7f95ac199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95b17f3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95ac107d20 0x7f95ac19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95aaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac101990 0x7f95ac198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95b17f3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1119a0 con 0x7f95ac101990 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95b17f3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac111820 con 0x7f95ac1073a0 2024-08-20T03:01:15.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95aaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac101990 0x7f95ac198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40402/0 (socket says 172.21.15.44:40402) 2024-08-20T03:01:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95b17f3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac111b20 con 0x7f95ac107d20 2024-08-20T03:01:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95aaffd640 1 -- 172.21.15.44:0/16601468 learned_addr learned my addr 172.21.15.44:0/16601468 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95aa7fc640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95ac1073a0 0x7f95ac199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95ab7fe640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95ac107d20 0x7f95ac19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95aaffd640 1 -- 172.21.15.44:0/16601468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95ac107d20 msgr2=0x7f95ac19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95aaffd640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95ac107d20 0x7f95ac19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95aaffd640 1 -- 172.21.15.44:0/16601468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95ac1073a0 msgr2=0x7f95ac199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95aaffd640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95ac1073a0 0x7f95ac199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95aaffd640 1 -- 172.21.15.44:0/16601468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ac19f070 con 0x7f95ac101990 2024-08-20T03:01:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95aa7fc640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95ac1073a0 0x7f95ac199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:01:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.079+0000 7f95aaffd640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac101990 0x7f95ac198fc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f959c00ea60 tx=0x7f959c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:15.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.080+0000 7f95a3fff640 1 -- 172.21.15.44:0/16601468 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00cde0 con 0x7f95ac101990 2024-08-20T03:01:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.080+0000 7f95b17f3640 1 -- 172.21.15.44:0/16601468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95ac19f360 con 0x7f95ac101990 2024-08-20T03:01:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.080+0000 7f95b17f3640 1 -- 172.21.15.44:0/16601468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95ac1a5bf0 con 0x7f95ac101990 2024-08-20T03:01:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.080+0000 7f95a3fff640 1 -- 172.21.15.44:0/16601468 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959c0035f0 con 0x7f95ac101990 2024-08-20T03:01:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.080+0000 7f95a3fff640 1 -- 172.21.15.44:0/16601468 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c018690 con 0x7f95ac101990 2024-08-20T03:01:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.081+0000 7f95b17f3640 1 -- 172.21.15.44:0/16601468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f958c005290 con 0x7f95ac101990 2024-08-20T03:01:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.083+0000 7f95a3fff640 1 -- 172.21.15.44:0/16601468 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f959c004020 con 0x7f95ac101990 2024-08-20T03:01:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.084+0000 7f95a3fff640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95980b3d30 0x7f95980b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.084+0000 7f95a3fff640 1 -- 172.21.15.44:0/16601468 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f959c0e3fa0 con 0x7f95ac101990 2024-08-20T03:01:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.084+0000 7f95aa7fc640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95980b3d30 0x7f95980b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.085+0000 7f95aa7fc640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95980b3d30 0x7f95980b61f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9594031440 tx=0x7f9594005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:15.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.086+0000 7f95a3fff640 1 -- 172.21.15.44:0/16601468 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f959c0ab470 con 0x7f95ac101990 2024-08-20T03:01:15.278 INFO:teuthology.orchestra.run.smithi044.stdout:223338299431 2024-08-20T03:01:15.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.275+0000 7f95b17f3640 1 -- 172.21.15.44:0/16601468 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f958c005850 con 0x7f95ac101990 2024-08-20T03:01:15.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.275+0000 7f95a3fff640 1 -- 172.21.15.44:0/16601468 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f959c0aad70 con 0x7f95ac101990 2024-08-20T03:01:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.277+0000 7f95b17f3640 1 -- 172.21.15.44:0/16601468 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95980b3d30 msgr2=0x7f95980b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.277+0000 7f95b17f3640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95980b3d30 0x7f95980b61f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9594031440 tx=0x7f9594005b50 comp rx=0 tx=0).stop 2024-08-20T03:01:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.277+0000 7f95b17f3640 1 -- 172.21.15.44:0/16601468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac101990 msgr2=0x7f95ac198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:15.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.277+0000 7f95b17f3640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac101990 0x7f95ac198fc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f959c00ea60 tx=0x7f959c00ef20 comp rx=0 tx=0).stop 2024-08-20T03:01:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.277+0000 7f95b17f3640 1 -- 172.21.15.44:0/16601468 shutdown_connections 2024-08-20T03:01:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.277+0000 7f95b17f3640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95980b3d30 0x7f95980b61f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.277+0000 7f95b17f3640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95ac107d20 0x7f95ac19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.277+0000 7f95b17f3640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95ac1073a0 0x7f95ac199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.277+0000 7f95b17f3640 1 --2- 172.21.15.44:0/16601468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95ac101990 0x7f95ac198fc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.277+0000 7f95b17f3640 1 -- 172.21.15.44:0/16601468 >> 172.21.15.44:0/16601468 conn(0x7f95ac0fd400 msgr2=0x7f95ac1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.277+0000 7f95b17f3640 1 -- 172.21.15.44:0/16601468 shutdown_connections 2024-08-20T03:01:15.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:15.277+0000 7f95b17f3640 1 -- 172.21.15.44:0/16601468 wait complete. 2024-08-20T03:01:15.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:15 smithi099 ceph-mon[21984]: pgmap v283: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:15.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:15 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:15.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:15 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:15.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:15 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:15.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:15 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:15.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:15 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/4250765515' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-20T03:01:15.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:15 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/16601468' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-20T03:01:16.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:15 smithi044 ceph-mon[18790]: pgmap v283: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:16.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:15 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:16.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:15 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:16.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:15 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:16.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:15 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:16.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:15 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/4250765515' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-20T03:01:16.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:15 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/16601468' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-20T03:01:16.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:15 smithi165 ceph-mon[22357]: pgmap v283: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:16.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:15 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:16.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:15 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:16.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:15 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:16.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:15 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:16.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:15 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/4250765515' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-20T03:01:16.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:15 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/16601468' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-20T03:01:16.253 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705738 got 38654705740 for osd.0 2024-08-20T03:01:16.253 DEBUG:teuthology.parallel:result is None 2024-08-20T03:01:16.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.427+0000 7fec0a7b6640 1 -- 172.21.15.44:0/2004961175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec04102ad0 msgr2=0x7fec04102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:16.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.427+0000 7fec0a7b6640 1 --2- 172.21.15.44:0/2004961175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec04102ad0 0x7fec04102eb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7febf4009900 tx=0x7febf402a410 comp rx=0 tx=0).stop 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.427+0000 7fec0a7b6640 1 -- 172.21.15.44:0/2004961175 shutdown_connections 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.427+0000 7fec0a7b6640 1 --2- 172.21.15.44:0/2004961175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec04108e60 0x7fec0410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.427+0000 7fec0a7b6640 1 --2- 172.21.15.44:0/2004961175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec041084e0 0x7fec04108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.427+0000 7fec0a7b6640 1 --2- 172.21.15.44:0/2004961175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec04102ad0 0x7fec04102eb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.427+0000 7fec0a7b6640 1 -- 172.21.15.44:0/2004961175 >> 172.21.15.44:0/2004961175 conn(0x7fec040fe540 msgr2=0x7fec04100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.427+0000 7fec0a7b6640 1 -- 172.21.15.44:0/2004961175 shutdown_connections 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.427+0000 7fec0a7b6640 1 -- 172.21.15.44:0/2004961175 wait complete. 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.428+0000 7fec0a7b6640 1 Processor -- start 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.428+0000 7fec0a7b6640 1 -- start start 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.428+0000 7fec0a7b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec04102ad0 0x7fec04198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.428+0000 7fec0a7b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec041084e0 0x7fec041994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.428+0000 7fec0a7b6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec04108e60 0x7fec0419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:16.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.428+0000 7fec0a7b6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fec04112ae0 con 0x7fec041084e0 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.428+0000 7fec0a7b6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fec04112960 con 0x7fec04102ad0 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.428+0000 7fec0a7b6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fec04112c60 con 0x7fec04108e60 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.428+0000 7fec037fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec041084e0 0x7fec041994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.428+0000 7fec037fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec041084e0 0x7fec041994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40426/0 (socket says 172.21.15.44:40426) 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.428+0000 7fec037fe640 1 -- 172.21.15.44:0/623473708 learned_addr learned my addr 172.21.15.44:0/623473708 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.428+0000 7fec08d53640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec04108e60 0x7fec0419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.429+0000 7fec03fff640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec04102ad0 0x7fec04198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.429+0000 7fec037fe640 1 -- 172.21.15.44:0/623473708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec04108e60 msgr2=0x7fec0419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.429+0000 7fec037fe640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec04108e60 0x7fec0419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.429+0000 7fec037fe640 1 -- 172.21.15.44:0/623473708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec04102ad0 msgr2=0x7fec04198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.429+0000 7fec037fe640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec04102ad0 0x7fec04198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.429+0000 7fec037fe640 1 -- 172.21.15.44:0/623473708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0419f040 con 0x7fec041084e0 2024-08-20T03:01:16.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.429+0000 7fec037fe640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec041084e0 0x7fec041994d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7febf800c890 tx=0x7febf800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:16.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.429+0000 7fec017fa640 1 -- 172.21.15.44:0/623473708 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8004220 con 0x7fec041084e0 2024-08-20T03:01:16.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.429+0000 7fec0a7b6640 1 -- 172.21.15.44:0/623473708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0419f330 con 0x7fec041084e0 2024-08-20T03:01:16.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.429+0000 7fec017fa640 1 -- 172.21.15.44:0/623473708 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf80043c0 con 0x7fec041084e0 2024-08-20T03:01:16.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.429+0000 7fec017fa640 1 -- 172.21.15.44:0/623473708 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8004a00 con 0x7fec041084e0 2024-08-20T03:01:16.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.429+0000 7fec0a7b6640 1 -- 172.21.15.44:0/623473708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041a59f0 con 0x7fec041084e0 2024-08-20T03:01:16.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.430+0000 7fec0a7b6640 1 -- 172.21.15.44:0/623473708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec04102eb0 con 0x7fec041084e0 2024-08-20T03:01:16.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.432+0000 7fec017fa640 1 -- 172.21.15.44:0/623473708 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7febf80079b0 con 0x7fec041084e0 2024-08-20T03:01:16.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.432+0000 7fec017fa640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7febec0b3cb0 0x7febec0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:16.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.432+0000 7fec03fff640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7febec0b3cb0 0x7febec0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:16.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.432+0000 7fec017fa640 1 -- 172.21.15.44:0/623473708 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7febf80e4060 con 0x7fec041084e0 2024-08-20T03:01:16.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.433+0000 7fec03fff640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7febec0b3cb0 0x7febec0b6170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7febf4031440 tx=0x7febf40095f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:16.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.435+0000 7fec017fa640 1 -- 172.21.15.44:0/623473708 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7febf8014030 con 0x7fec041084e0 2024-08-20T03:01:16.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.627+0000 7fec0a7b6640 1 -- 172.21.15.44:0/623473708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fec0419a0c0 con 0x7fec041084e0 2024-08-20T03:01:16.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.628+0000 7fec017fa640 1 -- 172.21.15.44:0/623473708 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7febf80ab440 con 0x7fec041084e0 2024-08-20T03:01:16.631 INFO:teuthology.orchestra.run.smithi044.stdout:270582939677 2024-08-20T03:01:16.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.630+0000 7fec0a7b6640 1 -- 172.21.15.44:0/623473708 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7febec0b3cb0 msgr2=0x7febec0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:16.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.630+0000 7fec0a7b6640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7febec0b3cb0 0x7febec0b6170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7febf4031440 tx=0x7febf40095f0 comp rx=0 tx=0).stop 2024-08-20T03:01:16.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.630+0000 7fec0a7b6640 1 -- 172.21.15.44:0/623473708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec041084e0 msgr2=0x7fec041994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:16.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.630+0000 7fec0a7b6640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec041084e0 0x7fec041994d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7febf800c890 tx=0x7febf800cd50 comp rx=0 tx=0).stop 2024-08-20T03:01:16.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.630+0000 7fec0a7b6640 1 -- 172.21.15.44:0/623473708 shutdown_connections 2024-08-20T03:01:16.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.630+0000 7fec0a7b6640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7febec0b3cb0 0x7febec0b6170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:16.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.630+0000 7fec0a7b6640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec04108e60 0x7fec0419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:16.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.630+0000 7fec0a7b6640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fec041084e0 0x7fec041994d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:16.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.630+0000 7fec0a7b6640 1 --2- 172.21.15.44:0/623473708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec04102ad0 0x7fec04198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:16.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.630+0000 7fec0a7b6640 1 -- 172.21.15.44:0/623473708 >> 172.21.15.44:0/623473708 conn(0x7fec040fe540 msgr2=0x7fec04100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:16.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.630+0000 7fec0a7b6640 1 -- 172.21.15.44:0/623473708 shutdown_connections 2024-08-20T03:01:16.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.630+0000 7fec0a7b6640 1 -- 172.21.15.44:0/623473708 wait complete. 2024-08-20T03:01:16.837 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:16 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/623473708' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-20T03:01:17.009 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299428 got 223338299431 for osd.8 2024-08-20T03:01:17.009 DEBUG:teuthology.parallel:result is None 2024-08-20T03:01:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.635+0000 7fda680e9640 1 -- 172.21.15.44:0/306276318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda60075c70 msgr2=0x7fda6010eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.635+0000 7fda680e9640 1 --2- 172.21.15.44:0/306276318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda60075c70 0x7fda6010eae0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fda50009900 tx=0x7fda5002a410 comp rx=0 tx=0).stop 2024-08-20T03:01:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.635+0000 7fda680e9640 1 -- 172.21.15.44:0/306276318 shutdown_connections 2024-08-20T03:01:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.635+0000 7fda680e9640 1 --2- 172.21.15.44:0/306276318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda6010f0b0 0x7fda601114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.635+0000 7fda680e9640 1 --2- 172.21.15.44:0/306276318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda60075c70 0x7fda6010eae0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:17.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.635+0000 7fda680e9640 1 --2- 172.21.15.44:0/306276318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda60075270 0x7fda60075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.635+0000 7fda680e9640 1 -- 172.21.15.44:0/306276318 >> 172.21.15.44:0/306276318 conn(0x7fda600fe540 msgr2=0x7fda60100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.635+0000 7fda680e9640 1 -- 172.21.15.44:0/306276318 shutdown_connections 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.635+0000 7fda680e9640 1 -- 172.21.15.44:0/306276318 wait complete. 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda680e9640 1 Processor -- start 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda680e9640 1 -- start start 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda680e9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda60075270 0x7fda6019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda680e9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda60075c70 0x7fda6019b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda680e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6010f0b0 0x7fda601a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda680e9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda60114c60 con 0x7fda6010f0b0 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda680e9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda60114ae0 con 0x7fda60075270 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda680e9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fda60114de0 con 0x7fda60075c70 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda66686640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6010f0b0 0x7fda601a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda66686640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6010f0b0 0x7fda601a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40450/0 (socket says 172.21.15.44:40450) 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda66686640 1 -- 172.21.15.44:0/354489095 learned_addr learned my addr 172.21.15.44:0/354489095 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:17.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda65e85640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda60075270 0x7fda6019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda65684640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda60075c70 0x7fda6019b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda66686640 1 -- 172.21.15.44:0/354489095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda60075c70 msgr2=0x7fda6019b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda66686640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda60075c70 0x7fda6019b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda66686640 1 -- 172.21.15.44:0/354489095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda60075270 msgr2=0x7fda6019b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda66686640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda60075270 0x7fda6019b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.636+0000 7fda66686640 1 -- 172.21.15.44:0/354489095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda601a1240 con 0x7fda6010f0b0 2024-08-20T03:01:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.637+0000 7fda66686640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6010f0b0 0x7fda601a0a70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fda5800e9d0 tx=0x7fda5800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.637+0000 7fda56ffd640 1 -- 172.21.15.44:0/354489095 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5800cd80 con 0x7fda6010f0b0 2024-08-20T03:01:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.637+0000 7fda680e9640 1 -- 172.21.15.44:0/354489095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda601a1530 con 0x7fda6010f0b0 2024-08-20T03:01:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.637+0000 7fda56ffd640 1 -- 172.21.15.44:0/354489095 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda58005020 con 0x7fda6010f0b0 2024-08-20T03:01:17.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.637+0000 7fda56ffd640 1 -- 172.21.15.44:0/354489095 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda580116d0 con 0x7fda6010f0b0 2024-08-20T03:01:17.085 INFO:teuthology.orchestra.run.smithi044.stdout:201863462957 2024-08-20T03:01:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.637+0000 7fda680e9640 1 -- 172.21.15.44:0/354489095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda601a7bf0 con 0x7fda6010f0b0 2024-08-20T03:01:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.638+0000 7fda680e9640 1 -- 172.21.15.44:0/354489095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda60075650 con 0x7fda6010f0b0 2024-08-20T03:01:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.639+0000 7fda56ffd640 1 -- 172.21.15.44:0/354489095 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fda580051d0 con 0x7fda6010f0b0 2024-08-20T03:01:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.640+0000 7fda56ffd640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fda4c0b3cb0 0x7fda4c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.640+0000 7fda56ffd640 1 -- 172.21.15.44:0/354489095 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fda580e4340 con 0x7fda6010f0b0 2024-08-20T03:01:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.640+0000 7fda65e85640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fda4c0b3cb0 0x7fda4c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:17.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.641+0000 7fda65e85640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fda4c0b3cb0 0x7fda4c0b6170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00bbe0 tx=0x7fda5c00b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.644+0000 7fda56ffd640 1 -- 172.21.15.44:0/354489095 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fda58016030 con 0x7fda6010f0b0 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.833+0000 7fda680e9640 1 -- 172.21.15.44:0/354489095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fda601a7ee0 con 0x7fda6010f0b0 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.834+0000 7fda56ffd640 1 -- 172.21.15.44:0/354489095 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fda580ab0d0 con 0x7fda6010f0b0 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.836+0000 7fda680e9640 1 -- 172.21.15.44:0/354489095 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fda4c0b3cb0 msgr2=0x7fda4c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.836+0000 7fda680e9640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fda4c0b3cb0 0x7fda4c0b6170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00bbe0 tx=0x7fda5c00b390 comp rx=0 tx=0).stop 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.837+0000 7fda680e9640 1 -- 172.21.15.44:0/354489095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6010f0b0 msgr2=0x7fda601a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.837+0000 7fda680e9640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6010f0b0 0x7fda601a0a70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fda5800e9d0 tx=0x7fda5800ee90 comp rx=0 tx=0).stop 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.837+0000 7fda680e9640 1 -- 172.21.15.44:0/354489095 shutdown_connections 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.837+0000 7fda680e9640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fda4c0b3cb0 0x7fda4c0b6170 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.837+0000 7fda680e9640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda6010f0b0 0x7fda601a0a70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.837+0000 7fda680e9640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda60075c70 0x7fda6019b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.837+0000 7fda680e9640 1 --2- 172.21.15.44:0/354489095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda60075270 0x7fda6019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:17.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.837+0000 7fda680e9640 1 -- 172.21.15.44:0/354489095 >> 172.21.15.44:0/354489095 conn(0x7fda600fe540 msgr2=0x7fda601112b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:17.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.837+0000 7fda680e9640 1 -- 172.21.15.44:0/354489095 shutdown_connections 2024-08-20T03:01:17.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:16.837+0000 7fda680e9640 1 -- 172.21.15.44:0/354489095 wait complete. 2024-08-20T03:01:17.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:16 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/623473708' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-20T03:01:17.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:16 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/623473708' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-20T03:01:17.940 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:17 smithi044 ceph-mon[18790]: pgmap v284: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:17.940 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:17 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/354489095' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-20T03:01:17.970 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462954 got 201863462957 for osd.7 2024-08-20T03:01:17.971 DEBUG:teuthology.parallel:result is None 2024-08-20T03:01:18.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.106+0000 7f485083d640 1 -- 172.21.15.44:0/4279229787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f484c108e60 msgr2=0x7f484c10f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:18.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.106+0000 7f485083d640 1 --2- 172.21.15.44:0/4279229787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f484c108e60 0x7f484c10f660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4844009ca0 tx=0x7f484402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:18.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.106+0000 7f485083d640 1 -- 172.21.15.44:0/4279229787 shutdown_connections 2024-08-20T03:01:18.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.106+0000 7f485083d640 1 --2- 172.21.15.44:0/4279229787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f484c108e60 0x7f484c10f660 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:18.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.106+0000 7f485083d640 1 --2- 172.21.15.44:0/4279229787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484c1084e0 0x7f484c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:18.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.106+0000 7f485083d640 1 --2- 172.21.15.44:0/4279229787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f484c102ad0 0x7f484c102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.106+0000 7f485083d640 1 -- 172.21.15.44:0/4279229787 >> 172.21.15.44:0/4279229787 conn(0x7f484c0fe540 msgr2=0x7f484c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.106+0000 7f485083d640 1 -- 172.21.15.44:0/4279229787 shutdown_connections 2024-08-20T03:01:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.106+0000 7f485083d640 1 -- 172.21.15.44:0/4279229787 wait complete. 2024-08-20T03:01:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f485083d640 1 Processor -- start 2024-08-20T03:01:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f485083d640 1 -- start start 2024-08-20T03:01:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f485083d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f484c102ad0 0x7f484c198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f485083d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484c1084e0 0x7f484c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f485083d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f484c108e60 0x7f484c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f485083d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f484c112b20 con 0x7f484c102ad0 2024-08-20T03:01:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f485083d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f484c1129a0 con 0x7f484c108e60 2024-08-20T03:01:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f485083d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f484c112ca0 con 0x7f484c1084e0 2024-08-20T03:01:18.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f484a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f484c102ad0 0x7f484c198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f484a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f484c102ad0 0x7f484c198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49186/0 (socket says 172.21.15.44:49186) 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f484a59c640 1 -- 172.21.15.44:0/1997797863 learned_addr learned my addr 172.21.15.44:0/1997797863 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f484ad9d640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f484c108e60 0x7f484c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.107+0000 7f4849d9b640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484c1084e0 0x7f484c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.108+0000 7f484a59c640 1 -- 172.21.15.44:0/1997797863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484c1084e0 msgr2=0x7f484c1994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.108+0000 7f484a59c640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484c1084e0 0x7f484c1994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.108+0000 7f484a59c640 1 -- 172.21.15.44:0/1997797863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f484c108e60 msgr2=0x7f484c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.108+0000 7f484a59c640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f484c108e60 0x7f484c19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.108+0000 7f484a59c640 1 -- 172.21.15.44:0/1997797863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484c19f020 con 0x7f484c102ad0 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.108+0000 7f4849d9b640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484c1084e0 0x7f484c1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.108+0000 7f484a59c640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f484c102ad0 0x7f484c198f70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f483c00d950 tx=0x7f483c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.108+0000 7f48437fe640 1 -- 172.21.15.44:0/1997797863 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c017070 con 0x7f484c102ad0 2024-08-20T03:01:18.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.108+0000 7f48437fe640 1 -- 172.21.15.44:0/1997797863 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f483c009c30 con 0x7f484c102ad0 2024-08-20T03:01:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.108+0000 7f48437fe640 1 -- 172.21.15.44:0/1997797863 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c00f620 con 0x7f484c102ad0 2024-08-20T03:01:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.108+0000 7f485083d640 1 -- 172.21.15.44:0/1997797863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f484c19f310 con 0x7f484c102ad0 2024-08-20T03:01:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.108+0000 7f485083d640 1 -- 172.21.15.44:0/1997797863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f484c1a5ba0 con 0x7f484c102ad0 2024-08-20T03:01:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.109+0000 7f485083d640 1 -- 172.21.15.44:0/1997797863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f484c102eb0 con 0x7f484c102ad0 2024-08-20T03:01:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.110+0000 7f48437fe640 1 -- 172.21.15.44:0/1997797863 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f483c010460 con 0x7f484c102ad0 2024-08-20T03:01:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.111+0000 7f48437fe640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f48380b3d30 0x7f48380b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.111+0000 7f4849d9b640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f48380b3d30 0x7f48380b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.111+0000 7f48437fe640 1 -- 172.21.15.44:0/1997797863 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f483c0e3a40 con 0x7f484c102ad0 2024-08-20T03:01:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.112+0000 7f4849d9b640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f48380b3d30 0x7f48380b61f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f48340058c0 tx=0x7f4834005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:18.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.114+0000 7f48437fe640 1 -- 172.21.15.44:0/1997797863 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f483c0aaf00 con 0x7f484c102ad0 2024-08-20T03:01:18.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:17 smithi165 ceph-mon[22357]: pgmap v284: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:18.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:17 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/354489095' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-20T03:01:18.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:17 smithi099 ceph-mon[21984]: pgmap v284: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:18.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:17 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/354489095' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-20T03:01:18.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.302+0000 7f485083d640 1 -- 172.21.15.44:0/1997797863 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f484c19a0a0 con 0x7f484c102ad0 2024-08-20T03:01:18.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.302+0000 7f48437fe640 1 -- 172.21.15.44:0/1997797863 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f483c009de0 con 0x7f484c102ad0 2024-08-20T03:01:18.305 INFO:teuthology.orchestra.run.smithi044.stdout:176093659185 2024-08-20T03:01:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.304+0000 7f485083d640 1 -- 172.21.15.44:0/1997797863 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f48380b3d30 msgr2=0x7f48380b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.304+0000 7f485083d640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f48380b3d30 0x7f48380b61f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f48340058c0 tx=0x7f4834005850 comp rx=0 tx=0).stop 2024-08-20T03:01:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.304+0000 7f485083d640 1 -- 172.21.15.44:0/1997797863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f484c102ad0 msgr2=0x7f484c198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.304+0000 7f485083d640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f484c102ad0 0x7f484c198f70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f483c00d950 tx=0x7f483c00de10 comp rx=0 tx=0).stop 2024-08-20T03:01:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.304+0000 7f485083d640 1 -- 172.21.15.44:0/1997797863 shutdown_connections 2024-08-20T03:01:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.304+0000 7f485083d640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f48380b3d30 0x7f48380b61f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:18.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.304+0000 7f485083d640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f484c108e60 0x7f484c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:18.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.304+0000 7f485083d640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f484c1084e0 0x7f484c1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:18.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.304+0000 7f485083d640 1 --2- 172.21.15.44:0/1997797863 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f484c102ad0 0x7f484c198f70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:18.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.304+0000 7f485083d640 1 -- 172.21.15.44:0/1997797863 >> 172.21.15.44:0/1997797863 conn(0x7f484c0fe540 msgr2=0x7f484c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:18.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.305+0000 7f485083d640 1 -- 172.21.15.44:0/1997797863 shutdown_connections 2024-08-20T03:01:18.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:18.305+0000 7f485083d640 1 -- 172.21.15.44:0/1997797863 wait complete. 2024-08-20T03:01:18.895 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822707 got 154618822710 for osd.5 2024-08-20T03:01:18.895 DEBUG:teuthology.parallel:result is None 2024-08-20T03:01:19.001 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:18 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1997797863' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-20T03:01:19.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.081+0000 7f406ff16640 1 -- 172.21.15.44:0/2752668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40681039e0 msgr2=0x7f4068103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:19.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.081+0000 7f406ff16640 1 --2- 172.21.15.44:0/2752668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40681039e0 0x7f4068103dc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4064009ca0 tx=0x7f406402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:19.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.081+0000 7f406ff16640 1 -- 172.21.15.44:0/2752668 shutdown_connections 2024-08-20T03:01:19.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.081+0000 7f406ff16640 1 --2- 172.21.15.44:0/2752668 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406810ce40 0x7f406810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:19.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.081+0000 7f406ff16640 1 --2- 172.21.15.44:0/2752668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4068104300 0x7f406810c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.081+0000 7f406ff16640 1 --2- 172.21.15.44:0/2752668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40681039e0 0x7f4068103dc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.081+0000 7f406ff16640 1 -- 172.21.15.44:0/2752668 >> 172.21.15.44:0/2752668 conn(0x7f40680fd440 msgr2=0x7f40680ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.081+0000 7f406ff16640 1 -- 172.21.15.44:0/2752668 shutdown_connections 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.081+0000 7f406ff16640 1 -- 172.21.15.44:0/2752668 wait complete. 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.082+0000 7f406ff16640 1 Processor -- start 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.082+0000 7f406ff16640 1 -- start start 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.082+0000 7f406ff16640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40681039e0 0x7f4068198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406ff16640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068104300 0x7f4068199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406ff16640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406810ce40 0x7f406819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406ff16640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4068112b40 con 0x7f4068104300 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406ff16640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f40681129c0 con 0x7f40681039e0 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406ff16640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4068112cc0 con 0x7f406810ce40 2024-08-20T03:01:19.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406d4b1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068104300 0x7f4068199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406d4b1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068104300 0x7f4068199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49198/0 (socket says 172.21.15.44:49198) 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406d4b1640 1 -- 172.21.15.44:0/171423569 learned_addr learned my addr 172.21.15.44:0/171423569 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406dcb2640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40681039e0 0x7f4068198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406e4b3640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406810ce40 0x7f406819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406d4b1640 1 -- 172.21.15.44:0/171423569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406810ce40 msgr2=0x7f406819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406d4b1640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406810ce40 0x7f406819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406d4b1640 1 -- 172.21.15.44:0/171423569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40681039e0 msgr2=0x7f4068198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406d4b1640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40681039e0 0x7f4068198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406d4b1640 1 -- 172.21.15.44:0/171423569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406819f080 con 0x7f4068104300 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406dcb2640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40681039e0 0x7f4068198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406d4b1640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068104300 0x7f4068199510 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f405800c970 tx=0x7f405800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f405effd640 1 -- 172.21.15.44:0/171423569 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058007bb0 con 0x7f4068104300 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f405effd640 1 -- 172.21.15.44:0/171423569 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4058007d50 con 0x7f4068104300 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406ff16640 1 -- 172.21.15.44:0/171423569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406819f370 con 0x7f4068104300 2024-08-20T03:01:19.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f405effd640 1 -- 172.21.15.44:0/171423569 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405800f7b0 con 0x7f4068104300 2024-08-20T03:01:19.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.083+0000 7f406ff16640 1 -- 172.21.15.44:0/171423569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40681a5ab0 con 0x7f4068104300 2024-08-20T03:01:19.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.084+0000 7f406ff16640 1 -- 172.21.15.44:0/171423569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4068103dc0 con 0x7f4068104300 2024-08-20T03:01:19.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.086+0000 7f405effd640 1 -- 172.21.15.44:0/171423569 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f4058018480 con 0x7f4068104300 2024-08-20T03:01:19.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.087+0000 7f405effd640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40540b3cb0 0x7f40540b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:19.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.087+0000 7f406dcb2640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40540b3cb0 0x7f40540b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:19.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.087+0000 7f405effd640 1 -- 172.21.15.44:0/171423569 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f40580e44e0 con 0x7f4068104300 2024-08-20T03:01:19.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.088+0000 7f406dcb2640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40540b3cb0 0x7f40540b6170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4064000f80 tx=0x7f4064031af0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:19.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.090+0000 7f405effd640 1 -- 172.21.15.44:0/171423569 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f40580ab9e0 con 0x7f4068104300 2024-08-20T03:01:19.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:18 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1997797863' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-20T03:01:19.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:18 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1997797863' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-20T03:01:19.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.277+0000 7f406ff16640 1 -- 172.21.15.44:0/171423569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f40681a5e90 con 0x7f4068104300 2024-08-20T03:01:19.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.277+0000 7f405effd640 1 -- 172.21.15.44:0/171423569 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f40580ab2e0 con 0x7f4068104300 2024-08-20T03:01:19.281 INFO:teuthology.orchestra.run.smithi044.stdout:296352743450 2024-08-20T03:01:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.280+0000 7f406ff16640 1 -- 172.21.15.44:0/171423569 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40540b3cb0 msgr2=0x7f40540b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.280+0000 7f406ff16640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40540b3cb0 0x7f40540b6170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4064000f80 tx=0x7f4064031af0 comp rx=0 tx=0).stop 2024-08-20T03:01:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.280+0000 7f406ff16640 1 -- 172.21.15.44:0/171423569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068104300 msgr2=0x7f4068199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.280+0000 7f406ff16640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068104300 0x7f4068199510 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f405800c970 tx=0x7f405800ce30 comp rx=0 tx=0).stop 2024-08-20T03:01:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.280+0000 7f406ff16640 1 -- 172.21.15.44:0/171423569 shutdown_connections 2024-08-20T03:01:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.280+0000 7f406ff16640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40540b3cb0 0x7f40540b6170 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.280+0000 7f406ff16640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406810ce40 0x7f406819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.280+0000 7f406ff16640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4068104300 0x7f4068199510 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.280+0000 7f406ff16640 1 --2- 172.21.15.44:0/171423569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40681039e0 0x7f4068198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.280+0000 7f406ff16640 1 -- 172.21.15.44:0/171423569 >> 172.21.15.44:0/171423569 conn(0x7f40680fd440 msgr2=0x7f40680fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.280+0000 7f406ff16640 1 -- 172.21.15.44:0/171423569 shutdown_connections 2024-08-20T03:01:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:19.280+0000 7f406ff16640 1 -- 172.21.15.44:0/171423569 wait complete. 2024-08-20T03:01:19.566 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986230 got 133143986233 for osd.4 2024-08-20T03:01:19.566 DEBUG:teuthology.parallel:result is None 2024-08-20T03:01:19.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:19 smithi044 ceph-mon[18790]: pgmap v285: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:01:19.806 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:19 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/171423569' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-20T03:01:20.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:19 smithi165 ceph-mon[22357]: pgmap v285: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:01:20.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:19 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/171423569' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-20T03:01:20.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:19 smithi099 ceph-mon[21984]: pgmap v285: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:01:20.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:19 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/171423569' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-20T03:01:20.568 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939676 got 270582939677 for osd.10 2024-08-20T03:01:20.569 DEBUG:teuthology.parallel:result is None 2024-08-20T03:01:20.995 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659184 got 176093659185 for osd.6 2024-08-20T03:01:20.996 DEBUG:teuthology.parallel:result is None 2024-08-20T03:01:21.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.155+0000 7f8a75d67640 1 -- 172.21.15.44:0/1903139941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a701074d0 msgr2=0x7f8a7010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:21.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.155+0000 7f8a75d67640 1 --2- 172.21.15.44:0/1903139941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a701074d0 0x7f8a7010dcd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009ca0 tx=0x7f8a6802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:21.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.155+0000 7f8a75d67640 1 -- 172.21.15.44:0/1903139941 shutdown_connections 2024-08-20T03:01:21.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.155+0000 7f8a75d67640 1 --2- 172.21.15.44:0/1903139941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a701074d0 0x7f8a7010dcd0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:21.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.155+0000 7f8a75d67640 1 --2- 172.21.15.44:0/1903139941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a70106b50 0x7f8a70106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:21.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.155+0000 7f8a75d67640 1 --2- 172.21.15.44:0/1903139941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a70101520 0x7f8a70101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.155+0000 7f8a75d67640 1 -- 172.21.15.44:0/1903139941 >> 172.21.15.44:0/1903139941 conn(0x7f8a70076c80 msgr2=0x7f8a70077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.155+0000 7f8a75d67640 1 -- 172.21.15.44:0/1903139941 shutdown_connections 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.155+0000 7f8a75d67640 1 -- 172.21.15.44:0/1903139941 wait complete. 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a75d67640 1 Processor -- start 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a75d67640 1 -- start start 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a75d67640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a70101520 0x7f8a70198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a75d67640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a70106b50 0x7f8a70199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a75d67640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a701074d0 0x7f8a7019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a75d67640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a70112b00 con 0x7f8a701074d0 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a75d67640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8a70112980 con 0x7f8a70101520 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a75d67640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8a70112c80 con 0x7f8a70106b50 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a701074d0 0x7f8a7019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:21.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a6ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a701074d0 0x7f8a7019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49208/0 (socket says 172.21.15.44:49208) 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a6ffff640 1 -- 172.21.15.44:0/2222056221 learned_addr learned my addr 172.21.15.44:0/2222056221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a6effd640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a70106b50 0x7f8a70199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.156+0000 7f8a6f7fe640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a70101520 0x7f8a70198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a6ffff640 1 -- 172.21.15.44:0/2222056221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a70106b50 msgr2=0x7f8a70199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a6ffff640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a70106b50 0x7f8a70199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a6ffff640 1 -- 172.21.15.44:0/2222056221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a70101520 msgr2=0x7f8a70198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a6ffff640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a70101520 0x7f8a70198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a6ffff640 1 -- 172.21.15.44:0/2222056221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7019f0a0 con 0x7f8a701074d0 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a6effd640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a70106b50 0x7f8a70199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a6f7fe640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a70101520 0x7f8a70198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a6ffff640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a701074d0 0x7f8a7019e8d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8a68005cc0 tx=0x7f8a6802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a6cff9640 1 -- 172.21.15.44:0/2222056221 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68003cf0 con 0x7f8a701074d0 2024-08-20T03:01:21.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a6cff9640 1 -- 172.21.15.44:0/2222056221 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a680044f0 con 0x7f8a701074d0 2024-08-20T03:01:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a75d67640 1 -- 172.21.15.44:0/2222056221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7019f330 con 0x7f8a701074d0 2024-08-20T03:01:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a6cff9640 1 -- 172.21.15.44:0/2222056221 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6802fd10 con 0x7f8a701074d0 2024-08-20T03:01:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.157+0000 7f8a75d67640 1 -- 172.21.15.44:0/2222056221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a701a5ad0 con 0x7f8a701074d0 2024-08-20T03:01:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.158+0000 7f8a75d67640 1 -- 172.21.15.44:0/2222056221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a70101900 con 0x7f8a701074d0 2024-08-20T03:01:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.160+0000 7f8a6cff9640 1 -- 172.21.15.44:0/2222056221 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f8a68004040 con 0x7f8a701074d0 2024-08-20T03:01:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.161+0000 7f8a6cff9640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8a580b3d30 0x7f8a580b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.161+0000 7f8a6f7fe640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8a580b3d30 0x7f8a580b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.161+0000 7f8a6cff9640 1 -- 172.21.15.44:0/2222056221 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f8a681057d0 con 0x7f8a701074d0 2024-08-20T03:01:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.161+0000 7f8a6f7fe640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8a580b3d30 0x7f8a580b61f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8a60006fd0 tx=0x7f8a60008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:21.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.164+0000 7f8a6cff9640 1 -- 172.21.15.44:0/2222056221 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8a680cccc0 con 0x7f8a701074d0 2024-08-20T03:01:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.352+0000 7f8a75d67640 1 -- 172.21.15.44:0/2222056221 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f8a7002e340 con 0x7f8a701074d0 2024-08-20T03:01:21.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.352+0000 7f8a6cff9640 1 -- 172.21.15.44:0/2222056221 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8a680cc5c0 con 0x7f8a701074d0 2024-08-20T03:01:21.355 INFO:teuthology.orchestra.run.smithi044.stdout:249108103204 2024-08-20T03:01:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.354+0000 7f8a75d67640 1 -- 172.21.15.44:0/2222056221 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8a580b3d30 msgr2=0x7f8a580b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:21.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.354+0000 7f8a75d67640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8a580b3d30 0x7f8a580b61f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8a60006fd0 tx=0x7f8a60008040 comp rx=0 tx=0).stop 2024-08-20T03:01:21.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.354+0000 7f8a75d67640 1 -- 172.21.15.44:0/2222056221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a701074d0 msgr2=0x7f8a7019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:21.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.354+0000 7f8a75d67640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a701074d0 0x7f8a7019e8d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8a68005cc0 tx=0x7f8a6802f470 comp rx=0 tx=0).stop 2024-08-20T03:01:21.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.354+0000 7f8a75d67640 1 -- 172.21.15.44:0/2222056221 shutdown_connections 2024-08-20T03:01:21.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.354+0000 7f8a75d67640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8a580b3d30 0x7f8a580b61f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:21.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.354+0000 7f8a75d67640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a701074d0 0x7f8a7019e8d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:21.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.354+0000 7f8a75d67640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a70106b50 0x7f8a70199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:21.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.354+0000 7f8a75d67640 1 --2- 172.21.15.44:0/2222056221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a70101520 0x7f8a70198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:21.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.354+0000 7f8a75d67640 1 -- 172.21.15.44:0/2222056221 >> 172.21.15.44:0/2222056221 conn(0x7f8a70076c80 msgr2=0x7f8a70104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:21.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.354+0000 7f8a75d67640 1 -- 172.21.15.44:0/2222056221 shutdown_connections 2024-08-20T03:01:21.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:21.354+0000 7f8a75d67640 1 -- 172.21.15.44:0/2222056221 wait complete. 2024-08-20T03:01:22.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:21 smithi044 ceph-mon[18790]: pgmap v286: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-08-20T03:01:22.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:21 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2222056221' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-20T03:01:22.106 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743447 got 296352743450 for osd.11 2024-08-20T03:01:22.106 DEBUG:teuthology.parallel:result is None 2024-08-20T03:01:22.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:21 smithi165 ceph-mon[22357]: pgmap v286: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-08-20T03:01:22.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:21 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2222056221' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-20T03:01:22.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:21 smithi099 ceph-mon[21984]: pgmap v286: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-08-20T03:01:22.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:21 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2222056221' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-20T03:01:22.567 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:22.587 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103201 got 249108103204 for osd.9 2024-08-20T03:01:22.588 DEBUG:teuthology.parallel:result is None 2024-08-20T03:01:23.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.341+0000 7f6408b6f640 1 -- 172.21.15.44:0/1165460238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6404107d20 msgr2=0x7f640410e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:23.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.341+0000 7f6408b6f640 1 --2- 172.21.15.44:0/1165460238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6404107d20 0x7f640410e520 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009ca0 tx=0x7f63fc02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:23.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.341+0000 7f6408b6f640 1 -- 172.21.15.44:0/1165460238 shutdown_connections 2024-08-20T03:01:23.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.341+0000 7f6408b6f640 1 --2- 172.21.15.44:0/1165460238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6404107d20 0x7f640410e520 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:23.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.341+0000 7f6408b6f640 1 --2- 172.21.15.44:0/1165460238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64041073a0 0x7f64041077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:23.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.341+0000 7f6408b6f640 1 --2- 172.21.15.44:0/1165460238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6404101990 0x7f6404101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:23.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.341+0000 7f6408b6f640 1 -- 172.21.15.44:0/1165460238 >> 172.21.15.44:0/1165460238 conn(0x7f64040fd400 msgr2=0x7f64040ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:23.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.341+0000 7f6408b6f640 1 -- 172.21.15.44:0/1165460238 shutdown_connections 2024-08-20T03:01:23.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.341+0000 7f6408b6f640 1 -- 172.21.15.44:0/1165460238 wait complete. 2024-08-20T03:01:23.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f6408b6f640 1 Processor -- start 2024-08-20T03:01:23.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f6408b6f640 1 -- start start 2024-08-20T03:01:23.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f6408b6f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6404101990 0x7f640419b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:23.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f6408b6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64041073a0 0x7f640419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:23.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f6408b6f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6404107d20 0x7f64041a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:23.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f6408b6f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f64041119a0 con 0x7f64041073a0 2024-08-20T03:01:23.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f6408b6f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6404111820 con 0x7f6404101990 2024-08-20T03:01:23.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f6408b6f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6404111b20 con 0x7f6404107d20 2024-08-20T03:01:23.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f63fbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64041073a0 0x7f640419b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:23.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f63fbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64041073a0 0x7f640419b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49240/0 (socket says 172.21.15.44:49240) 2024-08-20T03:01:23.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f63fbfff640 1 -- 172.21.15.44:0/275478052 learned_addr learned my addr 172.21.15.44:0/275478052 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:23.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f640259c640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6404101990 0x7f640419b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f6402d9d640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6404107d20 0x7f64041a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f63fbfff640 1 -- 172.21.15.44:0/275478052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6404107d20 msgr2=0x7f64041a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f63fbfff640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6404107d20 0x7f64041a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f63fbfff640 1 -- 172.21.15.44:0/275478052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6404101990 msgr2=0x7f640419b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f63fbfff640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6404101990 0x7f640419b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.342+0000 7f63fbfff640 1 -- 172.21.15.44:0/275478052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64041a1220 con 0x7f64041073a0 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.343+0000 7f63fbfff640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64041073a0 0x7f640419b6b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00c890 tx=0x7f63ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.343+0000 7f63fb7fe640 1 -- 172.21.15.44:0/275478052 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec004220 con 0x7f64041073a0 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.343+0000 7f63fb7fe640 1 -- 172.21.15.44:0/275478052 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ec0043c0 con 0x7f64041073a0 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.343+0000 7f6408b6f640 1 -- 172.21.15.44:0/275478052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64041a1510 con 0x7f64041073a0 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.343+0000 7f63fb7fe640 1 -- 172.21.15.44:0/275478052 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec004a00 con 0x7f64041073a0 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.343+0000 7f6408b6f640 1 -- 172.21.15.44:0/275478052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64041a7bd0 con 0x7f64041073a0 2024-08-20T03:01:23.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.344+0000 7f6408b6f640 1 -- 172.21.15.44:0/275478052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6404101d70 con 0x7f64041073a0 2024-08-20T03:01:23.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.345+0000 7f63fb7fe640 1 -- 172.21.15.44:0/275478052 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f63ec0079b0 con 0x7f64041073a0 2024-08-20T03:01:23.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.346+0000 7f63fb7fe640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f63f00b3cb0 0x7f63f00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:23.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.347+0000 7f640259c640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f63f00b3cb0 0x7f63f00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:23.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.347+0000 7f63fb7fe640 1 -- 172.21.15.44:0/275478052 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f63ec0e4060 con 0x7f64041073a0 2024-08-20T03:01:23.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.347+0000 7f640259c640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f63f00b3cb0 0x7f63f00b6170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f63f4005fd0 tx=0x7f63f400b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:23.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.350+0000 7f63fb7fe640 1 -- 172.21.15.44:0/275478052 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f63ec014030 con 0x7f64041073a0 2024-08-20T03:01:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.540+0000 7f6408b6f640 1 -- 172.21.15.44:0/275478052 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f64041a7f60 con 0x7f64041073a0 2024-08-20T03:01:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.540+0000 7f63fb7fe640 1 -- 172.21.15.44:0/275478052 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f63ec0ab440 con 0x7f64041073a0 2024-08-20T03:01:23.543 INFO:teuthology.orchestra.run.smithi044.stdout:60129542219 2024-08-20T03:01:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.542+0000 7f6408b6f640 1 -- 172.21.15.44:0/275478052 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f63f00b3cb0 msgr2=0x7f63f00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.542+0000 7f6408b6f640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f63f00b3cb0 0x7f63f00b6170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f63f4005fd0 tx=0x7f63f400b390 comp rx=0 tx=0).stop 2024-08-20T03:01:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.542+0000 7f6408b6f640 1 -- 172.21.15.44:0/275478052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64041073a0 msgr2=0x7f640419b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.542+0000 7f6408b6f640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64041073a0 0x7f640419b6b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00c890 tx=0x7f63ec00cd50 comp rx=0 tx=0).stop 2024-08-20T03:01:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.543+0000 7f6408b6f640 1 -- 172.21.15.44:0/275478052 shutdown_connections 2024-08-20T03:01:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.543+0000 7f6408b6f640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f63f00b3cb0 0x7f63f00b6170 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.543+0000 7f6408b6f640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6404107d20 0x7f64041a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.543+0000 7f6408b6f640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64041073a0 0x7f640419b6b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:23.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.543+0000 7f6408b6f640 1 --2- 172.21.15.44:0/275478052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6404101990 0x7f640419b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.543+0000 7f6408b6f640 1 -- 172.21.15.44:0/275478052 >> 172.21.15.44:0/275478052 conn(0x7f64040fd400 msgr2=0x7f640406a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.543+0000 7f6408b6f640 1 -- 172.21.15.44:0/275478052 shutdown_connections 2024-08-20T03:01:23.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:23.543+0000 7f6408b6f640 1 -- 172.21.15.44:0/275478052 wait complete. 2024-08-20T03:01:23.735 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:23 smithi044 ceph-mon[18790]: pgmap v287: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-08-20T03:01:23.735 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:23 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/275478052' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-20T03:01:24.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:23 smithi165 ceph-mon[22357]: pgmap v287: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-08-20T03:01:24.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:23 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/275478052' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-20T03:01:24.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:23 smithi099 ceph-mon[21984]: pgmap v287: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-08-20T03:01:24.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:23 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/275478052' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-20T03:01:24.491 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542216 got 60129542219 for osd.1 2024-08-20T03:01:24.492 DEBUG:teuthology.parallel:result is None 2024-08-20T03:01:24.492 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-20T03:01:24.492 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T03:01:24.774 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.257+0000 7f6532303640 1 -- 172.21.15.44:0/577877212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f652c1042e0 msgr2=0x7f652c10c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.257+0000 7f6532303640 1 --2- 172.21.15.44:0/577877212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f652c1042e0 0x7f652c10c7d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6520009900 tx=0x7f652002a410 comp rx=0 tx=0).stop 2024-08-20T03:01:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.258+0000 7f6532303640 1 -- 172.21.15.44:0/577877212 shutdown_connections 2024-08-20T03:01:25.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.258+0000 7f6532303640 1 --2- 172.21.15.44:0/577877212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f652c10ce20 0x7f652c10f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.258+0000 7f6532303640 1 --2- 172.21.15.44:0/577877212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f652c1042e0 0x7f652c10c7d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.258+0000 7f6532303640 1 --2- 172.21.15.44:0/577877212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f652c1039c0 0x7f652c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.258+0000 7f6532303640 1 -- 172.21.15.44:0/577877212 >> 172.21.15.44:0/577877212 conn(0x7f652c0fd400 msgr2=0x7f652c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.258+0000 7f6532303640 1 -- 172.21.15.44:0/577877212 shutdown_connections 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.258+0000 7f6532303640 1 -- 172.21.15.44:0/577877212 wait complete. 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.258+0000 7f6532303640 1 Processor -- start 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.258+0000 7f6532303640 1 -- start start 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f6532303640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f652c1039c0 0x7f652c101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f6532303640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f652c1042e0 0x7f652c101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f6532303640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f652c10ce20 0x7f652c1020f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f6532303640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f652c112a90 con 0x7f652c1042e0 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f6532303640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f652c112910 con 0x7f652c1039c0 2024-08-20T03:01:25.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f6532303640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f652c112c10 con 0x7f652c10ce20 2024-08-20T03:01:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f652b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f652c1042e0 0x7f652c101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f652b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f652c1042e0 0x7f652c101bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49262/0 (socket says 172.21.15.44:49262) 2024-08-20T03:01:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f652b7fe640 1 -- 172.21.15.44:0/3641298089 learned_addr learned my addr 172.21.15.44:0/3641298089 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f652bfff640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f652c1039c0 0x7f652c101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f65308a0640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f652c10ce20 0x7f652c1020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f652b7fe640 1 -- 172.21.15.44:0/3641298089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f652c10ce20 msgr2=0x7f652c1020f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f652b7fe640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f652c10ce20 0x7f652c1020f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f652b7fe640 1 -- 172.21.15.44:0/3641298089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f652c1039c0 msgr2=0x7f652c101670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f652b7fe640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f652c1039c0 0x7f652c101670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f652b7fe640 1 -- 172.21.15.44:0/3641298089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c1a77a0 con 0x7f652c1042e0 2024-08-20T03:01:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f652bfff640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f652c1039c0 0x7f652c101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:01:25.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f652b7fe640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f652c1042e0 0x7f652c101bb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6520032fd0 tx=0x7f6520031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:25.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f65297fa640 1 -- 172.21.15.44:0/3641298089 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652003f030 con 0x7f652c1042e0 2024-08-20T03:01:25.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f65297fa640 1 -- 172.21.15.44:0/3641298089 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6520004490 con 0x7f652c1042e0 2024-08-20T03:01:25.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f6532303640 1 -- 172.21.15.44:0/3641298089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652c1a7a30 con 0x7f652c1042e0 2024-08-20T03:01:25.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f65297fa640 1 -- 172.21.15.44:0/3641298089 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520035070 con 0x7f652c1042e0 2024-08-20T03:01:25.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.259+0000 7f6532303640 1 -- 172.21.15.44:0/3641298089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652c1a7f60 con 0x7f652c1042e0 2024-08-20T03:01:25.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.260+0000 7f6532303640 1 -- 172.21.15.44:0/3641298089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f652c103da0 con 0x7f652c1042e0 2024-08-20T03:01:25.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.262+0000 7f65297fa640 1 -- 172.21.15.44:0/3641298089 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f6520042030 con 0x7f652c1042e0 2024-08-20T03:01:25.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.262+0000 7f65297fa640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f65140b3d30 0x7f65140b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:25.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.263+0000 7f652bfff640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f65140b3d30 0x7f65140b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:25.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.263+0000 7f65297fa640 1 -- 172.21.15.44:0/3641298089 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f65201044b0 con 0x7f652c1042e0 2024-08-20T03:01:25.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.263+0000 7f652bfff640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f65140b3d30 0x7f65140b61f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f651c005fd0 tx=0x7f651c00b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:25.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.266+0000 7f65297fa640 1 -- 172.21.15.44:0/3641298089 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f65200cb2a0 con 0x7f652c1042e0 2024-08-20T03:01:25.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.448+0000 7f6532303640 1 -- 172.21.15.44:0/3641298089 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f652c06aeb0 con 0x7f65140b3d30 2024-08-20T03:01:25.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.452+0000 7f65297fa640 1 -- 172.21.15.44:0/3641298089 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259036 (secure 0 0 0) 0x7f652c06aeb0 con 0x7f65140b3d30 2024-08-20T03:01:25.457 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:25.458 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-08-20T03:01:25.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.456+0000 7f6532303640 1 -- 172.21.15.44:0/3641298089 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f65140b3d30 msgr2=0x7f65140b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:25.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.456+0000 7f6532303640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f65140b3d30 0x7f65140b61f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f651c005fd0 tx=0x7f651c00b390 comp rx=0 tx=0).stop 2024-08-20T03:01:25.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.456+0000 7f6532303640 1 -- 172.21.15.44:0/3641298089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f652c1042e0 msgr2=0x7f652c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:25.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.456+0000 7f6532303640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f652c1042e0 0x7f652c101bb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6520032fd0 tx=0x7f6520031a20 comp rx=0 tx=0).stop 2024-08-20T03:01:25.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.456+0000 7f6532303640 1 -- 172.21.15.44:0/3641298089 shutdown_connections 2024-08-20T03:01:25.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.456+0000 7f6532303640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f65140b3d30 0x7f65140b61f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:25.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.456+0000 7f6532303640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f652c10ce20 0x7f652c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:25.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.456+0000 7f6532303640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f652c1042e0 0x7f652c101bb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:25.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.456+0000 7f6532303640 1 --2- 172.21.15.44:0/3641298089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f652c1039c0 0x7f652c101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:25.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.456+0000 7f6532303640 1 -- 172.21.15.44:0/3641298089 >> 172.21.15.44:0/3641298089 conn(0x7f652c0fd400 msgr2=0x7f652c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:25.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.456+0000 7f6532303640 1 -- 172.21.15.44:0/3641298089 shutdown_connections 2024-08-20T03:01:25.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:25.456+0000 7f6532303640 1 -- 172.21.15.44:0/3641298089 wait complete. 2024-08-20T03:01:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:25 smithi165 ceph-mon[22357]: pgmap v288: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:01:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:01:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:25 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:26.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:25 smithi099 ceph-mon[21984]: pgmap v288: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:26.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:26.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:26.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:01:26.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:01:26.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:25 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:26.188 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:25 smithi044 ceph-mon[18790]: pgmap v288: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:26.188 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:26.188 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:26.188 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:01:26.188 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:01:26.189 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:25 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:01:26.218 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-08-20T03:01:24.404243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":633,"num_read_kb":415,"num_write":370,"num_write_kb":2286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":3011602,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363900,"kb_used_data":10420,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577924,"statfs":{"total":1151940427776,"available":1151567794176,"internally_reserved":0,"allocated":10670080,"data_stored":6972024,"data_compressed":186921,"data_compressed_allocated":245760,"data_compressed_original":2023424,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002623"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.488814+0000","last_change":"2024-08-20T03:00:18.238595+0000","last_active":"2024-08-20T03:00:59.488814+0000","last_peered":"2024-08-20T03:00:59.488814+0000","last_clean":"2024-08-20T03:00:59.488814+0000","last_became_active":"2024-08-20T03:00:18.238306+0000","last_became_peered":"2024-08-20T03:00:18.238306+0000","last_unstale":"2024-08-20T03:00:59.488814+0000","last_undegraded":"2024-08-20T03:00:59.488814+0000","last_fullsized":"2024-08-20T03:00:59.488814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:33:04.681614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542497+0000","last_change":"2024-08-20T03:00:18.239722+0000","last_active":"2024-08-20T03:00:59.542497+0000","last_peered":"2024-08-20T03:00:59.542497+0000","last_clean":"2024-08-20T03:00:59.542497+0000","last_became_active":"2024-08-20T03:00:18.239391+0000","last_became_peered":"2024-08-20T03:00:18.239391+0000","last_unstale":"2024-08-20T03:00:59.542497+0000","last_undegraded":"2024-08-20T03:00:59.542497+0000","last_fullsized":"2024-08-20T03:00:59.542497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:08:34.657173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542224+0000","last_change":"2024-08-20T03:00:18.242389+0000","last_active":"2024-08-20T03:00:59.542224+0000","last_peered":"2024-08-20T03:00:59.542224+0000","last_clean":"2024-08-20T03:00:59.542224+0000","last_became_active":"2024-08-20T03:00:18.242080+0000","last_became_peered":"2024-08-20T03:00:18.242080+0000","last_unstale":"2024-08-20T03:00:59.542224+0000","last_undegraded":"2024-08-20T03:00:59.542224+0000","last_fullsized":"2024-08-20T03:00:59.542224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:37.042894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.517384+0000","last_change":"2024-08-20T03:00:18.233709+0000","last_active":"2024-08-20T03:00:59.517384+0000","last_peered":"2024-08-20T03:00:59.517384+0000","last_clean":"2024-08-20T03:00:59.517384+0000","last_became_active":"2024-08-20T03:00:18.233402+0000","last_became_peered":"2024-08-20T03:00:18.233402+0000","last_unstale":"2024-08-20T03:00:59.517384+0000","last_undegraded":"2024-08-20T03:00:59.517384+0000","last_fullsized":"2024-08-20T03:00:59.517384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:30:01.308181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491456+0000","last_change":"2024-08-20T03:00:18.237770+0000","last_active":"2024-08-20T03:00:59.491456+0000","last_peered":"2024-08-20T03:00:59.491456+0000","last_clean":"2024-08-20T03:00:59.491456+0000","last_became_active":"2024-08-20T03:00:18.237475+0000","last_became_peered":"2024-08-20T03:00:18.237475+0000","last_unstale":"2024-08-20T03:00:59.491456+0000","last_undegraded":"2024-08-20T03:00:59.491456+0000","last_fullsized":"2024-08-20T03:00:59.491456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:37:12.707935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491276+0000","last_change":"2024-08-20T03:00:18.245400+0000","last_active":"2024-08-20T03:00:59.491276+0000","last_peered":"2024-08-20T03:00:59.491276+0000","last_clean":"2024-08-20T03:00:59.491276+0000","last_became_active":"2024-08-20T03:00:18.244735+0000","last_became_peered":"2024-08-20T03:00:18.244735+0000","last_unstale":"2024-08-20T03:00:59.491276+0000","last_undegraded":"2024-08-20T03:00:59.491276+0000","last_fullsized":"2024-08-20T03:00:59.491276+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:23:25.459441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490416+0000","last_change":"2024-08-20T03:00:18.247366+0000","last_active":"2024-08-20T03:00:59.490416+0000","last_peered":"2024-08-20T03:00:59.490416+0000","last_clean":"2024-08-20T03:00:59.490416+0000","last_became_active":"2024-08-20T03:00:18.247032+0000","last_became_peered":"2024-08-20T03:00:18.247032+0000","last_unstale":"2024-08-20T03:00:59.490416+0000","last_undegraded":"2024-08-20T03:00:59.490416+0000","last_fullsized":"2024-08-20T03:00:59.490416+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:02:18.070393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487313+0000","last_change":"2024-08-20T03:00:18.240845+0000","last_active":"2024-08-20T03:00:59.487313+0000","last_peered":"2024-08-20T03:00:59.487313+0000","last_clean":"2024-08-20T03:00:59.487313+0000","last_became_active":"2024-08-20T03:00:18.240531+0000","last_became_peered":"2024-08-20T03:00:18.240531+0000","last_unstale":"2024-08-20T03:00:59.487313+0000","last_undegraded":"2024-08-20T03:00:59.487313+0000","last_fullsized":"2024-08-20T03:00:59.487313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:15:18.975804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494688+0000","last_change":"2024-08-20T03:00:18.240817+0000","last_active":"2024-08-20T03:00:59.494688+0000","last_peered":"2024-08-20T03:00:59.494688+0000","last_clean":"2024-08-20T03:00:59.494688+0000","last_became_active":"2024-08-20T03:00:18.240262+0000","last_became_peered":"2024-08-20T03:00:18.240262+0000","last_unstale":"2024-08-20T03:00:59.494688+0000","last_undegraded":"2024-08-20T03:00:59.494688+0000","last_fullsized":"2024-08-20T03:00:59.494688+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:30.512597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491738+0000","last_change":"2024-08-20T03:00:18.242139+0000","last_active":"2024-08-20T03:00:59.491738+0000","last_peered":"2024-08-20T03:00:59.491738+0000","last_clean":"2024-08-20T03:00:59.491738+0000","last_became_active":"2024-08-20T03:00:18.241817+0000","last_became_peered":"2024-08-20T03:00:18.241817+0000","last_unstale":"2024-08-20T03:00:59.491738+0000","last_undegraded":"2024-08-20T03:00:59.491738+0000","last_fullsized":"2024-08-20T03:00:59.491738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:25.121306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491069+0000","last_change":"2024-08-20T03:00:18.243205+0000","last_active":"2024-08-20T03:00:59.491069+0000","last_peered":"2024-08-20T03:00:59.491069+0000","last_clean":"2024-08-20T03:00:59.491069+0000","last_became_active":"2024-08-20T03:00:18.242296+0000","last_became_peered":"2024-08-20T03:00:18.242296+0000","last_unstale":"2024-08-20T03:00:59.491069+0000","last_undegraded":"2024-08-20T03:00:59.491069+0000","last_fullsized":"2024-08-20T03:00:59.491069+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:50.490934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.517398+0000","last_change":"2024-08-20T03:00:18.237205+0000","last_active":"2024-08-20T03:00:59.517398+0000","last_peered":"2024-08-20T03:00:59.517398+0000","last_clean":"2024-08-20T03:00:59.517398+0000","last_became_active":"2024-08-20T03:00:18.236631+0000","last_became_peered":"2024-08-20T03:00:18.236631+0000","last_unstale":"2024-08-20T03:00:59.517398+0000","last_undegraded":"2024-08-20T03:00:59.517398+0000","last_fullsized":"2024-08-20T03:00:59.517398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:13.327598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490080+0000","last_change":"2024-08-20T03:00:18.235768+0000","last_active":"2024-08-20T03:00:59.490080+0000","last_peered":"2024-08-20T03:00:59.490080+0000","last_clean":"2024-08-20T03:00:59.490080+0000","last_became_active":"2024-08-20T03:00:18.235461+0000","last_became_peered":"2024-08-20T03:00:18.235461+0000","last_unstale":"2024-08-20T03:00:59.490080+0000","last_undegraded":"2024-08-20T03:00:59.490080+0000","last_fullsized":"2024-08-20T03:00:59.490080+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:08:22.238020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542285+0000","last_change":"2024-08-20T03:00:18.246560+0000","last_active":"2024-08-20T03:00:59.542285+0000","last_peered":"2024-08-20T03:00:59.542285+0000","last_clean":"2024-08-20T03:00:59.542285+0000","last_became_active":"2024-08-20T03:00:18.246270+0000","last_became_peered":"2024-08-20T03:00:18.246270+0000","last_unstale":"2024-08-20T03:00:59.542285+0000","last_undegraded":"2024-08-20T03:00:59.542285+0000","last_fullsized":"2024-08-20T03:00:59.542285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:47:13.402635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542123+0000","last_change":"2024-08-20T03:00:18.245907+0000","last_active":"2024-08-20T03:00:59.542123+0000","last_peered":"2024-08-20T03:00:59.542123+0000","last_clean":"2024-08-20T03:00:59.542123+0000","last_became_active":"2024-08-20T03:00:18.245429+0000","last_became_peered":"2024-08-20T03:00:18.245429+0000","last_unstale":"2024-08-20T03:00:59.542123+0000","last_undegraded":"2024-08-20T03:00:59.542123+0000","last_fullsized":"2024-08-20T03:00:59.542123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:46.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494222+0000","last_change":"2024-08-20T03:00:18.242561+0000","last_active":"2024-08-20T03:00:59.494222+0000","last_peered":"2024-08-20T03:00:59.494222+0000","last_clean":"2024-08-20T03:00:59.494222+0000","last_became_active":"2024-08-20T03:00:18.242048+0000","last_became_peered":"2024-08-20T03:00:18.242048+0000","last_unstale":"2024-08-20T03:00:59.494222+0000","last_undegraded":"2024-08-20T03:00:59.494222+0000","last_fullsized":"2024-08-20T03:00:59.494222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:27.784131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.488943+0000","last_change":"2024-08-20T03:00:18.238547+0000","last_active":"2024-08-20T03:00:59.488943+0000","last_peered":"2024-08-20T03:00:59.488943+0000","last_clean":"2024-08-20T03:00:59.488943+0000","last_became_active":"2024-08-20T03:00:18.238268+0000","last_became_peered":"2024-08-20T03:00:18.238268+0000","last_unstale":"2024-08-20T03:00:59.488943+0000","last_undegraded":"2024-08-20T03:00:59.488943+0000","last_fullsized":"2024-08-20T03:00:59.488943+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:15:11.923674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543347+0000","last_change":"2024-08-20T03:00:18.241995+0000","last_active":"2024-08-20T03:00:59.543347+0000","last_peered":"2024-08-20T03:00:59.543347+0000","last_clean":"2024-08-20T03:00:59.543347+0000","last_became_active":"2024-08-20T03:00:18.241705+0000","last_became_peered":"2024-08-20T03:00:18.241705+0000","last_unstale":"2024-08-20T03:00:59.543347+0000","last_undegraded":"2024-08-20T03:00:59.543347+0000","last_fullsized":"2024-08-20T03:00:59.543347+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:33:02.183712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491386+0000","last_change":"2024-08-20T03:00:18.236800+0000","last_active":"2024-08-20T03:00:59.491386+0000","last_peered":"2024-08-20T03:00:59.491386+0000","last_clean":"2024-08-20T03:00:59.491386+0000","last_became_active":"2024-08-20T03:00:18.235909+0000","last_became_peered":"2024-08-20T03:00:18.235909+0000","last_unstale":"2024-08-20T03:00:59.491386+0000","last_undegraded":"2024-08-20T03:00:59.491386+0000","last_fullsized":"2024-08-20T03:00:59.491386+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:12.427760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.544249+0000","last_change":"2024-08-20T03:00:59.544187+0000","last_active":"2024-08-20T03:00:59.544249+0000","last_peered":"2024-08-20T03:00:59.544249+0000","last_clean":"2024-08-20T03:00:59.544249+0000","last_became_active":"2024-08-20T03:00:59.543807+0000","last_became_peered":"2024-08-20T03:00:59.543807+0000","last_unstale":"2024-08-20T03:00:59.544249+0000","last_undegraded":"2024-08-20T03:00:59.544249+0000","last_fullsized":"2024-08-20T03:00:59.544249+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:16:12.425646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490151+0000","last_change":"2024-08-20T03:00:18.243425+0000","last_active":"2024-08-20T03:00:59.490151+0000","last_peered":"2024-08-20T03:00:59.490151+0000","last_clean":"2024-08-20T03:00:59.490151+0000","last_became_active":"2024-08-20T03:00:18.243082+0000","last_became_peered":"2024-08-20T03:00:18.243082+0000","last_unstale":"2024-08-20T03:00:59.490151+0000","last_undegraded":"2024-08-20T03:00:59.490151+0000","last_fullsized":"2024-08-20T03:00:59.490151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:50.008655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494515+0000","last_change":"2024-08-20T03:00:18.242765+0000","last_active":"2024-08-20T03:00:59.494515+0000","last_peered":"2024-08-20T03:00:59.494515+0000","last_clean":"2024-08-20T03:00:59.494515+0000","last_became_active":"2024-08-20T03:00:18.242459+0000","last_became_peered":"2024-08-20T03:00:18.242459+0000","last_unstale":"2024-08-20T03:00:59.494515+0000","last_undegraded":"2024-08-20T03:00:59.494515+0000","last_fullsized":"2024-08-20T03:00:59.494515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:16.467068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487326+0000","last_change":"2024-08-20T03:00:18.236518+0000","last_active":"2024-08-20T03:00:59.487326+0000","last_peered":"2024-08-20T03:00:59.487326+0000","last_clean":"2024-08-20T03:00:59.487326+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:00:59.487326+0000","last_undegraded":"2024-08-20T03:00:59.487326+0000","last_fullsized":"2024-08-20T03:00:59.487326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:55:55.387134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492066+0000","last_change":"2024-08-20T03:00:18.236730+0000","last_active":"2024-08-20T03:00:59.492066+0000","last_peered":"2024-08-20T03:00:59.492066+0000","last_clean":"2024-08-20T03:00:59.492066+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:00:59.492066+0000","last_undegraded":"2024-08-20T03:00:59.492066+0000","last_fullsized":"2024-08-20T03:00:59.492066+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:18.196161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487634+0000","last_change":"2024-08-20T03:00:18.245390+0000","last_active":"2024-08-20T03:00:59.487634+0000","last_peered":"2024-08-20T03:00:59.487634+0000","last_clean":"2024-08-20T03:00:59.487634+0000","last_became_active":"2024-08-20T03:00:18.241175+0000","last_became_peered":"2024-08-20T03:00:18.241175+0000","last_unstale":"2024-08-20T03:00:59.487634+0000","last_undegraded":"2024-08-20T03:00:59.487634+0000","last_fullsized":"2024-08-20T03:00:59.487634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:51:45.792846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492615+0000","last_change":"2024-08-20T03:00:18.242700+0000","last_active":"2024-08-20T03:00:59.492615+0000","last_peered":"2024-08-20T03:00:59.492615+0000","last_clean":"2024-08-20T03:00:59.492615+0000","last_became_active":"2024-08-20T03:00:18.242407+0000","last_became_peered":"2024-08-20T03:00:18.242407+0000","last_unstale":"2024-08-20T03:00:59.492615+0000","last_undegraded":"2024-08-20T03:00:59.492615+0000","last_fullsized":"2024-08-20T03:00:59.492615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:31:28.061514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.488695+0000","last_change":"2024-08-20T03:00:18.233894+0000","last_active":"2024-08-20T03:00:59.488695+0000","last_peered":"2024-08-20T03:00:59.488695+0000","last_clean":"2024-08-20T03:00:59.488695+0000","last_became_active":"2024-08-20T03:00:18.233584+0000","last_became_peered":"2024-08-20T03:00:18.233584+0000","last_unstale":"2024-08-20T03:00:59.488695+0000","last_undegraded":"2024-08-20T03:00:59.488695+0000","last_fullsized":"2024-08-20T03:00:59.488695+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:36:53.156215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490178+0000","last_change":"2024-08-20T03:00:18.239256+0000","last_active":"2024-08-20T03:00:59.490178+0000","last_peered":"2024-08-20T03:00:59.490178+0000","last_clean":"2024-08-20T03:00:59.490178+0000","last_became_active":"2024-08-20T03:00:18.238931+0000","last_became_peered":"2024-08-20T03:00:18.238931+0000","last_unstale":"2024-08-20T03:00:59.490178+0000","last_undegraded":"2024-08-20T03:00:59.490178+0000","last_fullsized":"2024-08-20T03:00:59.490178+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:38:42.536488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542368+0000","last_change":"2024-08-20T03:00:18.245187+0000","last_active":"2024-08-20T03:00:59.542368+0000","last_peered":"2024-08-20T03:00:59.542368+0000","last_clean":"2024-08-20T03:00:59.542368+0000","last_became_active":"2024-08-20T03:00:18.244894+0000","last_became_peered":"2024-08-20T03:00:18.244894+0000","last_unstale":"2024-08-20T03:00:59.542368+0000","last_undegraded":"2024-08-20T03:00:59.542368+0000","last_fullsized":"2024-08-20T03:00:59.542368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:16:58.002014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492240+0000","last_change":"2024-08-20T03:00:18.241634+0000","last_active":"2024-08-20T03:00:59.492240+0000","last_peered":"2024-08-20T03:00:59.492240+0000","last_clean":"2024-08-20T03:00:59.492240+0000","last_became_active":"2024-08-20T03:00:18.241341+0000","last_became_peered":"2024-08-20T03:00:18.241341+0000","last_unstale":"2024-08-20T03:00:59.492240+0000","last_undegraded":"2024-08-20T03:00:59.492240+0000","last_fullsized":"2024-08-20T03:00:59.492240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:37:16.185544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492125+0000","last_change":"2024-08-20T03:00:18.239116+0000","last_active":"2024-08-20T03:00:59.492125+0000","last_peered":"2024-08-20T03:00:59.492125+0000","last_clean":"2024-08-20T03:00:59.492125+0000","last_became_active":"2024-08-20T03:00:18.236125+0000","last_became_peered":"2024-08-20T03:00:18.236125+0000","last_unstale":"2024-08-20T03:00:59.492125+0000","last_undegraded":"2024-08-20T03:00:59.492125+0000","last_fullsized":"2024-08-20T03:00:59.492125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:57:00.332483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.488228+0000","last_change":"2024-08-20T03:00:18.235996+0000","last_active":"2024-08-20T03:00:59.488228+0000","last_peered":"2024-08-20T03:00:59.488228+0000","last_clean":"2024-08-20T03:00:59.488228+0000","last_became_active":"2024-08-20T03:00:18.235670+0000","last_became_peered":"2024-08-20T03:00:18.235670+0000","last_unstale":"2024-08-20T03:00:59.488228+0000","last_undegraded":"2024-08-20T03:00:59.488228+0000","last_fullsized":"2024-08-20T03:00:59.488228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:59.992914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491090+0000","last_change":"2024-08-20T03:00:18.239679+0000","last_active":"2024-08-20T03:00:59.491090+0000","last_peered":"2024-08-20T03:00:59.491090+0000","last_clean":"2024-08-20T03:00:59.491090+0000","last_became_active":"2024-08-20T03:00:18.239361+0000","last_became_peered":"2024-08-20T03:00:18.239361+0000","last_unstale":"2024-08-20T03:00:59.491090+0000","last_undegraded":"2024-08-20T03:00:59.491090+0000","last_fullsized":"2024-08-20T03:00:59.491090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:28:09.430535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":86,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:22.463590+0000","last_change":"2024-08-20T03:00:16.221221+0000","last_active":"2024-08-20T03:01:22.463590+0000","last_peered":"2024-08-20T03:01:22.463590+0000","last_clean":"2024-08-20T03:01:22.463590+0000","last_became_active":"2024-08-20T03:00:16.220186+0000","last_became_peered":"2024-08-20T03:00:16.220186+0000","last_unstale":"2024-08-20T03:01:22.463590+0000","last_undegraded":"2024-08-20T03:01:22.463590+0000","last_fullsized":"2024-08-20T03:01:22.463590+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:51.045235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492241+0000","last_change":"2024-08-20T03:00:18.234809+0000","last_active":"2024-08-20T03:00:59.492241+0000","last_peered":"2024-08-20T03:00:59.492241+0000","last_clean":"2024-08-20T03:00:59.492241+0000","last_became_active":"2024-08-20T03:00:18.234486+0000","last_became_peered":"2024-08-20T03:00:18.234486+0000","last_unstale":"2024-08-20T03:00:59.492241+0000","last_undegraded":"2024-08-20T03:00:59.492241+0000","last_fullsized":"2024-08-20T03:00:59.492241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:11:52.527224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491303+0000","last_change":"2024-08-20T03:00:16.216876+0000","last_active":"2024-08-20T03:00:59.491303+0000","last_peered":"2024-08-20T03:00:59.491303+0000","last_clean":"2024-08-20T03:00:59.491303+0000","last_became_active":"2024-08-20T03:00:16.216522+0000","last_became_peered":"2024-08-20T03:00:16.216522+0000","last_unstale":"2024-08-20T03:00:59.491303+0000","last_undegraded":"2024-08-20T03:00:59.491303+0000","last_fullsized":"2024-08-20T03:00:59.491303+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:22.363331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.489899+0000","last_change":"2024-08-20T03:00:16.213751+0000","last_active":"2024-08-20T03:00:59.489899+0000","last_peered":"2024-08-20T03:00:59.489899+0000","last_clean":"2024-08-20T03:00:59.489899+0000","last_became_active":"2024-08-20T03:00:16.213348+0000","last_became_peered":"2024-08-20T03:00:16.213348+0000","last_unstale":"2024-08-20T03:00:59.489899+0000","last_undegraded":"2024-08-20T03:00:59.489899+0000","last_fullsized":"2024-08-20T03:00:59.489899+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:01:08.637151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487384+0000","last_change":"2024-08-20T03:00:18.237972+0000","last_active":"2024-08-20T03:00:59.487384+0000","last_peered":"2024-08-20T03:00:59.487384+0000","last_clean":"2024-08-20T03:00:59.487384+0000","last_became_active":"2024-08-20T03:00:18.236769+0000","last_became_peered":"2024-08-20T03:00:18.236769+0000","last_unstale":"2024-08-20T03:00:59.487384+0000","last_undegraded":"2024-08-20T03:00:59.487384+0000","last_fullsized":"2024-08-20T03:00:59.487384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:35:21.001045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492196+0000","last_change":"2024-08-20T03:00:16.221399+0000","last_active":"2024-08-20T03:00:59.492196+0000","last_peered":"2024-08-20T03:00:59.492196+0000","last_clean":"2024-08-20T03:00:59.492196+0000","last_became_active":"2024-08-20T03:00:16.220285+0000","last_became_peered":"2024-08-20T03:00:16.220285+0000","last_unstale":"2024-08-20T03:00:59.492196+0000","last_undegraded":"2024-08-20T03:00:59.492196+0000","last_fullsized":"2024-08-20T03:00:59.492196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:17.034657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543041+0000","last_change":"2024-08-20T03:00:18.236722+0000","last_active":"2024-08-20T03:00:59.543041+0000","last_peered":"2024-08-20T03:00:59.543041+0000","last_clean":"2024-08-20T03:00:59.543041+0000","last_became_active":"2024-08-20T03:00:18.236365+0000","last_became_peered":"2024-08-20T03:00:18.236365+0000","last_unstale":"2024-08-20T03:00:59.543041+0000","last_undegraded":"2024-08-20T03:00:59.543041+0000","last_fullsized":"2024-08-20T03:00:59.543041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:17.344161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":129,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:20.459769+0000","last_change":"2024-08-20T03:00:16.222331+0000","last_active":"2024-08-20T03:01:20.459769+0000","last_peered":"2024-08-20T03:01:20.459769+0000","last_clean":"2024-08-20T03:01:20.459769+0000","last_became_active":"2024-08-20T03:00:16.222019+0000","last_became_peered":"2024-08-20T03:00:16.222019+0000","last_unstale":"2024-08-20T03:01:20.459769+0000","last_undegraded":"2024-08-20T03:01:20.459769+0000","last_fullsized":"2024-08-20T03:01:20.459769+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:46:55.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494478+0000","last_change":"2024-08-20T03:00:18.239020+0000","last_active":"2024-08-20T03:00:59.494478+0000","last_peered":"2024-08-20T03:00:59.494478+0000","last_clean":"2024-08-20T03:00:59.494478+0000","last_became_active":"2024-08-20T03:00:18.238620+0000","last_became_peered":"2024-08-20T03:00:18.238620+0000","last_unstale":"2024-08-20T03:00:59.494478+0000","last_undegraded":"2024-08-20T03:00:59.494478+0000","last_fullsized":"2024-08-20T03:00:59.494478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:17:12.632357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491087+0000","last_change":"2024-08-20T03:00:16.214907+0000","last_active":"2024-08-20T03:00:59.491087+0000","last_peered":"2024-08-20T03:00:59.491087+0000","last_clean":"2024-08-20T03:00:59.491087+0000","last_became_active":"2024-08-20T03:00:16.214573+0000","last_became_peered":"2024-08-20T03:00:16.214573+0000","last_unstale":"2024-08-20T03:00:59.491087+0000","last_undegraded":"2024-08-20T03:00:59.491087+0000","last_fullsized":"2024-08-20T03:00:59.491087+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:21:34.084149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542627+0000","last_change":"2024-08-20T03:00:18.245524+0000","last_active":"2024-08-20T03:00:59.542627+0000","last_peered":"2024-08-20T03:00:59.542627+0000","last_clean":"2024-08-20T03:00:59.542627+0000","last_became_active":"2024-08-20T03:00:18.244966+0000","last_became_peered":"2024-08-20T03:00:18.244966+0000","last_unstale":"2024-08-20T03:00:59.542627+0000","last_undegraded":"2024-08-20T03:00:59.542627+0000","last_fullsized":"2024-08-20T03:00:59.542627+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:03:21.584182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491504+0000","last_change":"2024-08-20T03:00:16.470353+0000","last_active":"2024-08-20T03:00:59.491504+0000","last_peered":"2024-08-20T03:00:59.491504+0000","last_clean":"2024-08-20T03:00:59.491504+0000","last_became_active":"2024-08-20T03:00:16.469976+0000","last_became_peered":"2024-08-20T03:00:16.469976+0000","last_unstale":"2024-08-20T03:00:59.491504+0000","last_undegraded":"2024-08-20T03:00:59.491504+0000","last_fullsized":"2024-08-20T03:00:59.491504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:25:03.595784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494432+0000","last_change":"2024-08-20T03:00:18.245244+0000","last_active":"2024-08-20T03:00:59.494432+0000","last_peered":"2024-08-20T03:00:59.494432+0000","last_clean":"2024-08-20T03:00:59.494432+0000","last_became_active":"2024-08-20T03:00:18.244934+0000","last_became_peered":"2024-08-20T03:00:18.244934+0000","last_unstale":"2024-08-20T03:00:59.494432+0000","last_undegraded":"2024-08-20T03:00:59.494432+0000","last_fullsized":"2024-08-20T03:00:59.494432+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:33:35.035039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'30","reported_seq":184,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:20.549317+0000","last_change":"2024-08-20T03:00:59.539537+0000","last_active":"2024-08-20T03:01:20.549317+0000","last_peered":"2024-08-20T03:01:20.549317+0000","last_clean":"2024-08-20T03:01:20.549317+0000","last_became_active":"2024-08-20T03:00:59.518110+0000","last_became_peered":"2024-08-20T03:00:59.518110+0000","last_unstale":"2024-08-20T03:01:20.549317+0000","last_undegraded":"2024-08-20T03:01:20.549317+0000","last_fullsized":"2024-08-20T03:01:20.549317+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:10:58.722741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":104,"num_read_kb":110,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59762,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492878+0000","last_change":"2024-08-20T03:00:18.242931+0000","last_active":"2024-08-20T03:00:59.492878+0000","last_peered":"2024-08-20T03:00:59.492878+0000","last_clean":"2024-08-20T03:00:59.492878+0000","last_became_active":"2024-08-20T03:00:18.242619+0000","last_became_peered":"2024-08-20T03:00:18.242619+0000","last_unstale":"2024-08-20T03:00:59.492878+0000","last_undegraded":"2024-08-20T03:00:59.492878+0000","last_fullsized":"2024-08-20T03:00:59.492878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:43:02.296990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492488+0000","last_change":"2024-08-20T03:00:16.214688+0000","last_active":"2024-08-20T03:00:59.492488+0000","last_peered":"2024-08-20T03:00:59.492488+0000","last_clean":"2024-08-20T03:00:59.492488+0000","last_became_active":"2024-08-20T03:00:16.214343+0000","last_became_peered":"2024-08-20T03:00:16.214343+0000","last_unstale":"2024-08-20T03:00:59.492488+0000","last_undegraded":"2024-08-20T03:00:59.492488+0000","last_fullsized":"2024-08-20T03:00:59.492488+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:10:56.587164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.489936+0000","last_change":"2024-08-20T03:00:18.244158+0000","last_active":"2024-08-20T03:00:59.489936+0000","last_peered":"2024-08-20T03:00:59.489936+0000","last_clean":"2024-08-20T03:00:59.489936+0000","last_became_active":"2024-08-20T03:00:18.243864+0000","last_became_peered":"2024-08-20T03:00:18.243864+0000","last_unstale":"2024-08-20T03:00:59.489936+0000","last_undegraded":"2024-08-20T03:00:59.489936+0000","last_fullsized":"2024-08-20T03:00:59.489936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:10:31.373759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491090+0000","last_change":"2024-08-20T03:00:16.220846+0000","last_active":"2024-08-20T03:00:59.491090+0000","last_peered":"2024-08-20T03:00:59.491090+0000","last_clean":"2024-08-20T03:00:59.491090+0000","last_became_active":"2024-08-20T03:00:16.220519+0000","last_became_peered":"2024-08-20T03:00:16.220519+0000","last_unstale":"2024-08-20T03:00:59.491090+0000","last_undegraded":"2024-08-20T03:00:59.491090+0000","last_fullsized":"2024-08-20T03:00:59.491090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:43:41.721522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490638+0000","last_change":"2024-08-20T03:00:18.246555+0000","last_active":"2024-08-20T03:00:59.490638+0000","last_peered":"2024-08-20T03:00:59.490638+0000","last_clean":"2024-08-20T03:00:59.490638+0000","last_became_active":"2024-08-20T03:00:18.246182+0000","last_became_peered":"2024-08-20T03:00:18.246182+0000","last_unstale":"2024-08-20T03:00:59.490638+0000","last_undegraded":"2024-08-20T03:00:59.490638+0000","last_fullsized":"2024-08-20T03:00:59.490638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:55.827649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:22.462848+0000","last_change":"2024-08-20T03:00:16.216521+0000","last_active":"2024-08-20T03:01:22.462848+0000","last_peered":"2024-08-20T03:01:22.462848+0000","last_clean":"2024-08-20T03:01:22.462848+0000","last_became_active":"2024-08-20T03:00:16.216187+0000","last_became_peered":"2024-08-20T03:00:16.216187+0000","last_unstale":"2024-08-20T03:01:22.462848+0000","last_undegraded":"2024-08-20T03:01:22.462848+0000","last_fullsized":"2024-08-20T03:01:22.462848+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:41:13.639218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.544391+0000","last_change":"2024-08-20T03:00:59.544335+0000","last_active":"2024-08-20T03:00:59.544391+0000","last_peered":"2024-08-20T03:00:59.544391+0000","last_clean":"2024-08-20T03:00:59.544391+0000","last_became_active":"2024-08-20T03:00:59.544063+0000","last_became_peered":"2024-08-20T03:00:59.544063+0000","last_unstale":"2024-08-20T03:00:59.544391+0000","last_undegraded":"2024-08-20T03:00:59.544391+0000","last_fullsized":"2024-08-20T03:00:59.544391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:48:36.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490874+0000","last_change":"2024-08-20T03:00:16.470129+0000","last_active":"2024-08-20T03:00:59.490874+0000","last_peered":"2024-08-20T03:00:59.490874+0000","last_clean":"2024-08-20T03:00:59.490874+0000","last_became_active":"2024-08-20T03:00:16.469697+0000","last_became_peered":"2024-08-20T03:00:16.469697+0000","last_unstale":"2024-08-20T03:00:59.490874+0000","last_undegraded":"2024-08-20T03:00:59.490874+0000","last_fullsized":"2024-08-20T03:00:59.490874+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:42:22.623141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490796+0000","last_change":"2024-08-20T03:00:18.245090+0000","last_active":"2024-08-20T03:00:59.490796+0000","last_peered":"2024-08-20T03:00:59.490796+0000","last_clean":"2024-08-20T03:00:59.490796+0000","last_became_active":"2024-08-20T03:00:18.244524+0000","last_became_peered":"2024-08-20T03:00:18.244524+0000","last_unstale":"2024-08-20T03:00:59.490796+0000","last_undegraded":"2024-08-20T03:00:59.490796+0000","last_fullsized":"2024-08-20T03:00:59.490796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:14.561183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492233+0000","last_change":"2024-08-20T03:00:16.220042+0000","last_active":"2024-08-20T03:00:59.492233+0000","last_peered":"2024-08-20T03:00:59.492233+0000","last_clean":"2024-08-20T03:00:59.492233+0000","last_became_active":"2024-08-20T03:00:16.219567+0000","last_became_peered":"2024-08-20T03:00:16.219567+0000","last_unstale":"2024-08-20T03:00:59.492233+0000","last_undegraded":"2024-08-20T03:00:59.492233+0000","last_fullsized":"2024-08-20T03:00:59.492233+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:04.834039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.488320+0000","last_change":"2024-08-20T03:00:18.237497+0000","last_active":"2024-08-20T03:00:59.488320+0000","last_peered":"2024-08-20T03:00:59.488320+0000","last_clean":"2024-08-20T03:00:59.488320+0000","last_became_active":"2024-08-20T03:00:18.237170+0000","last_became_peered":"2024-08-20T03:00:18.237170+0000","last_unstale":"2024-08-20T03:00:59.488320+0000","last_undegraded":"2024-08-20T03:00:59.488320+0000","last_fullsized":"2024-08-20T03:00:59.488320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:52.387152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490226+0000","last_change":"2024-08-20T03:00:16.214861+0000","last_active":"2024-08-20T03:00:59.490226+0000","last_peered":"2024-08-20T03:00:59.490226+0000","last_clean":"2024-08-20T03:00:59.490226+0000","last_became_active":"2024-08-20T03:00:16.214487+0000","last_became_peered":"2024-08-20T03:00:16.214487+0000","last_unstale":"2024-08-20T03:00:59.490226+0000","last_undegraded":"2024-08-20T03:00:59.490226+0000","last_fullsized":"2024-08-20T03:00:59.490226+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:50:09.933304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491146+0000","last_change":"2024-08-20T03:00:18.244685+0000","last_active":"2024-08-20T03:00:59.491146+0000","last_peered":"2024-08-20T03:00:59.491146+0000","last_clean":"2024-08-20T03:00:59.491146+0000","last_became_active":"2024-08-20T03:00:18.244372+0000","last_became_peered":"2024-08-20T03:00:18.244372+0000","last_unstale":"2024-08-20T03:00:59.491146+0000","last_undegraded":"2024-08-20T03:00:59.491146+0000","last_fullsized":"2024-08-20T03:00:59.491146+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:21.821712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491625+0000","last_change":"2024-08-20T03:00:16.222287+0000","last_active":"2024-08-20T03:00:59.491625+0000","last_peered":"2024-08-20T03:00:59.491625+0000","last_clean":"2024-08-20T03:00:59.491625+0000","last_became_active":"2024-08-20T03:00:16.221989+0000","last_became_peered":"2024-08-20T03:00:16.221989+0000","last_unstale":"2024-08-20T03:00:59.491625+0000","last_undegraded":"2024-08-20T03:00:59.491625+0000","last_fullsized":"2024-08-20T03:00:59.491625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:32:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490950+0000","last_change":"2024-08-20T03:00:18.250488+0000","last_active":"2024-08-20T03:00:59.490950+0000","last_peered":"2024-08-20T03:00:59.490950+0000","last_clean":"2024-08-20T03:00:59.490950+0000","last_became_active":"2024-08-20T03:00:18.250094+0000","last_became_peered":"2024-08-20T03:00:18.250094+0000","last_unstale":"2024-08-20T03:00:59.490950+0000","last_undegraded":"2024-08-20T03:00:59.490950+0000","last_fullsized":"2024-08-20T03:00:59.490950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:00:55.755920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":81,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:19.459159+0000","last_change":"2024-08-20T03:00:16.470223+0000","last_active":"2024-08-20T03:01:19.459159+0000","last_peered":"2024-08-20T03:01:19.459159+0000","last_clean":"2024-08-20T03:01:19.459159+0000","last_became_active":"2024-08-20T03:00:16.469871+0000","last_became_peered":"2024-08-20T03:00:16.469871+0000","last_unstale":"2024-08-20T03:01:19.459159+0000","last_undegraded":"2024-08-20T03:01:19.459159+0000","last_fullsized":"2024-08-20T03:01:19.459159+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:05:11.779164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491172+0000","last_change":"2024-08-20T03:00:18.242755+0000","last_active":"2024-08-20T03:00:59.491172+0000","last_peered":"2024-08-20T03:00:59.491172+0000","last_clean":"2024-08-20T03:00:59.491172+0000","last_became_active":"2024-08-20T03:00:18.242447+0000","last_became_peered":"2024-08-20T03:00:18.242447+0000","last_unstale":"2024-08-20T03:00:59.491172+0000","last_undegraded":"2024-08-20T03:00:59.491172+0000","last_fullsized":"2024-08-20T03:00:59.491172+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:55:40.107933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490574+0000","last_change":"2024-08-20T02:59:02.774484+0000","last_active":"2024-08-20T03:00:59.490574+0000","last_peered":"2024-08-20T03:00:59.490574+0000","last_clean":"2024-08-20T03:00:59.490574+0000","last_became_active":"2024-08-20T02:59:02.769208+0000","last_became_peered":"2024-08-20T02:59:02.769208+0000","last_unstale":"2024-08-20T03:00:59.490574+0000","last_undegraded":"2024-08-20T03:00:59.490574+0000","last_fullsized":"2024-08-20T03:00:59.490574+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_clean_scrub_stamp":"2024-08-20T02:55:46.827249+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:36:27.931786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.516703+0000","last_change":"2024-08-20T03:00:16.213869+0000","last_active":"2024-08-20T03:00:59.516703+0000","last_peered":"2024-08-20T03:00:59.516703+0000","last_clean":"2024-08-20T03:00:59.516703+0000","last_became_active":"2024-08-20T03:00:16.213478+0000","last_became_peered":"2024-08-20T03:00:16.213478+0000","last_unstale":"2024-08-20T03:00:59.516703+0000","last_undegraded":"2024-08-20T03:00:59.516703+0000","last_fullsized":"2024-08-20T03:00:59.516703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:27.830497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487652+0000","last_change":"2024-08-20T03:00:18.242283+0000","last_active":"2024-08-20T03:00:59.487652+0000","last_peered":"2024-08-20T03:00:59.487652+0000","last_clean":"2024-08-20T03:00:59.487652+0000","last_became_active":"2024-08-20T03:00:18.241943+0000","last_became_peered":"2024-08-20T03:00:18.241943+0000","last_unstale":"2024-08-20T03:00:59.487652+0000","last_undegraded":"2024-08-20T03:00:59.487652+0000","last_fullsized":"2024-08-20T03:00:59.487652+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:22:25.776964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.498057+0000","last_change":"2024-08-20T03:00:59.497552+0000","last_active":"2024-08-20T03:00:59.498057+0000","last_peered":"2024-08-20T03:00:59.498057+0000","last_clean":"2024-08-20T03:00:59.498057+0000","last_became_active":"2024-08-20T03:00:59.493978+0000","last_became_peered":"2024-08-20T03:00:59.493978+0000","last_unstale":"2024-08-20T03:00:59.498057+0000","last_undegraded":"2024-08-20T03:00:59.498057+0000","last_fullsized":"2024-08-20T03:00:59.498057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:38:12.908159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.544757+0000","last_change":"2024-08-20T03:00:59.544696+0000","last_active":"2024-08-20T03:00:59.544757+0000","last_peered":"2024-08-20T03:00:59.544757+0000","last_clean":"2024-08-20T03:00:59.544757+0000","last_became_active":"2024-08-20T03:00:59.544303+0000","last_became_peered":"2024-08-20T03:00:59.544303+0000","last_unstale":"2024-08-20T03:00:59.544757+0000","last_undegraded":"2024-08-20T03:00:59.544757+0000","last_fullsized":"2024-08-20T03:00:59.544757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:01:04.814695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":81,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:20.459509+0000","last_change":"2024-08-20T03:00:16.468073+0000","last_active":"2024-08-20T03:01:20.459509+0000","last_peered":"2024-08-20T03:01:20.459509+0000","last_clean":"2024-08-20T03:01:20.459509+0000","last_became_active":"2024-08-20T03:00:16.467560+0000","last_became_peered":"2024-08-20T03:00:16.467560+0000","last_unstale":"2024-08-20T03:01:20.459509+0000","last_undegraded":"2024-08-20T03:01:20.459509+0000","last_fullsized":"2024-08-20T03:01:20.459509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:42:24.499777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492439+0000","last_change":"2024-08-20T03:00:18.232537+0000","last_active":"2024-08-20T03:00:59.492439+0000","last_peered":"2024-08-20T03:00:59.492439+0000","last_clean":"2024-08-20T03:00:59.492439+0000","last_became_active":"2024-08-20T03:00:18.232273+0000","last_became_peered":"2024-08-20T03:00:18.232273+0000","last_unstale":"2024-08-20T03:00:59.492439+0000","last_undegraded":"2024-08-20T03:00:59.492439+0000","last_fullsized":"2024-08-20T03:00:59.492439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:36:04.981869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'6","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:21.460983+0000","last_change":"2024-08-20T03:00:16.214807+0000","last_active":"2024-08-20T03:01:21.460983+0000","last_peered":"2024-08-20T03:01:21.460983+0000","last_clean":"2024-08-20T03:01:21.460983+0000","last_became_active":"2024-08-20T03:00:16.214439+0000","last_became_peered":"2024-08-20T03:00:16.214439+0000","last_unstale":"2024-08-20T03:01:21.460983+0000","last_undegraded":"2024-08-20T03:01:21.460983+0000","last_fullsized":"2024-08-20T03:01:21.460983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:07.519151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491420+0000","last_change":"2024-08-20T03:00:18.247220+0000","last_active":"2024-08-20T03:00:59.491420+0000","last_peered":"2024-08-20T03:00:59.491420+0000","last_clean":"2024-08-20T03:00:59.491420+0000","last_became_active":"2024-08-20T03:00:18.246896+0000","last_became_peered":"2024-08-20T03:00:18.246896+0000","last_unstale":"2024-08-20T03:00:59.491420+0000","last_undegraded":"2024-08-20T03:00:59.491420+0000","last_fullsized":"2024-08-20T03:00:59.491420+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:43:16.690590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543240+0000","last_change":"2024-08-20T03:00:16.468280+0000","last_active":"2024-08-20T03:00:59.543240+0000","last_peered":"2024-08-20T03:00:59.543240+0000","last_clean":"2024-08-20T03:00:59.543240+0000","last_became_active":"2024-08-20T03:00:16.467930+0000","last_became_peered":"2024-08-20T03:00:16.467930+0000","last_unstale":"2024-08-20T03:00:59.543240+0000","last_undegraded":"2024-08-20T03:00:59.543240+0000","last_fullsized":"2024-08-20T03:00:59.543240+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:30.423563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491804+0000","last_change":"2024-08-20T03:00:18.240322+0000","last_active":"2024-08-20T03:00:59.491804+0000","last_peered":"2024-08-20T03:00:59.491804+0000","last_clean":"2024-08-20T03:00:59.491804+0000","last_became_active":"2024-08-20T03:00:18.239973+0000","last_became_peered":"2024-08-20T03:00:18.239973+0000","last_unstale":"2024-08-20T03:00:59.491804+0000","last_undegraded":"2024-08-20T03:00:59.491804+0000","last_fullsized":"2024-08-20T03:00:59.491804+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:41:28.259762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.493654+0000","last_change":"2024-08-20T03:00:16.468307+0000","last_active":"2024-08-20T03:00:59.493654+0000","last_peered":"2024-08-20T03:00:59.493654+0000","last_clean":"2024-08-20T03:00:59.493654+0000","last_became_active":"2024-08-20T03:00:16.467993+0000","last_became_peered":"2024-08-20T03:00:16.467993+0000","last_unstale":"2024-08-20T03:00:59.493654+0000","last_undegraded":"2024-08-20T03:00:59.493654+0000","last_fullsized":"2024-08-20T03:00:59.493654+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:34.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490183+0000","last_change":"2024-08-20T03:00:18.234009+0000","last_active":"2024-08-20T03:00:59.490183+0000","last_peered":"2024-08-20T03:00:59.490183+0000","last_clean":"2024-08-20T03:00:59.490183+0000","last_became_active":"2024-08-20T03:00:18.233311+0000","last_became_peered":"2024-08-20T03:00:18.233311+0000","last_unstale":"2024-08-20T03:00:59.490183+0000","last_undegraded":"2024-08-20T03:00:59.490183+0000","last_fullsized":"2024-08-20T03:00:59.490183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:53:10.759339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'16","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543372+0000","last_change":"2024-08-20T03:00:16.214932+0000","last_active":"2024-08-20T03:00:59.543372+0000","last_peered":"2024-08-20T03:00:59.543372+0000","last_clean":"2024-08-20T03:00:59.543372+0000","last_became_active":"2024-08-20T03:00:16.214584+0000","last_became_peered":"2024-08-20T03:00:16.214584+0000","last_unstale":"2024-08-20T03:00:59.543372+0000","last_undegraded":"2024-08-20T03:00:59.543372+0000","last_fullsized":"2024-08-20T03:00:59.543372+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:47.270685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487711+0000","last_change":"2024-08-20T03:00:18.247892+0000","last_active":"2024-08-20T03:00:59.487711+0000","last_peered":"2024-08-20T03:00:59.487711+0000","last_clean":"2024-08-20T03:00:59.487711+0000","last_became_active":"2024-08-20T03:00:18.245903+0000","last_became_peered":"2024-08-20T03:00:18.245903+0000","last_unstale":"2024-08-20T03:00:59.487711+0000","last_undegraded":"2024-08-20T03:00:59.487711+0000","last_fullsized":"2024-08-20T03:00:59.487711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:08:50.909638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.516720+0000","last_change":"2024-08-20T03:00:16.372503+0000","last_active":"2024-08-20T03:00:59.516720+0000","last_peered":"2024-08-20T03:00:59.516720+0000","last_clean":"2024-08-20T03:00:59.516720+0000","last_became_active":"2024-08-20T03:00:16.372218+0000","last_became_peered":"2024-08-20T03:00:16.372218+0000","last_unstale":"2024-08-20T03:00:59.516720+0000","last_undegraded":"2024-08-20T03:00:59.516720+0000","last_fullsized":"2024-08-20T03:00:59.516720+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:13.341900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542560+0000","last_change":"2024-08-20T03:00:18.244738+0000","last_active":"2024-08-20T03:00:59.542560+0000","last_peered":"2024-08-20T03:00:59.542560+0000","last_clean":"2024-08-20T03:00:59.542560+0000","last_became_active":"2024-08-20T03:00:18.244398+0000","last_became_peered":"2024-08-20T03:00:18.244398+0000","last_unstale":"2024-08-20T03:00:59.542560+0000","last_undegraded":"2024-08-20T03:00:59.542560+0000","last_fullsized":"2024-08-20T03:00:59.542560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:45:14.540125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494339+0000","last_change":"2024-08-20T03:00:16.468251+0000","last_active":"2024-08-20T03:00:59.494339+0000","last_peered":"2024-08-20T03:00:59.494339+0000","last_clean":"2024-08-20T03:00:59.494339+0000","last_became_active":"2024-08-20T03:00:16.467912+0000","last_became_peered":"2024-08-20T03:00:16.467912+0000","last_unstale":"2024-08-20T03:00:59.494339+0000","last_undegraded":"2024-08-20T03:00:59.494339+0000","last_fullsized":"2024-08-20T03:00:59.494339+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:43.799747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490539+0000","last_change":"2024-08-20T03:00:18.234624+0000","last_active":"2024-08-20T03:00:59.490539+0000","last_peered":"2024-08-20T03:00:59.490539+0000","last_clean":"2024-08-20T03:00:59.490539+0000","last_became_active":"2024-08-20T03:00:18.234331+0000","last_became_peered":"2024-08-20T03:00:18.234331+0000","last_unstale":"2024-08-20T03:00:59.490539+0000","last_undegraded":"2024-08-20T03:00:59.490539+0000","last_fullsized":"2024-08-20T03:00:59.490539+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:05:09.665645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487438+0000","last_change":"2024-08-20T03:00:16.216658+0000","last_active":"2024-08-20T03:00:59.487438+0000","last_peered":"2024-08-20T03:00:59.487438+0000","last_clean":"2024-08-20T03:00:59.487438+0000","last_became_active":"2024-08-20T03:00:16.216352+0000","last_became_peered":"2024-08-20T03:00:16.216352+0000","last_unstale":"2024-08-20T03:00:59.487438+0000","last_undegraded":"2024-08-20T03:00:59.487438+0000","last_fullsized":"2024-08-20T03:00:59.487438+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:20.763280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.488349+0000","last_change":"2024-08-20T03:00:18.240984+0000","last_active":"2024-08-20T03:00:59.488349+0000","last_peered":"2024-08-20T03:00:59.488349+0000","last_clean":"2024-08-20T03:00:59.488349+0000","last_became_active":"2024-08-20T03:00:18.240640+0000","last_became_peered":"2024-08-20T03:00:18.240640+0000","last_unstale":"2024-08-20T03:00:59.488349+0000","last_undegraded":"2024-08-20T03:00:59.488349+0000","last_fullsized":"2024-08-20T03:00:59.488349+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:05.589186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492342+0000","last_change":"2024-08-20T03:00:16.214818+0000","last_active":"2024-08-20T03:00:59.492342+0000","last_peered":"2024-08-20T03:00:59.492342+0000","last_clean":"2024-08-20T03:00:59.492342+0000","last_became_active":"2024-08-20T03:00:16.214493+0000","last_became_peered":"2024-08-20T03:00:16.214493+0000","last_unstale":"2024-08-20T03:00:59.492342+0000","last_undegraded":"2024-08-20T03:00:59.492342+0000","last_fullsized":"2024-08-20T03:00:59.492342+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:56:12.895142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487487+0000","last_change":"2024-08-20T03:00:18.240350+0000","last_active":"2024-08-20T03:00:59.487487+0000","last_peered":"2024-08-20T03:00:59.487487+0000","last_clean":"2024-08-20T03:00:59.487487+0000","last_became_active":"2024-08-20T03:00:18.239464+0000","last_became_peered":"2024-08-20T03:00:18.239464+0000","last_unstale":"2024-08-20T03:00:59.487487+0000","last_undegraded":"2024-08-20T03:00:59.487487+0000","last_fullsized":"2024-08-20T03:00:59.487487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:48.715130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491431+0000","last_change":"2024-08-20T03:00:16.216144+0000","last_active":"2024-08-20T03:00:59.491431+0000","last_peered":"2024-08-20T03:00:59.491431+0000","last_clean":"2024-08-20T03:00:59.491431+0000","last_became_active":"2024-08-20T03:00:16.215792+0000","last_became_peered":"2024-08-20T03:00:16.215792+0000","last_unstale":"2024-08-20T03:00:59.491431+0000","last_undegraded":"2024-08-20T03:00:59.491431+0000","last_fullsized":"2024-08-20T03:00:59.491431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:25:01.500878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490589+0000","last_change":"2024-08-20T03:00:18.245286+0000","last_active":"2024-08-20T03:00:59.490589+0000","last_peered":"2024-08-20T03:00:59.490589+0000","last_clean":"2024-08-20T03:00:59.490589+0000","last_became_active":"2024-08-20T03:00:18.244760+0000","last_became_peered":"2024-08-20T03:00:18.244760+0000","last_unstale":"2024-08-20T03:00:59.490589+0000","last_undegraded":"2024-08-20T03:00:59.490589+0000","last_fullsized":"2024-08-20T03:00:59.490589+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:09.415794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.496088+0000","last_change":"2024-08-20T03:00:59.495948+0000","last_active":"2024-08-20T03:00:59.496088+0000","last_peered":"2024-08-20T03:00:59.496088+0000","last_clean":"2024-08-20T03:00:59.496088+0000","last_became_active":"2024-08-20T03:00:59.495543+0000","last_became_peered":"2024-08-20T03:00:59.495543+0000","last_unstale":"2024-08-20T03:00:59.496088+0000","last_undegraded":"2024-08-20T03:00:59.496088+0000","last_fullsized":"2024-08-20T03:00:59.496088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:38:30.671235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492392+0000","last_change":"2024-08-20T03:00:16.220019+0000","last_active":"2024-08-20T03:00:59.492392+0000","last_peered":"2024-08-20T03:00:59.492392+0000","last_clean":"2024-08-20T03:00:59.492392+0000","last_became_active":"2024-08-20T03:00:16.219624+0000","last_became_peered":"2024-08-20T03:00:16.219624+0000","last_unstale":"2024-08-20T03:00:59.492392+0000","last_undegraded":"2024-08-20T03:00:59.492392+0000","last_fullsized":"2024-08-20T03:00:59.492392+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:33:24.926695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494355+0000","last_change":"2024-08-20T03:00:18.242156+0000","last_active":"2024-08-20T03:00:59.494355+0000","last_peered":"2024-08-20T03:00:59.494355+0000","last_clean":"2024-08-20T03:00:59.494355+0000","last_became_active":"2024-08-20T03:00:18.241811+0000","last_became_peered":"2024-08-20T03:00:18.241811+0000","last_unstale":"2024-08-20T03:00:59.494355+0000","last_undegraded":"2024-08-20T03:00:59.494355+0000","last_fullsized":"2024-08-20T03:00:59.494355+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:58.230658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.534283+0000","last_change":"2024-08-20T03:00:59.533906+0000","last_active":"2024-08-20T03:00:59.534283+0000","last_peered":"2024-08-20T03:00:59.534283+0000","last_clean":"2024-08-20T03:00:59.534283+0000","last_became_active":"2024-08-20T03:00:59.517824+0000","last_became_peered":"2024-08-20T03:00:59.517824+0000","last_unstale":"2024-08-20T03:00:59.534283+0000","last_undegraded":"2024-08-20T03:00:59.534283+0000","last_fullsized":"2024-08-20T03:00:59.534283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:25:28.219430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542643+0000","last_change":"2024-08-20T03:00:18.245256+0000","last_active":"2024-08-20T03:00:59.542643+0000","last_peered":"2024-08-20T03:00:59.542643+0000","last_clean":"2024-08-20T03:00:59.542643+0000","last_became_active":"2024-08-20T03:00:18.244339+0000","last_became_peered":"2024-08-20T03:00:18.244339+0000","last_unstale":"2024-08-20T03:00:59.542643+0000","last_undegraded":"2024-08-20T03:00:59.542643+0000","last_fullsized":"2024-08-20T03:00:59.542643+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:00.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487797+0000","last_change":"2024-08-20T03:00:16.470004+0000","last_active":"2024-08-20T03:00:59.487797+0000","last_peered":"2024-08-20T03:00:59.487797+0000","last_clean":"2024-08-20T03:00:59.487797+0000","last_became_active":"2024-08-20T03:00:16.469646+0000","last_became_peered":"2024-08-20T03:00:16.469646+0000","last_unstale":"2024-08-20T03:00:59.487797+0000","last_undegraded":"2024-08-20T03:00:59.487797+0000","last_fullsized":"2024-08-20T03:00:59.487797+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:35.244683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487684+0000","last_change":"2024-08-20T03:00:18.238053+0000","last_active":"2024-08-20T03:00:59.487684+0000","last_peered":"2024-08-20T03:00:59.487684+0000","last_clean":"2024-08-20T03:00:59.487684+0000","last_became_active":"2024-08-20T03:00:18.237186+0000","last_became_peered":"2024-08-20T03:00:18.237186+0000","last_unstale":"2024-08-20T03:00:59.487684+0000","last_undegraded":"2024-08-20T03:00:59.487684+0000","last_fullsized":"2024-08-20T03:00:59.487684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:14.425688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":89,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:20.459087+0000","last_change":"2024-08-20T03:00:16.214584+0000","last_active":"2024-08-20T03:01:20.459087+0000","last_peered":"2024-08-20T03:01:20.459087+0000","last_clean":"2024-08-20T03:01:20.459087+0000","last_became_active":"2024-08-20T03:00:16.214255+0000","last_became_peered":"2024-08-20T03:00:16.214255+0000","last_unstale":"2024-08-20T03:01:20.459087+0000","last_undegraded":"2024-08-20T03:01:20.459087+0000","last_fullsized":"2024-08-20T03:01:20.459087+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:16:30.983745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":552,"num_read_kb":346,"num_write":245,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":59762,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":200898,"data_compressed":9639,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281428,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743451,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":843776,"data_stored":417802,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939679,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":999424,"data_stored":1068593,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103205,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":868352,"data_stored":417732,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299433,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":864256,"data_stored":418110,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462959,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":1011019,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659187,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":851968,"data_stored":417840,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":1010306,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":477072,"data_compressed":15466,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":868352,"data_stored":417274,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378695,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":418973,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542220,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":913408,"data_stored":478515,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":418788,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":60462,"data_compressed":9639,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T03:01:26.222 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T03:01:26.479 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:26.884 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:26 smithi044 ceph-mon[18790]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:01:26.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.962+0000 7f4317fd0640 1 -- 172.21.15.44:0/3720908201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4310101520 msgr2=0x7f4310101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:26.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.962+0000 7f4317fd0640 1 --2- 172.21.15.44:0/3720908201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4310101520 0x7f4310101900 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f430c009900 tx=0x7f430c02a410 comp rx=0 tx=0).stop 2024-08-20T03:01:26.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 -- 172.21.15.44:0/3720908201 shutdown_connections 2024-08-20T03:01:26.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 --2- 172.21.15.44:0/3720908201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431010cf60 0x7f431010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:26.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 --2- 172.21.15.44:0/3720908201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f431010a630 0x7f431010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:26.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 --2- 172.21.15.44:0/3720908201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4310101520 0x7f4310101900 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:26.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 -- 172.21.15.44:0/3720908201 >> 172.21.15.44:0/3720908201 conn(0x7f4310076c80 msgr2=0x7f4310077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:26.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 -- 172.21.15.44:0/3720908201 shutdown_connections 2024-08-20T03:01:26.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 -- 172.21.15.44:0/3720908201 wait complete. 2024-08-20T03:01:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 Processor -- start 2024-08-20T03:01:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 -- start start 2024-08-20T03:01:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4310101520 0x7f43101024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431010a630 0x7f4310102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f431010cf60 0x7f4310102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4310112ac0 con 0x7f4310101520 2024-08-20T03:01:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4310112940 con 0x7f431010cf60 2024-08-20T03:01:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.963+0000 7f4317fd0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4310112c40 con 0x7f431010a630 2024-08-20T03:01:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4315d6c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4310101520 0x7f43101024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4315d6c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4310101520 0x7f43101024f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49290/0 (socket says 172.21.15.44:49290) 2024-08-20T03:01:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4315d6c640 1 -- 172.21.15.44:0/1590569473 learned_addr learned my addr 172.21.15.44:0/1590569473 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:26.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f431556b640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431010a630 0x7f4310102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f431656d640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f431010cf60 0x7f4310102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4315d6c640 1 -- 172.21.15.44:0/1590569473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431010a630 msgr2=0x7f4310102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4315d6c640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431010a630 0x7f4310102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4315d6c640 1 -- 172.21.15.44:0/1590569473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f431010cf60 msgr2=0x7f4310102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4315d6c640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f431010cf60 0x7f4310102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4315d6c640 1 -- 172.21.15.44:0/1590569473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43101a7410 con 0x7f4310101520 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4315d6c640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4310101520 0x7f43101024f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4306ffd640 1 -- 172.21.15.44:0/1590569473 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c03f030 con 0x7f4310101520 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4306ffd640 1 -- 172.21.15.44:0/1590569473 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f430c003800 con 0x7f4310101520 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4317fd0640 1 -- 172.21.15.44:0/1590569473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43101a76a0 con 0x7f4310101520 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4306ffd640 1 -- 172.21.15.44:0/1590569473 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c035070 con 0x7f4310101520 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.964+0000 7f4317fd0640 1 -- 172.21.15.44:0/1590569473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43101a7c00 con 0x7f4310101520 2024-08-20T03:01:26.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.965+0000 7f4317fd0640 1 -- 172.21.15.44:0/1590569473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4310101900 con 0x7f4310101520 2024-08-20T03:01:26.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.966+0000 7f4306ffd640 1 -- 172.21.15.44:0/1590569473 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f430c003cb0 con 0x7f4310101520 2024-08-20T03:01:26.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.967+0000 7f4306ffd640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42fc0b3cb0 0x7f42fc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:26.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.967+0000 7f431556b640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42fc0b3cb0 0x7f42fc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:26.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.967+0000 7f4306ffd640 1 -- 172.21.15.44:0/1590569473 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f430c1049b0 con 0x7f4310101520 2024-08-20T03:01:26.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.968+0000 7f431556b640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42fc0b3cb0 0x7f42fc0b6170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4310103fe0 tx=0x7f430000a4c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:26.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:26.971+0000 7f4306ffd640 1 -- 172.21.15.44:0/1590569473 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f430c0cbd90 con 0x7f4310101520 2024-08-20T03:01:27.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:26 smithi165 ceph-mon[22357]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:01:27.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.153+0000 7f4317fd0640 1 -- 172.21.15.44:0/1590569473 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43101009c0 con 0x7f42fc0b3cb0 2024-08-20T03:01:27.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.157+0000 7f4306ffd640 1 -- 172.21.15.44:0/1590569473 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259036 (secure 0 0 0) 0x7f43101009c0 con 0x7f42fc0b3cb0 2024-08-20T03:01:27.161 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:27.162 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-08-20T03:01:27.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.160+0000 7f4317fd0640 1 -- 172.21.15.44:0/1590569473 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42fc0b3cb0 msgr2=0x7f42fc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:27.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.160+0000 7f4317fd0640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42fc0b3cb0 0x7f42fc0b6170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4310103fe0 tx=0x7f430000a4c0 comp rx=0 tx=0).stop 2024-08-20T03:01:27.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.160+0000 7f4317fd0640 1 -- 172.21.15.44:0/1590569473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4310101520 msgr2=0x7f43101024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:27.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.160+0000 7f4317fd0640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4310101520 0x7f43101024f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c0042f0 comp rx=0 tx=0).stop 2024-08-20T03:01:27.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.160+0000 7f4317fd0640 1 -- 172.21.15.44:0/1590569473 shutdown_connections 2024-08-20T03:01:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.160+0000 7f4317fd0640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42fc0b3cb0 0x7f42fc0b6170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.160+0000 7f4317fd0640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f431010cf60 0x7f4310102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.160+0000 7f4317fd0640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431010a630 0x7f4310102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.160+0000 7f4317fd0640 1 --2- 172.21.15.44:0/1590569473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4310101520 0x7f43101024f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.160+0000 7f4317fd0640 1 -- 172.21.15.44:0/1590569473 >> 172.21.15.44:0/1590569473 conn(0x7f4310076c80 msgr2=0x7f43101002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.160+0000 7f4317fd0640 1 -- 172.21.15.44:0/1590569473 shutdown_connections 2024-08-20T03:01:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:27.160+0000 7f4317fd0640 1 -- 172.21.15.44:0/1590569473 wait complete. 2024-08-20T03:01:27.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:26 smithi099 ceph-mon[21984]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:01:27.855 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-08-20T03:01:26.404660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":658,"num_read_kb":430,"num_write":370,"num_write_kb":2286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":3011602,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363900,"kb_used_data":10420,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577924,"statfs":{"total":1151940427776,"available":1151567794176,"internally_reserved":0,"allocated":10670080,"data_stored":6972024,"data_compressed":186921,"data_compressed_allocated":245760,"data_compressed_original":2023424,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002611"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.488814+0000","last_change":"2024-08-20T03:00:18.238595+0000","last_active":"2024-08-20T03:00:59.488814+0000","last_peered":"2024-08-20T03:00:59.488814+0000","last_clean":"2024-08-20T03:00:59.488814+0000","last_became_active":"2024-08-20T03:00:18.238306+0000","last_became_peered":"2024-08-20T03:00:18.238306+0000","last_unstale":"2024-08-20T03:00:59.488814+0000","last_undegraded":"2024-08-20T03:00:59.488814+0000","last_fullsized":"2024-08-20T03:00:59.488814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:33:04.681614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542497+0000","last_change":"2024-08-20T03:00:18.239722+0000","last_active":"2024-08-20T03:00:59.542497+0000","last_peered":"2024-08-20T03:00:59.542497+0000","last_clean":"2024-08-20T03:00:59.542497+0000","last_became_active":"2024-08-20T03:00:18.239391+0000","last_became_peered":"2024-08-20T03:00:18.239391+0000","last_unstale":"2024-08-20T03:00:59.542497+0000","last_undegraded":"2024-08-20T03:00:59.542497+0000","last_fullsized":"2024-08-20T03:00:59.542497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:08:34.657173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542224+0000","last_change":"2024-08-20T03:00:18.242389+0000","last_active":"2024-08-20T03:00:59.542224+0000","last_peered":"2024-08-20T03:00:59.542224+0000","last_clean":"2024-08-20T03:00:59.542224+0000","last_became_active":"2024-08-20T03:00:18.242080+0000","last_became_peered":"2024-08-20T03:00:18.242080+0000","last_unstale":"2024-08-20T03:00:59.542224+0000","last_undegraded":"2024-08-20T03:00:59.542224+0000","last_fullsized":"2024-08-20T03:00:59.542224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:37.042894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.517384+0000","last_change":"2024-08-20T03:00:18.233709+0000","last_active":"2024-08-20T03:00:59.517384+0000","last_peered":"2024-08-20T03:00:59.517384+0000","last_clean":"2024-08-20T03:00:59.517384+0000","last_became_active":"2024-08-20T03:00:18.233402+0000","last_became_peered":"2024-08-20T03:00:18.233402+0000","last_unstale":"2024-08-20T03:00:59.517384+0000","last_undegraded":"2024-08-20T03:00:59.517384+0000","last_fullsized":"2024-08-20T03:00:59.517384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:30:01.308181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491456+0000","last_change":"2024-08-20T03:00:18.237770+0000","last_active":"2024-08-20T03:00:59.491456+0000","last_peered":"2024-08-20T03:00:59.491456+0000","last_clean":"2024-08-20T03:00:59.491456+0000","last_became_active":"2024-08-20T03:00:18.237475+0000","last_became_peered":"2024-08-20T03:00:18.237475+0000","last_unstale":"2024-08-20T03:00:59.491456+0000","last_undegraded":"2024-08-20T03:00:59.491456+0000","last_fullsized":"2024-08-20T03:00:59.491456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:37:12.707935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491276+0000","last_change":"2024-08-20T03:00:18.245400+0000","last_active":"2024-08-20T03:00:59.491276+0000","last_peered":"2024-08-20T03:00:59.491276+0000","last_clean":"2024-08-20T03:00:59.491276+0000","last_became_active":"2024-08-20T03:00:18.244735+0000","last_became_peered":"2024-08-20T03:00:18.244735+0000","last_unstale":"2024-08-20T03:00:59.491276+0000","last_undegraded":"2024-08-20T03:00:59.491276+0000","last_fullsized":"2024-08-20T03:00:59.491276+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:23:25.459441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490416+0000","last_change":"2024-08-20T03:00:18.247366+0000","last_active":"2024-08-20T03:00:59.490416+0000","last_peered":"2024-08-20T03:00:59.490416+0000","last_clean":"2024-08-20T03:00:59.490416+0000","last_became_active":"2024-08-20T03:00:18.247032+0000","last_became_peered":"2024-08-20T03:00:18.247032+0000","last_unstale":"2024-08-20T03:00:59.490416+0000","last_undegraded":"2024-08-20T03:00:59.490416+0000","last_fullsized":"2024-08-20T03:00:59.490416+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:02:18.070393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487313+0000","last_change":"2024-08-20T03:00:18.240845+0000","last_active":"2024-08-20T03:00:59.487313+0000","last_peered":"2024-08-20T03:00:59.487313+0000","last_clean":"2024-08-20T03:00:59.487313+0000","last_became_active":"2024-08-20T03:00:18.240531+0000","last_became_peered":"2024-08-20T03:00:18.240531+0000","last_unstale":"2024-08-20T03:00:59.487313+0000","last_undegraded":"2024-08-20T03:00:59.487313+0000","last_fullsized":"2024-08-20T03:00:59.487313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:15:18.975804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494688+0000","last_change":"2024-08-20T03:00:18.240817+0000","last_active":"2024-08-20T03:00:59.494688+0000","last_peered":"2024-08-20T03:00:59.494688+0000","last_clean":"2024-08-20T03:00:59.494688+0000","last_became_active":"2024-08-20T03:00:18.240262+0000","last_became_peered":"2024-08-20T03:00:18.240262+0000","last_unstale":"2024-08-20T03:00:59.494688+0000","last_undegraded":"2024-08-20T03:00:59.494688+0000","last_fullsized":"2024-08-20T03:00:59.494688+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:30.512597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491738+0000","last_change":"2024-08-20T03:00:18.242139+0000","last_active":"2024-08-20T03:00:59.491738+0000","last_peered":"2024-08-20T03:00:59.491738+0000","last_clean":"2024-08-20T03:00:59.491738+0000","last_became_active":"2024-08-20T03:00:18.241817+0000","last_became_peered":"2024-08-20T03:00:18.241817+0000","last_unstale":"2024-08-20T03:00:59.491738+0000","last_undegraded":"2024-08-20T03:00:59.491738+0000","last_fullsized":"2024-08-20T03:00:59.491738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:25.121306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491069+0000","last_change":"2024-08-20T03:00:18.243205+0000","last_active":"2024-08-20T03:00:59.491069+0000","last_peered":"2024-08-20T03:00:59.491069+0000","last_clean":"2024-08-20T03:00:59.491069+0000","last_became_active":"2024-08-20T03:00:18.242296+0000","last_became_peered":"2024-08-20T03:00:18.242296+0000","last_unstale":"2024-08-20T03:00:59.491069+0000","last_undegraded":"2024-08-20T03:00:59.491069+0000","last_fullsized":"2024-08-20T03:00:59.491069+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:50.490934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.517398+0000","last_change":"2024-08-20T03:00:18.237205+0000","last_active":"2024-08-20T03:00:59.517398+0000","last_peered":"2024-08-20T03:00:59.517398+0000","last_clean":"2024-08-20T03:00:59.517398+0000","last_became_active":"2024-08-20T03:00:18.236631+0000","last_became_peered":"2024-08-20T03:00:18.236631+0000","last_unstale":"2024-08-20T03:00:59.517398+0000","last_undegraded":"2024-08-20T03:00:59.517398+0000","last_fullsized":"2024-08-20T03:00:59.517398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:13.327598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490080+0000","last_change":"2024-08-20T03:00:18.235768+0000","last_active":"2024-08-20T03:00:59.490080+0000","last_peered":"2024-08-20T03:00:59.490080+0000","last_clean":"2024-08-20T03:00:59.490080+0000","last_became_active":"2024-08-20T03:00:18.235461+0000","last_became_peered":"2024-08-20T03:00:18.235461+0000","last_unstale":"2024-08-20T03:00:59.490080+0000","last_undegraded":"2024-08-20T03:00:59.490080+0000","last_fullsized":"2024-08-20T03:00:59.490080+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:08:22.238020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542285+0000","last_change":"2024-08-20T03:00:18.246560+0000","last_active":"2024-08-20T03:00:59.542285+0000","last_peered":"2024-08-20T03:00:59.542285+0000","last_clean":"2024-08-20T03:00:59.542285+0000","last_became_active":"2024-08-20T03:00:18.246270+0000","last_became_peered":"2024-08-20T03:00:18.246270+0000","last_unstale":"2024-08-20T03:00:59.542285+0000","last_undegraded":"2024-08-20T03:00:59.542285+0000","last_fullsized":"2024-08-20T03:00:59.542285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:47:13.402635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542123+0000","last_change":"2024-08-20T03:00:18.245907+0000","last_active":"2024-08-20T03:00:59.542123+0000","last_peered":"2024-08-20T03:00:59.542123+0000","last_clean":"2024-08-20T03:00:59.542123+0000","last_became_active":"2024-08-20T03:00:18.245429+0000","last_became_peered":"2024-08-20T03:00:18.245429+0000","last_unstale":"2024-08-20T03:00:59.542123+0000","last_undegraded":"2024-08-20T03:00:59.542123+0000","last_fullsized":"2024-08-20T03:00:59.542123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:46.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494222+0000","last_change":"2024-08-20T03:00:18.242561+0000","last_active":"2024-08-20T03:00:59.494222+0000","last_peered":"2024-08-20T03:00:59.494222+0000","last_clean":"2024-08-20T03:00:59.494222+0000","last_became_active":"2024-08-20T03:00:18.242048+0000","last_became_peered":"2024-08-20T03:00:18.242048+0000","last_unstale":"2024-08-20T03:00:59.494222+0000","last_undegraded":"2024-08-20T03:00:59.494222+0000","last_fullsized":"2024-08-20T03:00:59.494222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:27.784131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.488943+0000","last_change":"2024-08-20T03:00:18.238547+0000","last_active":"2024-08-20T03:00:59.488943+0000","last_peered":"2024-08-20T03:00:59.488943+0000","last_clean":"2024-08-20T03:00:59.488943+0000","last_became_active":"2024-08-20T03:00:18.238268+0000","last_became_peered":"2024-08-20T03:00:18.238268+0000","last_unstale":"2024-08-20T03:00:59.488943+0000","last_undegraded":"2024-08-20T03:00:59.488943+0000","last_fullsized":"2024-08-20T03:00:59.488943+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:15:11.923674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543347+0000","last_change":"2024-08-20T03:00:18.241995+0000","last_active":"2024-08-20T03:00:59.543347+0000","last_peered":"2024-08-20T03:00:59.543347+0000","last_clean":"2024-08-20T03:00:59.543347+0000","last_became_active":"2024-08-20T03:00:18.241705+0000","last_became_peered":"2024-08-20T03:00:18.241705+0000","last_unstale":"2024-08-20T03:00:59.543347+0000","last_undegraded":"2024-08-20T03:00:59.543347+0000","last_fullsized":"2024-08-20T03:00:59.543347+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:33:02.183712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491386+0000","last_change":"2024-08-20T03:00:18.236800+0000","last_active":"2024-08-20T03:00:59.491386+0000","last_peered":"2024-08-20T03:00:59.491386+0000","last_clean":"2024-08-20T03:00:59.491386+0000","last_became_active":"2024-08-20T03:00:18.235909+0000","last_became_peered":"2024-08-20T03:00:18.235909+0000","last_unstale":"2024-08-20T03:00:59.491386+0000","last_undegraded":"2024-08-20T03:00:59.491386+0000","last_fullsized":"2024-08-20T03:00:59.491386+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:12.427760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.544249+0000","last_change":"2024-08-20T03:00:59.544187+0000","last_active":"2024-08-20T03:00:59.544249+0000","last_peered":"2024-08-20T03:00:59.544249+0000","last_clean":"2024-08-20T03:00:59.544249+0000","last_became_active":"2024-08-20T03:00:59.543807+0000","last_became_peered":"2024-08-20T03:00:59.543807+0000","last_unstale":"2024-08-20T03:00:59.544249+0000","last_undegraded":"2024-08-20T03:00:59.544249+0000","last_fullsized":"2024-08-20T03:00:59.544249+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:16:12.425646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490151+0000","last_change":"2024-08-20T03:00:18.243425+0000","last_active":"2024-08-20T03:00:59.490151+0000","last_peered":"2024-08-20T03:00:59.490151+0000","last_clean":"2024-08-20T03:00:59.490151+0000","last_became_active":"2024-08-20T03:00:18.243082+0000","last_became_peered":"2024-08-20T03:00:18.243082+0000","last_unstale":"2024-08-20T03:00:59.490151+0000","last_undegraded":"2024-08-20T03:00:59.490151+0000","last_fullsized":"2024-08-20T03:00:59.490151+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:50.008655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494515+0000","last_change":"2024-08-20T03:00:18.242765+0000","last_active":"2024-08-20T03:00:59.494515+0000","last_peered":"2024-08-20T03:00:59.494515+0000","last_clean":"2024-08-20T03:00:59.494515+0000","last_became_active":"2024-08-20T03:00:18.242459+0000","last_became_peered":"2024-08-20T03:00:18.242459+0000","last_unstale":"2024-08-20T03:00:59.494515+0000","last_undegraded":"2024-08-20T03:00:59.494515+0000","last_fullsized":"2024-08-20T03:00:59.494515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:16.467068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487326+0000","last_change":"2024-08-20T03:00:18.236518+0000","last_active":"2024-08-20T03:00:59.487326+0000","last_peered":"2024-08-20T03:00:59.487326+0000","last_clean":"2024-08-20T03:00:59.487326+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:00:59.487326+0000","last_undegraded":"2024-08-20T03:00:59.487326+0000","last_fullsized":"2024-08-20T03:00:59.487326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:55:55.387134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492066+0000","last_change":"2024-08-20T03:00:18.236730+0000","last_active":"2024-08-20T03:00:59.492066+0000","last_peered":"2024-08-20T03:00:59.492066+0000","last_clean":"2024-08-20T03:00:59.492066+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:00:59.492066+0000","last_undegraded":"2024-08-20T03:00:59.492066+0000","last_fullsized":"2024-08-20T03:00:59.492066+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:18.196161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487634+0000","last_change":"2024-08-20T03:00:18.245390+0000","last_active":"2024-08-20T03:00:59.487634+0000","last_peered":"2024-08-20T03:00:59.487634+0000","last_clean":"2024-08-20T03:00:59.487634+0000","last_became_active":"2024-08-20T03:00:18.241175+0000","last_became_peered":"2024-08-20T03:00:18.241175+0000","last_unstale":"2024-08-20T03:00:59.487634+0000","last_undegraded":"2024-08-20T03:00:59.487634+0000","last_fullsized":"2024-08-20T03:00:59.487634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:51:45.792846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492615+0000","last_change":"2024-08-20T03:00:18.242700+0000","last_active":"2024-08-20T03:00:59.492615+0000","last_peered":"2024-08-20T03:00:59.492615+0000","last_clean":"2024-08-20T03:00:59.492615+0000","last_became_active":"2024-08-20T03:00:18.242407+0000","last_became_peered":"2024-08-20T03:00:18.242407+0000","last_unstale":"2024-08-20T03:00:59.492615+0000","last_undegraded":"2024-08-20T03:00:59.492615+0000","last_fullsized":"2024-08-20T03:00:59.492615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:31:28.061514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.488695+0000","last_change":"2024-08-20T03:00:18.233894+0000","last_active":"2024-08-20T03:00:59.488695+0000","last_peered":"2024-08-20T03:00:59.488695+0000","last_clean":"2024-08-20T03:00:59.488695+0000","last_became_active":"2024-08-20T03:00:18.233584+0000","last_became_peered":"2024-08-20T03:00:18.233584+0000","last_unstale":"2024-08-20T03:00:59.488695+0000","last_undegraded":"2024-08-20T03:00:59.488695+0000","last_fullsized":"2024-08-20T03:00:59.488695+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:36:53.156215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490178+0000","last_change":"2024-08-20T03:00:18.239256+0000","last_active":"2024-08-20T03:00:59.490178+0000","last_peered":"2024-08-20T03:00:59.490178+0000","last_clean":"2024-08-20T03:00:59.490178+0000","last_became_active":"2024-08-20T03:00:18.238931+0000","last_became_peered":"2024-08-20T03:00:18.238931+0000","last_unstale":"2024-08-20T03:00:59.490178+0000","last_undegraded":"2024-08-20T03:00:59.490178+0000","last_fullsized":"2024-08-20T03:00:59.490178+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:38:42.536488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542368+0000","last_change":"2024-08-20T03:00:18.245187+0000","last_active":"2024-08-20T03:00:59.542368+0000","last_peered":"2024-08-20T03:00:59.542368+0000","last_clean":"2024-08-20T03:00:59.542368+0000","last_became_active":"2024-08-20T03:00:18.244894+0000","last_became_peered":"2024-08-20T03:00:18.244894+0000","last_unstale":"2024-08-20T03:00:59.542368+0000","last_undegraded":"2024-08-20T03:00:59.542368+0000","last_fullsized":"2024-08-20T03:00:59.542368+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:16:58.002014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492240+0000","last_change":"2024-08-20T03:00:18.241634+0000","last_active":"2024-08-20T03:00:59.492240+0000","last_peered":"2024-08-20T03:00:59.492240+0000","last_clean":"2024-08-20T03:00:59.492240+0000","last_became_active":"2024-08-20T03:00:18.241341+0000","last_became_peered":"2024-08-20T03:00:18.241341+0000","last_unstale":"2024-08-20T03:00:59.492240+0000","last_undegraded":"2024-08-20T03:00:59.492240+0000","last_fullsized":"2024-08-20T03:00:59.492240+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:37:16.185544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492125+0000","last_change":"2024-08-20T03:00:18.239116+0000","last_active":"2024-08-20T03:00:59.492125+0000","last_peered":"2024-08-20T03:00:59.492125+0000","last_clean":"2024-08-20T03:00:59.492125+0000","last_became_active":"2024-08-20T03:00:18.236125+0000","last_became_peered":"2024-08-20T03:00:18.236125+0000","last_unstale":"2024-08-20T03:00:59.492125+0000","last_undegraded":"2024-08-20T03:00:59.492125+0000","last_fullsized":"2024-08-20T03:00:59.492125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:57:00.332483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.488228+0000","last_change":"2024-08-20T03:00:18.235996+0000","last_active":"2024-08-20T03:00:59.488228+0000","last_peered":"2024-08-20T03:00:59.488228+0000","last_clean":"2024-08-20T03:00:59.488228+0000","last_became_active":"2024-08-20T03:00:18.235670+0000","last_became_peered":"2024-08-20T03:00:18.235670+0000","last_unstale":"2024-08-20T03:00:59.488228+0000","last_undegraded":"2024-08-20T03:00:59.488228+0000","last_fullsized":"2024-08-20T03:00:59.488228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:59.992914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491090+0000","last_change":"2024-08-20T03:00:18.239679+0000","last_active":"2024-08-20T03:00:59.491090+0000","last_peered":"2024-08-20T03:00:59.491090+0000","last_clean":"2024-08-20T03:00:59.491090+0000","last_became_active":"2024-08-20T03:00:18.239361+0000","last_became_peered":"2024-08-20T03:00:18.239361+0000","last_unstale":"2024-08-20T03:00:59.491090+0000","last_undegraded":"2024-08-20T03:00:59.491090+0000","last_fullsized":"2024-08-20T03:00:59.491090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:28:09.430535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":86,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:22.463590+0000","last_change":"2024-08-20T03:00:16.221221+0000","last_active":"2024-08-20T03:01:22.463590+0000","last_peered":"2024-08-20T03:01:22.463590+0000","last_clean":"2024-08-20T03:01:22.463590+0000","last_became_active":"2024-08-20T03:00:16.220186+0000","last_became_peered":"2024-08-20T03:00:16.220186+0000","last_unstale":"2024-08-20T03:01:22.463590+0000","last_undegraded":"2024-08-20T03:01:22.463590+0000","last_fullsized":"2024-08-20T03:01:22.463590+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:51.045235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492241+0000","last_change":"2024-08-20T03:00:18.234809+0000","last_active":"2024-08-20T03:00:59.492241+0000","last_peered":"2024-08-20T03:00:59.492241+0000","last_clean":"2024-08-20T03:00:59.492241+0000","last_became_active":"2024-08-20T03:00:18.234486+0000","last_became_peered":"2024-08-20T03:00:18.234486+0000","last_unstale":"2024-08-20T03:00:59.492241+0000","last_undegraded":"2024-08-20T03:00:59.492241+0000","last_fullsized":"2024-08-20T03:00:59.492241+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:11:52.527224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491303+0000","last_change":"2024-08-20T03:00:16.216876+0000","last_active":"2024-08-20T03:00:59.491303+0000","last_peered":"2024-08-20T03:00:59.491303+0000","last_clean":"2024-08-20T03:00:59.491303+0000","last_became_active":"2024-08-20T03:00:16.216522+0000","last_became_peered":"2024-08-20T03:00:16.216522+0000","last_unstale":"2024-08-20T03:00:59.491303+0000","last_undegraded":"2024-08-20T03:00:59.491303+0000","last_fullsized":"2024-08-20T03:00:59.491303+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:22.363331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.489899+0000","last_change":"2024-08-20T03:00:16.213751+0000","last_active":"2024-08-20T03:00:59.489899+0000","last_peered":"2024-08-20T03:00:59.489899+0000","last_clean":"2024-08-20T03:00:59.489899+0000","last_became_active":"2024-08-20T03:00:16.213348+0000","last_became_peered":"2024-08-20T03:00:16.213348+0000","last_unstale":"2024-08-20T03:00:59.489899+0000","last_undegraded":"2024-08-20T03:00:59.489899+0000","last_fullsized":"2024-08-20T03:00:59.489899+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:01:08.637151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487384+0000","last_change":"2024-08-20T03:00:18.237972+0000","last_active":"2024-08-20T03:00:59.487384+0000","last_peered":"2024-08-20T03:00:59.487384+0000","last_clean":"2024-08-20T03:00:59.487384+0000","last_became_active":"2024-08-20T03:00:18.236769+0000","last_became_peered":"2024-08-20T03:00:18.236769+0000","last_unstale":"2024-08-20T03:00:59.487384+0000","last_undegraded":"2024-08-20T03:00:59.487384+0000","last_fullsized":"2024-08-20T03:00:59.487384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:35:21.001045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492196+0000","last_change":"2024-08-20T03:00:16.221399+0000","last_active":"2024-08-20T03:00:59.492196+0000","last_peered":"2024-08-20T03:00:59.492196+0000","last_clean":"2024-08-20T03:00:59.492196+0000","last_became_active":"2024-08-20T03:00:16.220285+0000","last_became_peered":"2024-08-20T03:00:16.220285+0000","last_unstale":"2024-08-20T03:00:59.492196+0000","last_undegraded":"2024-08-20T03:00:59.492196+0000","last_fullsized":"2024-08-20T03:00:59.492196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:17.034657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543041+0000","last_change":"2024-08-20T03:00:18.236722+0000","last_active":"2024-08-20T03:00:59.543041+0000","last_peered":"2024-08-20T03:00:59.543041+0000","last_clean":"2024-08-20T03:00:59.543041+0000","last_became_active":"2024-08-20T03:00:18.236365+0000","last_became_peered":"2024-08-20T03:00:18.236365+0000","last_unstale":"2024-08-20T03:00:59.543041+0000","last_undegraded":"2024-08-20T03:00:59.543041+0000","last_fullsized":"2024-08-20T03:00:59.543041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:17.344161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":139,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:25.466561+0000","last_change":"2024-08-20T03:00:16.222331+0000","last_active":"2024-08-20T03:01:25.466561+0000","last_peered":"2024-08-20T03:01:25.466561+0000","last_clean":"2024-08-20T03:01:25.466561+0000","last_became_active":"2024-08-20T03:00:16.222019+0000","last_became_peered":"2024-08-20T03:00:16.222019+0000","last_unstale":"2024-08-20T03:01:25.466561+0000","last_undegraded":"2024-08-20T03:01:25.466561+0000","last_fullsized":"2024-08-20T03:01:25.466561+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:46:55.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494478+0000","last_change":"2024-08-20T03:00:18.239020+0000","last_active":"2024-08-20T03:00:59.494478+0000","last_peered":"2024-08-20T03:00:59.494478+0000","last_clean":"2024-08-20T03:00:59.494478+0000","last_became_active":"2024-08-20T03:00:18.238620+0000","last_became_peered":"2024-08-20T03:00:18.238620+0000","last_unstale":"2024-08-20T03:00:59.494478+0000","last_undegraded":"2024-08-20T03:00:59.494478+0000","last_fullsized":"2024-08-20T03:00:59.494478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:17:12.632357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491087+0000","last_change":"2024-08-20T03:00:16.214907+0000","last_active":"2024-08-20T03:00:59.491087+0000","last_peered":"2024-08-20T03:00:59.491087+0000","last_clean":"2024-08-20T03:00:59.491087+0000","last_became_active":"2024-08-20T03:00:16.214573+0000","last_became_peered":"2024-08-20T03:00:16.214573+0000","last_unstale":"2024-08-20T03:00:59.491087+0000","last_undegraded":"2024-08-20T03:00:59.491087+0000","last_fullsized":"2024-08-20T03:00:59.491087+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:21:34.084149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542627+0000","last_change":"2024-08-20T03:00:18.245524+0000","last_active":"2024-08-20T03:00:59.542627+0000","last_peered":"2024-08-20T03:00:59.542627+0000","last_clean":"2024-08-20T03:00:59.542627+0000","last_became_active":"2024-08-20T03:00:18.244966+0000","last_became_peered":"2024-08-20T03:00:18.244966+0000","last_unstale":"2024-08-20T03:00:59.542627+0000","last_undegraded":"2024-08-20T03:00:59.542627+0000","last_fullsized":"2024-08-20T03:00:59.542627+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:03:21.584182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491504+0000","last_change":"2024-08-20T03:00:16.470353+0000","last_active":"2024-08-20T03:00:59.491504+0000","last_peered":"2024-08-20T03:00:59.491504+0000","last_clean":"2024-08-20T03:00:59.491504+0000","last_became_active":"2024-08-20T03:00:16.469976+0000","last_became_peered":"2024-08-20T03:00:16.469976+0000","last_unstale":"2024-08-20T03:00:59.491504+0000","last_undegraded":"2024-08-20T03:00:59.491504+0000","last_fullsized":"2024-08-20T03:00:59.491504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:25:03.595784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494432+0000","last_change":"2024-08-20T03:00:18.245244+0000","last_active":"2024-08-20T03:00:59.494432+0000","last_peered":"2024-08-20T03:00:59.494432+0000","last_clean":"2024-08-20T03:00:59.494432+0000","last_became_active":"2024-08-20T03:00:18.244934+0000","last_became_peered":"2024-08-20T03:00:18.244934+0000","last_unstale":"2024-08-20T03:00:59.494432+0000","last_undegraded":"2024-08-20T03:00:59.494432+0000","last_fullsized":"2024-08-20T03:00:59.494432+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:33:35.035039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'30","reported_seq":194,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:25.555925+0000","last_change":"2024-08-20T03:00:59.539537+0000","last_active":"2024-08-20T03:01:25.555925+0000","last_peered":"2024-08-20T03:01:25.555925+0000","last_clean":"2024-08-20T03:01:25.555925+0000","last_became_active":"2024-08-20T03:00:59.518110+0000","last_became_peered":"2024-08-20T03:00:59.518110+0000","last_unstale":"2024-08-20T03:01:25.555925+0000","last_undegraded":"2024-08-20T03:01:25.555925+0000","last_fullsized":"2024-08-20T03:01:25.555925+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:10:58.722741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":115,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59762,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492878+0000","last_change":"2024-08-20T03:00:18.242931+0000","last_active":"2024-08-20T03:00:59.492878+0000","last_peered":"2024-08-20T03:00:59.492878+0000","last_clean":"2024-08-20T03:00:59.492878+0000","last_became_active":"2024-08-20T03:00:18.242619+0000","last_became_peered":"2024-08-20T03:00:18.242619+0000","last_unstale":"2024-08-20T03:00:59.492878+0000","last_undegraded":"2024-08-20T03:00:59.492878+0000","last_fullsized":"2024-08-20T03:00:59.492878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:43:02.296990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492488+0000","last_change":"2024-08-20T03:00:16.214688+0000","last_active":"2024-08-20T03:00:59.492488+0000","last_peered":"2024-08-20T03:00:59.492488+0000","last_clean":"2024-08-20T03:00:59.492488+0000","last_became_active":"2024-08-20T03:00:16.214343+0000","last_became_peered":"2024-08-20T03:00:16.214343+0000","last_unstale":"2024-08-20T03:00:59.492488+0000","last_undegraded":"2024-08-20T03:00:59.492488+0000","last_fullsized":"2024-08-20T03:00:59.492488+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:10:56.587164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.489936+0000","last_change":"2024-08-20T03:00:18.244158+0000","last_active":"2024-08-20T03:00:59.489936+0000","last_peered":"2024-08-20T03:00:59.489936+0000","last_clean":"2024-08-20T03:00:59.489936+0000","last_became_active":"2024-08-20T03:00:18.243864+0000","last_became_peered":"2024-08-20T03:00:18.243864+0000","last_unstale":"2024-08-20T03:00:59.489936+0000","last_undegraded":"2024-08-20T03:00:59.489936+0000","last_fullsized":"2024-08-20T03:00:59.489936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:10:31.373759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491090+0000","last_change":"2024-08-20T03:00:16.220846+0000","last_active":"2024-08-20T03:00:59.491090+0000","last_peered":"2024-08-20T03:00:59.491090+0000","last_clean":"2024-08-20T03:00:59.491090+0000","last_became_active":"2024-08-20T03:00:16.220519+0000","last_became_peered":"2024-08-20T03:00:16.220519+0000","last_unstale":"2024-08-20T03:00:59.491090+0000","last_undegraded":"2024-08-20T03:00:59.491090+0000","last_fullsized":"2024-08-20T03:00:59.491090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:43:41.721522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490638+0000","last_change":"2024-08-20T03:00:18.246555+0000","last_active":"2024-08-20T03:00:59.490638+0000","last_peered":"2024-08-20T03:00:59.490638+0000","last_clean":"2024-08-20T03:00:59.490638+0000","last_became_active":"2024-08-20T03:00:18.246182+0000","last_became_peered":"2024-08-20T03:00:18.246182+0000","last_unstale":"2024-08-20T03:00:59.490638+0000","last_undegraded":"2024-08-20T03:00:59.490638+0000","last_fullsized":"2024-08-20T03:00:59.490638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:55.827649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'5","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:22.462848+0000","last_change":"2024-08-20T03:00:16.216521+0000","last_active":"2024-08-20T03:01:22.462848+0000","last_peered":"2024-08-20T03:01:22.462848+0000","last_clean":"2024-08-20T03:01:22.462848+0000","last_became_active":"2024-08-20T03:00:16.216187+0000","last_became_peered":"2024-08-20T03:00:16.216187+0000","last_unstale":"2024-08-20T03:01:22.462848+0000","last_undegraded":"2024-08-20T03:01:22.462848+0000","last_fullsized":"2024-08-20T03:01:22.462848+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:41:13.639218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.544391+0000","last_change":"2024-08-20T03:00:59.544335+0000","last_active":"2024-08-20T03:00:59.544391+0000","last_peered":"2024-08-20T03:00:59.544391+0000","last_clean":"2024-08-20T03:00:59.544391+0000","last_became_active":"2024-08-20T03:00:59.544063+0000","last_became_peered":"2024-08-20T03:00:59.544063+0000","last_unstale":"2024-08-20T03:00:59.544391+0000","last_undegraded":"2024-08-20T03:00:59.544391+0000","last_fullsized":"2024-08-20T03:00:59.544391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:48:36.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490874+0000","last_change":"2024-08-20T03:00:16.470129+0000","last_active":"2024-08-20T03:00:59.490874+0000","last_peered":"2024-08-20T03:00:59.490874+0000","last_clean":"2024-08-20T03:00:59.490874+0000","last_became_active":"2024-08-20T03:00:16.469697+0000","last_became_peered":"2024-08-20T03:00:16.469697+0000","last_unstale":"2024-08-20T03:00:59.490874+0000","last_undegraded":"2024-08-20T03:00:59.490874+0000","last_fullsized":"2024-08-20T03:00:59.490874+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:42:22.623141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490796+0000","last_change":"2024-08-20T03:00:18.245090+0000","last_active":"2024-08-20T03:00:59.490796+0000","last_peered":"2024-08-20T03:00:59.490796+0000","last_clean":"2024-08-20T03:00:59.490796+0000","last_became_active":"2024-08-20T03:00:18.244524+0000","last_became_peered":"2024-08-20T03:00:18.244524+0000","last_unstale":"2024-08-20T03:00:59.490796+0000","last_undegraded":"2024-08-20T03:00:59.490796+0000","last_fullsized":"2024-08-20T03:00:59.490796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:14.561183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492233+0000","last_change":"2024-08-20T03:00:16.220042+0000","last_active":"2024-08-20T03:00:59.492233+0000","last_peered":"2024-08-20T03:00:59.492233+0000","last_clean":"2024-08-20T03:00:59.492233+0000","last_became_active":"2024-08-20T03:00:16.219567+0000","last_became_peered":"2024-08-20T03:00:16.219567+0000","last_unstale":"2024-08-20T03:00:59.492233+0000","last_undegraded":"2024-08-20T03:00:59.492233+0000","last_fullsized":"2024-08-20T03:00:59.492233+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:04.834039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.488320+0000","last_change":"2024-08-20T03:00:18.237497+0000","last_active":"2024-08-20T03:00:59.488320+0000","last_peered":"2024-08-20T03:00:59.488320+0000","last_clean":"2024-08-20T03:00:59.488320+0000","last_became_active":"2024-08-20T03:00:18.237170+0000","last_became_peered":"2024-08-20T03:00:18.237170+0000","last_unstale":"2024-08-20T03:00:59.488320+0000","last_undegraded":"2024-08-20T03:00:59.488320+0000","last_fullsized":"2024-08-20T03:00:59.488320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:52.387152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490226+0000","last_change":"2024-08-20T03:00:16.214861+0000","last_active":"2024-08-20T03:00:59.490226+0000","last_peered":"2024-08-20T03:00:59.490226+0000","last_clean":"2024-08-20T03:00:59.490226+0000","last_became_active":"2024-08-20T03:00:16.214487+0000","last_became_peered":"2024-08-20T03:00:16.214487+0000","last_unstale":"2024-08-20T03:00:59.490226+0000","last_undegraded":"2024-08-20T03:00:59.490226+0000","last_fullsized":"2024-08-20T03:00:59.490226+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:50:09.933304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491146+0000","last_change":"2024-08-20T03:00:18.244685+0000","last_active":"2024-08-20T03:00:59.491146+0000","last_peered":"2024-08-20T03:00:59.491146+0000","last_clean":"2024-08-20T03:00:59.491146+0000","last_became_active":"2024-08-20T03:00:18.244372+0000","last_became_peered":"2024-08-20T03:00:18.244372+0000","last_unstale":"2024-08-20T03:00:59.491146+0000","last_undegraded":"2024-08-20T03:00:59.491146+0000","last_fullsized":"2024-08-20T03:00:59.491146+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:21.821712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491625+0000","last_change":"2024-08-20T03:00:16.222287+0000","last_active":"2024-08-20T03:00:59.491625+0000","last_peered":"2024-08-20T03:00:59.491625+0000","last_clean":"2024-08-20T03:00:59.491625+0000","last_became_active":"2024-08-20T03:00:16.221989+0000","last_became_peered":"2024-08-20T03:00:16.221989+0000","last_unstale":"2024-08-20T03:00:59.491625+0000","last_undegraded":"2024-08-20T03:00:59.491625+0000","last_fullsized":"2024-08-20T03:00:59.491625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:32:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490950+0000","last_change":"2024-08-20T03:00:18.250488+0000","last_active":"2024-08-20T03:00:59.490950+0000","last_peered":"2024-08-20T03:00:59.490950+0000","last_clean":"2024-08-20T03:00:59.490950+0000","last_became_active":"2024-08-20T03:00:18.250094+0000","last_became_peered":"2024-08-20T03:00:18.250094+0000","last_unstale":"2024-08-20T03:00:59.490950+0000","last_undegraded":"2024-08-20T03:00:59.490950+0000","last_fullsized":"2024-08-20T03:00:59.490950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:00:55.755920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'5","reported_seq":86,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:24.465960+0000","last_change":"2024-08-20T03:00:16.470223+0000","last_active":"2024-08-20T03:01:24.465960+0000","last_peered":"2024-08-20T03:01:24.465960+0000","last_clean":"2024-08-20T03:01:24.465960+0000","last_became_active":"2024-08-20T03:00:16.469871+0000","last_became_peered":"2024-08-20T03:00:16.469871+0000","last_unstale":"2024-08-20T03:01:24.465960+0000","last_undegraded":"2024-08-20T03:01:24.465960+0000","last_fullsized":"2024-08-20T03:01:24.465960+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:05:11.779164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491172+0000","last_change":"2024-08-20T03:00:18.242755+0000","last_active":"2024-08-20T03:00:59.491172+0000","last_peered":"2024-08-20T03:00:59.491172+0000","last_clean":"2024-08-20T03:00:59.491172+0000","last_became_active":"2024-08-20T03:00:18.242447+0000","last_became_peered":"2024-08-20T03:00:18.242447+0000","last_unstale":"2024-08-20T03:00:59.491172+0000","last_undegraded":"2024-08-20T03:00:59.491172+0000","last_fullsized":"2024-08-20T03:00:59.491172+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:55:40.107933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490574+0000","last_change":"2024-08-20T02:59:02.774484+0000","last_active":"2024-08-20T03:00:59.490574+0000","last_peered":"2024-08-20T03:00:59.490574+0000","last_clean":"2024-08-20T03:00:59.490574+0000","last_became_active":"2024-08-20T02:59:02.769208+0000","last_became_peered":"2024-08-20T02:59:02.769208+0000","last_unstale":"2024-08-20T03:00:59.490574+0000","last_undegraded":"2024-08-20T03:00:59.490574+0000","last_fullsized":"2024-08-20T03:00:59.490574+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_clean_scrub_stamp":"2024-08-20T02:55:46.827249+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:36:27.931786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.516703+0000","last_change":"2024-08-20T03:00:16.213869+0000","last_active":"2024-08-20T03:00:59.516703+0000","last_peered":"2024-08-20T03:00:59.516703+0000","last_clean":"2024-08-20T03:00:59.516703+0000","last_became_active":"2024-08-20T03:00:16.213478+0000","last_became_peered":"2024-08-20T03:00:16.213478+0000","last_unstale":"2024-08-20T03:00:59.516703+0000","last_undegraded":"2024-08-20T03:00:59.516703+0000","last_fullsized":"2024-08-20T03:00:59.516703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:27.830497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487652+0000","last_change":"2024-08-20T03:00:18.242283+0000","last_active":"2024-08-20T03:00:59.487652+0000","last_peered":"2024-08-20T03:00:59.487652+0000","last_clean":"2024-08-20T03:00:59.487652+0000","last_became_active":"2024-08-20T03:00:18.241943+0000","last_became_peered":"2024-08-20T03:00:18.241943+0000","last_unstale":"2024-08-20T03:00:59.487652+0000","last_undegraded":"2024-08-20T03:00:59.487652+0000","last_fullsized":"2024-08-20T03:00:59.487652+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:22:25.776964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.498057+0000","last_change":"2024-08-20T03:00:59.497552+0000","last_active":"2024-08-20T03:00:59.498057+0000","last_peered":"2024-08-20T03:00:59.498057+0000","last_clean":"2024-08-20T03:00:59.498057+0000","last_became_active":"2024-08-20T03:00:59.493978+0000","last_became_peered":"2024-08-20T03:00:59.493978+0000","last_unstale":"2024-08-20T03:00:59.498057+0000","last_undegraded":"2024-08-20T03:00:59.498057+0000","last_fullsized":"2024-08-20T03:00:59.498057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:38:12.908159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.544757+0000","last_change":"2024-08-20T03:00:59.544696+0000","last_active":"2024-08-20T03:00:59.544757+0000","last_peered":"2024-08-20T03:00:59.544757+0000","last_clean":"2024-08-20T03:00:59.544757+0000","last_became_active":"2024-08-20T03:00:59.544303+0000","last_became_peered":"2024-08-20T03:00:59.544303+0000","last_unstale":"2024-08-20T03:00:59.544757+0000","last_undegraded":"2024-08-20T03:00:59.544757+0000","last_fullsized":"2024-08-20T03:00:59.544757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:01:04.814695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":81,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:20.459509+0000","last_change":"2024-08-20T03:00:16.468073+0000","last_active":"2024-08-20T03:01:20.459509+0000","last_peered":"2024-08-20T03:01:20.459509+0000","last_clean":"2024-08-20T03:01:20.459509+0000","last_became_active":"2024-08-20T03:00:16.467560+0000","last_became_peered":"2024-08-20T03:00:16.467560+0000","last_unstale":"2024-08-20T03:01:20.459509+0000","last_undegraded":"2024-08-20T03:01:20.459509+0000","last_fullsized":"2024-08-20T03:01:20.459509+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:42:24.499777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492439+0000","last_change":"2024-08-20T03:00:18.232537+0000","last_active":"2024-08-20T03:00:59.492439+0000","last_peered":"2024-08-20T03:00:59.492439+0000","last_clean":"2024-08-20T03:00:59.492439+0000","last_became_active":"2024-08-20T03:00:18.232273+0000","last_became_peered":"2024-08-20T03:00:18.232273+0000","last_unstale":"2024-08-20T03:00:59.492439+0000","last_undegraded":"2024-08-20T03:00:59.492439+0000","last_fullsized":"2024-08-20T03:00:59.492439+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:36:04.981869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'6","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:21.460983+0000","last_change":"2024-08-20T03:00:16.214807+0000","last_active":"2024-08-20T03:01:21.460983+0000","last_peered":"2024-08-20T03:01:21.460983+0000","last_clean":"2024-08-20T03:01:21.460983+0000","last_became_active":"2024-08-20T03:00:16.214439+0000","last_became_peered":"2024-08-20T03:00:16.214439+0000","last_unstale":"2024-08-20T03:01:21.460983+0000","last_undegraded":"2024-08-20T03:01:21.460983+0000","last_fullsized":"2024-08-20T03:01:21.460983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:07.519151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491420+0000","last_change":"2024-08-20T03:00:18.247220+0000","last_active":"2024-08-20T03:00:59.491420+0000","last_peered":"2024-08-20T03:00:59.491420+0000","last_clean":"2024-08-20T03:00:59.491420+0000","last_became_active":"2024-08-20T03:00:18.246896+0000","last_became_peered":"2024-08-20T03:00:18.246896+0000","last_unstale":"2024-08-20T03:00:59.491420+0000","last_undegraded":"2024-08-20T03:00:59.491420+0000","last_fullsized":"2024-08-20T03:00:59.491420+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:43:16.690590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543240+0000","last_change":"2024-08-20T03:00:16.468280+0000","last_active":"2024-08-20T03:00:59.543240+0000","last_peered":"2024-08-20T03:00:59.543240+0000","last_clean":"2024-08-20T03:00:59.543240+0000","last_became_active":"2024-08-20T03:00:16.467930+0000","last_became_peered":"2024-08-20T03:00:16.467930+0000","last_unstale":"2024-08-20T03:00:59.543240+0000","last_undegraded":"2024-08-20T03:00:59.543240+0000","last_fullsized":"2024-08-20T03:00:59.543240+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:30.423563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491804+0000","last_change":"2024-08-20T03:00:18.240322+0000","last_active":"2024-08-20T03:00:59.491804+0000","last_peered":"2024-08-20T03:00:59.491804+0000","last_clean":"2024-08-20T03:00:59.491804+0000","last_became_active":"2024-08-20T03:00:18.239973+0000","last_became_peered":"2024-08-20T03:00:18.239973+0000","last_unstale":"2024-08-20T03:00:59.491804+0000","last_undegraded":"2024-08-20T03:00:59.491804+0000","last_fullsized":"2024-08-20T03:00:59.491804+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:41:28.259762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.493654+0000","last_change":"2024-08-20T03:00:16.468307+0000","last_active":"2024-08-20T03:00:59.493654+0000","last_peered":"2024-08-20T03:00:59.493654+0000","last_clean":"2024-08-20T03:00:59.493654+0000","last_became_active":"2024-08-20T03:00:16.467993+0000","last_became_peered":"2024-08-20T03:00:16.467993+0000","last_unstale":"2024-08-20T03:00:59.493654+0000","last_undegraded":"2024-08-20T03:00:59.493654+0000","last_fullsized":"2024-08-20T03:00:59.493654+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:34.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490183+0000","last_change":"2024-08-20T03:00:18.234009+0000","last_active":"2024-08-20T03:00:59.490183+0000","last_peered":"2024-08-20T03:00:59.490183+0000","last_clean":"2024-08-20T03:00:59.490183+0000","last_became_active":"2024-08-20T03:00:18.233311+0000","last_became_peered":"2024-08-20T03:00:18.233311+0000","last_unstale":"2024-08-20T03:00:59.490183+0000","last_undegraded":"2024-08-20T03:00:59.490183+0000","last_fullsized":"2024-08-20T03:00:59.490183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:53:10.759339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'16","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543372+0000","last_change":"2024-08-20T03:00:16.214932+0000","last_active":"2024-08-20T03:00:59.543372+0000","last_peered":"2024-08-20T03:00:59.543372+0000","last_clean":"2024-08-20T03:00:59.543372+0000","last_became_active":"2024-08-20T03:00:16.214584+0000","last_became_peered":"2024-08-20T03:00:16.214584+0000","last_unstale":"2024-08-20T03:00:59.543372+0000","last_undegraded":"2024-08-20T03:00:59.543372+0000","last_fullsized":"2024-08-20T03:00:59.543372+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:47.270685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487711+0000","last_change":"2024-08-20T03:00:18.247892+0000","last_active":"2024-08-20T03:00:59.487711+0000","last_peered":"2024-08-20T03:00:59.487711+0000","last_clean":"2024-08-20T03:00:59.487711+0000","last_became_active":"2024-08-20T03:00:18.245903+0000","last_became_peered":"2024-08-20T03:00:18.245903+0000","last_unstale":"2024-08-20T03:00:59.487711+0000","last_undegraded":"2024-08-20T03:00:59.487711+0000","last_fullsized":"2024-08-20T03:00:59.487711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:08:50.909638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.516720+0000","last_change":"2024-08-20T03:00:16.372503+0000","last_active":"2024-08-20T03:00:59.516720+0000","last_peered":"2024-08-20T03:00:59.516720+0000","last_clean":"2024-08-20T03:00:59.516720+0000","last_became_active":"2024-08-20T03:00:16.372218+0000","last_became_peered":"2024-08-20T03:00:16.372218+0000","last_unstale":"2024-08-20T03:00:59.516720+0000","last_undegraded":"2024-08-20T03:00:59.516720+0000","last_fullsized":"2024-08-20T03:00:59.516720+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:13.341900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542560+0000","last_change":"2024-08-20T03:00:18.244738+0000","last_active":"2024-08-20T03:00:59.542560+0000","last_peered":"2024-08-20T03:00:59.542560+0000","last_clean":"2024-08-20T03:00:59.542560+0000","last_became_active":"2024-08-20T03:00:18.244398+0000","last_became_peered":"2024-08-20T03:00:18.244398+0000","last_unstale":"2024-08-20T03:00:59.542560+0000","last_undegraded":"2024-08-20T03:00:59.542560+0000","last_fullsized":"2024-08-20T03:00:59.542560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:45:14.540125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494339+0000","last_change":"2024-08-20T03:00:16.468251+0000","last_active":"2024-08-20T03:00:59.494339+0000","last_peered":"2024-08-20T03:00:59.494339+0000","last_clean":"2024-08-20T03:00:59.494339+0000","last_became_active":"2024-08-20T03:00:16.467912+0000","last_became_peered":"2024-08-20T03:00:16.467912+0000","last_unstale":"2024-08-20T03:00:59.494339+0000","last_undegraded":"2024-08-20T03:00:59.494339+0000","last_fullsized":"2024-08-20T03:00:59.494339+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:43.799747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490539+0000","last_change":"2024-08-20T03:00:18.234624+0000","last_active":"2024-08-20T03:00:59.490539+0000","last_peered":"2024-08-20T03:00:59.490539+0000","last_clean":"2024-08-20T03:00:59.490539+0000","last_became_active":"2024-08-20T03:00:18.234331+0000","last_became_peered":"2024-08-20T03:00:18.234331+0000","last_unstale":"2024-08-20T03:00:59.490539+0000","last_undegraded":"2024-08-20T03:00:59.490539+0000","last_fullsized":"2024-08-20T03:00:59.490539+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:05:09.665645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487438+0000","last_change":"2024-08-20T03:00:16.216658+0000","last_active":"2024-08-20T03:00:59.487438+0000","last_peered":"2024-08-20T03:00:59.487438+0000","last_clean":"2024-08-20T03:00:59.487438+0000","last_became_active":"2024-08-20T03:00:16.216352+0000","last_became_peered":"2024-08-20T03:00:16.216352+0000","last_unstale":"2024-08-20T03:00:59.487438+0000","last_undegraded":"2024-08-20T03:00:59.487438+0000","last_fullsized":"2024-08-20T03:00:59.487438+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:20.763280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.488349+0000","last_change":"2024-08-20T03:00:18.240984+0000","last_active":"2024-08-20T03:00:59.488349+0000","last_peered":"2024-08-20T03:00:59.488349+0000","last_clean":"2024-08-20T03:00:59.488349+0000","last_became_active":"2024-08-20T03:00:18.240640+0000","last_became_peered":"2024-08-20T03:00:18.240640+0000","last_unstale":"2024-08-20T03:00:59.488349+0000","last_undegraded":"2024-08-20T03:00:59.488349+0000","last_fullsized":"2024-08-20T03:00:59.488349+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:05.589186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492342+0000","last_change":"2024-08-20T03:00:16.214818+0000","last_active":"2024-08-20T03:00:59.492342+0000","last_peered":"2024-08-20T03:00:59.492342+0000","last_clean":"2024-08-20T03:00:59.492342+0000","last_became_active":"2024-08-20T03:00:16.214493+0000","last_became_peered":"2024-08-20T03:00:16.214493+0000","last_unstale":"2024-08-20T03:00:59.492342+0000","last_undegraded":"2024-08-20T03:00:59.492342+0000","last_fullsized":"2024-08-20T03:00:59.492342+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:56:12.895142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487487+0000","last_change":"2024-08-20T03:00:18.240350+0000","last_active":"2024-08-20T03:00:59.487487+0000","last_peered":"2024-08-20T03:00:59.487487+0000","last_clean":"2024-08-20T03:00:59.487487+0000","last_became_active":"2024-08-20T03:00:18.239464+0000","last_became_peered":"2024-08-20T03:00:18.239464+0000","last_unstale":"2024-08-20T03:00:59.487487+0000","last_undegraded":"2024-08-20T03:00:59.487487+0000","last_fullsized":"2024-08-20T03:00:59.487487+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:48.715130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491431+0000","last_change":"2024-08-20T03:00:16.216144+0000","last_active":"2024-08-20T03:00:59.491431+0000","last_peered":"2024-08-20T03:00:59.491431+0000","last_clean":"2024-08-20T03:00:59.491431+0000","last_became_active":"2024-08-20T03:00:16.215792+0000","last_became_peered":"2024-08-20T03:00:16.215792+0000","last_unstale":"2024-08-20T03:00:59.491431+0000","last_undegraded":"2024-08-20T03:00:59.491431+0000","last_fullsized":"2024-08-20T03:00:59.491431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:25:01.500878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490589+0000","last_change":"2024-08-20T03:00:18.245286+0000","last_active":"2024-08-20T03:00:59.490589+0000","last_peered":"2024-08-20T03:00:59.490589+0000","last_clean":"2024-08-20T03:00:59.490589+0000","last_became_active":"2024-08-20T03:00:18.244760+0000","last_became_peered":"2024-08-20T03:00:18.244760+0000","last_unstale":"2024-08-20T03:00:59.490589+0000","last_undegraded":"2024-08-20T03:00:59.490589+0000","last_fullsized":"2024-08-20T03:00:59.490589+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:09.415794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.496088+0000","last_change":"2024-08-20T03:00:59.495948+0000","last_active":"2024-08-20T03:00:59.496088+0000","last_peered":"2024-08-20T03:00:59.496088+0000","last_clean":"2024-08-20T03:00:59.496088+0000","last_became_active":"2024-08-20T03:00:59.495543+0000","last_became_peered":"2024-08-20T03:00:59.495543+0000","last_unstale":"2024-08-20T03:00:59.496088+0000","last_undegraded":"2024-08-20T03:00:59.496088+0000","last_fullsized":"2024-08-20T03:00:59.496088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:38:30.671235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492392+0000","last_change":"2024-08-20T03:00:16.220019+0000","last_active":"2024-08-20T03:00:59.492392+0000","last_peered":"2024-08-20T03:00:59.492392+0000","last_clean":"2024-08-20T03:00:59.492392+0000","last_became_active":"2024-08-20T03:00:16.219624+0000","last_became_peered":"2024-08-20T03:00:16.219624+0000","last_unstale":"2024-08-20T03:00:59.492392+0000","last_undegraded":"2024-08-20T03:00:59.492392+0000","last_fullsized":"2024-08-20T03:00:59.492392+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:33:24.926695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494355+0000","last_change":"2024-08-20T03:00:18.242156+0000","last_active":"2024-08-20T03:00:59.494355+0000","last_peered":"2024-08-20T03:00:59.494355+0000","last_clean":"2024-08-20T03:00:59.494355+0000","last_became_active":"2024-08-20T03:00:18.241811+0000","last_became_peered":"2024-08-20T03:00:18.241811+0000","last_unstale":"2024-08-20T03:00:59.494355+0000","last_undegraded":"2024-08-20T03:00:59.494355+0000","last_fullsized":"2024-08-20T03:00:59.494355+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:58.230658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.534283+0000","last_change":"2024-08-20T03:00:59.533906+0000","last_active":"2024-08-20T03:00:59.534283+0000","last_peered":"2024-08-20T03:00:59.534283+0000","last_clean":"2024-08-20T03:00:59.534283+0000","last_became_active":"2024-08-20T03:00:59.517824+0000","last_became_peered":"2024-08-20T03:00:59.517824+0000","last_unstale":"2024-08-20T03:00:59.534283+0000","last_undegraded":"2024-08-20T03:00:59.534283+0000","last_fullsized":"2024-08-20T03:00:59.534283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:25:28.219430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.542643+0000","last_change":"2024-08-20T03:00:18.245256+0000","last_active":"2024-08-20T03:00:59.542643+0000","last_peered":"2024-08-20T03:00:59.542643+0000","last_clean":"2024-08-20T03:00:59.542643+0000","last_became_active":"2024-08-20T03:00:18.244339+0000","last_became_peered":"2024-08-20T03:00:18.244339+0000","last_unstale":"2024-08-20T03:00:59.542643+0000","last_undegraded":"2024-08-20T03:00:59.542643+0000","last_fullsized":"2024-08-20T03:00:59.542643+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:00.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487797+0000","last_change":"2024-08-20T03:00:16.470004+0000","last_active":"2024-08-20T03:00:59.487797+0000","last_peered":"2024-08-20T03:00:59.487797+0000","last_clean":"2024-08-20T03:00:59.487797+0000","last_became_active":"2024-08-20T03:00:16.469646+0000","last_became_peered":"2024-08-20T03:00:16.469646+0000","last_unstale":"2024-08-20T03:00:59.487797+0000","last_undegraded":"2024-08-20T03:00:59.487797+0000","last_fullsized":"2024-08-20T03:00:59.487797+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:35.244683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487684+0000","last_change":"2024-08-20T03:00:18.238053+0000","last_active":"2024-08-20T03:00:59.487684+0000","last_peered":"2024-08-20T03:00:59.487684+0000","last_clean":"2024-08-20T03:00:59.487684+0000","last_became_active":"2024-08-20T03:00:18.237186+0000","last_became_peered":"2024-08-20T03:00:18.237186+0000","last_unstale":"2024-08-20T03:00:59.487684+0000","last_undegraded":"2024-08-20T03:00:59.487684+0000","last_fullsized":"2024-08-20T03:00:59.487684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:14.425688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":89,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:20.459087+0000","last_change":"2024-08-20T03:00:16.214584+0000","last_active":"2024-08-20T03:01:20.459087+0000","last_peered":"2024-08-20T03:01:20.459087+0000","last_clean":"2024-08-20T03:01:20.459087+0000","last_became_active":"2024-08-20T03:00:16.214255+0000","last_became_peered":"2024-08-20T03:00:16.214255+0000","last_unstale":"2024-08-20T03:01:20.459087+0000","last_undegraded":"2024-08-20T03:01:20.459087+0000","last_fullsized":"2024-08-20T03:01:20.459087+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:16:30.983745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":577,"num_read_kb":361,"num_write":245,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":59762,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":200898,"data_compressed":9639,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281428,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743451,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":843776,"data_stored":417802,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939680,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":999424,"data_stored":1068593,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103205,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":868352,"data_stored":417732,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299433,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":864256,"data_stored":418110,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462959,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":1011019,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659187,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713128,"statfs":{"total":95995035648,"available":95962243072,"internally_reserved":0,"allocated":851968,"data_stored":417840,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":1010306,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":477072,"data_compressed":15466,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":868352,"data_stored":417274,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":418973,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542220,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":913408,"data_stored":478515,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705744,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":418788,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":60462,"data_compressed":9639,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T03:01:27.859 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-20T03:01:27.859 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-20T03:01:27.859 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-08-20T03:01:27.860 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph health --format=json 2024-08-20T03:01:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:27 smithi044 ceph-mon[18790]: pgmap v289: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:27 smithi044 ceph-mon[18790]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:01:28.113 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:27 smithi165 ceph-mon[22357]: pgmap v289: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:27 smithi165 ceph-mon[22357]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:01:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:27 smithi099 ceph-mon[21984]: pgmap v289: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:27 smithi099 ceph-mon[21984]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:01:28.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.622+0000 7f0b697f7640 1 -- 172.21.15.44:0/2696264681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b64108e40 msgr2=0x7f0b6410f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:28.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.622+0000 7f0b697f7640 1 --2- 172.21.15.44:0/2696264681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b64108e40 0x7f0b6410f640 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009ca0 tx=0x7f0b5c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:28.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.622+0000 7f0b697f7640 1 -- 172.21.15.44:0/2696264681 shutdown_connections 2024-08-20T03:01:28.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.622+0000 7f0b697f7640 1 --2- 172.21.15.44:0/2696264681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b64108e40 0x7f0b6410f640 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:28.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.622+0000 7f0b697f7640 1 --2- 172.21.15.44:0/2696264681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b641084c0 0x7f0b64108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:28.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.622+0000 7f0b697f7640 1 --2- 172.21.15.44:0/2696264681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b64102ab0 0x7f0b64102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:28.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.622+0000 7f0b697f7640 1 -- 172.21.15.44:0/2696264681 >> 172.21.15.44:0/2696264681 conn(0x7f0b640fe540 msgr2=0x7f0b64100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:28.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.622+0000 7f0b697f7640 1 -- 172.21.15.44:0/2696264681 shutdown_connections 2024-08-20T03:01:28.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.622+0000 7f0b697f7640 1 -- 172.21.15.44:0/2696264681 wait complete. 2024-08-20T03:01:28.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.622+0000 7f0b697f7640 1 Processor -- start 2024-08-20T03:01:28.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b697f7640 1 -- start start 2024-08-20T03:01:28.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b697f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b64102ab0 0x7f0b64198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:28.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b697f7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b641084c0 0x7f0b641994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:28.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b697f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b64108e40 0x7f0b6419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:28.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b697f7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0b64112b00 con 0x7f0b64108e40 2024-08-20T03:01:28.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b697f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0b64112980 con 0x7f0b64102ab0 2024-08-20T03:01:28.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b697f7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0b64112c80 con 0x7f0b641084c0 2024-08-20T03:01:28.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b637fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b64108e40 0x7f0b6419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b637fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b64108e40 0x7f0b6419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58094/0 (socket says 172.21.15.44:58094) 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b637fe640 1 -- 172.21.15.44:0/3218826394 learned_addr learned my addr 172.21.15.44:0/3218826394 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b62ffd640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b64102ab0 0x7f0b64198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b5a7fc640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b641084c0 0x7f0b641994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b637fe640 1 -- 172.21.15.44:0/3218826394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b641084c0 msgr2=0x7f0b641994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b637fe640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b641084c0 0x7f0b641994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b637fe640 1 -- 172.21.15.44:0/3218826394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b64102ab0 msgr2=0x7f0b64198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b637fe640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b64102ab0 0x7f0b64198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b637fe640 1 -- 172.21.15.44:0/3218826394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6419f020 con 0x7f0b64108e40 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.623+0000 7f0b5a7fc640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b641084c0 0x7f0b641994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.624+0000 7f0b637fe640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b64108e40 0x7f0b6419e850 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c005cc0 tx=0x7f0b5c02f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.624+0000 7f0b60ff9640 1 -- 172.21.15.44:0/3218826394 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c003cf0 con 0x7f0b64108e40 2024-08-20T03:01:28.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.624+0000 7f0b697f7640 1 -- 172.21.15.44:0/3218826394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6419f2b0 con 0x7f0b64108e40 2024-08-20T03:01:28.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.624+0000 7f0b60ff9640 1 -- 172.21.15.44:0/3218826394 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0044f0 con 0x7f0b64108e40 2024-08-20T03:01:28.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.624+0000 7f0b60ff9640 1 -- 172.21.15.44:0/3218826394 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c02fd10 con 0x7f0b64108e40 2024-08-20T03:01:28.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.624+0000 7f0b697f7640 1 -- 172.21.15.44:0/3218826394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b641a5a50 con 0x7f0b64108e40 2024-08-20T03:01:28.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.625+0000 7f0b697f7640 1 -- 172.21.15.44:0/3218826394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b64102e90 con 0x7f0b64108e40 2024-08-20T03:01:28.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.626+0000 7f0b60ff9640 1 -- 172.21.15.44:0/3218826394 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f0b5c004040 con 0x7f0b64108e40 2024-08-20T03:01:28.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.627+0000 7f0b60ff9640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0b500b3cb0 0x7f0b500b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:28.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.627+0000 7f0b62ffd640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0b500b3cb0 0x7f0b500b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:28.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.627+0000 7f0b60ff9640 1 -- 172.21.15.44:0/3218826394 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0b5c105660 con 0x7f0b64108e40 2024-08-20T03:01:28.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.627+0000 7f0b62ffd640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0b500b3cb0 0x7f0b500b6170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0b540058c0 tx=0x7f0b5400a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:28.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.631+0000 7f0b60ff9640 1 -- 172.21.15.44:0/3218826394 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0b5c0ccb50 con 0x7f0b64108e40 2024-08-20T03:01:28.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.855+0000 7f0b697f7640 1 -- 172.21.15.44:0/3218826394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0b6402e340 con 0x7f0b64108e40 2024-08-20T03:01:28.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.855+0000 7f0b60ff9640 1 -- 172.21.15.44:0/3218826394 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0b5c0cc450 con 0x7f0b64108e40 2024-08-20T03:01:28.858 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:28.858 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-20T03:01:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.857+0000 7f0b697f7640 1 -- 172.21.15.44:0/3218826394 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0b500b3cb0 msgr2=0x7f0b500b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.857+0000 7f0b697f7640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0b500b3cb0 0x7f0b500b6170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0b540058c0 tx=0x7f0b5400a2d0 comp rx=0 tx=0).stop 2024-08-20T03:01:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.857+0000 7f0b697f7640 1 -- 172.21.15.44:0/3218826394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b64108e40 msgr2=0x7f0b6419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.857+0000 7f0b697f7640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b64108e40 0x7f0b6419e850 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c005cc0 tx=0x7f0b5c02f470 comp rx=0 tx=0).stop 2024-08-20T03:01:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.857+0000 7f0b697f7640 1 -- 172.21.15.44:0/3218826394 shutdown_connections 2024-08-20T03:01:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.857+0000 7f0b697f7640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0b500b3cb0 0x7f0b500b6170 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.857+0000 7f0b697f7640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0b64108e40 0x7f0b6419e850 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.857+0000 7f0b697f7640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b641084c0 0x7f0b641994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:28.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.857+0000 7f0b697f7640 1 --2- 172.21.15.44:0/3218826394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b64102ab0 0x7f0b64198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:28.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.857+0000 7f0b697f7640 1 -- 172.21.15.44:0/3218826394 >> 172.21.15.44:0/3218826394 conn(0x7f0b640fe540 msgr2=0x7f0b64100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:28.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.858+0000 7f0b697f7640 1 -- 172.21.15.44:0/3218826394 shutdown_connections 2024-08-20T03:01:28.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:28.858+0000 7f0b697f7640 1 -- 172.21.15.44:0/3218826394 wait complete. 2024-08-20T03:01:29.565 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-08-20T03:01:29.565 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:01:29.819 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:30.096 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:29 smithi044 ceph-mon[18790]: pgmap v290: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:01:30.096 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:29 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3218826394' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-20T03:01:30.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:29 smithi165 ceph-mon[22357]: pgmap v290: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:01:30.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:29 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3218826394' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-20T03:01:30.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:29 smithi099 ceph-mon[21984]: pgmap v290: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:01:30.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:29 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3218826394' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-20T03:01:30.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.302+0000 7fb992513640 1 -- 172.21.15.44:0/2152890119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb98c1019b0 msgr2=0x7fb98c101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:30.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.302+0000 7fb992513640 1 --2- 172.21.15.44:0/2152890119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb98c1019b0 0x7fb98c101d90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009900 tx=0x7fb97c02a3f0 comp rx=0 tx=0).stop 2024-08-20T03:01:30.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 -- 172.21.15.44:0/2152890119 shutdown_connections 2024-08-20T03:01:30.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 --2- 172.21.15.44:0/2152890119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98c107d40 0x7fb98c10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:30.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 --2- 172.21.15.44:0/2152890119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb98c1073c0 0x7fb98c107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:30.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 --2- 172.21.15.44:0/2152890119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb98c1019b0 0x7fb98c101d90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:30.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 -- 172.21.15.44:0/2152890119 >> 172.21.15.44:0/2152890119 conn(0x7fb98c0fd400 msgr2=0x7fb98c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:30.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 -- 172.21.15.44:0/2152890119 shutdown_connections 2024-08-20T03:01:30.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 -- 172.21.15.44:0/2152890119 wait complete. 2024-08-20T03:01:30.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 Processor -- start 2024-08-20T03:01:30.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 -- start start 2024-08-20T03:01:30.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98c1019b0 0x7fb98c198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:30.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb98c1073c0 0x7fb98c1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:30.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb98c107d40 0x7fb98c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:30.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c1119c0 con 0x7fb98c1073c0 2024-08-20T03:01:30.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c111840 con 0x7fb98c107d40 2024-08-20T03:01:30.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.303+0000 7fb992513640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c111b40 con 0x7fb98c1019b0 2024-08-20T03:01:30.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb983fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb98c1073c0 0x7fb98c1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:30.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb983fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb98c1073c0 0x7fb98c1994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58116/0 (socket says 172.21.15.44:58116) 2024-08-20T03:01:30.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb983fff640 1 -- 172.21.15.44:0/942796688 learned_addr learned my addr 172.21.15.44:0/942796688 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:30.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb98bfff640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98c1019b0 0x7fb98c198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:30.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb990ab0640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb98c107d40 0x7fb98c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:30.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb983fff640 1 -- 172.21.15.44:0/942796688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98c1019b0 msgr2=0x7fb98c198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:30.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb983fff640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98c1019b0 0x7fb98c198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:30.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb983fff640 1 -- 172.21.15.44:0/942796688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb98c107d40 msgr2=0x7fb98c19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:30.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb983fff640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb98c107d40 0x7fb98c19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:30.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb983fff640 1 -- 172.21.15.44:0/942796688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c19f040 con 0x7fb98c1073c0 2024-08-20T03:01:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb98bfff640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98c1019b0 0x7fb98c198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb983fff640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb98c1073c0 0x7fb98c1994d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb97400c920 tx=0x7fb97400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb989ffb640 1 -- 172.21.15.44:0/942796688 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974007c70 con 0x7fb98c1073c0 2024-08-20T03:01:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb992513640 1 -- 172.21.15.44:0/942796688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c19f330 con 0x7fb98c1073c0 2024-08-20T03:01:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb989ffb640 1 -- 172.21.15.44:0/942796688 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb974007e10 con 0x7fb98c1073c0 2024-08-20T03:01:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb989ffb640 1 -- 172.21.15.44:0/942796688 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974018820 con 0x7fb98c1073c0 2024-08-20T03:01:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.304+0000 7fb992513640 1 -- 172.21.15.44:0/942796688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c1a59f0 con 0x7fb98c1073c0 2024-08-20T03:01:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.305+0000 7fb992513640 1 -- 172.21.15.44:0/942796688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98c101d90 con 0x7fb98c1073c0 2024-08-20T03:01:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.307+0000 7fb989ffb640 1 -- 172.21.15.44:0/942796688 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb97400f450 con 0x7fb98c1073c0 2024-08-20T03:01:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.307+0000 7fb989ffb640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb9780b3cb0 0x7fb9780b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.307+0000 7fb98bfff640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb9780b3cb0 0x7fb9780b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:30.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.307+0000 7fb989ffb640 1 -- 172.21.15.44:0/942796688 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb9740e4330 con 0x7fb98c1073c0 2024-08-20T03:01:30.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.308+0000 7fb98bfff640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb9780b3cb0 0x7fb9780b6170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb97c031440 tx=0x7fb97c0095f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:30.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.310+0000 7fb989ffb640 1 -- 172.21.15.44:0/942796688 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb974014030 con 0x7fb98c1073c0 2024-08-20T03:01:30.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.532+0000 7fb992513640 1 -- 172.21.15.44:0/942796688 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb98c19a0c0 con 0x7fb98c1073c0 2024-08-20T03:01:30.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.533+0000 7fb989ffb640 1 -- 172.21.15.44:0/942796688 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11965 (secure 0 0 0) 0x7fb9740ab800 con 0x7fb98c1073c0 2024-08-20T03:01:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:30.536 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"btime":"2024-08-20T03:00:30:384117+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:00:30.384102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24604,24589,24596,24583,24569]},"id":1}]} 2024-08-20T03:01:30.536 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-08-20T03:01:30.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.535+0000 7fb992513640 1 -- 172.21.15.44:0/942796688 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb9780b3cb0 msgr2=0x7fb9780b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.535+0000 7fb992513640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb9780b3cb0 0x7fb9780b6170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb97c031440 tx=0x7fb97c0095f0 comp rx=0 tx=0).stop 2024-08-20T03:01:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.535+0000 7fb992513640 1 -- 172.21.15.44:0/942796688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb98c1073c0 msgr2=0x7fb98c1994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.535+0000 7fb992513640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb98c1073c0 0x7fb98c1994d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb97400c920 tx=0x7fb97400cde0 comp rx=0 tx=0).stop 2024-08-20T03:01:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.535+0000 7fb992513640 1 -- 172.21.15.44:0/942796688 shutdown_connections 2024-08-20T03:01:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.535+0000 7fb992513640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb9780b3cb0 0x7fb9780b6170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.535+0000 7fb992513640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb98c107d40 0x7fb98c19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.535+0000 7fb992513640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb98c1073c0 0x7fb98c1994d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.535+0000 7fb992513640 1 --2- 172.21.15.44:0/942796688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb98c1019b0 0x7fb98c198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.535+0000 7fb992513640 1 -- 172.21.15.44:0/942796688 >> 172.21.15.44:0/942796688 conn(0x7fb98c0fd400 msgr2=0x7fb98c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.535+0000 7fb992513640 1 -- 172.21.15.44:0/942796688 shutdown_connections 2024-08-20T03:01:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:30.535+0000 7fb992513640 1 -- 172.21.15.44:0/942796688 wait complete. 2024-08-20T03:01:31.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:30 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/942796688' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:31.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:30 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/942796688' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:31.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:30 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/942796688' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:31.392 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:01:31.651 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:32.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:31 smithi044 ceph-mon[18790]: pgmap v291: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:32.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:31 smithi165 ceph-mon[22357]: pgmap v291: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:32.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:31 smithi099 ceph-mon[21984]: pgmap v291: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:32.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.142+0000 7f9613fb4640 1 -- 172.21.15.44:0/3849300518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c1073a0 msgr2=0x7f960c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:32.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.142+0000 7f9613fb4640 1 --2- 172.21.15.44:0/3849300518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c1073a0 0x7f960c1077e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009900 tx=0x7f95fc02a410 comp rx=0 tx=0).stop 2024-08-20T03:01:32.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.142+0000 7f9613fb4640 1 -- 172.21.15.44:0/3849300518 shutdown_connections 2024-08-20T03:01:32.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.142+0000 7f9613fb4640 1 --2- 172.21.15.44:0/3849300518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f960c107d20 0x7f960c10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:32.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.142+0000 7f9613fb4640 1 --2- 172.21.15.44:0/3849300518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c1073a0 0x7f960c1077e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:32.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.142+0000 7f9613fb4640 1 --2- 172.21.15.44:0/3849300518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f960c101990 0x7f960c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:32.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.142+0000 7f9613fb4640 1 -- 172.21.15.44:0/3849300518 >> 172.21.15.44:0/3849300518 conn(0x7f960c0fd400 msgr2=0x7f960c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.142+0000 7f9613fb4640 1 -- 172.21.15.44:0/3849300518 shutdown_connections 2024-08-20T03:01:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.142+0000 7f9613fb4640 1 -- 172.21.15.44:0/3849300518 wait complete. 2024-08-20T03:01:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.142+0000 7f9613fb4640 1 Processor -- start 2024-08-20T03:01:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9613fb4640 1 -- start start 2024-08-20T03:01:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9613fb4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c101990 0x7f960c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9613fb4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f960c1073a0 0x7f960c199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9613fb4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f960c107d20 0x7f960c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9613fb4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f960c1119a0 con 0x7f960c101990 2024-08-20T03:01:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9613fb4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f960c111820 con 0x7f960c1073a0 2024-08-20T03:01:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9613fb4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f960c111b20 con 0x7f960c107d20 2024-08-20T03:01:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9611d50640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c101990 0x7f960c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:32.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9611d50640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c101990 0x7f960c198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58142/0 (socket says 172.21.15.44:58142) 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9611d50640 1 -- 172.21.15.44:0/3998791371 learned_addr learned my addr 172.21.15.44:0/3998791371 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f961154f640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f960c1073a0 0x7f960c199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9612551640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f960c107d20 0x7f960c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9611d50640 1 -- 172.21.15.44:0/3998791371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f960c107d20 msgr2=0x7f960c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9611d50640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f960c107d20 0x7f960c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9611d50640 1 -- 172.21.15.44:0/3998791371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f960c1073a0 msgr2=0x7f960c199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9611d50640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f960c1073a0 0x7f960c199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9611d50640 1 -- 172.21.15.44:0/3998791371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c19eff0 con 0x7f960c101990 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9612551640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f960c107d20 0x7f960c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9611d50640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c101990 0x7f960c198f40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f960800c920 tx=0x7f960800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.143+0000 7f9602ffd640 1 -- 172.21.15.44:0/3998791371 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608007c70 con 0x7f960c101990 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.144+0000 7f9613fb4640 1 -- 172.21.15.44:0/3998791371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c19f2e0 con 0x7f960c101990 2024-08-20T03:01:32.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.144+0000 7f9602ffd640 1 -- 172.21.15.44:0/3998791371 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9608007e10 con 0x7f960c101990 2024-08-20T03:01:32.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.144+0000 7f9602ffd640 1 -- 172.21.15.44:0/3998791371 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608018820 con 0x7f960c101990 2024-08-20T03:01:32.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.144+0000 7f9613fb4640 1 -- 172.21.15.44:0/3998791371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c1a5b70 con 0x7f960c101990 2024-08-20T03:01:32.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.145+0000 7f9613fb4640 1 -- 172.21.15.44:0/3998791371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960c101d70 con 0x7f960c101990 2024-08-20T03:01:32.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.146+0000 7f9602ffd640 1 -- 172.21.15.44:0/3998791371 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f960800f450 con 0x7f960c101990 2024-08-20T03:01:32.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.147+0000 7f9602ffd640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95f80b3cb0 0x7f95f80b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:32.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.147+0000 7f961154f640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95f80b3cb0 0x7f95f80b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:32.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.147+0000 7f9602ffd640 1 -- 172.21.15.44:0/3998791371 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f96080e4270 con 0x7f960c101990 2024-08-20T03:01:32.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.147+0000 7f961154f640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95f80b3cb0 0x7f95f80b6170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f95fc031440 tx=0x7f95fc005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:32.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.150+0000 7f9602ffd640 1 -- 172.21.15.44:0/3998791371 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f9608014030 con 0x7f960c101990 2024-08-20T03:01:32.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.372+0000 7f9613fb4640 1 -- 172.21.15.44:0/3998791371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f960c1a5d40 con 0x7f960c101990 2024-08-20T03:01:32.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.373+0000 7f9602ffd640 1 -- 172.21.15.44:0/3998791371 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11965 (secure 0 0 0) 0x7f96080ab740 con 0x7f960c101990 2024-08-20T03:01:32.375 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:32.376 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"btime":"2024-08-20T03:00:30:384117+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:00:30.384102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24604,24589,24596,24583,24569]},"id":1}]} 2024-08-20T03:01:32.376 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-08-20T03:01:32.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.376+0000 7f9613fb4640 1 -- 172.21.15.44:0/3998791371 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95f80b3cb0 msgr2=0x7f95f80b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:32.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.376+0000 7f9613fb4640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95f80b3cb0 0x7f95f80b6170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f95fc031440 tx=0x7f95fc005b50 comp rx=0 tx=0).stop 2024-08-20T03:01:32.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.376+0000 7f9613fb4640 1 -- 172.21.15.44:0/3998791371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c101990 msgr2=0x7f960c198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:32.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.376+0000 7f9613fb4640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c101990 0x7f960c198f40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f960800c920 tx=0x7f960800cde0 comp rx=0 tx=0).stop 2024-08-20T03:01:32.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.376+0000 7f9613fb4640 1 -- 172.21.15.44:0/3998791371 shutdown_connections 2024-08-20T03:01:32.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.376+0000 7f9613fb4640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95f80b3cb0 0x7f95f80b6170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:32.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.376+0000 7f9613fb4640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f960c107d20 0x7f960c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:32.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.376+0000 7f9613fb4640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f960c1073a0 0x7f960c199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:32.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.376+0000 7f9613fb4640 1 --2- 172.21.15.44:0/3998791371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f960c101990 0x7f960c198f40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:32.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.376+0000 7f9613fb4640 1 -- 172.21.15.44:0/3998791371 >> 172.21.15.44:0/3998791371 conn(0x7f960c0fd400 msgr2=0x7f960c104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:32.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.376+0000 7f9613fb4640 1 -- 172.21.15.44:0/3998791371 shutdown_connections 2024-08-20T03:01:32.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:32.376+0000 7f9613fb4640 1 -- 172.21.15.44:0/3998791371 wait complete. 2024-08-20T03:01:33.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:32 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3998791371' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:33.066 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T03:01:33.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:32 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3998791371' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:33.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:32 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3998791371' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:33.321 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:33.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.767+0000 7ff26b03d640 1 -- 172.21.15.44:0/3256628795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2641073d0 msgr2=0x7ff264107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:33.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.767+0000 7ff26b03d640 1 --2- 172.21.15.44:0/3256628795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2641073d0 0x7ff264107810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff250009900 tx=0x7ff25002a410 comp rx=0 tx=0).stop 2024-08-20T03:01:33.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.767+0000 7ff26b03d640 1 -- 172.21.15.44:0/3256628795 shutdown_connections 2024-08-20T03:01:33.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.767+0000 7ff26b03d640 1 --2- 172.21.15.44:0/3256628795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff264107d50 0x7ff26410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:33.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.767+0000 7ff26b03d640 1 --2- 172.21.15.44:0/3256628795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2641073d0 0x7ff264107810 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:33.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.767+0000 7ff26b03d640 1 --2- 172.21.15.44:0/3256628795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2641019c0 0x7ff264101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:33.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.767+0000 7ff26b03d640 1 -- 172.21.15.44:0/3256628795 >> 172.21.15.44:0/3256628795 conn(0x7ff2640fd410 msgr2=0x7ff2640ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:33.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.767+0000 7ff26b03d640 1 -- 172.21.15.44:0/3256628795 shutdown_connections 2024-08-20T03:01:33.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.767+0000 7ff26b03d640 1 -- 172.21.15.44:0/3256628795 wait complete. 2024-08-20T03:01:33.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff26b03d640 1 Processor -- start 2024-08-20T03:01:33.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff26b03d640 1 -- start start 2024-08-20T03:01:33.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff26b03d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2641019c0 0x7ff264198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:33.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff26b03d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2641073d0 0x7ff264198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:33.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff26b03d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff264107d50 0x7ff26419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:33.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff26b03d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff264111a10 con 0x7ff264107d50 2024-08-20T03:01:33.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff26b03d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff264111890 con 0x7ff2641073d0 2024-08-20T03:01:33.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff26b03d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff264111b90 con 0x7ff2641019c0 2024-08-20T03:01:33.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff2695da640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff264107d50 0x7ff26419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:33.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff2695da640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff264107d50 0x7ff26419d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58168/0 (socket says 172.21.15.44:58168) 2024-08-20T03:01:33.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff2695da640 1 -- 172.21.15.44:0/2967749133 learned_addr learned my addr 172.21.15.44:0/2967749133 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:33.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff268dd9640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2641019c0 0x7ff264198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:33.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.768+0000 7ff263fff640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2641073d0 0x7ff264198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:33.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.769+0000 7ff2695da640 1 -- 172.21.15.44:0/2967749133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2641019c0 msgr2=0x7ff264198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:33.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.769+0000 7ff2695da640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2641019c0 0x7ff264198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:33.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.769+0000 7ff2695da640 1 -- 172.21.15.44:0/2967749133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2641073d0 msgr2=0x7ff264198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:33.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.769+0000 7ff2695da640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2641073d0 0x7ff264198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:33.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.769+0000 7ff2695da640 1 -- 172.21.15.44:0/2967749133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26419e1c0 con 0x7ff264107d50 2024-08-20T03:01:33.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.769+0000 7ff263fff640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2641073d0 0x7ff264198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:33.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.769+0000 7ff2695da640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff264107d50 0x7ff26419d9f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00cce0 tx=0x7ff25c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:33.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.769+0000 7ff261ffb640 1 -- 172.21.15.44:0/2967749133 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00f450 con 0x7ff264107d50 2024-08-20T03:01:33.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.769+0000 7ff261ffb640 1 -- 172.21.15.44:0/2967749133 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff25c00faa0 con 0x7ff264107d50 2024-08-20T03:01:33.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.769+0000 7ff26b03d640 1 -- 172.21.15.44:0/2967749133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26419e4b0 con 0x7ff264107d50 2024-08-20T03:01:33.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.769+0000 7ff261ffb640 1 -- 172.21.15.44:0/2967749133 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c018b70 con 0x7ff264107d50 2024-08-20T03:01:33.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.769+0000 7ff26b03d640 1 -- 172.21.15.44:0/2967749133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2641a4b70 con 0x7ff264107d50 2024-08-20T03:01:33.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.770+0000 7ff26b03d640 1 -- 172.21.15.44:0/2967749133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff264101da0 con 0x7ff264107d50 2024-08-20T03:01:33.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.771+0000 7ff261ffb640 1 -- 172.21.15.44:0/2967749133 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7ff25c018d10 con 0x7ff264107d50 2024-08-20T03:01:33.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.772+0000 7ff261ffb640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2540b3cb0 0x7ff2540b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:33.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.772+0000 7ff268dd9640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2540b3cb0 0x7ff2540b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:33.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.772+0000 7ff261ffb640 1 -- 172.21.15.44:0/2967749133 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff25c0e4340 con 0x7ff264107d50 2024-08-20T03:01:33.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.773+0000 7ff268dd9640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2540b3cb0 0x7ff2540b6170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff258009bd0 tx=0x7ff258005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:33.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.776+0000 7ff261ffb640 1 -- 172.21.15.44:0/2967749133 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff25c014030 con 0x7ff264107d50 2024-08-20T03:01:33.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.966+0000 7ff26b03d640 1 -- 172.21.15.44:0/2967749133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff26406ae90 con 0x7ff264107d50 2024-08-20T03:01:33.968 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:33 smithi044 ceph-mon[18790]: pgmap v292: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-08-20T03:01:33.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.967+0000 7ff261ffb640 1 -- 172.21.15.44:0/2967749133 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22954 (secure 0 0 0) 0x7ff25c0ab720 con 0x7ff264107d50 2024-08-20T03:01:33.970 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:33.970 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":77,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","created":"2024-08-20T02:52:10.979264+0000","modified":"2024-08-20T03:00:59.484512+0000","last_up_change":"2024-08-20T02:59:26.404917+0000","last_in_change":"2024-08-20T02:59:03.707380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T02:55:46.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T03:00:14.354568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T03:00:16.869335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7708f8c-a799-4b8f-ab1e-78482d73ca1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6803","nonce":858898316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6805","nonce":858898316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6809","nonce":858898316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6807","nonce":858898316}]},"public_addr":"172.21.15.44:6803/858898316","cluster_addr":"172.21.15.44:6805/858898316","heartbeat_back_addr":"172.21.15.44:6809/858898316","heartbeat_front_addr":"172.21.15.44:6807/858898316","state":["exists","up"]},{"osd":1,"uuid":"12f07986-60bf-498f-a012-cd7ea607b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6801","nonce":4252911663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6803","nonce":4252911663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6807","nonce":4252911663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6805","nonce":4252911663}]},"public_addr":"172.21.15.99:6801/4252911663","cluster_addr":"172.21.15.99:6803/4252911663","heartbeat_back_addr":"172.21.15.99:6807/4252911663","heartbeat_front_addr":"172.21.15.99:6805/4252911663","state":["exists","up"]},{"osd":2,"uuid":"0f189000-ba7f-4412-b0d3-75e8265f6685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6801","nonce":1492740465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6803","nonce":1492740465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6807","nonce":1492740465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6805","nonce":1492740465}]},"public_addr":"172.21.15.165:6801/1492740465","cluster_addr":"172.21.15.165:6803/1492740465","heartbeat_back_addr":"172.21.15.165:6807/1492740465","heartbeat_front_addr":"172.21.15.165:6805/1492740465","state":["exists","up"]},{"osd":3,"uuid":"8f9b042c-d6b5-4eee-aa29-ffc41a48c20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6811","nonce":3423983702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6813","nonce":3423983702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6817","nonce":3423983702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6815","nonce":3423983702}]},"public_addr":"172.21.15.44:6811/3423983702","cluster_addr":"172.21.15.44:6813/3423983702","heartbeat_back_addr":"172.21.15.44:6817/3423983702","heartbeat_front_addr":"172.21.15.44:6815/3423983702","state":["exists","up"]},{"osd":4,"uuid":"4edebfdb-af10-49ee-b724-2a63b9219221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6809","nonce":3099617325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6811","nonce":3099617325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6815","nonce":3099617325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6813","nonce":3099617325}]},"public_addr":"172.21.15.99:6809/3099617325","cluster_addr":"172.21.15.99:6811/3099617325","heartbeat_back_addr":"172.21.15.99:6815/3099617325","heartbeat_front_addr":"172.21.15.99:6813/3099617325","state":["exists","up"]},{"osd":5,"uuid":"6d22c21f-cc43-4ef5-a75c-d4fc1a361c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6809","nonce":523245255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6811","nonce":523245255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6815","nonce":523245255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6813","nonce":523245255}]},"public_addr":"172.21.15.165:6809/523245255","cluster_addr":"172.21.15.165:6811/523245255","heartbeat_back_addr":"172.21.15.165:6815/523245255","heartbeat_front_addr":"172.21.15.165:6813/523245255","state":["exists","up"]},{"osd":6,"uuid":"03df0c2f-1b7c-4242-888c-e047322fd8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6819","nonce":1039098986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6821","nonce":1039098986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6825","nonce":1039098986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6823","nonce":1039098986}]},"public_addr":"172.21.15.44:6819/1039098986","cluster_addr":"172.21.15.44:6821/1039098986","heartbeat_back_addr":"172.21.15.44:6825/1039098986","heartbeat_front_addr":"172.21.15.44:6823/1039098986","state":["exists","up"]},{"osd":7,"uuid":"84bbc683-9b13-448e-897c-114458dae33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6817","nonce":1438613564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6819","nonce":1438613564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6823","nonce":1438613564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6821","nonce":1438613564}]},"public_addr":"172.21.15.99:6817/1438613564","cluster_addr":"172.21.15.99:6819/1438613564","heartbeat_back_addr":"172.21.15.99:6823/1438613564","heartbeat_front_addr":"172.21.15.99:6821/1438613564","state":["exists","up"]},{"osd":8,"uuid":"14d7ffd3-db05-4228-a12e-4c69ef65ef8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6817","nonce":632957688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6819","nonce":632957688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6823","nonce":632957688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6821","nonce":632957688}]},"public_addr":"172.21.15.165:6817/632957688","cluster_addr":"172.21.15.165:6819/632957688","heartbeat_back_addr":"172.21.15.165:6823/632957688","heartbeat_front_addr":"172.21.15.165:6821/632957688","state":["exists","up"]},{"osd":9,"uuid":"1ae468a5-3d72-4eb0-a1a6-cd87eae740b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6827","nonce":2426940093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6829","nonce":2426940093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6833","nonce":2426940093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6831","nonce":2426940093}]},"public_addr":"172.21.15.44:6827/2426940093","cluster_addr":"172.21.15.44:6829/2426940093","heartbeat_back_addr":"172.21.15.44:6833/2426940093","heartbeat_front_addr":"172.21.15.44:6831/2426940093","state":["exists","up"]},{"osd":10,"uuid":"fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6825","nonce":3275793038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6827","nonce":3275793038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6831","nonce":3275793038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6829","nonce":3275793038}]},"public_addr":"172.21.15.99:6825/3275793038","cluster_addr":"172.21.15.99:6827/3275793038","heartbeat_back_addr":"172.21.15.99:6831/3275793038","heartbeat_front_addr":"172.21.15.99:6829/3275793038","state":["exists","up"]},{"osd":11,"uuid":"50715410-1fc2-4818-b600-604a4b7cce9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6825","nonce":107688415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6827","nonce":107688415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6831","nonce":107688415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6829","nonce":107688415}]},"public_addr":"172.21.15.165:6825/107688415","cluster_addr":"172.21.15.165:6827/107688415","heartbeat_back_addr":"172.21.15.165:6831/107688415","heartbeat_front_addr":"172.21.15.165:6829/107688415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:54:56.697075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:20.486846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:43.353376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:08.333879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:32.186492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:55.891366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:19.681421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:45.475926+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:10.109795+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:34.444169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:59.147646+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:59:25.403071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2597827045":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/3582612619":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/1916809746":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2919033419":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:6800/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2331034359":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2572185146":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/4176802819":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/3654439062":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6801/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/2376733275":"2024-08-21T02:53:58.312659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T03:01:33.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.970+0000 7ff26b03d640 1 -- 172.21.15.44:0/2967749133 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2540b3cb0 msgr2=0x7ff2540b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:33.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.970+0000 7ff26b03d640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2540b3cb0 0x7ff2540b6170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff258009bd0 tx=0x7ff258005f90 comp rx=0 tx=0).stop 2024-08-20T03:01:33.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.970+0000 7ff26b03d640 1 -- 172.21.15.44:0/2967749133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff264107d50 msgr2=0x7ff26419d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:33.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.970+0000 7ff26b03d640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff264107d50 0x7ff26419d9f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00cce0 tx=0x7ff25c00a620 comp rx=0 tx=0).stop 2024-08-20T03:01:33.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.970+0000 7ff26b03d640 1 -- 172.21.15.44:0/2967749133 shutdown_connections 2024-08-20T03:01:33.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.970+0000 7ff26b03d640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2540b3cb0 0x7ff2540b6170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.970+0000 7ff26b03d640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff264107d50 0x7ff26419d9f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.970+0000 7ff26b03d640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2641073d0 0x7ff264198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.970+0000 7ff26b03d640 1 --2- 172.21.15.44:0/2967749133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2641019c0 0x7ff264198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.970+0000 7ff26b03d640 1 -- 172.21.15.44:0/2967749133 >> 172.21.15.44:0/2967749133 conn(0x7ff2640fd410 msgr2=0x7ff2640ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.970+0000 7ff26b03d640 1 -- 172.21.15.44:0/2967749133 shutdown_connections 2024-08-20T03:01:33.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:33.970+0000 7ff26b03d640 1 -- 172.21.15.44:0/2967749133 wait complete. 2024-08-20T03:01:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:33 smithi165 ceph-mon[22357]: pgmap v292: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-08-20T03:01:34.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:33 smithi099 ceph-mon[21984]: pgmap v292: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-08-20T03:01:34.676 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:01:34.931 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:35.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:34 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2967749133' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:01:35.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:34 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2967749133' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:01:35.209 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:34 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2967749133' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:01:35.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.412+0000 7f350973c640 1 -- 172.21.15.44:0/2721043188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35041084e0 msgr2=0x7f3504108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.412+0000 7f350973c640 1 --2- 172.21.15.44:0/2721043188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35041084e0 0x7f3504108920 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009900 tx=0x7f34f802a410 comp rx=0 tx=0).stop 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.412+0000 7f350973c640 1 -- 172.21.15.44:0/2721043188 shutdown_connections 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.412+0000 7f350973c640 1 --2- 172.21.15.44:0/2721043188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3504108e60 0x7f350410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.412+0000 7f350973c640 1 --2- 172.21.15.44:0/2721043188 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35041084e0 0x7f3504108920 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.412+0000 7f350973c640 1 --2- 172.21.15.44:0/2721043188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3504102ad0 0x7f3504102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.412+0000 7f350973c640 1 -- 172.21.15.44:0/2721043188 >> 172.21.15.44:0/2721043188 conn(0x7f35040fe540 msgr2=0x7f3504100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.412+0000 7f350973c640 1 -- 172.21.15.44:0/2721043188 shutdown_connections 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.412+0000 7f350973c640 1 -- 172.21.15.44:0/2721043188 wait complete. 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f350973c640 1 Processor -- start 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f350973c640 1 -- start start 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f350973c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3504102ad0 0x7f3504199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f350973c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35041084e0 0x7f3504199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:35.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f350973c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3504108e60 0x7f350419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f350973c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3504112b20 con 0x7f35041084e0 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f350973c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f35041129a0 con 0x7f3504108e60 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f350973c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3504112ca0 con 0x7f3504102ad0 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f35027fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35041084e0 0x7f3504199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f35027fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35041084e0 0x7f3504199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58200/0 (socket says 172.21.15.44:58200) 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f35027fc640 1 -- 172.21.15.44:0/3143232798 learned_addr learned my addr 172.21.15.44:0/3143232798 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f3502ffd640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3504102ad0 0x7f3504199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f35037fe640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3504108e60 0x7f350419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f35027fc640 1 -- 172.21.15.44:0/3143232798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3504102ad0 msgr2=0x7f3504199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f35027fc640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3504102ad0 0x7f3504199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f35027fc640 1 -- 172.21.15.44:0/3143232798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3504108e60 msgr2=0x7f350419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f35027fc640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3504108e60 0x7f350419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:35.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f35027fc640 1 -- 172.21.15.44:0/3143232798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350419f0e0 con 0x7f35041084e0 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.413+0000 7f35037fe640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3504108e60 0x7f350419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.414+0000 7f35027fc640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35041084e0 0x7f3504199570 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f34f8005940 tx=0x7f34f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.414+0000 7f34ebfff640 1 -- 172.21.15.44:0/3143232798 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f803f030 con 0x7f35041084e0 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.414+0000 7f350973c640 1 -- 172.21.15.44:0/3143232798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350419f370 con 0x7f35041084e0 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.414+0000 7f34ebfff640 1 -- 172.21.15.44:0/3143232798 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f8003cb0 con 0x7f35041084e0 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.414+0000 7f34ebfff640 1 -- 172.21.15.44:0/3143232798 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8035070 con 0x7f35041084e0 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.414+0000 7f350973c640 1 -- 172.21.15.44:0/3143232798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35041a5a90 con 0x7f35041084e0 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.415+0000 7f350973c640 1 -- 172.21.15.44:0/3143232798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3504102eb0 con 0x7f35041084e0 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.416+0000 7f34ebfff640 1 -- 172.21.15.44:0/3143232798 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f34f8042030 con 0x7f35041084e0 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.417+0000 7f34ebfff640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f34ec0b3cb0 0x7f34ec0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.417+0000 7f3502ffd640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f34ec0b3cb0 0x7f34ec0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.417+0000 7f34ebfff640 1 -- 172.21.15.44:0/3143232798 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f34f8104ac0 con 0x7f35041084e0 2024-08-20T03:01:35.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.418+0000 7f3502ffd640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f34ec0b3cb0 0x7f34ec0b6170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005fd0 tx=0x7f34f4005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:35.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.420+0000 7f34ebfff640 1 -- 172.21.15.44:0/3143232798 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f34f80cbf00 con 0x7f35041084e0 2024-08-20T03:01:35.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.642+0000 7f350973c640 1 -- 172.21.15.44:0/3143232798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f350419a160 con 0x7f35041084e0 2024-08-20T03:01:35.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.643+0000 7f34ebfff640 1 -- 172.21.15.44:0/3143232798 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11965 (secure 0 0 0) 0x7f34f80d41d0 con 0x7f35041084e0 2024-08-20T03:01:35.645 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:35.645 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"btime":"2024-08-20T03:00:30:384117+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:00:30.384102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24604,24589,24596,24583,24569]},"id":1}]} 2024-08-20T03:01:35.645 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-08-20T03:01:35.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.645+0000 7f350973c640 1 -- 172.21.15.44:0/3143232798 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f34ec0b3cb0 msgr2=0x7f34ec0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:35.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.645+0000 7f350973c640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f34ec0b3cb0 0x7f34ec0b6170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f34f4005fd0 tx=0x7f34f4005850 comp rx=0 tx=0).stop 2024-08-20T03:01:35.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.645+0000 7f350973c640 1 -- 172.21.15.44:0/3143232798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35041084e0 msgr2=0x7f3504199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:35.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.645+0000 7f350973c640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35041084e0 0x7f3504199570 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f34f8005940 tx=0x7f34f80042f0 comp rx=0 tx=0).stop 2024-08-20T03:01:35.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.645+0000 7f350973c640 1 -- 172.21.15.44:0/3143232798 shutdown_connections 2024-08-20T03:01:35.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.645+0000 7f350973c640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f34ec0b3cb0 0x7f34ec0b6170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:35.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.645+0000 7f350973c640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3504108e60 0x7f350419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:35.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.645+0000 7f350973c640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35041084e0 0x7f3504199570 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:35.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.645+0000 7f350973c640 1 --2- 172.21.15.44:0/3143232798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3504102ad0 0x7f3504199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:35.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.645+0000 7f350973c640 1 -- 172.21.15.44:0/3143232798 >> 172.21.15.44:0/3143232798 conn(0x7f35040fe540 msgr2=0x7f3504100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:35.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.645+0000 7f350973c640 1 -- 172.21.15.44:0/3143232798 shutdown_connections 2024-08-20T03:01:35.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:35.645+0000 7f350973c640 1 -- 172.21.15.44:0/3143232798 wait complete. 2024-08-20T03:01:36.021 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:35 smithi044 ceph-mon[18790]: pgmap v293: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:36.022 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:35 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3143232798' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:36.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:35 smithi165 ceph-mon[22357]: pgmap v293: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:36.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:35 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3143232798' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:36.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:35 smithi099 ceph-mon[21984]: pgmap v293: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:36.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:35 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3143232798' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:36.394 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-20T03:00:18.935328+0000', 'modified': '2024-08-20T03:00:30.384102+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24604, 'mds_1': 24596, 'mds_2': 24589, 'mds_3': 24583, 'mds_4': 24569}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14796': {'gid': 14796, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6839/335802127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 335802127}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 335802127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14808': {'gid': 14808, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6841/370123956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6840', 'nonce': 370123956}, {'type': 'v1', 'addr': '172.21.15.44:6841', 'nonce': 370123956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6833/2092228440', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 2092228440}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 2092228440}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/1910345968', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 1910345968}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 1910345968}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/797787628', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 797787628}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 797787628}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.99:6837/728486889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 728486889}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 728486889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24583': {'gid': 24583, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.165:6835/3019941967', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 3019941967}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 3019941967}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24589': {'gid': 24589, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.165:6837/3478633280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 3478633280}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 3478633280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.99:6839/1272675547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 1272675547}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 1272675547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24604': {'gid': 24604, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.165:6839/3029973590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 3029973590}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 3029973590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24604, 'qdb_cluster': [24604, 24589, 24596, 24583, 24569]} 2024-08-20T03:01:36.394 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-08-20T03:01:36.394 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-20T03:01:36.648 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:37.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.115+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3674123419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c0108e40 msgr2=0x7fd8c010f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:37.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.115+0000 7fd8c64fc640 1 --2- 172.21.15.44:0/3674123419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c0108e40 0x7fd8c010f640 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009ca0 tx=0x7fd8b802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.115+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3674123419 shutdown_connections 2024-08-20T03:01:37.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.115+0000 7fd8c64fc640 1 --2- 172.21.15.44:0/3674123419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c0108e40 0x7fd8c010f640 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.115+0000 7fd8c64fc640 1 --2- 172.21.15.44:0/3674123419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8c01084c0 0x7fd8c0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.115+0000 7fd8c64fc640 1 --2- 172.21.15.44:0/3674123419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8c0102ab0 0x7fd8c0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.115+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3674123419 >> 172.21.15.44:0/3674123419 conn(0x7fd8c00fe500 msgr2=0x7fd8c0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:37.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.115+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3674123419 shutdown_connections 2024-08-20T03:01:37.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.115+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3674123419 wait complete. 2024-08-20T03:01:37.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.115+0000 7fd8c64fc640 1 Processor -- start 2024-08-20T03:01:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c64fc640 1 -- start start 2024-08-20T03:01:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c64fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8c0102ab0 0x7fd8c00751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c64fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8c01084c0 0x7fd8c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c64fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c0108e40 0x7fd8c0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c64fc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0112ac0 con 0x7fd8c0108e40 2024-08-20T03:01:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c64fc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0112940 con 0x7fd8c0102ab0 2024-08-20T03:01:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c64fc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0112c40 con 0x7fd8c01084c0 2024-08-20T03:01:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c4a99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c0108e40 0x7fd8c0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c4a99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c0108e40 0x7fd8c0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58214/0 (socket says 172.21.15.44:58214) 2024-08-20T03:01:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c4a99640 1 -- 172.21.15.44:0/3287789102 learned_addr learned my addr 172.21.15.44:0/3287789102 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8bffff640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8c0102ab0 0x7fd8c00751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8c01084c0 0x7fd8c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c4a99640 1 -- 172.21.15.44:0/3287789102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8c01084c0 msgr2=0x7fd8c0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c4a99640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8c01084c0 0x7fd8c0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c4a99640 1 -- 172.21.15.44:0/3287789102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8c0102ab0 msgr2=0x7fd8c00751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c4a99640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8c0102ab0 0x7fd8c00751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c4a99640 1 -- 172.21.15.44:0/3287789102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c01a72f0 con 0x7fd8c0108e40 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8c4a99640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c0108e40 0x7fd8c0075c50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8005cc0 tx=0x7fd8b802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.116+0000 7fd8bdffb640 1 -- 172.21.15.44:0/3287789102 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8003bf0 con 0x7fd8c0108e40 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.117+0000 7fd8bdffb640 1 -- 172.21.15.44:0/3287789102 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b8003d90 con 0x7fd8c0108e40 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.117+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3287789102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c01a7580 con 0x7fd8c0108e40 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.117+0000 7fd8bdffb640 1 -- 172.21.15.44:0/3287789102 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b802fb20 con 0x7fd8c0108e40 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.117+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3287789102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c01a7ac0 con 0x7fd8c0108e40 2024-08-20T03:01:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.119+0000 7fd8c88f5740 1 Processor -- start 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.119+0000 7fd8bdffb640 1 -- 172.21.15.44:0/3287789102 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fd8b8004580 con 0x7fd8c0108e40 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.119+0000 7fd8c88f5740 1 -- start start 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8c88f5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 0x55c5d738cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8c88f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d738d510 0x55c5d7398f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8c88f5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d73994a0 0x55c5d739b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8c88f5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c5d73036a0 con 0x55c5d738cbf0 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8c88f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c5d72f2ba0 con 0x55c5d738d510 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8c88f5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5d72f6be0 con 0x55c5d73994a0 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 0x55c5d738cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 0x55c5d738cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58228/0 (socket says 172.21.15.44:58228) 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8b7fff640 1 -- 172.21.15.44:0/561358118 learned_addr learned my addr 172.21.15.44:0/561358118 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8bffff640 1 --2- 172.21.15.44:0/561358118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d738d510 0x55c5d7398f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8b7fff640 1 -- 172.21.15.44:0/561358118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d73994a0 msgr2=0x55c5d739b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/561358118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d73994a0 0x55c5d739b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8b7fff640 1 -- 172.21.15.44:0/561358118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d738d510 msgr2=0x55c5d7398f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/561358118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d738d510 0x55c5d7398f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8b7fff640 1 -- 172.21.15.44:0/561358118 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5d7052540 con 0x55c5d738cbf0 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8bdffb640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd8ac0b3cb0 0x7fd8ac0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/561358118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 0x55c5d738cfd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800b7f0 tx=0x7fd8a800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12e2082f1fb83dd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8bffff640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd8ac0b3cb0 0x7fd8ac0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8bdffb640 1 -- 172.21.15.44:0/3287789102 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd8b8105410 con 0x7fd8c0108e40 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.120+0000 7fd8b67fc640 1 -- 172.21.15.44:0/561358118 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8003dd0 con 0x55c5d738cbf0 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.121+0000 7fd8b67fc640 1 -- 172.21.15.44:0/561358118 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8a80054d0 con 0x55c5d738cbf0 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.121+0000 7fd8b67fc640 1 -- 172.21.15.44:0/561358118 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8010430 con 0x55c5d738cbf0 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.121+0000 7fd8c88f5740 1 -- 172.21.15.44:0/561358118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 msgr2=0x55c5d738cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:37.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.121+0000 7fd8bffff640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd8ac0b3cb0 0x7fd8ac0b6170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55c5d738fd70 tx=0x7fd8b000a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.121+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/561358118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 0x55c5d738cfd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800b7f0 tx=0x7fd8a800bcb0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.121+0000 7fd8c88f5740 1 -- 172.21.15.44:0/561358118 shutdown_connections 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.121+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/561358118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d73994a0 0x55c5d739b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.121+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/561358118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d738d510 0x55c5d7398f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.121+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/561358118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 0x55c5d738cfd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.121+0000 7fd8c88f5740 1 -- 172.21.15.44:0/561358118 >> 172.21.15.44:0/561358118 conn(0x55c5d738aa20 msgr2=0x55c5d738bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.121+0000 7fd8c88f5740 1 -- 172.21.15.44:0/561358118 shutdown_connections 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.121+0000 7fd8c88f5740 1 -- 172.21.15.44:0/561358118 wait complete. 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8c88f5740 1 -- start start 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8c88f5740 1 Processor -- start 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8c88f5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 0x55c5d73963d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8c88f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d738d510 0x55c5d7396910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8c88f5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d73994a0 0x55c5d7396e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8c88f5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c5d71555f0 con 0x55c5d738cbf0 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8c88f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c5d72796c0 con 0x55c5d738d510 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8c88f5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5d739c1a0 con 0x55c5d73994a0 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8c88f5740 10 client.0 _ll_register_callbacks cb 0x55c5d72b61e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8bffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 0x55c5d73963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8bffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 0x55c5d73963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58232/0 (socket says 172.21.15.44:58232) 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8bffff640 1 -- 172.21.15.44:0/1707933033 learned_addr learned my addr 172.21.15.44:0/1707933033 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d738d510 0x55c5d7396910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.122+0000 7fd8c4a99640 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d73994a0 0x55c5d7396e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd8bffff640 1 -- 172.21.15.44:0/1707933033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d73994a0 msgr2=0x55c5d7396e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd8bffff640 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d73994a0 0x55c5d7396e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd8bffff640 1 -- 172.21.15.44:0/1707933033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d738d510 msgr2=0x55c5d7396910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd8bffff640 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d738d510 0x55c5d7396910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd8bffff640 1 -- 172.21.15.44:0/1707933033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5d73a5500 con 0x55c5d738cbf0 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd8bffff640 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 0x55c5d73963d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000a7a0 tx=0x7fd8b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5968e4bf05613e85 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd89f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd89f7fe640 1 -- 172.21.15.44:0/1707933033 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b000f8c0 con 0x55c5d738cbf0 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd89f7fe640 1 -- 172.21.15.44:0/1707933033 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd8b000fa60 con 0x55c5d738cbf0 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd8c88f5740 1 -- 172.21.15.44:0/1707933033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8a80054d0 con 0x55c5d738cbf0 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd89f7fe640 1 -- 172.21.15.44:0/1707933033 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd8b000fc60 con 0x55c5d738cbf0 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd8c88f5740 10 client.15168 fetch_fsmap learned FSMap version 27 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd8c88f5740 1 -- 172.21.15.44:0/1707933033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5d73a7ef0 con 0x55c5d738cbf0 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd89f7fe640 1 -- 172.21.15.44:0/1707933033 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8a80054d0 con 0x55c5d738cbf0 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.123+0000 7fd89f7fe640 1 -- 172.21.15.44:0/1707933033 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd8b001d430 con 0x55c5d738cbf0 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.124+0000 7fd8c88f5740 10 client.15168 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:01:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.124+0000 7fd8c88f5740 10 client.15168 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.124+0000 7fd8c88f5740 20 client.15168 populate_metadata read hostname 'smithi044' 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.124+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c5d73aa580 0x55c5d73ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.124+0000 7fd8c88f5740 4 client.15168 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.124+0000 7fd8c88f5740 1 -- 172.21.15.44:0/1707933033 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5d73055f0 con 0x55c5d73aa580 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.124+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c5d73aa580 0x55c5d73ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.124+0000 7fd89f7fe640 10 client.15168 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.124+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c5d73aa580 0x55c5d73ab9f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f3363c4c8e8c5f39 server_cookie=bbbcd0fc63886c40 in_seq=0 out_seq=0 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.125+0000 7fd89f7fe640 1 -- 172.21.15.44:0/1707933033 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5d73055f0 con 0x55c5d73aa580 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.125+0000 7fd89f7fe640 10 client.15168 handle_command_reply: tid=0 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.125+0000 7fd8c88f5740 1 client.15168 shutdown 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.126+0000 7fd8c88f5740 1 -- 172.21.15.44:0/1707933033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 msgr2=0x55c5d73963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.126+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 0x55c5d73963d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000a7a0 tx=0x7fd8b00046c0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.126+0000 7fd8c88f5740 1 -- 172.21.15.44:0/1707933033 shutdown_connections 2024-08-20T03:01:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.126+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d73994a0 0x55c5d7396e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.126+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c5d73aa580 0x55c5d73ab9f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.126+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d738d510 0x55c5d7396910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.126+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/1707933033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738cbf0 0x55c5d73963d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.126+0000 7fd89f7fe640 0 client.15168 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:01:37.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.126+0000 7fd8c88f5740 1 -- 172.21.15.44:0/1707933033 >> 172.21.15.44:0/1707933033 conn(0x55c5d738aa20 msgr2=0x55c5d73905c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:37.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.126+0000 7fd8c88f5740 1 -- 172.21.15.44:0/1707933033 shutdown_connections 2024-08-20T03:01:37.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.126+0000 7fd8c88f5740 1 -- 172.21.15.44:0/1707933033 wait complete. 2024-08-20T03:01:37.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.126+0000 7fd8c88f5740 20 client.15168 trim_cache size 0 max 16384 2024-08-20T03:01:37.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.144+0000 7fd8c88f5740 1 Processor -- start 2024-08-20T03:01:37.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.144+0000 7fd8c88f5740 1 -- start start 2024-08-20T03:01:37.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.144+0000 7fd8c88f5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d738cbf0 0x55c5d7390db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.144+0000 7fd8c88f5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 0x55c5d73958c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.144+0000 7fd8c88f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d73994a0 0x55c5d73935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.144+0000 7fd8c88f5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c5d73a9d30 con 0x55c5d738d510 2024-08-20T03:01:37.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.144+0000 7fd8c88f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c5d73aa400 con 0x55c5d73994a0 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.144+0000 7fd8c88f5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5d73aaad0 con 0x55c5d738cbf0 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.144+0000 7fd8b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 0x55c5d73958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.144+0000 7fd8b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 0x55c5d73958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58240/0 (socket says 172.21.15.44:58240) 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.144+0000 7fd8b7fff640 1 -- 172.21.15.44:0/708759605 learned_addr learned my addr 172.21.15.44:0/708759605 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.145+0000 7fd8c4a99640 1 --2- 172.21.15.44:0/708759605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d73994a0 0x55c5d73935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.145+0000 7fd8bffff640 1 --2- 172.21.15.44:0/708759605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d738cbf0 0x55c5d7390db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.145+0000 7fd8b7fff640 1 -- 172.21.15.44:0/708759605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d738cbf0 msgr2=0x55c5d7390db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.145+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/708759605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d738cbf0 0x55c5d7390db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.145+0000 7fd8b7fff640 1 -- 172.21.15.44:0/708759605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d73994a0 msgr2=0x55c5d73935e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.145+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/708759605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d73994a0 0x55c5d73935e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.145+0000 7fd8b7fff640 1 -- 172.21.15.44:0/708759605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5d738b4a0 con 0x55c5d738d510 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.145+0000 7fd8bffff640 1 --2- 172.21.15.44:0/708759605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d738cbf0 0x55c5d7390db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.145+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/708759605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 0x55c5d73958c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8006010 tx=0x7fd8a800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e9d77d3cdf991d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:37.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.145+0000 7fd8b5ffb640 1 -- 172.21.15.44:0/708759605 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a802d020 con 0x55c5d738d510 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.145+0000 7fd8b5ffb640 1 -- 172.21.15.44:0/708759605 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8a8014040 con 0x55c5d738d510 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.145+0000 7fd8b5ffb640 1 -- 172.21.15.44:0/708759605 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a801c450 con 0x55c5d738d510 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.146+0000 7fd8c88f5740 1 -- 172.21.15.44:0/708759605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 msgr2=0x55c5d73958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.146+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/708759605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 0x55c5d73958c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8006010 tx=0x7fd8a800b390 comp rx=0 tx=0).stop 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.146+0000 7fd8c88f5740 1 -- 172.21.15.44:0/708759605 shutdown_connections 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.146+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/708759605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d73994a0 0x55c5d73935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.146+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/708759605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 0x55c5d73958c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.146+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/708759605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d738cbf0 0x55c5d7390db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.146+0000 7fd8c88f5740 1 -- 172.21.15.44:0/708759605 >> 172.21.15.44:0/708759605 conn(0x55c5d738aa20 msgr2=0x55c5d7394c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.146+0000 7fd8c88f5740 1 -- 172.21.15.44:0/708759605 shutdown_connections 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.146+0000 7fd8c88f5740 1 -- 172.21.15.44:0/708759605 wait complete. 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.146+0000 7fd8c88f5740 1 -- start start 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.146+0000 7fd8c88f5740 1 Processor -- start 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8c88f5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d738cbf0 0x55c5d73b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8c88f5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 0x55c5d73ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8c88f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d73994a0 0x55c5d73adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8c88f5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c5d73ab870 con 0x55c5d738d510 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8c88f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c5d73ab1a0 con 0x55c5d73994a0 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8c88f5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c5d73abf40 con 0x55c5d738cbf0 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8c88f5740 10 client.0 _ll_register_callbacks cb 0x7fd8b000a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 0x55c5d73ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 0x55c5d73ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58254/0 (socket says 172.21.15.44:58254) 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8b7fff640 1 -- 172.21.15.44:0/298062703 learned_addr learned my addr 172.21.15.44:0/298062703 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8bffff640 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d738cbf0 0x55c5d73b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8c4a99640 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d73994a0 0x55c5d73adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8b7fff640 1 -- 172.21.15.44:0/298062703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d738cbf0 msgr2=0x55c5d73b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d738cbf0 0x55c5d73b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8b7fff640 1 -- 172.21.15.44:0/298062703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d73994a0 msgr2=0x55c5d73adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d73994a0 0x55c5d73adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8b7fff640 1 -- 172.21.15.44:0/298062703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5d73ae0e0 con 0x55c5d738d510 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8b7fff640 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 0x55c5d73ad5b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8028a70 tx=0x7fd8a8028aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9131457e2ecff0e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd89f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd89f7fe640 1 -- 172.21.15.44:0/298062703 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a803c060 con 0x55c5d738d510 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd8c88f5740 1 -- 172.21.15.44:0/298062703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8a8014040 con 0x55c5d738d510 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.147+0000 7fd89f7fe640 1 -- 172.21.15.44:0/298062703 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd8a80283b0 con 0x55c5d738d510 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.148+0000 7fd89f7fe640 1 -- 172.21.15.44:0/298062703 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd8a802d030 con 0x55c5d738d510 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.148+0000 7fd8c88f5740 10 client.15174 fetch_fsmap learned FSMap version 27 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.148+0000 7fd8c88f5740 1 -- 172.21.15.44:0/298062703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5d73b3ad0 con 0x55c5d738d510 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.148+0000 7fd89f7fe640 1 -- 172.21.15.44:0/298062703 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8a8014040 con 0x55c5d738d510 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.148+0000 7fd89f7fe640 1 -- 172.21.15.44:0/298062703 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd8a803f020 con 0x55c5d738d510 2024-08-20T03:01:37.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.148+0000 7fd8c88f5740 10 client.15174 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.148+0000 7fd8c88f5740 10 client.15174 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.148+0000 7fd8c88f5740 20 client.15174 populate_metadata read hostname 'smithi044' 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.148+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c5d7418ec0 0x55c5d7417bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.148+0000 7fd8c88f5740 4 client.15174 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.148+0000 7fd8c88f5740 1 -- 172.21.15.44:0/298062703 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "status"}) -- 0x55c5d7301090 con 0x55c5d7418ec0 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd8bffff640 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c5d7418ec0 0x55c5d7417bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd8bffff640 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c5d7418ec0 0x55c5d7417bd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b8d8cc89df5888b0 server_cookie=a4b0019fabb7ccb9 in_seq=0 out_seq=0 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd89f7fe640 10 client.15174 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd89f7fe640 1 -- 172.21.15.44:0/298062703 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c5d7301090 con 0x55c5d7418ec0 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd89f7fe640 10 client.15174 handle_command_reply: tid=0 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd8c88f5740 1 client.15174 shutdown 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd8c88f5740 1 -- 172.21.15.44:0/298062703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 msgr2=0x55c5d73ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 0x55c5d73ad5b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8028a70 tx=0x7fd8a8028aa0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd8c88f5740 1 -- 172.21.15.44:0/298062703 shutdown_connections 2024-08-20T03:01:37.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c5d7418ec0 0x55c5d7417bd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c5d73994a0 0x55c5d73adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c5d738d510 0x55c5d73ad5b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd8c88f5740 1 --2- 172.21.15.44:0/298062703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c5d738cbf0 0x55c5d73b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd89f7fe640 0 client.15174 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:01:37.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd8c88f5740 1 -- 172.21.15.44:0/298062703 >> 172.21.15.44:0/298062703 conn(0x55c5d738aa20 msgr2=0x55c5d73a4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:37.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.149+0000 7fd8c88f5740 1 -- 172.21.15.44:0/298062703 shutdown_connections 2024-08-20T03:01:37.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c88f5740 1 -- 172.21.15.44:0/298062703 wait complete. 2024-08-20T03:01:37.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c88f5740 20 client.15174 trim_cache size 0 max 16384 2024-08-20T03:01:37.165 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:01:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T03:01:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 4, 2024-08-20T03:01:37.165 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24569, 2024-08-20T03:01:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-08-20T03:01:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-08-20T03:01:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-08-20T03:01:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 67.769143982000003, 2024-08-20T03:01:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2024-08-20T03:01:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 77, 2024-08-20T03:01:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 75, 2024-08-20T03:01:37.166 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 100.46513682200001 2024-08-20T03:01:37.166 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:01:37.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3287789102 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd8ac0b3cb0 msgr2=0x7fd8ac0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:37.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c64fc640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd8ac0b3cb0 0x7fd8ac0b6170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55c5d738fd70 tx=0x7fd8b000a2d0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3287789102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c0108e40 msgr2=0x7fd8c0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:37.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c64fc640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c0108e40 0x7fd8c0075c50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8005cc0 tx=0x7fd8b802f380 comp rx=0 tx=0).stop 2024-08-20T03:01:37.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3287789102 shutdown_connections 2024-08-20T03:01:37.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c64fc640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd8ac0b3cb0 0x7fd8ac0b6170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c64fc640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8c0108e40 0x7fd8c0075c50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c64fc640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8c01084c0 0x7fd8c0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c64fc640 1 --2- 172.21.15.44:0/3287789102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8c0102ab0 0x7fd8c00751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:37.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3287789102 >> 172.21.15.44:0/3287789102 conn(0x7fd8c00fe500 msgr2=0x7fd8c01008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:37.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3287789102 shutdown_connections 2024-08-20T03:01:37.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:37.150+0000 7fd8c64fc640 1 -- 172.21.15.44:0/3287789102 wait complete. 2024-08-20T03:01:37.831 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.f status 2024-08-20T03:01:38.085 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:38.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:37 smithi165 ceph-mon[22357]: pgmap v294: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:38.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:37 smithi099 ceph-mon[21984]: pgmap v294: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:38.207 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:37 smithi044 ceph-mon[18790]: pgmap v294: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:38.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.558+0000 7efeec613640 1 -- 172.21.15.44:0/4176621089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee4107d60 msgr2=0x7efee410e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:38.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.558+0000 7efeec613640 1 --2- 172.21.15.44:0/4176621089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee4107d60 0x7efee410e560 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efedc009ca0 tx=0x7efedc02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.558+0000 7efeec613640 1 -- 172.21.15.44:0/4176621089 shutdown_connections 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.558+0000 7efeec613640 1 --2- 172.21.15.44:0/4176621089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee4107d60 0x7efee410e560 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.558+0000 7efeec613640 1 --2- 172.21.15.44:0/4176621089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee41073e0 0x7efee4107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.558+0000 7efeec613640 1 --2- 172.21.15.44:0/4176621089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee41019d0 0x7efee4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.558+0000 7efeec613640 1 -- 172.21.15.44:0/4176621089 >> 172.21.15.44:0/4176621089 conn(0x7efee40fd440 msgr2=0x7efee40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.558+0000 7efeec613640 1 -- 172.21.15.44:0/4176621089 shutdown_connections 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.558+0000 7efeec613640 1 -- 172.21.15.44:0/4176621089 wait complete. 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.559+0000 7efeec613640 1 Processor -- start 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.559+0000 7efeec613640 1 -- start start 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.559+0000 7efeec613640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee41019d0 0x7efee4199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.559+0000 7efeec613640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee41073e0 0x7efee419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.559+0000 7efeec613640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee4107d60 0x7efee419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.559+0000 7efeec613640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efee4111a60 con 0x7efee41073e0 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.559+0000 7efeec613640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efee41118e0 con 0x7efee41019d0 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.559+0000 7efeec613640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efee4111be0 con 0x7efee4107d60 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efee9bae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee41073e0 0x7efee419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efee9bae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee41073e0 0x7efee419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46674/0 (socket says 172.21.15.44:46674) 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efee9bae640 1 -- 172.21.15.44:0/706000294 learned_addr learned my addr 172.21.15.44:0/706000294 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efeea3af640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee41019d0 0x7efee4199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efeeabb0640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee4107d60 0x7efee419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efee9bae640 1 -- 172.21.15.44:0/706000294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee4107d60 msgr2=0x7efee419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efee9bae640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee4107d60 0x7efee419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efee9bae640 1 -- 172.21.15.44:0/706000294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee41019d0 msgr2=0x7efee4199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efee9bae640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee41019d0 0x7efee4199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efee9bae640 1 -- 172.21.15.44:0/706000294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee41a5330 con 0x7efee41073e0 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efeeabb0640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee4107d60 0x7efee419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efeea3af640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee41019d0 0x7efee4199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:38.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efee9bae640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee41073e0 0x7efee419a490 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efed800b4a0 tx=0x7efed800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:38.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efed77fe640 1 -- 172.21.15.44:0/706000294 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8014070 con 0x7efee41073e0 2024-08-20T03:01:38.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efeec613640 1 -- 172.21.15.44:0/706000294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee41a5620 con 0x7efee41073e0 2024-08-20T03:01:38.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efed77fe640 1 -- 172.21.15.44:0/706000294 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed8010ac0 con 0x7efee41073e0 2024-08-20T03:01:38.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efed77fe640 1 -- 172.21.15.44:0/706000294 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8017c30 con 0x7efee41073e0 2024-08-20T03:01:38.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.560+0000 7efeec613640 1 -- 172.21.15.44:0/706000294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee41a5b10 con 0x7efee41073e0 2024-08-20T03:01:38.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.563+0000 7efed77fe640 1 -- 172.21.15.44:0/706000294 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7efed8017070 con 0x7efee41073e0 2024-08-20T03:01:38.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.563+0000 7efeeea0c740 1 Processor -- start 2024-08-20T03:01:38.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.563+0000 7efeeea0c740 1 -- start start 2024-08-20T03:01:38.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.563+0000 7efeeea0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 0x556055efcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.563+0000 7efeeea0c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055f0d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.563+0000 7efeeea0c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055f0d8b0 0x556055f0fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.563+0000 7efeeea0c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556055e736a0 con 0x556055efd510 2024-08-20T03:01:38.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.563+0000 7efeeea0c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556055e62ba0 con 0x556055f0d8b0 2024-08-20T03:01:38.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.563+0000 7efeeea0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556055e66be0 con 0x556055efcbf0 2024-08-20T03:01:38.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.563+0000 7efeea3af640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055f0d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.563+0000 7efeea3af640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055f0d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46678/0 (socket says 172.21.15.44:46678) 2024-08-20T03:01:38.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.563+0000 7efeea3af640 1 -- 172.21.15.44:0/495329487 learned_addr learned my addr 172.21.15.44:0/495329487 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:38.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efee9bae640 1 --2- 172.21.15.44:0/495329487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055f0d8b0 0x556055f0fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efeeabb0640 1 --2- 172.21.15.44:0/495329487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 0x556055efcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efed77fe640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efed00b3cb0 0x7efed00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efeea3af640 1 -- 172.21.15.44:0/495329487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 msgr2=0x556055efcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:38.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efeea3af640 1 --2- 172.21.15.44:0/495329487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 0x556055efcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efeea3af640 1 -- 172.21.15.44:0/495329487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055f0d8b0 msgr2=0x556055f0fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:38.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efeea3af640 1 --2- 172.21.15.44:0/495329487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055f0d8b0 0x556055f0fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efeea3af640 1 -- 172.21.15.44:0/495329487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556055bc2540 con 0x556055efd510 2024-08-20T03:01:38.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efee9bae640 1 --2- 172.21.15.44:0/495329487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055f0d8b0 0x556055f0fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:38.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efeeabb0640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efed00b3cb0 0x7efed00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efed77fe640 1 -- 172.21.15.44:0/706000294 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efed80e3200 con 0x7efee41073e0 2024-08-20T03:01:38.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efeea3af640 1 --2- 172.21.15.44:0/495329487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055f0d370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efee0007850 tx=0x7efee0007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c6682432a6db2b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:38.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efed4ff9640 1 -- 172.21.15.44:0/495329487 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee000d950 con 0x556055efd510 2024-08-20T03:01:38.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efed4ff9640 1 -- 172.21.15.44:0/495329487 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee0018070 con 0x556055efd510 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.564+0000 7efed4ff9640 1 -- 172.21.15.44:0/495329487 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0013d90 con 0x556055efd510 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.565+0000 7efeeea0c740 1 -- 172.21.15.44:0/495329487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 msgr2=0x556055f0d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.565+0000 7efeeabb0640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efed00b3cb0 0x7efed00b6170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efedc009a60 tx=0x7efedc0095c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.565+0000 7efeeea0c740 1 --2- 172.21.15.44:0/495329487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055f0d370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efee0007850 tx=0x7efee0007d10 comp rx=0 tx=0).stop 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.565+0000 7efeeea0c740 1 -- 172.21.15.44:0/495329487 shutdown_connections 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.565+0000 7efeeea0c740 1 --2- 172.21.15.44:0/495329487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055f0d8b0 0x556055f0fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.565+0000 7efeeea0c740 1 --2- 172.21.15.44:0/495329487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055f0d370 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.565+0000 7efeeea0c740 1 --2- 172.21.15.44:0/495329487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 0x556055efcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.565+0000 7efeeea0c740 1 -- 172.21.15.44:0/495329487 >> 172.21.15.44:0/495329487 conn(0x556055efaa20 msgr2=0x556055efbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.565+0000 7efeeea0c740 1 -- 172.21.15.44:0/495329487 shutdown_connections 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.565+0000 7efeeea0c740 1 -- 172.21.15.44:0/495329487 wait complete. 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.565+0000 7efeeea0c740 1 -- start start 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.565+0000 7efeeea0c740 1 Processor -- start 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeea0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 0x556055f0d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeea0c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055efd510 0x556055f0a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeea0c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055f0d8b0 0x556055f0aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeea0c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556055cc55f0 con 0x556055f0d8b0 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeea0c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556055de96c0 con 0x556055efd510 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeea0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556055f10560 con 0x556055efcbf0 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeea0c740 10 client.0 _ll_register_callbacks cb 0x556055e261e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeabb0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055f0d8b0 0x556055f0aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeabb0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055f0d8b0 0x556055f0aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46682/0 (socket says 172.21.15.44:46682) 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeabb0640 1 -- 172.21.15.44:0/179752103 learned_addr learned my addr 172.21.15.44:0/179752103 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeea3af640 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 0x556055f0d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efee9bae640 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055efd510 0x556055f0a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeabb0640 1 -- 172.21.15.44:0/179752103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 msgr2=0x556055f0d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeabb0640 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 0x556055f0d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeabb0640 1 -- 172.21.15.44:0/179752103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055efd510 msgr2=0x556055f0a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeabb0640 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055efd510 0x556055f0a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeabb0640 1 -- 172.21.15.44:0/179752103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556055f19890 con 0x556055f0d8b0 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efeeabb0640 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055f0d8b0 0x556055f0aee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efedc0317a0 tx=0x7efedc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dea0e290ae8cbd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efecdffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.566+0000 7efecdffb640 1 -- 172.21.15.44:0/179752103 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc004240 con 0x556055f0d8b0 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efecdffb640 1 -- 172.21.15.44:0/179752103 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efedc0043e0 con 0x556055f0d8b0 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efeeea0c740 1 -- 172.21.15.44:0/179752103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee0018070 con 0x556055f0d8b0 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efecdffb640 1 -- 172.21.15.44:0/179752103 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efedc0045e0 con 0x556055f0d8b0 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efeeea0c740 10 client.15186 fetch_fsmap learned FSMap version 27 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efeeea0c740 1 -- 172.21.15.44:0/179752103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556055f1c1c0 con 0x556055f0d8b0 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efecdffb640 1 -- 172.21.15.44:0/179752103 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee0018070 con 0x556055f0d8b0 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efecdffb640 1 -- 172.21.15.44:0/179752103 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efedc0306e0 con 0x556055f0d8b0 2024-08-20T03:01:38.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efeeea0c740 10 client.15186 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efeeea0c740 10 client.15186 resolve_mds: resolved name 'f' to daemon mds.f 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efeeea0c740 20 client.15186 populate_metadata read hostname 'smithi044' 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efeeea0c740 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x556055f1e8e0 0x556055f1fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efeeea0c740 4 client.15186 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efeeea0c740 1 -- 172.21.15.44:0/179752103 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556055e755f0 con 0x556055f1e8e0 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.567+0000 7efeea3af640 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x556055f1e8e0 0x556055f1fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.568+0000 7efecdffb640 10 client.15186 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.568+0000 7efeea3af640 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x556055f1e8e0 0x556055f1fd50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ffaae80c5c295180 server_cookie=3cd2ea3182d29ab in_seq=0 out_seq=0 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efecdffb640 1 -- 172.21.15.44:0/179752103 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556055e755f0 con 0x556055f1e8e0 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efecdffb640 10 client.15186 handle_command_reply: tid=0 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efeeea0c740 1 client.15186 shutdown 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efeeea0c740 1 -- 172.21.15.44:0/179752103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055f0d8b0 msgr2=0x556055f0aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:38.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efeeea0c740 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055f0d8b0 0x556055f0aee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efedc0317a0 tx=0x7efedc02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efeeea0c740 1 -- 172.21.15.44:0/179752103 shutdown_connections 2024-08-20T03:01:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efeeea0c740 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x556055f1e8e0 0x556055f1fd50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efeeea0c740 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055f0d8b0 0x556055f0aee0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efeeea0c740 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055efd510 0x556055f0a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efecdffb640 0 client.15186 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:01:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efeeea0c740 1 --2- 172.21.15.44:0/179752103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 0x556055f0d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efeeea0c740 1 -- 172.21.15.44:0/179752103 >> 172.21.15.44:0/179752103 conn(0x556055efaa20 msgr2=0x556055f00550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efeeea0c740 1 -- 172.21.15.44:0/179752103 shutdown_connections 2024-08-20T03:01:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efeeea0c740 1 -- 172.21.15.44:0/179752103 wait complete. 2024-08-20T03:01:38.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.569+0000 7efeeea0c740 20 client.15186 trim_cache size 0 max 16384 2024-08-20T03:01:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.587+0000 7efeeea0c740 1 Processor -- start 2024-08-20T03:01:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.587+0000 7efeeea0c740 1 -- start start 2024-08-20T03:01:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.587+0000 7efeeea0c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055efcbf0 0x556055f08340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.587+0000 7efeeea0c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055f0dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.587+0000 7efeeea0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055f0d8b0 0x556055f08fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.587+0000 7efeeea0c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556055f1e090 con 0x556055efd510 2024-08-20T03:01:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.587+0000 7efeeea0c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556055f1e760 con 0x556055efcbf0 2024-08-20T03:01:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.587+0000 7efeeea0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556055f1ee30 con 0x556055f0d8b0 2024-08-20T03:01:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.587+0000 7efee9bae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055f0dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.587+0000 7efee9bae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055f0dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46690/0 (socket says 172.21.15.44:46690) 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.587+0000 7efee9bae640 1 -- 172.21.15.44:0/2413933309 learned_addr learned my addr 172.21.15.44:0/2413933309 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.587+0000 7efeea3af640 1 --2- 172.21.15.44:0/2413933309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055efcbf0 0x556055f08340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.588+0000 7efeeabb0640 1 --2- 172.21.15.44:0/2413933309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055f0d8b0 0x556055f08fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.588+0000 7efee9bae640 1 -- 172.21.15.44:0/2413933309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055f0d8b0 msgr2=0x556055f08fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.588+0000 7efee9bae640 1 --2- 172.21.15.44:0/2413933309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055f0d8b0 0x556055f08fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.588+0000 7efee9bae640 1 -- 172.21.15.44:0/2413933309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055efcbf0 msgr2=0x556055f08340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.588+0000 7efee9bae640 1 --2- 172.21.15.44:0/2413933309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055efcbf0 0x556055f08340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.588+0000 7efee9bae640 1 -- 172.21.15.44:0/2413933309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556055f0cf30 con 0x556055efd510 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.588+0000 7efee9bae640 1 --2- 172.21.15.44:0/2413933309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055f0dc90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efed800be60 tx=0x7efed80a67d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570182d3ec63ad20 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.588+0000 7efecffff640 1 -- 172.21.15.44:0/2413933309 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed80e3030 con 0x556055efd510 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.588+0000 7efecffff640 1 -- 172.21.15.44:0/2413933309 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed80a9b60 con 0x556055efd510 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.588+0000 7efecffff640 1 -- 172.21.15.44:0/2413933309 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed801e040 con 0x556055efd510 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 -- 172.21.15.44:0/2413933309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 msgr2=0x556055f0dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:38.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 --2- 172.21.15.44:0/2413933309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055f0dc90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efed800be60 tx=0x7efed80a67d0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 -- 172.21.15.44:0/2413933309 shutdown_connections 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 --2- 172.21.15.44:0/2413933309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055f0d8b0 0x556055f08fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 --2- 172.21.15.44:0/2413933309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055f0dc90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 --2- 172.21.15.44:0/2413933309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055efcbf0 0x556055f08340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 -- 172.21.15.44:0/2413933309 >> 172.21.15.44:0/2413933309 conn(0x556055efaa20 msgr2=0x556055f1d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 -- 172.21.15.44:0/2413933309 shutdown_connections 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 -- 172.21.15.44:0/2413933309 wait complete. 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 -- start start 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 Processor -- start 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 0x556055ef94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055ef99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055f0d8b0 0x556055f26fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556055f1fbd0 con 0x556055efd510 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556055f1f500 con 0x556055f0d8b0 2024-08-20T03:01:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556055f202a0 con 0x556055efcbf0 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efeeea0c740 10 client.0 _ll_register_callbacks cb 0x7efedc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.589+0000 7efee9bae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055ef99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efee9bae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055ef99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46698/0 (socket says 172.21.15.44:46698) 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efee9bae640 1 -- 172.21.15.44:0/318288184 learned_addr learned my addr 172.21.15.44:0/318288184 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efeea3af640 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 0x556055ef94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efee9bae640 1 -- 172.21.15.44:0/318288184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 msgr2=0x556055ef94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efee9bae640 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 0x556055ef94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efeeabb0640 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055f0d8b0 0x556055f26fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efee9bae640 1 -- 172.21.15.44:0/318288184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055f0d8b0 msgr2=0x556055f26fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efee9bae640 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055f0d8b0 0x556055f26fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efee9bae640 1 -- 172.21.15.44:0/318288184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556055f275b0 con 0x556055efd510 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efeeabb0640 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055f0d8b0 0x556055f26fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efee9bae640 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055ef99e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efed80a60d0 tx=0x7efed80e6780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea97094f198215b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efecdffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efecdffb640 1 -- 172.21.15.44:0/318288184 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed80e3030 con 0x556055efd510 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efecdffb640 1 -- 172.21.15.44:0/318288184 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efed8010c70 con 0x556055efd510 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efeeea0c740 1 -- 172.21.15.44:0/318288184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed80a9b60 con 0x556055efd510 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efecdffb640 1 -- 172.21.15.44:0/318288184 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efed80aa750 con 0x556055efd510 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efeeea0c740 10 client.15192 fetch_fsmap learned FSMap version 27 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efeeea0c740 1 -- 172.21.15.44:0/318288184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556055f288b0 con 0x556055efd510 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.590+0000 7efecdffb640 1 -- 172.21.15.44:0/318288184 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed80a9b60 con 0x556055efd510 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.591+0000 7efecdffb640 1 -- 172.21.15.44:0/318288184 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efed806f200 con 0x556055efd510 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.591+0000 7efeeea0c740 10 client.15192 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.591+0000 7efeeea0c740 10 client.15192 resolve_mds: resolved name 'f' to daemon mds.f 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.591+0000 7efeeea0c740 20 client.15192 populate_metadata read hostname 'smithi044' 2024-08-20T03:01:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.591+0000 7efeeea0c740 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x556055f1e8e0 0x556055f8bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.591+0000 7efeeea0c740 4 client.15192 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.591+0000 7efeeea0c740 1 -- 172.21.15.44:0/318288184 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "status"}) -- 0x556055e71090 con 0x556055f1e8e0 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.591+0000 7efeea3af640 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x556055f1e8e0 0x556055f8bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.591+0000 7efeea3af640 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x556055f1e8e0 0x556055f8bd60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cdd09b153daf0a8 server_cookie=3e023a0b10596cbb in_seq=0 out_seq=0 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.591+0000 7efecdffb640 10 client.15192 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.591+0000 7efecdffb640 1 -- 172.21.15.44:0/318288184 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556055e71090 con 0x556055f1e8e0 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efecdffb640 10 client.15192 handle_command_reply: tid=0 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeeea0c740 1 client.15192 shutdown 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeeea0c740 1 -- 172.21.15.44:0/318288184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 msgr2=0x556055ef99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeeea0c740 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055ef99e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efed80a60d0 tx=0x7efed80e6780 comp rx=0 tx=0).stop 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeeea0c740 1 -- 172.21.15.44:0/318288184 shutdown_connections 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeeea0c740 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x556055f1e8e0 0x556055f8bd60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeeea0c740 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556055f0d8b0 0x556055f26fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeeea0c740 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556055efd510 0x556055ef99e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeeea0c740 1 --2- 172.21.15.44:0/318288184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556055efcbf0 0x556055ef94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeeea0c740 1 -- 172.21.15.44:0/318288184 >> 172.21.15.44:0/318288184 conn(0x556055efaa20 msgr2=0x556055efba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:38.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efecdffb640 0 client.15192 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:01:38.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeeea0c740 1 -- 172.21.15.44:0/318288184 shutdown_connections 2024-08-20T03:01:38.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeeea0c740 1 -- 172.21.15.44:0/318288184 wait complete. 2024-08-20T03:01:38.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeeea0c740 20 client.15192 trim_cache size 0 max 16384 2024-08-20T03:01:38.601 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:01:38.601 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T03:01:38.601 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 3, 2024-08-20T03:01:38.601 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24583, 2024-08-20T03:01:38.601 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-08-20T03:01:38.602 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-08-20T03:01:38.602 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-08-20T03:01:38.602 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 70.223832071999993, 2024-08-20T03:01:38.602 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2024-08-20T03:01:38.602 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 77, 2024-08-20T03:01:38.602 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 75, 2024-08-20T03:01:38.602 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 107.199460063 2024-08-20T03:01:38.602 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:01:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeec613640 1 -- 172.21.15.44:0/706000294 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efed00b3cb0 msgr2=0x7efed00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeec613640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efed00b3cb0 0x7efed00b6170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efedc009a60 tx=0x7efedc0095c0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeec613640 1 -- 172.21.15.44:0/706000294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee41073e0 msgr2=0x7efee419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeec613640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee41073e0 0x7efee419a490 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efed800b4a0 tx=0x7efed800b960 comp rx=0 tx=0).stop 2024-08-20T03:01:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeec613640 1 -- 172.21.15.44:0/706000294 shutdown_connections 2024-08-20T03:01:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeec613640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efed00b3cb0 0x7efed00b6170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeec613640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efee4107d60 0x7efee419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeec613640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efee41073e0 0x7efee419a490 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeec613640 1 --2- 172.21.15.44:0/706000294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee41019d0 0x7efee4199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.592+0000 7efeec613640 1 -- 172.21.15.44:0/706000294 >> 172.21.15.44:0/706000294 conn(0x7efee40fd440 msgr2=0x7efee40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.593+0000 7efeec613640 1 -- 172.21.15.44:0/706000294 shutdown_connections 2024-08-20T03:01:38.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:38.593+0000 7efeec613640 1 -- 172.21.15.44:0/706000294 wait complete. 2024-08-20T03:01:39.347 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.c status 2024-08-20T03:01:39.599 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:39.988 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:39 smithi044 ceph-mon[18790]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:01:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.067+0000 7fd6983d0640 1 -- 172.21.15.44:0/592743079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6901073e0 msgr2=0x7fd690107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.067+0000 7fd6983d0640 1 --2- 172.21.15.44:0/592743079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6901073e0 0x7fd690107820 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd680009900 tx=0x7fd68002a410 comp rx=0 tx=0).stop 2024-08-20T03:01:40.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.067+0000 7fd6983d0640 1 -- 172.21.15.44:0/592743079 shutdown_connections 2024-08-20T03:01:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.067+0000 7fd6983d0640 1 --2- 172.21.15.44:0/592743079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd690107d60 0x7fd69010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.067+0000 7fd6983d0640 1 --2- 172.21.15.44:0/592743079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6901073e0 0x7fd690107820 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.067+0000 7fd6983d0640 1 --2- 172.21.15.44:0/592743079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6901019d0 0x7fd690101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.067+0000 7fd6983d0640 1 -- 172.21.15.44:0/592743079 >> 172.21.15.44:0/592743079 conn(0x7fd6900fd440 msgr2=0x7fd6900ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.067+0000 7fd6983d0640 1 -- 172.21.15.44:0/592743079 shutdown_connections 2024-08-20T03:01:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.067+0000 7fd6983d0640 1 -- 172.21.15.44:0/592743079 wait complete. 2024-08-20T03:01:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.067+0000 7fd6983d0640 1 Processor -- start 2024-08-20T03:01:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.067+0000 7fd6983d0640 1 -- start start 2024-08-20T03:01:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd6983d0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6901019d0 0x7fd69019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd6983d0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6901073e0 0x7fd69019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd6983d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690107d60 0x7fd6901a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd6983d0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd690111a20 con 0x7fd690107d60 2024-08-20T03:01:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd6983d0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6901118a0 con 0x7fd6901073e0 2024-08-20T03:01:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd6983d0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd690111ba0 con 0x7fd6901019d0 2024-08-20T03:01:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd69696d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690107d60 0x7fd6901a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd69696d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690107d60 0x7fd6901a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46724/0 (socket says 172.21.15.44:46724) 2024-08-20T03:01:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd69696d640 1 -- 172.21.15.44:0/3598023798 learned_addr learned my addr 172.21.15.44:0/3598023798 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd69616c640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6901019d0 0x7fd69019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd69596b640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6901073e0 0x7fd69019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd69696d640 1 -- 172.21.15.44:0/3598023798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6901019d0 msgr2=0x7fd69019b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd69696d640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6901019d0 0x7fd69019b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd69696d640 1 -- 172.21.15.44:0/3598023798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6901073e0 msgr2=0x7fd69019b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:40.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd69696d640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6901073e0 0x7fd69019b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd69696d640 1 -- 172.21.15.44:0/3598023798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6901a12b0 con 0x7fd690107d60 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd69616c640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6901019d0 0x7fd69019b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.068+0000 7fd69696d640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690107d60 0x7fd6901a0ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd68800cad0 tx=0x7fd68800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.069+0000 7fd6877fe640 1 -- 172.21.15.44:0/3598023798 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68800ae20 con 0x7fd690107d60 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.069+0000 7fd6877fe640 1 -- 172.21.15.44:0/3598023798 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd688005020 con 0x7fd690107d60 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.069+0000 7fd6877fe640 1 -- 172.21.15.44:0/3598023798 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68800f6b0 con 0x7fd690107d60 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.069+0000 7fd6983d0640 1 -- 172.21.15.44:0/3598023798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6901a15a0 con 0x7fd690107d60 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.069+0000 7fd6983d0640 1 -- 172.21.15.44:0/3598023798 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6901a7c60 con 0x7fd690107d60 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.071+0000 7fd69a7c9740 1 Processor -- start 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.072+0000 7fd6877fe640 1 -- 172.21.15.44:0/3598023798 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fd6880051d0 con 0x7fd690107d60 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.072+0000 7fd69a7c9740 1 -- start start 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.072+0000 7fd69a7c9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 0x55d84fa59fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.072+0000 7fd69a7c9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa5a510 0x55d84fa65f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.072+0000 7fd69a7c9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa664a0 0x55d84fa68890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.072+0000 7fd69a7c9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d84f9d06a0 con 0x55d84fa59bf0 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.072+0000 7fd69a7c9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d84f9bfba0 con 0x55d84fa664a0 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.072+0000 7fd69a7c9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d84f9c3be0 con 0x55d84fa5a510 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.072+0000 7fd69596b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 0x55d84fa59fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.072+0000 7fd69596b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 0x55d84fa59fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46730/0 (socket says 172.21.15.44:46730) 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.072+0000 7fd69596b640 1 -- 172.21.15.44:0/1011670291 learned_addr learned my addr 172.21.15.44:0/1011670291 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.072+0000 7fd69616c640 1 --2- 172.21.15.44:0/1011670291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa5a510 0x55d84fa65f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd69596b640 1 --2- 172.21.15.44:0/1011670291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa664a0 0x55d84fa68890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd69596b640 1 -- 172.21.15.44:0/1011670291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa5a510 msgr2=0x55d84fa65f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd69596b640 1 --2- 172.21.15.44:0/1011670291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa5a510 0x55d84fa65f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd69596b640 1 -- 172.21.15.44:0/1011670291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa664a0 msgr2=0x55d84fa68890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd69596b640 1 --2- 172.21.15.44:0/1011670291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa664a0 0x55d84fa68890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd69596b640 1 -- 172.21.15.44:0/1011670291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d84f71f540 con 0x55d84fa59bf0 2024-08-20T03:01:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd6877fe640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd67c0b3cb0 0x7fd67c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd69596b640 1 --2- 172.21.15.44:0/1011670291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 0x55d84fa59fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd68002a930 tx=0x7fd6800058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b751a831c6cbb65 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd69616c640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd67c0b3cb0 0x7fd67c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd6877fe640 1 -- 172.21.15.44:0/3598023798 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd6880e3fe0 con 0x7fd690107d60 2024-08-20T03:01:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd684de6640 1 -- 172.21.15.44:0/1011670291 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680043020 con 0x55d84fa59bf0 2024-08-20T03:01:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd684de6640 1 -- 172.21.15.44:0/1011670291 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd680031870 con 0x55d84fa59bf0 2024-08-20T03:01:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.073+0000 7fd684de6640 1 -- 172.21.15.44:0/1011670291 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680035050 con 0x55d84fa59bf0 2024-08-20T03:01:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.074+0000 7fd69a7c9740 1 -- 172.21.15.44:0/1011670291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 msgr2=0x55d84fa59fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.074+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/1011670291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 0x55d84fa59fd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd68002a930 tx=0x7fd6800058b0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.074+0000 7fd69616c640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd67c0b3cb0 0x7fd67c0b6170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55d84fa5cd70 tx=0x7fd68c008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.074+0000 7fd69a7c9740 1 -- 172.21.15.44:0/1011670291 shutdown_connections 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.074+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/1011670291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa664a0 0x55d84fa68890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.074+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/1011670291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa5a510 0x55d84fa65f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.074+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/1011670291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 0x55d84fa59fd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.074+0000 7fd69a7c9740 1 -- 172.21.15.44:0/1011670291 >> 172.21.15.44:0/1011670291 conn(0x55d84fa57a20 msgr2=0x55d84fa58bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.074+0000 7fd69a7c9740 1 -- 172.21.15.44:0/1011670291 shutdown_connections 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.074+0000 7fd69a7c9740 1 -- 172.21.15.44:0/1011670291 wait complete. 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.074+0000 7fd69a7c9740 1 -- start start 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69a7c9740 1 Processor -- start 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69a7c9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 0x55d84fa634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69a7c9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa5a510 0x55d84fa639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69a7c9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa664a0 0x55d84fa63f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69a7c9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d84f8225f0 con 0x55d84fa59bf0 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69a7c9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d84f9466c0 con 0x55d84fa664a0 2024-08-20T03:01:40.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69a7c9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d84fa690f0 con 0x55d84fa5a510 2024-08-20T03:01:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69a7c9740 10 client.0 _ll_register_callbacks cb 0x55d84f9831e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:01:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69616c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 0x55d84fa634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69616c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 0x55d84fa634a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46734/0 (socket says 172.21.15.44:46734) 2024-08-20T03:01:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69616c640 1 -- 172.21.15.44:0/876541437 learned_addr learned my addr 172.21.15.44:0/876541437 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69596b640 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa5a510 0x55d84fa639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69616c640 1 -- 172.21.15.44:0/876541437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa5a510 msgr2=0x55d84fa639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69616c640 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa5a510 0x55d84fa639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69696d640 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa664a0 0x55d84fa63f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69616c640 1 -- 172.21.15.44:0/876541437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa664a0 msgr2=0x55d84fa63f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69616c640 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa664a0 0x55d84fa63f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69616c640 1 -- 172.21.15.44:0/876541437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d84fa72550 con 0x55d84fa59bf0 2024-08-20T03:01:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69696d640 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa664a0 0x55d84fa63f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd69616c640 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 0x55d84fa634a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd68c007450 tx=0x7fd68c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c995c6ffebcf20 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.075+0000 7fd679ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd679ffb640 1 -- 172.21.15.44:0/876541437 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c01c050 con 0x55d84fa59bf0 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd679ffb640 1 -- 172.21.15.44:0/876541437 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd68c014b40 con 0x55d84fa59bf0 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd69a7c9740 1 -- 172.21.15.44:0/876541437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd680031870 con 0x55d84fa59bf0 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd679ffb640 1 -- 172.21.15.44:0/876541437 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd68c014d40 con 0x55d84fa59bf0 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd69a7c9740 10 client.15204 fetch_fsmap learned FSMap version 27 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd69a7c9740 1 -- 172.21.15.44:0/876541437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d84fa74f40 con 0x55d84fa59bf0 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd679ffb640 1 -- 172.21.15.44:0/876541437 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd680031870 con 0x55d84fa59bf0 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd679ffb640 1 -- 172.21.15.44:0/876541437 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd68c0064d0 con 0x55d84fa59bf0 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd69a7c9740 10 client.15204 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd69a7c9740 10 client.15204 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd69a7c9740 20 client.15204 populate_metadata read hostname 'smithi044' 2024-08-20T03:01:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d84fa77600 0x55d84fa78a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd69a7c9740 4 client.15204 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:01:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.076+0000 7fd69a7c9740 1 -- 172.21.15.44:0/876541437 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d84f9d25f0 con 0x55d84fa77600 2024-08-20T03:01:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.077+0000 7fd69596b640 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d84fa77600 0x55d84fa78a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.077+0000 7fd69596b640 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d84fa77600 0x55d84fa78a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e3f0efc62e6e159 server_cookie=be8a0557b4a6715 in_seq=0 out_seq=0 2024-08-20T03:01:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.077+0000 7fd679ffb640 10 client.15204 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:01:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd679ffb640 1 -- 172.21.15.44:0/876541437 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d84f9d25f0 con 0x55d84fa77600 2024-08-20T03:01:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd679ffb640 10 client.15204 handle_command_reply: tid=0 2024-08-20T03:01:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd69a7c9740 1 client.15204 shutdown 2024-08-20T03:01:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd69a7c9740 1 -- 172.21.15.44:0/876541437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 msgr2=0x55d84fa634a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 0x55d84fa634a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd68c007450 tx=0x7fd68c0046c0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd69a7c9740 1 -- 172.21.15.44:0/876541437 shutdown_connections 2024-08-20T03:01:40.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa664a0 0x55d84fa63f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d84fa77600 0x55d84fa78a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa5a510 0x55d84fa639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/876541437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa59bf0 0x55d84fa634a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd679ffb640 0 client.15204 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:01:40.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd69a7c9740 1 -- 172.21.15.44:0/876541437 >> 172.21.15.44:0/876541437 conn(0x55d84fa57a20 msgr2=0x55d84fa5d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:40.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd69a7c9740 1 -- 172.21.15.44:0/876541437 shutdown_connections 2024-08-20T03:01:40.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd69a7c9740 1 -- 172.21.15.44:0/876541437 wait complete. 2024-08-20T03:01:40.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.078+0000 7fd69a7c9740 20 client.15204 trim_cache size 0 max 16384 2024-08-20T03:01:40.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.096+0000 7fd69a7c9740 1 Processor -- start 2024-08-20T03:01:40.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.096+0000 7fd69a7c9740 1 -- start start 2024-08-20T03:01:40.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69a7c9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa59bf0 0x55d84fa5ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69a7c9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 0x55d84fa61d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69a7c9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa664a0 0x55d84fa62260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69a7c9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d84fa76db0 con 0x55d84fa5a510 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69a7c9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d84fa77480 con 0x55d84fa59bf0 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69a7c9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d84fa77b50 con 0x55d84fa664a0 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69596b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 0x55d84fa61d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69596b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 0x55d84fa61d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46736/0 (socket says 172.21.15.44:46736) 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69596b640 1 -- 172.21.15.44:0/3553798661 learned_addr learned my addr 172.21.15.44:0/3553798661 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69616c640 1 --2- 172.21.15.44:0/3553798661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa59bf0 0x55d84fa5ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69596b640 1 -- 172.21.15.44:0/3553798661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa664a0 msgr2=0x55d84fa62260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69596b640 1 --2- 172.21.15.44:0/3553798661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa664a0 0x55d84fa62260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69596b640 1 -- 172.21.15.44:0/3553798661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa59bf0 msgr2=0x55d84fa5ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69596b640 1 --2- 172.21.15.44:0/3553798661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa59bf0 0x55d84fa5ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69596b640 1 -- 172.21.15.44:0/3553798661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d84fa57850 con 0x55d84fa5a510 2024-08-20T03:01:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd69596b640 1 --2- 172.21.15.44:0/3553798661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 0x55d84fa61d20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd680005ab0 tx=0x7fd6800054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5216bc735d3a947 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.097+0000 7fd67bfff640 1 -- 172.21.15.44:0/3553798661 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680035040 con 0x55d84fa5a510 2024-08-20T03:01:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.098+0000 7fd67bfff640 1 -- 172.21.15.44:0/3553798661 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd68003a930 con 0x55d84fa5a510 2024-08-20T03:01:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.098+0000 7fd67bfff640 1 -- 172.21.15.44:0/3553798661 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680043040 con 0x55d84fa5a510 2024-08-20T03:01:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.098+0000 7fd69a7c9740 1 -- 172.21.15.44:0/3553798661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 msgr2=0x55d84fa61d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.098+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/3553798661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 0x55d84fa61d20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd680005ab0 tx=0x7fd6800054b0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.098+0000 7fd69a7c9740 1 -- 172.21.15.44:0/3553798661 shutdown_connections 2024-08-20T03:01:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.098+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/3553798661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa664a0 0x55d84fa62260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.098+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/3553798661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 0x55d84fa61d20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.098+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/3553798661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa59bf0 0x55d84fa5ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.098+0000 7fd69a7c9740 1 -- 172.21.15.44:0/3553798661 >> 172.21.15.44:0/3553798661 conn(0x55d84fa57a20 msgr2=0x55d84fa5e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:40.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.098+0000 7fd69a7c9740 1 -- 172.21.15.44:0/3553798661 shutdown_connections 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.098+0000 7fd69a7c9740 1 -- 172.21.15.44:0/3553798661 wait complete. 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.098+0000 7fd69a7c9740 1 -- start start 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69a7c9740 1 Processor -- start 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69a7c9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa59bf0 0x55d84fa7a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69a7c9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 0x55d84fa7a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69a7c9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa664a0 0x55d84fa7f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69a7c9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d84fa788f0 con 0x55d84fa5a510 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69a7c9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d84fa78220 con 0x55d84fa59bf0 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69a7c9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d84fa78fc0 con 0x55d84fa664a0 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69a7c9740 10 client.0 _ll_register_callbacks cb 0x7fd68c0077c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69596b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 0x55d84fa7a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69596b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 0x55d84fa7a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46738/0 (socket says 172.21.15.44:46738) 2024-08-20T03:01:40.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69596b640 1 -- 172.21.15.44:0/368288097 learned_addr learned my addr 172.21.15.44:0/368288097 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69616c640 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa59bf0 0x55d84fa7a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69696d640 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa664a0 0x55d84fa7f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69596b640 1 -- 172.21.15.44:0/368288097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa664a0 msgr2=0x55d84fa7f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69596b640 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa664a0 0x55d84fa7f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69596b640 1 -- 172.21.15.44:0/368288097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa59bf0 msgr2=0x55d84fa7a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69596b640 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa59bf0 0x55d84fa7a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69596b640 1 -- 172.21.15.44:0/368288097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d84fa7fe30 con 0x55d84fa5a510 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd69596b640 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 0x55d84fa7a800 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd6800059d0 tx=0x7fd680005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674cab7217e862c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.099+0000 7fd679ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd679ffb640 1 -- 172.21.15.44:0/368288097 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680043070 con 0x55d84fa5a510 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd69a7c9740 1 -- 172.21.15.44:0/368288097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68003a930 con 0x55d84fa5a510 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd679ffb640 1 -- 172.21.15.44:0/368288097 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd68003a680 con 0x55d84fa5a510 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd679ffb640 1 -- 172.21.15.44:0/368288097 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd680035030 con 0x55d84fa5a510 2024-08-20T03:01:40.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd69a7c9740 10 client.15210 fetch_fsmap learned FSMap version 27 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd69a7c9740 1 -- 172.21.15.44:0/368288097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d84fa81290 con 0x55d84fa5a510 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd679ffb640 1 -- 172.21.15.44:0/368288097 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd68003a930 con 0x55d84fa5a510 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd679ffb640 1 -- 172.21.15.44:0/368288097 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd680003680 con 0x55d84fa5a510 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd69a7c9740 10 client.15210 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd69a7c9740 10 client.15210 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd69a7c9740 20 client.15210 populate_metadata read hostname 'smithi044' 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d84fae5930 0x55d84fae4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd69a7c9740 4 client.15210 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.100+0000 7fd69a7c9740 1 -- 172.21.15.44:0/368288097 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "status"}) -- 0x55d84f9ce090 con 0x55d84fae5930 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.101+0000 7fd69616c640 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d84fae5930 0x55d84fae4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.101+0000 7fd69616c640 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d84fae5930 0x55d84fae4640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7b65c908c18109d server_cookie=4a633fce4a582e72 in_seq=0 out_seq=0 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.101+0000 7fd679ffb640 10 client.15210 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:01:40.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.101+0000 7fd679ffb640 1 -- 172.21.15.44:0/368288097 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d84f9ce090 con 0x55d84fae5930 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.101+0000 7fd679ffb640 10 client.15210 handle_command_reply: tid=0 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.101+0000 7fd69a7c9740 1 client.15210 shutdown 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.101+0000 7fd69a7c9740 1 -- 172.21.15.44:0/368288097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 msgr2=0x55d84fa7a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.101+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 0x55d84fa7a800 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd6800059d0 tx=0x7fd680005d50 comp rx=0 tx=0).stop 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd69a7c9740 1 -- 172.21.15.44:0/368288097 shutdown_connections 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d84fae5930 0x55d84fae4640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d84fa664a0 0x55d84fa7f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d84fa5a510 0x55d84fa7a800 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd679ffb640 0 client.15210 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd69a7c9740 1 --2- 172.21.15.44:0/368288097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d84fa59bf0 0x55d84fa7a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd69a7c9740 1 -- 172.21.15.44:0/368288097 >> 172.21.15.44:0/368288097 conn(0x55d84fa57a20 msgr2=0x55d84fa61c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd69a7c9740 1 -- 172.21.15.44:0/368288097 shutdown_connections 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd69a7c9740 1 -- 172.21.15.44:0/368288097 wait complete. 2024-08-20T03:01:40.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd69a7c9740 20 client.15210 trim_cache size 0 max 16384 2024-08-20T03:01:40.118 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:01:40.118 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T03:01:40.118 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 2, 2024-08-20T03:01:40.118 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24589, 2024-08-20T03:01:40.118 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-08-20T03:01:40.118 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-08-20T03:01:40.118 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-08-20T03:01:40.118 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 72.740446386000002, 2024-08-20T03:01:40.118 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2024-08-20T03:01:40.119 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 77, 2024-08-20T03:01:40.119 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 75, 2024-08-20T03:01:40.119 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 106.121380266 2024-08-20T03:01:40.119 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:01:40.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd6983d0640 1 -- 172.21.15.44:0/3598023798 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd67c0b3cb0 msgr2=0x7fd67c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:40.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd6983d0640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd67c0b3cb0 0x7fd67c0b6170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55d84fa5cd70 tx=0x7fd68c008040 comp rx=0 tx=0).stop 2024-08-20T03:01:40.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd6983d0640 1 -- 172.21.15.44:0/3598023798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690107d60 msgr2=0x7fd6901a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:40.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd6983d0640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690107d60 0x7fd6901a0ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd68800cad0 tx=0x7fd68800cf90 comp rx=0 tx=0).stop 2024-08-20T03:01:40.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd6983d0640 1 -- 172.21.15.44:0/3598023798 shutdown_connections 2024-08-20T03:01:40.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd6983d0640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd67c0b3cb0 0x7fd67c0b6170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd6983d0640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd690107d60 0x7fd6901a0ae0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd6983d0640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6901073e0 0x7fd69019b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd6983d0640 1 --2- 172.21.15.44:0/3598023798 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6901019d0 0x7fd69019b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:40.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd6983d0640 1 -- 172.21.15.44:0/3598023798 >> 172.21.15.44:0/3598023798 conn(0x7fd6900fd440 msgr2=0x7fd6901049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:40.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd6983d0640 1 -- 172.21.15.44:0/3598023798 shutdown_connections 2024-08-20T03:01:40.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:40.102+0000 7fd6983d0640 1 -- 172.21.15.44:0/3598023798 wait complete. 2024-08-20T03:01:40.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:39 smithi165 ceph-mon[22357]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:01:40.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:39 smithi099 ceph-mon[21984]: pgmap v295: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:01:40.811 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-20T03:01:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:41.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.537+0000 7fb19e618640 1 -- 172.21.15.44:0/305646870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb198107d20 msgr2=0x7fb19810e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:41.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.537+0000 7fb19e618640 1 --2- 172.21.15.44:0/305646870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb198107d20 0x7fb19810e520 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb190009ca0 tx=0x7fb19002a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.537+0000 7fb19e618640 1 -- 172.21.15.44:0/305646870 shutdown_connections 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.537+0000 7fb19e618640 1 --2- 172.21.15.44:0/305646870 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb198107d20 0x7fb19810e520 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.537+0000 7fb19e618640 1 --2- 172.21.15.44:0/305646870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1981073a0 0x7fb1981077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.537+0000 7fb19e618640 1 --2- 172.21.15.44:0/305646870 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb198101990 0x7fb198101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.537+0000 7fb19e618640 1 -- 172.21.15.44:0/305646870 >> 172.21.15.44:0/305646870 conn(0x7fb1980fd400 msgr2=0x7fb1980ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.537+0000 7fb19e618640 1 -- 172.21.15.44:0/305646870 shutdown_connections 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.537+0000 7fb19e618640 1 -- 172.21.15.44:0/305646870 wait complete. 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.537+0000 7fb19e618640 1 Processor -- start 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19e618640 1 -- start start 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19e618640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb198101990 0x7fb198198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19e618640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1981073a0 0x7fb198199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19e618640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb198107d20 0x7fb19819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19e618640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb1981119a0 con 0x7fb198107d20 2024-08-20T03:01:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19e618640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb198111820 con 0x7fb198101990 2024-08-20T03:01:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19e618640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb198111b20 con 0x7fb1981073a0 2024-08-20T03:01:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19cbb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb198107d20 0x7fb19819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19cbb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb198107d20 0x7fb19819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46756/0 (socket says 172.21.15.44:46756) 2024-08-20T03:01:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19cbb5640 1 -- 172.21.15.44:0/275307875 learned_addr learned my addr 172.21.15.44:0/275307875 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb1977fe640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1981073a0 0x7fb198199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb197fff640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb198101990 0x7fb198198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19cbb5640 1 -- 172.21.15.44:0/275307875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1981073a0 msgr2=0x7fb198199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19cbb5640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1981073a0 0x7fb198199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19cbb5640 1 -- 172.21.15.44:0/275307875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb198101990 msgr2=0x7fb198198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:41.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19cbb5640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb198101990 0x7fb198198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.538+0000 7fb19cbb5640 1 -- 172.21.15.44:0/275307875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19819ef90 con 0x7fb198107d20 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.539+0000 7fb19cbb5640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb198107d20 0x7fb19819e7c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb190005cc0 tx=0x7fb19002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.539+0000 7fb1957fa640 1 -- 172.21.15.44:0/275307875 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190003bf0 con 0x7fb198107d20 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.539+0000 7fb1957fa640 1 -- 172.21.15.44:0/275307875 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb190003d90 con 0x7fb198107d20 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.539+0000 7fb19e618640 1 -- 172.21.15.44:0/275307875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19819f220 con 0x7fb198107d20 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.539+0000 7fb1957fa640 1 -- 172.21.15.44:0/275307875 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19002fb20 con 0x7fb198107d20 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.539+0000 7fb19e618640 1 -- 172.21.15.44:0/275307875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1981a5940 con 0x7fb198107d20 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.541+0000 7fb1957fa640 1 -- 172.21.15.44:0/275307875 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb190004580 con 0x7fb198107d20 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.541+0000 7fb1a0a11740 1 Processor -- start 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.541+0000 7fb1a0a11740 1 -- start start 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1a0a11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628facbcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1a0a11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628facc8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1a0a11740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facc94a0 0x5628faccb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1a0a11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5628fac336a0 con 0x5628facbcbf0 2024-08-20T03:01:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1a0a11740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5628fac22ba0 con 0x5628facc94a0 2024-08-20T03:01:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1a0a11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5628fac26be0 con 0x5628facbd510 2024-08-20T03:01:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1977fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628facbcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1977fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628facbcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46770/0 (socket says 172.21.15.44:46770) 2024-08-20T03:01:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1977fe640 1 -- 172.21.15.44:0/1408736445 learned_addr learned my addr 172.21.15.44:0/1408736445 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1977fe640 1 --2- 172.21.15.44:0/1408736445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facc94a0 0x5628faccb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb197fff640 1 --2- 172.21.15.44:0/1408736445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628facc8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1977fe640 1 -- 172.21.15.44:0/1408736445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 msgr2=0x5628facc8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1977fe640 1 --2- 172.21.15.44:0/1408736445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628facc8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1977fe640 1 -- 172.21.15.44:0/1408736445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facc94a0 msgr2=0x5628faccb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:41.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1977fe640 1 --2- 172.21.15.44:0/1408736445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facc94a0 0x5628faccb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1977fe640 1 -- 172.21.15.44:0/1408736445 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628fa982540 con 0x5628facbcbf0 2024-08-20T03:01:41.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb197fff640 1 --2- 172.21.15.44:0/1408736445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628facc8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:41.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1957fa640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb1840b3cb0 0x7fb1840b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1977fe640 1 --2- 172.21.15.44:0/1408736445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628facbcfd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb18000bfd0 tx=0x7fb18000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882e575614189817 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:41.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb197fff640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb1840b3cb0 0x7fb1840b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb1957fa640 1 -- 172.21.15.44:0/275307875 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb190105250 con 0x7fb198107d20 2024-08-20T03:01:41.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.542+0000 7fb18e7fc640 1 -- 172.21.15.44:0/1408736445 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180004030 con 0x5628facbcbf0 2024-08-20T03:01:41.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.543+0000 7fb18e7fc640 1 -- 172.21.15.44:0/1408736445 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1800041d0 con 0x5628facbcbf0 2024-08-20T03:01:41.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.543+0000 7fb18e7fc640 1 -- 172.21.15.44:0/1408736445 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180010460 con 0x5628facbcbf0 2024-08-20T03:01:41.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.543+0000 7fb1a0a11740 1 -- 172.21.15.44:0/1408736445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 msgr2=0x5628facbcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:41.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.543+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/1408736445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628facbcfd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb18000bfd0 tx=0x7fb18000b840 comp rx=0 tx=0).stop 2024-08-20T03:01:41.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.543+0000 7fb197fff640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb1840b3cb0 0x7fb1840b6170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x5628facbfd70 tx=0x7fb18800a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.543+0000 7fb1a0a11740 1 -- 172.21.15.44:0/1408736445 shutdown_connections 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.543+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/1408736445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facc94a0 0x5628faccb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.543+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/1408736445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628facc8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.543+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/1408736445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628facbcfd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.543+0000 7fb1a0a11740 1 -- 172.21.15.44:0/1408736445 >> 172.21.15.44:0/1408736445 conn(0x5628facbaa20 msgr2=0x5628facbbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.543+0000 7fb1a0a11740 1 -- 172.21.15.44:0/1408736445 shutdown_connections 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.543+0000 7fb1a0a11740 1 -- 172.21.15.44:0/1408736445 wait complete. 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb1a0a11740 1 -- start start 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb1a0a11740 1 Processor -- start 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb1a0a11740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facbcbf0 0x5628facc6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb1a0a11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628facc6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb1a0a11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facc94a0 0x5628facc6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb1a0a11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5628faa855f0 con 0x5628facc94a0 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb1a0a11740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5628faba96c0 con 0x5628facbcbf0 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb1a0a11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5628faccc150 con 0x5628facbd510 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb1a0a11740 10 client.0 _ll_register_callbacks cb 0x5628fabe61e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb19cbb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facc94a0 0x5628facc6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb19cbb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facc94a0 0x5628facc6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46784/0 (socket says 172.21.15.44:46784) 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb19cbb5640 1 -- 172.21.15.44:0/2102291563 learned_addr learned my addr 172.21.15.44:0/2102291563 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.544+0000 7fb1977fe640 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628facc6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb197fff640 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facbcbf0 0x5628facc6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb19cbb5640 1 -- 172.21.15.44:0/2102291563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 msgr2=0x5628facc6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb19cbb5640 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628facc6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb19cbb5640 1 -- 172.21.15.44:0/2102291563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facbcbf0 msgr2=0x5628facc6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb19cbb5640 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facbcbf0 0x5628facc6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb19cbb5640 1 -- 172.21.15.44:0/2102291563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628facd5590 con 0x5628facc94a0 2024-08-20T03:01:41.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb19cbb5640 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facc94a0 0x5628facc6ed0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb1900cc030 tx=0x7fb190102fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d79909cda24d654 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:41.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb1777fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:01:41.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb1777fe640 1 -- 172.21.15.44:0/2102291563 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1900e6060 con 0x5628facc94a0 2024-08-20T03:01:41.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb1777fe640 1 -- 172.21.15.44:0/2102291563 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb19002fcc0 con 0x5628facc94a0 2024-08-20T03:01:41.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb1a0a11740 1 -- 172.21.15.44:0/2102291563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1800041d0 con 0x5628facc94a0 2024-08-20T03:01:41.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb1777fe640 1 -- 172.21.15.44:0/2102291563 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb19002fec0 con 0x5628facc94a0 2024-08-20T03:01:41.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb1a0a11740 10 client.15222 fetch_fsmap learned FSMap version 27 2024-08-20T03:01:41.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb1a0a11740 1 -- 172.21.15.44:0/2102291563 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5628facd7f20 con 0x5628facc94a0 2024-08-20T03:01:41.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb1777fe640 1 -- 172.21.15.44:0/2102291563 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1800041d0 con 0x5628facc94a0 2024-08-20T03:01:41.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.545+0000 7fb1777fe640 1 -- 172.21.15.44:0/2102291563 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb1900cc300 con 0x5628facc94a0 2024-08-20T03:01:41.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.546+0000 7fb1a0a11740 10 client.15222 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:01:41.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.546+0000 7fb1a0a11740 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T03:01:41.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.546+0000 7fb1a0a11740 20 client.15222 populate_metadata read hostname 'smithi044' 2024-08-20T03:01:41.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.546+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5628facda550 0x5628facdb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.546+0000 7fb1a0a11740 4 client.15222 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:01:41.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.546+0000 7fb1a0a11740 1 -- 172.21.15.44:0/2102291563 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628fac355f0 con 0x5628facda550 2024-08-20T03:01:41.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.546+0000 7fb197fff640 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5628facda550 0x5628facdb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.546+0000 7fb1777fe640 10 client.15222 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:01:41.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.546+0000 7fb197fff640 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5628facda550 0x5628facdb9c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=afd8debc4ead998e server_cookie=bd14479b0883d2f1 in_seq=0 out_seq=0 2024-08-20T03:01:41.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.547+0000 7fb1777fe640 1 -- 172.21.15.44:0/2102291563 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628fac355f0 con 0x5628facda550 2024-08-20T03:01:41.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.547+0000 7fb1777fe640 10 client.15222 handle_command_reply: tid=0 2024-08-20T03:01:41.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.547+0000 7fb1a0a11740 1 client.15222 shutdown 2024-08-20T03:01:41.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.547+0000 7fb1a0a11740 1 -- 172.21.15.44:0/2102291563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facc94a0 msgr2=0x5628facc6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:41.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.547+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facc94a0 0x5628facc6ed0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb1900cc030 tx=0x7fb190102fd0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.547+0000 7fb1a0a11740 1 -- 172.21.15.44:0/2102291563 shutdown_connections 2024-08-20T03:01:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.547+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5628facda550 0x5628facdb9c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.547+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facc94a0 0x5628facc6ed0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.547+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628facc6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.547+0000 7fb1777fe640 0 client.15222 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:01:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.547+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/2102291563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facbcbf0 0x5628facc6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.547+0000 7fb1a0a11740 1 -- 172.21.15.44:0/2102291563 >> 172.21.15.44:0/2102291563 conn(0x5628facbaa20 msgr2=0x5628facc05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.548+0000 7fb1a0a11740 1 -- 172.21.15.44:0/2102291563 shutdown_connections 2024-08-20T03:01:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.548+0000 7fb1a0a11740 1 -- 172.21.15.44:0/2102291563 wait complete. 2024-08-20T03:01:41.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.548+0000 7fb1a0a11740 20 client.15222 trim_cache size 0 max 16384 2024-08-20T03:01:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb1a0a11740 1 Processor -- start 2024-08-20T03:01:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb1a0a11740 1 -- start start 2024-08-20T03:01:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb1a0a11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628facc1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb1a0a11740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facbd510 0x5628facc1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb1a0a11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facc94a0 0x5628facc4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb1a0a11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5628facd9d00 con 0x5628facbcbf0 2024-08-20T03:01:41.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb1a0a11740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5628facda3d0 con 0x5628facbd510 2024-08-20T03:01:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb1a0a11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5628facdaaa0 con 0x5628facc94a0 2024-08-20T03:01:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb197fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628facc1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb197fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628facc1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46786/0 (socket says 172.21.15.44:46786) 2024-08-20T03:01:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb197fff640 1 -- 172.21.15.44:0/908331613 learned_addr learned my addr 172.21.15.44:0/908331613 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb197fff640 1 -- 172.21.15.44:0/908331613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facc94a0 msgr2=0x5628facc4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb197fff640 1 --2- 172.21.15.44:0/908331613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facc94a0 0x5628facc4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb197fff640 1 -- 172.21.15.44:0/908331613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facbd510 msgr2=0x5628facc1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb197fff640 1 --2- 172.21.15.44:0/908331613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facbd510 0x5628facc1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb197fff640 1 -- 172.21.15.44:0/908331613 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628facc9000 con 0x5628facbcbf0 2024-08-20T03:01:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb197fff640 1 --2- 172.21.15.44:0/908331613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628facc1700 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb188012cb0 tx=0x7fb188018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d94a4de5d574ce0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.566+0000 7fb18dffb640 1 -- 172.21.15.44:0/908331613 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18802b030 con 0x5628facbcbf0 2024-08-20T03:01:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.567+0000 7fb18dffb640 1 -- 172.21.15.44:0/908331613 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb18801fdc0 con 0x5628facbcbf0 2024-08-20T03:01:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.567+0000 7fb18dffb640 1 -- 172.21.15.44:0/908331613 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1880106a0 con 0x5628facbcbf0 2024-08-20T03:01:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.567+0000 7fb1a0a11740 1 -- 172.21.15.44:0/908331613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 msgr2=0x5628facc1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.567+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/908331613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628facc1700 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb188012cb0 tx=0x7fb188018000 comp rx=0 tx=0).stop 2024-08-20T03:01:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.567+0000 7fb1a0a11740 1 -- 172.21.15.44:0/908331613 shutdown_connections 2024-08-20T03:01:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.567+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/908331613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facc94a0 0x5628facc4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.567+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/908331613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facbd510 0x5628facc1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.567+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/908331613 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628facc1700 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.567+0000 7fb1a0a11740 1 -- 172.21.15.44:0/908331613 >> 172.21.15.44:0/908331613 conn(0x5628facbaa20 msgr2=0x5628facbbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.567+0000 7fb1a0a11740 1 -- 172.21.15.44:0/908331613 shutdown_connections 2024-08-20T03:01:41.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.567+0000 7fb1a0a11740 1 -- 172.21.15.44:0/908331613 wait complete. 2024-08-20T03:01:41.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb1a0a11740 1 -- start start 2024-08-20T03:01:41.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb1a0a11740 1 Processor -- start 2024-08-20T03:01:41.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb1a0a11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628face2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb1a0a11740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628face3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb1a0a11740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facc94a0 0x5628facde710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb1a0a11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5628facdb840 con 0x5628facbcbf0 2024-08-20T03:01:41.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb1a0a11740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5628facdb170 con 0x5628facc94a0 2024-08-20T03:01:41.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb1a0a11740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5628facdbf10 con 0x5628facbd510 2024-08-20T03:01:41.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb1a0a11740 10 client.0 _ll_register_callbacks cb 0x7fb190004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:01:41.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb197fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628face2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb197fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628face2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46798/0 (socket says 172.21.15.44:46798) 2024-08-20T03:01:41.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb197fff640 1 -- 172.21.15.44:0/3413615512 learned_addr learned my addr 172.21.15.44:0/3413615512 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:41.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb19cbb5640 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facc94a0 0x5628facde710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb197fff640 1 -- 172.21.15.44:0/3413615512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 msgr2=0x5628face3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:41.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb1977fe640 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628face3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb197fff640 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628face3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb197fff640 1 -- 172.21.15.44:0/3413615512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facc94a0 msgr2=0x5628facde710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:41.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb197fff640 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facc94a0 0x5628facde710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb197fff640 1 -- 172.21.15.44:0/3413615512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628facdee50 con 0x5628facbcbf0 2024-08-20T03:01:41.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.568+0000 7fb19cbb5640 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facc94a0 0x5628facde710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:41.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1977fe640 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628face3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:41.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb197fff640 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628face2ba0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb188027c20 tx=0x7fb1880053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854da7724524113b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:41.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1777fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:01:41.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1777fe640 1 -- 172.21.15.44:0/3413615512 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb188034030 con 0x5628facbcbf0 2024-08-20T03:01:41.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1777fe640 1 -- 172.21.15.44:0/3413615512 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb1880272c0 con 0x5628facbcbf0 2024-08-20T03:01:41.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1a0a11740 1 -- 172.21.15.44:0/3413615512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb18801fdc0 con 0x5628facbcbf0 2024-08-20T03:01:41.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1777fe640 1 -- 172.21.15.44:0/3413615512 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb18802b070 con 0x5628facbcbf0 2024-08-20T03:01:41.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1a0a11740 10 client.15228 fetch_fsmap learned FSMap version 27 2024-08-20T03:01:41.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1a0a11740 1 -- 172.21.15.44:0/3413615512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5628face2290 con 0x5628facbcbf0 2024-08-20T03:01:41.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1777fe640 1 -- 172.21.15.44:0/3413615512 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb18801fdc0 con 0x5628facbcbf0 2024-08-20T03:01:41.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1777fe640 1 -- 172.21.15.44:0/3413615512 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb18803b4d0 con 0x5628facbcbf0 2024-08-20T03:01:41.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1a0a11740 10 client.15228 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:01:41.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1a0a11740 10 client.15228 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T03:01:41.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1a0a11740 20 client.15228 populate_metadata read hostname 'smithi044' 2024-08-20T03:01:41.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5628fad48f30 0x5628fad47c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:41.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1a0a11740 4 client.15228 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T03:01:41.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.569+0000 7fb1a0a11740 1 -- 172.21.15.44:0/3413615512 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "status"}) -- 0x5628fac31090 con 0x5628fad48f30 2024-08-20T03:01:41.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.570+0000 7fb1977fe640 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5628fad48f30 0x5628fad47c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:41.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.570+0000 7fb1977fe640 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5628fad48f30 0x5628fad47c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e80e2f5256650f1 server_cookie=7d711d15249631b in_seq=0 out_seq=0 2024-08-20T03:01:41.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.570+0000 7fb1777fe640 10 client.15228 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:01:41.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.570+0000 7fb1777fe640 1 -- 172.21.15.44:0/3413615512 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5628fac31090 con 0x5628fad48f30 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.570+0000 7fb1777fe640 10 client.15228 handle_command_reply: tid=0 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.570+0000 7fb1a0a11740 1 client.15228 shutdown 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb1a0a11740 1 -- 172.21.15.44:0/3413615512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 msgr2=0x5628face2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628face2ba0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb188027c20 tx=0x7fb1880053b0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb1a0a11740 1 -- 172.21.15.44:0/3413615512 shutdown_connections 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5628fad48f30 0x5628fad47c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628facc94a0 0x5628facde710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628facbd510 0x5628face3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb1a0a11740 1 --2- 172.21.15.44:0/3413615512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628facbcbf0 0x5628face2ba0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb1a0a11740 1 -- 172.21.15.44:0/3413615512 >> 172.21.15.44:0/3413615512 conn(0x5628facbaa20 msgr2=0x5628facc1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb1777fe640 0 client.15228 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb1a0a11740 1 -- 172.21.15.44:0/3413615512 shutdown_connections 2024-08-20T03:01:41.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb1a0a11740 1 -- 172.21.15.44:0/3413615512 wait complete. 2024-08-20T03:01:41.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb1a0a11740 20 client.15228 trim_cache size 0 max 16384 2024-08-20T03:01:41.589 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:01:41.589 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T03:01:41.589 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 1, 2024-08-20T03:01:41.589 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24596, 2024-08-20T03:01:41.589 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-08-20T03:01:41.589 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-08-20T03:01:41.589 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-08-20T03:01:41.589 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 75.219881224999995, 2024-08-20T03:01:41.589 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2024-08-20T03:01:41.590 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 77, 2024-08-20T03:01:41.590 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 75, 2024-08-20T03:01:41.590 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 93.447326969000002 2024-08-20T03:01:41.590 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:01:41.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb19e618640 1 -- 172.21.15.44:0/275307875 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb1840b3cb0 msgr2=0x7fb1840b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:41.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb19e618640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb1840b3cb0 0x7fb1840b6170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x5628facbfd70 tx=0x7fb18800a2d0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb19e618640 1 -- 172.21.15.44:0/275307875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb198107d20 msgr2=0x7fb19819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:41.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.571+0000 7fb19e618640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb198107d20 0x7fb19819e7c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb190005cc0 tx=0x7fb19002f380 comp rx=0 tx=0).stop 2024-08-20T03:01:41.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.572+0000 7fb19e618640 1 -- 172.21.15.44:0/275307875 shutdown_connections 2024-08-20T03:01:41.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.572+0000 7fb19e618640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb1840b3cb0 0x7fb1840b6170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.572+0000 7fb19e618640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb198107d20 0x7fb19819e7c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.572+0000 7fb19e618640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1981073a0 0x7fb198199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.572+0000 7fb19e618640 1 --2- 172.21.15.44:0/275307875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb198101990 0x7fb198198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:41.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.572+0000 7fb19e618640 1 -- 172.21.15.44:0/275307875 >> 172.21.15.44:0/275307875 conn(0x7fb1980fd400 msgr2=0x7fb19806aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:41.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.572+0000 7fb19e618640 1 -- 172.21.15.44:0/275307875 shutdown_connections 2024-08-20T03:01:41.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:41.572+0000 7fb19e618640 1 -- 172.21.15.44:0/275307875 wait complete. 2024-08-20T03:01:42.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:41 smithi165 ceph-mon[22357]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:42.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:41 smithi099 ceph-mon[21984]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:42.264 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:41 smithi044 ceph-mon[18790]: pgmap v296: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:42.296 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.i status 2024-08-20T03:01:42.552 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:43.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.031+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/2385656582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d28108e40 msgr2=0x7f7d2810f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:43.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.031+0000 7f7d2cdf7640 1 --2- 172.21.15.44:0/2385656582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d28108e40 0x7f7d2810f640 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009900 tx=0x7f7d2002a410 comp rx=0 tx=0).stop 2024-08-20T03:01:43.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.031+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/2385656582 shutdown_connections 2024-08-20T03:01:43.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.031+0000 7f7d2cdf7640 1 --2- 172.21.15.44:0/2385656582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d28108e40 0x7f7d2810f640 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.031+0000 7f7d2cdf7640 1 --2- 172.21.15.44:0/2385656582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d281084c0 0x7f7d28108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.031+0000 7f7d2cdf7640 1 --2- 172.21.15.44:0/2385656582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d28102ab0 0x7f7d28102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.031+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/2385656582 >> 172.21.15.44:0/2385656582 conn(0x7f7d280fe500 msgr2=0x7f7d28100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:43.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.031+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/2385656582 shutdown_connections 2024-08-20T03:01:43.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.031+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/2385656582 wait complete. 2024-08-20T03:01:43.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d2cdf7640 1 Processor -- start 2024-08-20T03:01:43.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d2cdf7640 1 -- start start 2024-08-20T03:01:43.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d2cdf7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d28102ab0 0x7f7d28198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d2cdf7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d281084c0 0x7f7d281994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d2cdf7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d28108e40 0x7f7d2819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d2cdf7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28112b00 con 0x7f7d28108e40 2024-08-20T03:01:43.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d2cdf7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28112980 con 0x7f7d281084c0 2024-08-20T03:01:43.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d2cdf7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28112c80 con 0x7f7d28102ab0 2024-08-20T03:01:43.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d26d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d28108e40 0x7f7d2819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d26d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d28108e40 0x7f7d2819e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46806/0 (socket says 172.21.15.44:46806) 2024-08-20T03:01:43.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d26d9d640 1 -- 172.21.15.44:0/3305699945 learned_addr learned my addr 172.21.15.44:0/3305699945 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:43.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d2659c640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d28102ab0 0x7f7d28198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.032+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d281084c0 0x7f7d281994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.033+0000 7f7d26d9d640 1 -- 172.21.15.44:0/3305699945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d28102ab0 msgr2=0x7f7d28198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:43.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.033+0000 7f7d26d9d640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d28102ab0 0x7f7d28198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.033+0000 7f7d26d9d640 1 -- 172.21.15.44:0/3305699945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d281084c0 msgr2=0x7f7d281994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:43.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.033+0000 7f7d26d9d640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d281084c0 0x7f7d281994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.033+0000 7f7d26d9d640 1 -- 172.21.15.44:0/3305699945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2819f060 con 0x7f7d28108e40 2024-08-20T03:01:43.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.033+0000 7f7d26d9d640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d28108e40 0x7f7d2819e890 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7d20004770 tx=0x7f7d20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:43.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.033+0000 7f7d177fe640 1 -- 172.21.15.44:0/3305699945 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2003f030 con 0x7f7d28108e40 2024-08-20T03:01:43.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.033+0000 7f7d177fe640 1 -- 172.21.15.44:0/3305699945 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d20003aa0 con 0x7f7d28108e40 2024-08-20T03:01:43.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.033+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/3305699945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2819f2f0 con 0x7f7d28108e40 2024-08-20T03:01:43.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.033+0000 7f7d177fe640 1 -- 172.21.15.44:0/3305699945 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20035070 con 0x7f7d28108e40 2024-08-20T03:01:43.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.033+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/3305699945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d281a5a10 con 0x7f7d28108e40 2024-08-20T03:01:43.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.035+0000 7f7d177fe640 1 -- 172.21.15.44:0/3305699945 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f7d20042030 con 0x7f7d28108e40 2024-08-20T03:01:43.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.035+0000 7f7d2f1f0740 1 Processor -- start 2024-08-20T03:01:43.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d2f1f0740 1 -- start start 2024-08-20T03:01:43.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d2f1f0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383735bf0 0x560383735fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d2f1f0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560383736510 0x560383746370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d2f1f0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603837468b0 0x560383748ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d2f1f0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5603836ac6a0 con 0x560383735bf0 2024-08-20T03:01:43.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d2f1f0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56038369bba0 con 0x560383736510 2024-08-20T03:01:43.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d2f1f0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56038369fbe0 con 0x5603837468b0 2024-08-20T03:01:43.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d25d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383735bf0 0x560383735fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d25d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383735bf0 0x560383735fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46814/0 (socket says 172.21.15.44:46814) 2024-08-20T03:01:43.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d25d9b640 1 -- 172.21.15.44:0/3466811416 learned_addr learned my addr 172.21.15.44:0/3466811416 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:43.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d2659c640 1 --2- 172.21.15.44:0/3466811416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560383736510 0x560383746370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/3466811416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603837468b0 0x560383748ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d25d9b640 1 -- 172.21.15.44:0/3466811416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603837468b0 msgr2=0x560383748ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:43.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/3466811416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603837468b0 0x560383748ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d25d9b640 1 -- 172.21.15.44:0/3466811416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560383736510 msgr2=0x560383746370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:43.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/3466811416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560383736510 0x560383746370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d25d9b640 1 -- 172.21.15.44:0/3466811416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603833fb540 con 0x560383735bf0 2024-08-20T03:01:43.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.036+0000 7f7d177fe640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7d100b3cb0 0x7f7d100b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.037+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/3466811416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383735bf0 0x560383735fd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00bf80 tx=0x7f7d1c00b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcbd5c6ab537066 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:43.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.037+0000 7f7d2659c640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7d100b3cb0 0x7f7d100b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.037+0000 7f7d177fe640 1 -- 172.21.15.44:0/3305699945 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7d201045e0 con 0x7f7d28108e40 2024-08-20T03:01:43.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.037+0000 7f7d14ff9640 1 -- 172.21.15.44:0/3466811416 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c019030 con 0x560383735bf0 2024-08-20T03:01:43.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.037+0000 7f7d14ff9640 1 -- 172.21.15.44:0/3466811416 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d1c005bf0 con 0x560383735bf0 2024-08-20T03:01:43.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.037+0000 7f7d14ff9640 1 -- 172.21.15.44:0/3466811416 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c01d740 con 0x560383735bf0 2024-08-20T03:01:43.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/3466811416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383735bf0 msgr2=0x560383735fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:43.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/3466811416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383735bf0 0x560383735fd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00bf80 tx=0x7f7d1c00b610 comp rx=0 tx=0).stop 2024-08-20T03:01:43.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2659c640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7d100b3cb0 0x7f7d100b6170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x560383734900 tx=0x7f7d18009680 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:43.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/3466811416 shutdown_connections 2024-08-20T03:01:43.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/3466811416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603837468b0 0x560383748ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/3466811416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560383736510 0x560383746370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/3466811416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383735bf0 0x560383735fd0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/3466811416 >> 172.21.15.44:0/3466811416 conn(0x560383733a20 msgr2=0x560383734bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:43.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/3466811416 shutdown_connections 2024-08-20T03:01:43.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/3466811416 wait complete. 2024-08-20T03:01:43.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 -- start start 2024-08-20T03:01:43.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 Processor -- start 2024-08-20T03:01:43.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560383735bf0 0x560383743890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383743dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603837468b0 0x560383744310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5603834fe5f0 con 0x560383736510 2024-08-20T03:01:43.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5603836226c0 con 0x5603837468b0 2024-08-20T03:01:43.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.038+0000 7f7d2f1f0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560383749530 con 0x560383735bf0 2024-08-20T03:01:43.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d2f1f0740 10 client.0 _ll_register_callbacks cb 0x56038365f1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:01:43.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d25d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383743dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d25d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383743dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46822/0 (socket says 172.21.15.44:46822) 2024-08-20T03:01:43.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d25d9b640 1 -- 172.21.15.44:0/1523174517 learned_addr learned my addr 172.21.15.44:0/1523174517 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:43.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d2659c640 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560383735bf0 0x560383743890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d26d9d640 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603837468b0 0x560383744310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d25d9b640 1 -- 172.21.15.44:0/1523174517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560383735bf0 msgr2=0x560383743890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:43.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560383735bf0 0x560383743890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d25d9b640 1 -- 172.21.15.44:0/1523174517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603837468b0 msgr2=0x560383744310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:43.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603837468b0 0x560383744310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d25d9b640 1 -- 172.21.15.44:0/1523174517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560383752930 con 0x560383736510 2024-08-20T03:01:43.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d2659c640 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560383735bf0 0x560383743890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:43.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d26d9d640 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603837468b0 0x560383744310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:43.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383743dd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005f70 tx=0x7f7d1c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9ea54eea74fce2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:01:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d0dffb640 1 -- 172.21.15.44:0/1523174517 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c019040 con 0x560383736510 2024-08-20T03:01:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d0dffb640 1 -- 172.21.15.44:0/1523174517 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7d1c0046c0 con 0x560383736510 2024-08-20T03:01:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1523174517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603836ae5f0 con 0x560383736510 2024-08-20T03:01:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d0dffb640 1 -- 172.21.15.44:0/1523174517 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d1c00b2c0 con 0x560383736510 2024-08-20T03:01:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d2f1f0740 10 client.15240 fetch_fsmap learned FSMap version 27 2024-08-20T03:01:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1523174517 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5603837552e0 con 0x560383736510 2024-08-20T03:01:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.039+0000 7f7d0dffb640 1 -- 172.21.15.44:0/1523174517 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5603836ae5f0 con 0x560383736510 2024-08-20T03:01:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.040+0000 7f7d0dffb640 1 -- 172.21.15.44:0/1523174517 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7d1c025070 con 0x560383736510 2024-08-20T03:01:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.040+0000 7f7d2f1f0740 10 client.15240 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:01:43.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.040+0000 7f7d2f1f0740 10 client.15240 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-20T03:01:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.040+0000 7f7d2f1f0740 20 client.15240 populate_metadata read hostname 'smithi044' 2024-08-20T03:01:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.040+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x560383757620 0x560383758a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.040+0000 7f7d2f1f0740 4 client.15240 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:01:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.040+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1523174517 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603836aa090 con 0x560383757620 2024-08-20T03:01:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.040+0000 7f7d2659c640 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x560383757620 0x560383758a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.041+0000 7f7d2659c640 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x560383757620 0x560383758a90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c6c2d7eb2d9ff server_cookie=eb5fd6f514b74564 in_seq=0 out_seq=0 2024-08-20T03:01:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.041+0000 7f7d0dffb640 10 client.15240 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:01:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.041+0000 7f7d0dffb640 1 -- 172.21.15.44:0/1523174517 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603836aa090 con 0x560383757620 2024-08-20T03:01:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.041+0000 7f7d0dffb640 10 client.15240 handle_command_reply: tid=0 2024-08-20T03:01:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.041+0000 7f7d2f1f0740 1 client.15240 shutdown 2024-08-20T03:01:43.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.041+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1523174517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 msgr2=0x560383743dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.041+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383743dd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005f70 tx=0x7f7d1c0052f0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.042+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1523174517 shutdown_connections 2024-08-20T03:01:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.042+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x560383757620 0x560383758a90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.042+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603837468b0 0x560383744310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.042+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383743dd0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.042+0000 7f7d0dffb640 0 client.15240 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:01:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.042+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/1523174517 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560383735bf0 0x560383743890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.042+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1523174517 >> 172.21.15.44:0/1523174517 conn(0x560383733a20 msgr2=0x5603837395c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.042+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1523174517 shutdown_connections 2024-08-20T03:01:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.042+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1523174517 wait complete. 2024-08-20T03:01:43.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.042+0000 7f7d2f1f0740 20 client.15240 trim_cache size 0 max 16384 2024-08-20T03:01:43.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d2f1f0740 1 Processor -- start 2024-08-20T03:01:43.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d2f1f0740 1 -- start start 2024-08-20T03:01:43.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d2f1f0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560383735bf0 0x560383758a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d2f1f0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383735240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d2f1f0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603837468b0 0x560383741340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d2f1f0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560383756dd0 con 0x560383736510 2024-08-20T03:01:43.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d2f1f0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5603837574a0 con 0x560383735bf0 2024-08-20T03:01:43.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d2f1f0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560383757b70 con 0x5603837468b0 2024-08-20T03:01:43.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d25d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383735240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d25d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383735240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46824/0 (socket says 172.21.15.44:46824) 2024-08-20T03:01:43.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d25d9b640 1 -- 172.21.15.44:0/307986068 learned_addr learned my addr 172.21.15.44:0/307986068 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:43.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d2659c640 1 --2- 172.21.15.44:0/307986068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560383735bf0 0x560383758a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d26d9d640 1 --2- 172.21.15.44:0/307986068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603837468b0 0x560383741340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d25d9b640 1 -- 172.21.15.44:0/307986068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603837468b0 msgr2=0x560383741340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/307986068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603837468b0 0x560383741340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d25d9b640 1 -- 172.21.15.44:0/307986068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560383735bf0 msgr2=0x560383758a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/307986068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560383735bf0 0x560383758a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d25d9b640 1 -- 172.21.15.44:0/307986068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560383741880 con 0x560383736510 2024-08-20T03:01:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d26d9d640 1 --2- 172.21.15.44:0/307986068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603837468b0 0x560383741340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d2659c640 1 --2- 172.21.15.44:0/307986068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560383735bf0 0x560383758a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.060+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/307986068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383735240 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005fc0 tx=0x7f7d1c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78ee49c0e4ca228 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.061+0000 7f7d0ffff640 1 -- 172.21.15.44:0/307986068 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c034020 con 0x560383736510 2024-08-20T03:01:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.061+0000 7f7d0ffff640 1 -- 172.21.15.44:0/307986068 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d1c0116c0 con 0x560383736510 2024-08-20T03:01:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.061+0000 7f7d0ffff640 1 -- 172.21.15.44:0/307986068 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c004430 con 0x560383736510 2024-08-20T03:01:43.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.061+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/307986068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 msgr2=0x560383735240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:43.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.061+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/307986068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383735240 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c005fc0 tx=0x7f7d1c0056a0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.061+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/307986068 shutdown_connections 2024-08-20T03:01:43.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.061+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/307986068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603837468b0 0x560383741340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.061+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/307986068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383735240 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.061+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/307986068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560383735bf0 0x560383758a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.061+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/307986068 >> 172.21.15.44:0/307986068 conn(0x560383733a20 msgr2=0x560383734b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:43.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.061+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/307986068 shutdown_connections 2024-08-20T03:01:43.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.061+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/307986068 wait complete. 2024-08-20T03:01:43.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d2f1f0740 1 -- start start 2024-08-20T03:01:43.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d2f1f0740 1 Processor -- start 2024-08-20T03:01:43.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d2f1f0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560383735bf0 0x560383738410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d2f1f0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383736b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d2f1f0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603837468b0 0x5603837371e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d2f1f0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560383758910 con 0x560383736510 2024-08-20T03:01:43.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d2f1f0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560383758240 con 0x5603837468b0 2024-08-20T03:01:43.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d2f1f0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560383758fe0 con 0x560383735bf0 2024-08-20T03:01:43.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d2f1f0740 10 client.0 _ll_register_callbacks cb 0x7f7d18009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:01:43.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d25d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383736b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d25d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383736b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46826/0 (socket says 172.21.15.44:46826) 2024-08-20T03:01:43.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d25d9b640 1 -- 172.21.15.44:0/1211711004 learned_addr learned my addr 172.21.15.44:0/1211711004 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:43.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d2659c640 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560383735bf0 0x560383738410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d26d9d640 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603837468b0 0x5603837371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d25d9b640 1 -- 172.21.15.44:0/1211711004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560383735bf0 msgr2=0x560383738410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560383735bf0 0x560383738410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d25d9b640 1 -- 172.21.15.44:0/1211711004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603837468b0 msgr2=0x5603837371e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:01:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603837468b0 0x5603837371e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.062+0000 7f7d25d9b640 1 -- 172.21.15.44:0/1211711004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56038375fc90 con 0x560383736510 2024-08-20T03:01:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d25d9b640 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383736b10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c004110 tx=0x7f7d1c011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c3181869bae02d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:01:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d0dffb640 1 -- 172.21.15.44:0/1211711004 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c034030 con 0x560383736510 2024-08-20T03:01:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d0dffb640 1 -- 172.21.15.44:0/1211711004 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7d1c0114c0 con 0x560383736510 2024-08-20T03:01:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1211711004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d1c0116c0 con 0x560383736510 2024-08-20T03:01:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d0dffb640 1 -- 172.21.15.44:0/1211711004 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7d1c048050 con 0x560383736510 2024-08-20T03:01:43.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d2f1f0740 10 client.15246 fetch_fsmap learned FSMap version 27 2024-08-20T03:01:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1211711004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560383760f90 con 0x560383736510 2024-08-20T03:01:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d0dffb640 1 -- 172.21.15.44:0/1211711004 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d1c0116c0 con 0x560383736510 2024-08-20T03:01:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d0dffb640 1 -- 172.21.15.44:0/1211711004 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7d1c025040 con 0x560383736510 2024-08-20T03:01:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d2f1f0740 10 client.15246 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:01:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d2f1f0740 10 client.15246 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-20T03:01:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d2f1f0740 20 client.15246 populate_metadata read hostname 'smithi044' 2024-08-20T03:01:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5603837c5900 0x5603837c4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d2f1f0740 4 client.15246 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T03:01:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.063+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1211711004 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "status"}) -- 0x5603836a77d0 con 0x5603837c5900 2024-08-20T03:01:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d2659c640 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5603837c5900 0x5603837c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d2659c640 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5603837c5900 0x5603837c4610 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb58e13042f008f2 server_cookie=3798aa6f31590f0f in_seq=0 out_seq=0 2024-08-20T03:01:43.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d0dffb640 10 client.15246 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:01:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d0dffb640 1 -- 172.21.15.44:0/1211711004 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5603836a77d0 con 0x5603837c5900 2024-08-20T03:01:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d0dffb640 10 client.15246 handle_command_reply: tid=0 2024-08-20T03:01:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d2f1f0740 1 client.15246 shutdown 2024-08-20T03:01:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1211711004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 msgr2=0x560383736b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383736b10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c004110 tx=0x7f7d1c011e30 comp rx=0 tx=0).stop 2024-08-20T03:01:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1211711004 shutdown_connections 2024-08-20T03:01:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5603837c5900 0x5603837c4610 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603837468b0 0x5603837371e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d0dffb640 0 client.15246 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:01:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560383736510 0x560383736b10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d2f1f0740 1 --2- 172.21.15.44:0/1211711004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560383735bf0 0x560383738410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.064+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1211711004 >> 172.21.15.44:0/1211711004 conn(0x560383733a20 msgr2=0x5603837517c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:43.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1211711004 shutdown_connections 2024-08-20T03:01:43.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2f1f0740 1 -- 172.21.15.44:0/1211711004 wait complete. 2024-08-20T03:01:43.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2f1f0740 20 client.15246 trim_cache size 0 max 16384 2024-08-20T03:01:43.075 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:01:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T03:01:43.075 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-08-20T03:01:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24604, 2024-08-20T03:01:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-08-20T03:01:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-08-20T03:01:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-08-20T03:01:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 83.826613484000006, 2024-08-20T03:01:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2024-08-20T03:01:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 77, 2024-08-20T03:01:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 75, 2024-08-20T03:01:43.076 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 103.74008348 2024-08-20T03:01:43.076 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:01:43.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/3305699945 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7d100b3cb0 msgr2=0x7f7d100b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:43.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2cdf7640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7d100b3cb0 0x7f7d100b6170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x560383734900 tx=0x7f7d18009680 comp rx=0 tx=0).stop 2024-08-20T03:01:43.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/3305699945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d28108e40 msgr2=0x7f7d2819e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:43.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2cdf7640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d28108e40 0x7f7d2819e890 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7d20004770 tx=0x7f7d20004270 comp rx=0 tx=0).stop 2024-08-20T03:01:43.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/3305699945 shutdown_connections 2024-08-20T03:01:43.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2cdf7640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7d100b3cb0 0x7f7d100b6170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2cdf7640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d28108e40 0x7f7d2819e890 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2cdf7640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d281084c0 0x7f7d281994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2cdf7640 1 --2- 172.21.15.44:0/3305699945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d28102ab0 0x7f7d28198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/3305699945 >> 172.21.15.44:0/3305699945 conn(0x7f7d280fe500 msgr2=0x7f7d281008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/3305699945 shutdown_connections 2024-08-20T03:01:43.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:43.065+0000 7f7d2cdf7640 1 -- 172.21.15.44:0/3305699945 wait complete. 2024-08-20T03:01:43.777 INFO:tasks.cephadm:Setup complete, yielding 2024-08-20T03:01:43.778 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-20T03:01:43.788 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-08-20T03:01:43.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- bash -c 'ceph orch status' 2024-08-20T03:01:44.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:43 smithi044 ceph-mon[18790]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-08-20T03:01:44.041 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:43 smithi165 ceph-mon[22357]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-08-20T03:01:44.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:43 smithi099 ceph-mon[21984]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-08-20T03:01:44.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.517+0000 7fb01788a640 1 -- 172.21.15.44:0/1228510556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0101084c0 msgr2=0x7fb010108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:44.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.517+0000 7fb01788a640 1 --2- 172.21.15.44:0/1228510556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0101084c0 0x7fb010108900 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb004009900 tx=0x7fb00402a410 comp rx=0 tx=0).stop 2024-08-20T03:01:44.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.517+0000 7fb01788a640 1 -- 172.21.15.44:0/1228510556 shutdown_connections 2024-08-20T03:01:44.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.517+0000 7fb01788a640 1 --2- 172.21.15.44:0/1228510556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb010108e40 0x7fb01010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:44.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.517+0000 7fb01788a640 1 --2- 172.21.15.44:0/1228510556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0101084c0 0x7fb010108900 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:44.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.517+0000 7fb01788a640 1 --2- 172.21.15.44:0/1228510556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb010102ab0 0x7fb010102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:44.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.517+0000 7fb01788a640 1 -- 172.21.15.44:0/1228510556 >> 172.21.15.44:0/1228510556 conn(0x7fb0100fe540 msgr2=0x7fb010100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:44.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.517+0000 7fb01788a640 1 -- 172.21.15.44:0/1228510556 shutdown_connections 2024-08-20T03:01:44.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.517+0000 7fb01788a640 1 -- 172.21.15.44:0/1228510556 wait complete. 2024-08-20T03:01:44.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.517+0000 7fb01788a640 1 Processor -- start 2024-08-20T03:01:44.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.517+0000 7fb01788a640 1 -- start start 2024-08-20T03:01:44.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb01788a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb010102ab0 0x7fb010198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:44.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb01788a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0101084c0 0x7fb010199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:44.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb01788a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb010108e40 0x7fb01019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:44.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb01788a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb010112b10 con 0x7fb010102ab0 2024-08-20T03:01:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb01788a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb010112990 con 0x7fb0101084c0 2024-08-20T03:01:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb01788a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb010112c90 con 0x7fb010108e40 2024-08-20T03:01:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb015626640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb010102ab0 0x7fb010198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb015626640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb010102ab0 0x7fb010198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46840/0 (socket says 172.21.15.44:46840) 2024-08-20T03:01:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb015626640 1 -- 172.21.15.44:0/2046009642 learned_addr learned my addr 172.21.15.44:0/2046009642 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb014e25640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0101084c0 0x7fb010199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb015e27640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb010108e40 0x7fb01019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb015626640 1 -- 172.21.15.44:0/2046009642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb010108e40 msgr2=0x7fb01019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb015626640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb010108e40 0x7fb01019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb015626640 1 -- 172.21.15.44:0/2046009642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0101084c0 msgr2=0x7fb010199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb015626640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0101084c0 0x7fb010199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:44.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb015626640 1 -- 172.21.15.44:0/2046009642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01019f070 con 0x7fb010102ab0 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb014e25640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0101084c0 0x7fb010199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb015626640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb010102ab0 0x7fb010198fc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb00000eab0 tx=0x7fb00000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb00e7fc640 1 -- 172.21.15.44:0/2046009642 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00000ce40 con 0x7fb010102ab0 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb00e7fc640 1 -- 172.21.15.44:0/2046009642 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb000011920 con 0x7fb010102ab0 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb01788a640 1 -- 172.21.15.44:0/2046009642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01019f360 con 0x7fb010102ab0 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb00e7fc640 1 -- 172.21.15.44:0/2046009642 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000010a30 con 0x7fb010102ab0 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.518+0000 7fb01788a640 1 -- 172.21.15.44:0/2046009642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0101a5bf0 con 0x7fb010102ab0 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.519+0000 7fb01788a640 1 -- 172.21.15.44:0/2046009642 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb010102e90 con 0x7fb010102ab0 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.521+0000 7fb00e7fc640 1 -- 172.21.15.44:0/2046009642 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb000010bd0 con 0x7fb010102ab0 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.522+0000 7fb00e7fc640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faff80b3d30 0x7faff80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.522+0000 7fb014e25640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faff80b3d30 0x7faff80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.522+0000 7fb00e7fc640 1 -- 172.21.15.44:0/2046009642 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb0000e44e0 con 0x7fb010102ab0 2024-08-20T03:01:44.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.522+0000 7fb014e25640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faff80b3d30 0x7faff80b61f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb004000c00 tx=0x7fb004005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:44.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.525+0000 7fb00e7fc640 1 -- 172.21.15.44:0/2046009642 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb000016030 con 0x7fb010102ab0 2024-08-20T03:01:44.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.726+0000 7fb01788a640 1 -- 172.21.15.44:0/2046009642 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fb0100632c0 con 0x7faff80b3d30 2024-08-20T03:01:44.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.727+0000 7fb00e7fc640 1 -- 172.21.15.44:0/2046009642 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fb0100632c0 con 0x7faff80b3d30 2024-08-20T03:01:44.730 INFO:teuthology.orchestra.run.smithi044.stdout:Backend: cephadm 2024-08-20T03:01:44.730 INFO:teuthology.orchestra.run.smithi044.stdout:Available: Yes 2024-08-20T03:01:44.730 INFO:teuthology.orchestra.run.smithi044.stdout:Paused: No 2024-08-20T03:01:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.729+0000 7fb01788a640 1 -- 172.21.15.44:0/2046009642 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faff80b3d30 msgr2=0x7faff80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.729+0000 7fb01788a640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faff80b3d30 0x7faff80b61f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb004000c00 tx=0x7fb004005a20 comp rx=0 tx=0).stop 2024-08-20T03:01:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.729+0000 7fb01788a640 1 -- 172.21.15.44:0/2046009642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb010102ab0 msgr2=0x7fb010198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.729+0000 7fb01788a640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb010102ab0 0x7fb010198fc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb00000eab0 tx=0x7fb00000ef70 comp rx=0 tx=0).stop 2024-08-20T03:01:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.730+0000 7fb01788a640 1 -- 172.21.15.44:0/2046009642 shutdown_connections 2024-08-20T03:01:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.730+0000 7fb01788a640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faff80b3d30 0x7faff80b61f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.730+0000 7fb01788a640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb010108e40 0x7fb01019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.730+0000 7fb01788a640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0101084c0 0x7fb010199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:44.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.730+0000 7fb01788a640 1 --2- 172.21.15.44:0/2046009642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb010102ab0 0x7fb010198fc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:44.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.730+0000 7fb01788a640 1 -- 172.21.15.44:0/2046009642 >> 172.21.15.44:0/2046009642 conn(0x7fb0100fe540 msgr2=0x7fb010100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:44.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.730+0000 7fb01788a640 1 -- 172.21.15.44:0/2046009642 shutdown_connections 2024-08-20T03:01:44.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:44.730+0000 7fb01788a640 1 -- 172.21.15.44:0/2046009642 wait complete. 2024-08-20T03:01:45.455 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ps' 2024-08-20T03:01:45.710 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:46.121 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:45 smithi044 ceph-mon[18790]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:46.121 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:45 smithi044 ceph-mon[18790]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:46.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:45 smithi165 ceph-mon[22357]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:46.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:45 smithi165 ceph-mon[22357]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:46.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:45 smithi099 ceph-mon[21984]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:46.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:45 smithi099 ceph-mon[21984]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:46.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.199+0000 7f3bb7b69640 1 -- 172.21.15.44:0/1231680923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb0101520 msgr2=0x7f3bb0101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:46.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.199+0000 7f3bb7b69640 1 --2- 172.21.15.44:0/1231680923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb0101520 0x7f3bb0101900 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009900 tx=0x7f3bac02a410 comp rx=0 tx=0).stop 2024-08-20T03:01:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.199+0000 7f3bb7b69640 1 -- 172.21.15.44:0/1231680923 shutdown_connections 2024-08-20T03:01:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.199+0000 7f3bb7b69640 1 --2- 172.21.15.44:0/1231680923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb010cf60 0x7f3bb010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.199+0000 7f3bb7b69640 1 --2- 172.21.15.44:0/1231680923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb010a630 0x7f3bb010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.199+0000 7f3bb7b69640 1 --2- 172.21.15.44:0/1231680923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb0101520 0x7f3bb0101900 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.199+0000 7f3bb7b69640 1 -- 172.21.15.44:0/1231680923 >> 172.21.15.44:0/1231680923 conn(0x7f3bb0076cc0 msgr2=0x7f3bb00770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:46.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.199+0000 7f3bb7b69640 1 -- 172.21.15.44:0/1231680923 shutdown_connections 2024-08-20T03:01:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.199+0000 7f3bb7b69640 1 -- 172.21.15.44:0/1231680923 wait complete. 2024-08-20T03:01:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb7b69640 1 Processor -- start 2024-08-20T03:01:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb7b69640 1 -- start start 2024-08-20T03:01:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb7b69640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb0101520 0x7f3bb01024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb7b69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb010a630 0x7f3bb0102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb7b69640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb010cf60 0x7f3bb0102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb7b69640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0112aa0 con 0x7f3bb010a630 2024-08-20T03:01:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb7b69640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0112920 con 0x7f3bb010cf60 2024-08-20T03:01:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb7b69640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0112c20 con 0x7f3bb0101520 2024-08-20T03:01:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb5104640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb010a630 0x7f3bb0102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb5104640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb010a630 0x7f3bb0102a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46868/0 (socket says 172.21.15.44:46868) 2024-08-20T03:01:46.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb5104640 1 -- 172.21.15.44:0/2429360167 learned_addr learned my addr 172.21.15.44:0/2429360167 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb6106640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb010cf60 0x7f3bb0102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb5905640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb0101520 0x7f3bb01024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb5104640 1 -- 172.21.15.44:0/2429360167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb0101520 msgr2=0x7f3bb01024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb5104640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb0101520 0x7f3bb01024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb5104640 1 -- 172.21.15.44:0/2429360167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb010cf60 msgr2=0x7f3bb0102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb5104640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb010cf60 0x7f3bb0102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb5104640 1 -- 172.21.15.44:0/2429360167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb01a7310 con 0x7f3bb010a630 2024-08-20T03:01:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.200+0000 7f3bb5905640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb0101520 0x7f3bb01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:01:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.201+0000 7f3bb5104640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb010a630 0x7f3bb0102a30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400d950 tx=0x7f3ba400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.201+0000 7f3ba2ffd640 1 -- 172.21.15.44:0/2429360167 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4017070 con 0x7f3bb010a630 2024-08-20T03:01:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.201+0000 7f3ba2ffd640 1 -- 172.21.15.44:0/2429360167 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ba4009c30 con 0x7f3bb010a630 2024-08-20T03:01:46.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.201+0000 7f3bb7b69640 1 -- 172.21.15.44:0/2429360167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb01a7600 con 0x7f3bb010a630 2024-08-20T03:01:46.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.201+0000 7f3ba2ffd640 1 -- 172.21.15.44:0/2429360167 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400f620 con 0x7f3bb010a630 2024-08-20T03:01:46.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.201+0000 7f3bb7b69640 1 -- 172.21.15.44:0/2429360167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb01a7b90 con 0x7f3bb010a630 2024-08-20T03:01:46.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.202+0000 7f3bb7b69640 1 -- 172.21.15.44:0/2429360167 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb0101900 con 0x7f3bb010a630 2024-08-20T03:01:46.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.203+0000 7f3ba2ffd640 1 -- 172.21.15.44:0/2429360167 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f3ba4010460 con 0x7f3bb010a630 2024-08-20T03:01:46.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.204+0000 7f3ba2ffd640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3b9c0b3d30 0x7f3b9c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:46.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.204+0000 7f3bb5905640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3b9c0b3d30 0x7f3b9c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:46.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.204+0000 7f3ba2ffd640 1 -- 172.21.15.44:0/2429360167 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3ba40e3a40 con 0x7f3bb010a630 2024-08-20T03:01:46.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.205+0000 7f3bb5905640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3b9c0b3d30 0x7f3b9c0b61f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3bac031440 tx=0x7f3bac005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:46.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.207+0000 7f3ba2ffd640 1 -- 172.21.15.44:0/2429360167 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f3ba40aaf00 con 0x7f3bb010a630 2024-08-20T03:01:46.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.402+0000 7f3bb7b69640 1 -- 172.21.15.44:0/2429360167 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3bb01009c0 con 0x7f3b9c0b3d30 2024-08-20T03:01:46.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.416+0000 7f3ba2ffd640 1 -- 172.21.15.44:0/2429360167 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f3bb01009c0 con 0x7f3b9c0b3d30 2024-08-20T03:01:46.418 INFO:teuthology.orchestra.run.smithi044.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-08-20T03:01:46.418 INFO:teuthology.orchestra.run.smithi044.stdout:mds.a smithi044 running (101s) 21s ago 100s 17.2M - 19.0.0-5872-gef427541 9f5c88d94bb6 37ec51dad20b 2024-08-20T03:01:46.418 INFO:teuthology.orchestra.run.smithi044.stdout:mds.b smithi099 running (98s) 31s ago 98s 17.4M - 19.0.0-5872-gef427541 9f5c88d94bb6 2a1257101341 2024-08-20T03:01:46.418 INFO:teuthology.orchestra.run.smithi044.stdout:mds.c smithi165 running (112s) 31s ago 112s 18.3M - 19.0.0-5872-gef427541 9f5c88d94bb6 3c27108e7bb3 2024-08-20T03:01:46.418 INFO:teuthology.orchestra.run.smithi044.stdout:mds.d smithi044 running (2m) 21s ago 2m 13.7M - 19.0.0-5872-gef427541 9f5c88d94bb6 91ad41b7d956 2024-08-20T03:01:46.418 INFO:teuthology.orchestra.run.smithi044.stdout:mds.e smithi099 running (117s) 31s ago 117s 19.7M - 19.0.0-5872-gef427541 9f5c88d94bb6 595d45342c70 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mds.f smithi165 running (115s) 31s ago 114s 18.4M - 19.0.0-5872-gef427541 9f5c88d94bb6 0783fe6121a0 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mds.g smithi044 running (104s) 21s ago 104s 19.7M - 19.0.0-5872-gef427541 9f5c88d94bb6 98b40491673a 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mds.h smithi099 running (110s) 31s ago 109s 18.4M - 19.0.0-5872-gef427541 9f5c88d94bb6 02ec440a0d71 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mds.i smithi165 running (107s) 31s ago 107s 26.2M - 19.0.0-5872-gef427541 9f5c88d94bb6 ef18f09e186f 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mds.j smithi044 running (2m) 21s ago 2m 16.9M - 19.0.0-5872-gef427541 9f5c88d94bb6 48ce209b7fab 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mds.k smithi099 running (2m) 31s ago 2m 17.3M - 19.0.0-5872-gef427541 9f5c88d94bb6 b692a8f7e019 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mds.l smithi165 running (2m) 31s ago 2m 17.0M - 19.0.0-5872-gef427541 9f5c88d94bb6 99599d5447be 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mgr.x smithi044 *:9283,8765 running (9m) 21s ago 9m 516M - 19.0.0-5872-gef427541 9f5c88d94bb6 4db061448d48 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mgr.y smithi099 *:8443,8765 running (7m) 31s ago 7m 457M - 19.0.0-5872-gef427541 9f5c88d94bb6 78cf05a7f6f0 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mgr.z smithi165 *:8443,8765 running (7m) 31s ago 7m 457M - 19.0.0-5872-gef427541 9f5c88d94bb6 1a4c2ed740a4 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mon.a smithi044 running (9m) 21s ago 9m 52.8M 2048M 19.0.0-5872-gef427541 9f5c88d94bb6 b0653d702393 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mon.b smithi099 running (8m) 31s ago 8m 42.0M 2048M 19.0.0-5872-gef427541 9f5c88d94bb6 79fb5c25db4d 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:mon.c smithi165 running (8m) 31s ago 8m 42.3M 2048M 19.0.0-5872-gef427541 9f5c88d94bb6 b9c558999519 2024-08-20T03:01:46.419 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 smithi044 running (6m) 21s ago 6m 45.1M 4273M 19.0.0-5872-gef427541 9f5c88d94bb6 ded8da4ebf47 2024-08-20T03:01:46.420 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 smithi099 running (6m) 31s ago 6m 54.6M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 2bc4321bd17b 2024-08-20T03:01:46.420 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 smithi165 running (6m) 31s ago 6m 45.9M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 4f787bbc0eb6 2024-08-20T03:01:46.420 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 smithi044 running (5m) 21s ago 5m 52.8M 4273M 19.0.0-5872-gef427541 9f5c88d94bb6 a56414a543fb 2024-08-20T03:01:46.420 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 smithi099 running (5m) 31s ago 5m 42.7M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 d75d62af8a8b 2024-08-20T03:01:46.420 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 smithi165 running (4m) 31s ago 4m 54.5M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 fe942d45eace 2024-08-20T03:01:46.420 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 smithi044 running (4m) 21s ago 4m 50.6M 4273M 19.0.0-5872-gef427541 9f5c88d94bb6 79e044a853fc 2024-08-20T03:01:46.420 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 smithi099 running (4m) 31s ago 4m 54.2M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 40e93aabc69f 2024-08-20T03:01:46.420 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8 smithi165 running (3m) 31s ago 3m 55.0M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 bca2bd2f2506 2024-08-20T03:01:46.420 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9 smithi044 running (3m) 21s ago 3m 42.2M 4273M 19.0.0-5872-gef427541 9f5c88d94bb6 cf6bf4a563a9 2024-08-20T03:01:46.420 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10 smithi099 running (2m) 31s ago 2m 45.0M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 b0ea8d8553c9 2024-08-20T03:01:46.420 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11 smithi165 running (2m) 31s ago 2m 52.0M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 8e1af7e4602a 2024-08-20T03:01:46.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.418+0000 7f3bb7b69640 1 -- 172.21.15.44:0/2429360167 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3b9c0b3d30 msgr2=0x7f3b9c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:46.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.418+0000 7f3bb7b69640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3b9c0b3d30 0x7f3b9c0b61f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3bac031440 tx=0x7f3bac005a20 comp rx=0 tx=0).stop 2024-08-20T03:01:46.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.418+0000 7f3bb7b69640 1 -- 172.21.15.44:0/2429360167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb010a630 msgr2=0x7f3bb0102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:46.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.418+0000 7f3bb7b69640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb010a630 0x7f3bb0102a30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400d950 tx=0x7f3ba400de10 comp rx=0 tx=0).stop 2024-08-20T03:01:46.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.418+0000 7f3bb7b69640 1 -- 172.21.15.44:0/2429360167 shutdown_connections 2024-08-20T03:01:46.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.418+0000 7f3bb7b69640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3b9c0b3d30 0x7f3b9c0b61f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:46.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.418+0000 7f3bb7b69640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bb010cf60 0x7f3bb0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:46.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.418+0000 7f3bb7b69640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3bb010a630 0x7f3bb0102a30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:46.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.418+0000 7f3bb7b69640 1 --2- 172.21.15.44:0/2429360167 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb0101520 0x7f3bb01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:46.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.418+0000 7f3bb7b69640 1 -- 172.21.15.44:0/2429360167 >> 172.21.15.44:0/2429360167 conn(0x7f3bb0076cc0 msgr2=0x7f3bb01002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:46.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.419+0000 7f3bb7b69640 1 -- 172.21.15.44:0/2429360167 shutdown_connections 2024-08-20T03:01:46.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:46.419+0000 7f3bb7b69640 1 -- 172.21.15.44:0/2429360167 wait complete. 2024-08-20T03:01:47.254 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ls' 2024-08-20T03:01:47.513 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:47.906 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:47 smithi044 ceph-mon[18790]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:47.906 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:47 smithi044 ceph-mon[18790]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.985+0000 7fd28559e640 1 -- 172.21.15.44:0/3596260929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd280108310 msgr2=0x7fd280108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.985+0000 7fd28559e640 1 --2- 172.21.15.44:0/3596260929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd280108310 0x7fd280108750 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd274009900 tx=0x7fd27402a410 comp rx=0 tx=0).stop 2024-08-20T03:01:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.985+0000 7fd28559e640 1 -- 172.21.15.44:0/3596260929 shutdown_connections 2024-08-20T03:01:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.985+0000 7fd28559e640 1 --2- 172.21.15.44:0/3596260929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd280108c90 0x7fd28010f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.985+0000 7fd28559e640 1 --2- 172.21.15.44:0/3596260929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd280108310 0x7fd280108750 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.985+0000 7fd28559e640 1 --2- 172.21.15.44:0/3596260929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd280102900 0x7fd280102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.985+0000 7fd28559e640 1 -- 172.21.15.44:0/3596260929 >> 172.21.15.44:0/3596260929 conn(0x7fd2800fe390 msgr2=0x7fd2801007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.985+0000 7fd28559e640 1 -- 172.21.15.44:0/3596260929 shutdown_connections 2024-08-20T03:01:48.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.985+0000 7fd28559e640 1 -- 172.21.15.44:0/3596260929 wait complete. 2024-08-20T03:01:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.986+0000 7fd28559e640 1 Processor -- start 2024-08-20T03:01:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.986+0000 7fd28559e640 1 -- start start 2024-08-20T03:01:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.986+0000 7fd28559e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd280102900 0x7fd280198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.986+0000 7fd28559e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd280108310 0x7fd280199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.986+0000 7fd28559e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd280108c90 0x7fd28019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.986+0000 7fd28559e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd280112950 con 0x7fd280102900 2024-08-20T03:01:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.986+0000 7fd28559e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801127d0 con 0x7fd280108c90 2024-08-20T03:01:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.986+0000 7fd28559e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd280112ad0 con 0x7fd280108310 2024-08-20T03:01:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.986+0000 7fd27ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd280102900 0x7fd280198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.986+0000 7fd27ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd280102900 0x7fd280198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44568/0 (socket says 172.21.15.44:44568) 2024-08-20T03:01:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.986+0000 7fd27ffff640 1 -- 172.21.15.44:0/1428543189 learned_addr learned my addr 172.21.15.44:0/1428543189 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:48.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd27f7fe640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd280108310 0x7fd280199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd284d9d640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd280108c90 0x7fd28019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd27ffff640 1 -- 172.21.15.44:0/1428543189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd280108310 msgr2=0x7fd280199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd27ffff640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd280108310 0x7fd280199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd27ffff640 1 -- 172.21.15.44:0/1428543189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd280108c90 msgr2=0x7fd28019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd27ffff640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd280108c90 0x7fd28019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd27ffff640 1 -- 172.21.15.44:0/1428543189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28019edd0 con 0x7fd280102900 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd284d9d640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd280108c90 0x7fd28019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd27ffff640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd280102900 0x7fd280198d20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd27000d770 tx=0x7fd27000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd27d7fa640 1 -- 172.21.15.44:0/1428543189 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270017070 con 0x7fd280102900 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd28559e640 1 -- 172.21.15.44:0/1428543189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28019f0c0 con 0x7fd280102900 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd27d7fa640 1 -- 172.21.15.44:0/1428543189 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2700054d0 con 0x7fd280102900 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd27d7fa640 1 -- 172.21.15.44:0/1428543189 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270004c30 con 0x7fd280102900 2024-08-20T03:01:48.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.987+0000 7fd28559e640 1 -- 172.21.15.44:0/1428543189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2801a5780 con 0x7fd280102900 2024-08-20T03:01:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.988+0000 7fd28559e640 1 -- 172.21.15.44:0/1428543189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd280102ce0 con 0x7fd280102900 2024-08-20T03:01:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.989+0000 7fd27d7fa640 1 -- 172.21.15.44:0/1428543189 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fd270004e10 con 0x7fd280102900 2024-08-20T03:01:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.990+0000 7fd27d7fa640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd2680b3d60 0x7fd2680b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.991+0000 7fd27f7fe640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd2680b3d60 0x7fd2680b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.991+0000 7fd27d7fa640 1 -- 172.21.15.44:0/1428543189 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd2700e39d0 con 0x7fd280102900 2024-08-20T03:01:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.991+0000 7fd27f7fe640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd2680b3d60 0x7fd2680b6220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd274031440 tx=0x7fd274005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:48.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:47.995+0000 7fd27d7fa640 1 -- 172.21.15.44:0/1428543189 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd2700aadf0 con 0x7fd280102900 2024-08-20T03:01:48.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:47 smithi165 ceph-mon[22357]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:48.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:47 smithi165 ceph-mon[22357]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:48.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:47 smithi099 ceph-mon[21984]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:48.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:47 smithi099 ceph-mon[21984]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:48.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.187+0000 7fd28559e640 1 -- 172.21.15.44:0/1428543189 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fd2801058d0 con 0x7fd2680b3d60 2024-08-20T03:01:48.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.193+0000 7fd27d7fa640 1 -- 172.21.15.44:0/1428543189 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fd2801058d0 con 0x7fd2680b3d60 2024-08-20T03:01:48.195 INFO:teuthology.orchestra.run.smithi044.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-08-20T03:01:48.195 INFO:teuthology.orchestra.run.smithi044.stdout:mds.all 12/12 33s ago 2m smithi044=a;smithi044=d;smithi044=g;smithi044=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi165=c;smithi165=f;smithi165=i;smithi165=l;count:12 2024-08-20T03:01:48.195 INFO:teuthology.orchestra.run.smithi044.stdout:mgr 3/3 33s ago 7m smithi044=x;smithi099=y;smithi165=z;count:3 2024-08-20T03:01:48.195 INFO:teuthology.orchestra.run.smithi044.stdout:mon 3/3 33s ago 8m smithi044:172.21.15.44=a;smithi099:172.21.15.99=b;smithi165:172.21.15.165=c;count:3 2024-08-20T03:01:48.195 INFO:teuthology.orchestra.run.smithi044.stdout:osd 12 33s ago - 2024-08-20T03:01:48.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.195+0000 7fd28559e640 1 -- 172.21.15.44:0/1428543189 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd2680b3d60 msgr2=0x7fd2680b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:48.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.195+0000 7fd28559e640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd2680b3d60 0x7fd2680b6220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd274031440 tx=0x7fd274005b50 comp rx=0 tx=0).stop 2024-08-20T03:01:48.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.195+0000 7fd28559e640 1 -- 172.21.15.44:0/1428543189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd280102900 msgr2=0x7fd280198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:48.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.195+0000 7fd28559e640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd280102900 0x7fd280198d20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd27000d770 tx=0x7fd27000dc30 comp rx=0 tx=0).stop 2024-08-20T03:01:48.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.195+0000 7fd28559e640 1 -- 172.21.15.44:0/1428543189 shutdown_connections 2024-08-20T03:01:48.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.195+0000 7fd28559e640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd2680b3d60 0x7fd2680b6220 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:48.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.195+0000 7fd28559e640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd280108c90 0x7fd28019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:48.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.195+0000 7fd28559e640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd280108310 0x7fd280199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:48.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.195+0000 7fd28559e640 1 --2- 172.21.15.44:0/1428543189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd280102900 0x7fd280198d20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:48.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.195+0000 7fd28559e640 1 -- 172.21.15.44:0/1428543189 >> 172.21.15.44:0/1428543189 conn(0x7fd2800fe390 msgr2=0x7fd280101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:48.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.195+0000 7fd28559e640 1 -- 172.21.15.44:0/1428543189 shutdown_connections 2024-08-20T03:01:48.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:48.195+0000 7fd28559e640 1 -- 172.21.15.44:0/1428543189 wait complete. 2024-08-20T03:01:48.882 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- bash -c 'ceph orch host ls' 2024-08-20T03:01:49.133 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:49.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:48 smithi099 ceph-mon[21984]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:49.253 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:48 smithi044 ceph-mon[18790]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:49.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:48 smithi165 ceph-mon[22357]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:49.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.634+0000 7f1087b37640 1 -- 172.21.15.44:0/2263451920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1080107d60 msgr2=0x7f108010e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:49.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.634+0000 7f1087b37640 1 --2- 172.21.15.44:0/2263451920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1080107d60 0x7f108010e560 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1078009ca0 tx=0x7f107802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:49.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.634+0000 7f1087b37640 1 -- 172.21.15.44:0/2263451920 shutdown_connections 2024-08-20T03:01:49.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.634+0000 7f1087b37640 1 --2- 172.21.15.44:0/2263451920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1080107d60 0x7f108010e560 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:49.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.634+0000 7f1087b37640 1 --2- 172.21.15.44:0/2263451920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10801073e0 0x7f1080107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:49.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.634+0000 7f1087b37640 1 --2- 172.21.15.44:0/2263451920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10801019d0 0x7f1080101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:49.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.634+0000 7f1087b37640 1 -- 172.21.15.44:0/2263451920 >> 172.21.15.44:0/2263451920 conn(0x7f10800fd440 msgr2=0x7f10800ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:49.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.634+0000 7f1087b37640 1 -- 172.21.15.44:0/2263451920 shutdown_connections 2024-08-20T03:01:49.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.634+0000 7f1087b37640 1 -- 172.21.15.44:0/2263451920 wait complete. 2024-08-20T03:01:49.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f1087b37640 1 Processor -- start 2024-08-20T03:01:49.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f1087b37640 1 -- start start 2024-08-20T03:01:49.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f1087b37640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10801019d0 0x7f1080199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:49.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f1087b37640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10801073e0 0x7f108019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f1087b37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1080107d60 0x7f108019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f1087b37640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f10801119e0 con 0x7f1080107d60 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f1087b37640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1080111860 con 0x7f10801073e0 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f1087b37640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1080111b60 con 0x7f10801019d0 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f10860d4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1080107d60 0x7f108019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f10860d4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1080107d60 0x7f108019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44590/0 (socket says 172.21.15.44:44590) 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f10860d4640 1 -- 172.21.15.44:0/2041154745 learned_addr learned my addr 172.21.15.44:0/2041154745 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f10850d2640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10801073e0 0x7f108019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f10858d3640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10801019d0 0x7f1080199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f10860d4640 1 -- 172.21.15.44:0/2041154745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10801019d0 msgr2=0x7f1080199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f10860d4640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10801019d0 0x7f1080199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f10860d4640 1 -- 172.21.15.44:0/2041154745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10801073e0 msgr2=0x7f108019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f10860d4640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10801073e0 0x7f108019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:49.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f10860d4640 1 -- 172.21.15.44:0/2041154745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10801a5130 con 0x7f1080107d60 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.635+0000 7f10860d4640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1080107d60 0x7f108019a7f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1078005cc0 tx=0x7f107802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.636+0000 7f1076ffd640 1 -- 172.21.15.44:0/2041154745 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078003bf0 con 0x7f1080107d60 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.636+0000 7f1076ffd640 1 -- 172.21.15.44:0/2041154745 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1078003d90 con 0x7f1080107d60 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.636+0000 7f1076ffd640 1 -- 172.21.15.44:0/2041154745 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107802fb20 con 0x7f1080107d60 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.636+0000 7f1087b37640 1 -- 172.21.15.44:0/2041154745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10801a53c0 con 0x7f1080107d60 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.636+0000 7f1087b37640 1 -- 172.21.15.44:0/2041154745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10801a58b0 con 0x7f1080107d60 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.637+0000 7f1087b37640 1 -- 172.21.15.44:0/2041154745 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1080101db0 con 0x7f1080107d60 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.638+0000 7f1076ffd640 1 -- 172.21.15.44:0/2041154745 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f1078004580 con 0x7f1080107d60 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.639+0000 7f1076ffd640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f106c0b3d30 0x7f106c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.639+0000 7f10858d3640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f106c0b3d30 0x7f106c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.639+0000 7f1076ffd640 1 -- 172.21.15.44:0/2041154745 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1078105340 con 0x7f1080107d60 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.639+0000 7f10858d3640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f106c0b3d30 0x7f106c0b61f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f107c00abe0 tx=0x7f107c00a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:49.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.642+0000 7f1076ffd640 1 -- 172.21.15.44:0/2041154745 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f10780cc920 con 0x7f1080107d60 2024-08-20T03:01:49.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.833+0000 7f1087b37640 1 -- 172.21.15.44:0/2041154745 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f10800632c0 con 0x7f106c0b3d30 2024-08-20T03:01:49.836 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.834+0000 7f1076ffd640 1 -- 172.21.15.44:0/2041154745 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f10800632c0 con 0x7f106c0b3d30 2024-08-20T03:01:49.837 INFO:teuthology.orchestra.run.smithi044.stdout:HOST ADDR LABELS STATUS 2024-08-20T03:01:49.837 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044 172.21.15.44 2024-08-20T03:01:49.837 INFO:teuthology.orchestra.run.smithi044.stdout:smithi099 172.21.15.99 2024-08-20T03:01:49.837 INFO:teuthology.orchestra.run.smithi044.stdout:smithi165 172.21.15.165 2024-08-20T03:01:49.837 INFO:teuthology.orchestra.run.smithi044.stdout:3 hosts in cluster 2024-08-20T03:01:49.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.836+0000 7f1087b37640 1 -- 172.21.15.44:0/2041154745 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f106c0b3d30 msgr2=0x7f106c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:49.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.836+0000 7f1087b37640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f106c0b3d30 0x7f106c0b61f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f107c00abe0 tx=0x7f107c00a2d0 comp rx=0 tx=0).stop 2024-08-20T03:01:49.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.836+0000 7f1087b37640 1 -- 172.21.15.44:0/2041154745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1080107d60 msgr2=0x7f108019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:49.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.836+0000 7f1087b37640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1080107d60 0x7f108019a7f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1078005cc0 tx=0x7f107802f380 comp rx=0 tx=0).stop 2024-08-20T03:01:49.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.837+0000 7f1087b37640 1 -- 172.21.15.44:0/2041154745 shutdown_connections 2024-08-20T03:01:49.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.837+0000 7f1087b37640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f106c0b3d30 0x7f106c0b61f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:49.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.837+0000 7f1087b37640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1080107d60 0x7f108019a7f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:49.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.837+0000 7f1087b37640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10801073e0 0x7f108019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:49.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.837+0000 7f1087b37640 1 --2- 172.21.15.44:0/2041154745 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10801019d0 0x7f1080199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:49.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.837+0000 7f1087b37640 1 -- 172.21.15.44:0/2041154745 >> 172.21.15.44:0/2041154745 conn(0x7f10800fd440 msgr2=0x7f10800fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:49.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.837+0000 7f1087b37640 1 -- 172.21.15.44:0/2041154745 shutdown_connections 2024-08-20T03:01:49.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:49.837+0000 7f1087b37640 1 -- 172.21.15.44:0/2041154745 wait complete. 2024-08-20T03:01:50.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:49 smithi165 ceph-mon[22357]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:01:50.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:49 smithi099 ceph-mon[21984]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:01:50.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:49 smithi044 ceph-mon[18790]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:01:50.451 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- bash -c 'ceph orch device ls' 2024-08-20T03:01:50.708 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:51.109 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:50 smithi044 ceph-mon[18790]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:51.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:50 smithi099 ceph-mon[21984]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:51.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.189+0000 7fb071a48640 1 -- 172.21.15.44:0/2601174062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c104270 msgr2=0x7fb06c10c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:51.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.189+0000 7fb071a48640 1 --2- 172.21.15.44:0/2601174062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c104270 0x7fb06c10c850 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb054009900 tx=0x7fb05402a410 comp rx=0 tx=0).stop 2024-08-20T03:01:51.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.189+0000 7fb071a48640 1 -- 172.21.15.44:0/2601174062 shutdown_connections 2024-08-20T03:01:51.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.189+0000 7fb071a48640 1 --2- 172.21.15.44:0/2601174062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb06c10ce20 0x7fb06c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:51.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.189+0000 7fb071a48640 1 --2- 172.21.15.44:0/2601174062 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c104270 0x7fb06c10c850 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:51.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.189+0000 7fb071a48640 1 --2- 172.21.15.44:0/2601174062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb06c103870 0x7fb06c103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:51.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.189+0000 7fb071a48640 1 -- 172.21.15.44:0/2601174062 >> 172.21.15.44:0/2601174062 conn(0x7fb06c0fd400 msgr2=0x7fb06c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.189+0000 7fb071a48640 1 -- 172.21.15.44:0/2601174062 shutdown_connections 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.189+0000 7fb071a48640 1 -- 172.21.15.44:0/2601174062 wait complete. 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.189+0000 7fb071a48640 1 Processor -- start 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.189+0000 7fb071a48640 1 -- start start 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb071a48640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb06c103870 0x7fb06c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb071a48640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb06c104270 0x7fb06c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb071a48640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c10ce20 0x7fb06c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb071a48640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c112a60 con 0x7fb06c10ce20 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb071a48640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1128e0 con 0x7fb06c103870 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb071a48640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c112be0 con 0x7fb06c104270 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c10ce20 0x7fb06c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c10ce20 0x7fb06c19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44604/0 (socket says 172.21.15.44:44604) 2024-08-20T03:01:51.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb06b7fe640 1 -- 172.21.15.44:0/3632817973 learned_addr learned my addr 172.21.15.44:0/3632817973 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb06a7fc640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb06c104270 0x7fb06c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb06affd640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb06c103870 0x7fb06c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb06b7fe640 1 -- 172.21.15.44:0/3632817973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb06c104270 msgr2=0x7fb06c199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb06b7fe640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb06c104270 0x7fb06c199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb06b7fe640 1 -- 172.21.15.44:0/3632817973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb06c103870 msgr2=0x7fb06c198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb06b7fe640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb06c103870 0x7fb06c198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb06b7fe640 1 -- 172.21.15.44:0/3632817973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c19ef80 con 0x7fb06c10ce20 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb06b7fe640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c10ce20 0x7fb06c19e7b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb06400d750 tx=0x7fb06400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb070a46640 1 -- 172.21.15.44:0/3632817973 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064017070 con 0x7fb06c10ce20 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb070a46640 1 -- 172.21.15.44:0/3632817973 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0640054d0 con 0x7fb06c10ce20 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb071a48640 1 -- 172.21.15.44:0/3632817973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c19f270 con 0x7fb06c10ce20 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.190+0000 7fb070a46640 1 -- 172.21.15.44:0/3632817973 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064004c60 con 0x7fb06c10ce20 2024-08-20T03:01:51.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.191+0000 7fb071a48640 1 -- 172.21.15.44:0/3632817973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c1a5930 con 0x7fb06c10ce20 2024-08-20T03:01:51.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.191+0000 7fb071a48640 1 -- 172.21.15.44:0/3632817973 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb06c103c50 con 0x7fb06c10ce20 2024-08-20T03:01:51.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.193+0000 7fb070a46640 1 -- 172.21.15.44:0/3632817973 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb064009bd0 con 0x7fb06c10ce20 2024-08-20T03:01:51.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.194+0000 7fb070a46640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0580b3d30 0x7fb0580b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:51.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.194+0000 7fb06affd640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0580b3d30 0x7fb0580b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:51.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.194+0000 7fb070a46640 1 -- 172.21.15.44:0/3632817973 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb0640e3be0 con 0x7fb06c10ce20 2024-08-20T03:01:51.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.194+0000 7fb06affd640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0580b3d30 0x7fb0580b61f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb05c007350 tx=0x7fb05c0072e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:51.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.196+0000 7fb070a46640 1 -- 172.21.15.44:0/3632817973 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb0640ab0a0 con 0x7fb06c10ce20 2024-08-20T03:01:51.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:50 smithi165 ceph-mon[22357]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:51.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.393+0000 7fb071a48640 1 -- 172.21.15.44:0/3632817973 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb06c0ff010 con 0x7fb0580b3d30 2024-08-20T03:01:51.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.395+0000 7fb070a46640 1 -- 172.21.15.44:0/3632817973 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb06c0ff010 con 0x7fb0580b3d30 2024-08-20T03:01:51.397 INFO:teuthology.orchestra.run.smithi044.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-08-20T03:01:51.397 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001K400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-20T03:01:51.397 INFO:teuthology.orchestra.run.smithi044.stdout:smithi099 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040101400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-20T03:01:51.397 INFO:teuthology.orchestra.run.smithi044.stdout:smithi165 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CE400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-20T03:01:51.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.398+0000 7fb071a48640 1 -- 172.21.15.44:0/3632817973 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0580b3d30 msgr2=0x7fb0580b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:51.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.398+0000 7fb071a48640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0580b3d30 0x7fb0580b61f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb05c007350 tx=0x7fb05c0072e0 comp rx=0 tx=0).stop 2024-08-20T03:01:51.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.398+0000 7fb071a48640 1 -- 172.21.15.44:0/3632817973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c10ce20 msgr2=0x7fb06c19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:51.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.398+0000 7fb071a48640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c10ce20 0x7fb06c19e7b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb06400d750 tx=0x7fb06400dc10 comp rx=0 tx=0).stop 2024-08-20T03:01:51.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.398+0000 7fb071a48640 1 -- 172.21.15.44:0/3632817973 shutdown_connections 2024-08-20T03:01:51.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.398+0000 7fb071a48640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0580b3d30 0x7fb0580b61f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:51.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.398+0000 7fb071a48640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c10ce20 0x7fb06c19e7b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:51.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.398+0000 7fb071a48640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb06c104270 0x7fb06c199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:51.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.398+0000 7fb071a48640 1 --2- 172.21.15.44:0/3632817973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb06c103870 0x7fb06c198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:51.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.398+0000 7fb071a48640 1 -- 172.21.15.44:0/3632817973 >> 172.21.15.44:0/3632817973 conn(0x7fb06c0fd400 msgr2=0x7fb06c10f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:51.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.398+0000 7fb071a48640 1 -- 172.21.15.44:0/3632817973 shutdown_connections 2024-08-20T03:01:51.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:51.398+0000 7fb071a48640 1 -- 172.21.15.44:0/3632817973 wait complete. 2024-08-20T03:01:52.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:51 smithi044 ceph-mon[18790]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:52.086 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-20T03:01:52.094 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-08-20T03:01:52.095 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- bash -c 'ceph fs dump' 2024-08-20T03:01:52.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:51 smithi099 ceph-mon[21984]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:52.347 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:52.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:51 smithi165 ceph-mon[22357]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:52.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.798+0000 7fb00efaf640 1 -- 172.21.15.44:0/2894312727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0081073e0 msgr2=0x7fb008107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:52.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.798+0000 7fb00efaf640 1 --2- 172.21.15.44:0/2894312727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0081073e0 0x7fb008107820 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faff4009900 tx=0x7faff402a3f0 comp rx=0 tx=0).stop 2024-08-20T03:01:52.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.798+0000 7fb00efaf640 1 -- 172.21.15.44:0/2894312727 shutdown_connections 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.798+0000 7fb00efaf640 1 --2- 172.21.15.44:0/2894312727 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb008107d60 0x7fb00810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.798+0000 7fb00efaf640 1 --2- 172.21.15.44:0/2894312727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0081073e0 0x7fb008107820 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.798+0000 7fb00efaf640 1 --2- 172.21.15.44:0/2894312727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0081019d0 0x7fb008101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.798+0000 7fb00efaf640 1 -- 172.21.15.44:0/2894312727 >> 172.21.15.44:0/2894312727 conn(0x7fb0080fd440 msgr2=0x7fb0080ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.798+0000 7fb00efaf640 1 -- 172.21.15.44:0/2894312727 shutdown_connections 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.798+0000 7fb00efaf640 1 -- 172.21.15.44:0/2894312727 wait complete. 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00efaf640 1 Processor -- start 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00efaf640 1 -- start start 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00efaf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0081019d0 0x7fb008198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00efaf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0081073e0 0x7fb0081994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00efaf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb008107d60 0x7fb00819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00efaf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb008112b40 con 0x7fb008107d60 2024-08-20T03:01:52.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00efaf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0081129c0 con 0x7fb0081019d0 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00efaf640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb008112cc0 con 0x7fb0081073e0 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00d54c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb008107d60 0x7fb00819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00d54c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb008107d60 0x7fb00819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44634/0 (socket says 172.21.15.44:44634) 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00d54c640 1 -- 172.21.15.44:0/229371886 learned_addr learned my addr 172.21.15.44:0/229371886 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb007fff640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0081073e0 0x7fb0081994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00d54c640 1 -- 172.21.15.44:0/229371886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0081073e0 msgr2=0x7fb0081994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00cd4b640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0081019d0 0x7fb008198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00d54c640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0081073e0 0x7fb0081994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00d54c640 1 -- 172.21.15.44:0/229371886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0081019d0 msgr2=0x7fb008198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.799+0000 7fb00d54c640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0081019d0 0x7fb008198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.800+0000 7fb00d54c640 1 -- 172.21.15.44:0/229371886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00819f020 con 0x7fb008107d60 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.800+0000 7fb007fff640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0081073e0 0x7fb0081994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.800+0000 7fb00cd4b640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0081019d0 0x7fb008198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:01:52.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.800+0000 7fb00d54c640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb008107d60 0x7fb00819e850 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb00000cad0 tx=0x7fb00000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:52.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.800+0000 7fb005ffb640 1 -- 172.21.15.44:0/229371886 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00000ae20 con 0x7fb008107d60 2024-08-20T03:01:52.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.800+0000 7fb00efaf640 1 -- 172.21.15.44:0/229371886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00819f310 con 0x7fb008107d60 2024-08-20T03:01:52.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.800+0000 7fb005ffb640 1 -- 172.21.15.44:0/229371886 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb000005020 con 0x7fb008107d60 2024-08-20T03:01:52.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.800+0000 7fb005ffb640 1 -- 172.21.15.44:0/229371886 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00000f6b0 con 0x7fb008107d60 2024-08-20T03:01:52.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.800+0000 7fb00efaf640 1 -- 172.21.15.44:0/229371886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0081a5ba0 con 0x7fb008107d60 2024-08-20T03:01:52.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.801+0000 7fb00efaf640 1 -- 172.21.15.44:0/229371886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb008101db0 con 0x7fb008107d60 2024-08-20T03:01:52.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.802+0000 7fb005ffb640 1 -- 172.21.15.44:0/229371886 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb0000051d0 con 0x7fb008107d60 2024-08-20T03:01:52.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.803+0000 7fb005ffb640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faff80b3cb0 0x7faff80b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:52.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.803+0000 7fb00cd4b640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faff80b3cb0 0x7faff80b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:52.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.803+0000 7fb005ffb640 1 -- 172.21.15.44:0/229371886 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb0000e3f20 con 0x7fb008107d60 2024-08-20T03:01:52.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.804+0000 7fb00cd4b640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faff80b3cb0 0x7faff80b6170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faffc00baa0 tx=0x7faffc005d90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:52.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:52.807+0000 7fb005ffb640 1 -- 172.21.15.44:0/229371886 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb000014030 con 0x7fb008107d60 2024-08-20T03:01:53.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.028+0000 7fb00efaf640 1 -- 172.21.15.44:0/229371886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb008109b40 con 0x7fb008107d60 2024-08-20T03:01:53.031 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:52 smithi044 ceph-mon[18790]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:53.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.028+0000 7fb005ffb640 1 -- 172.21.15.44:0/229371886 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3123 (secure 0 0 0) 0x7fb0000ab360 con 0x7fb008107d60 2024-08-20T03:01:53.031 INFO:teuthology.orchestra.run.smithi044.stdout:e27 2024-08-20T03:01:53.031 INFO:teuthology.orchestra.run.smithi044.stdout:btime 2024-08-20T03:00:30:384117+0000 2024-08-20T03:01:53.031 INFO:teuthology.orchestra.run.smithi044.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-08-20T03:01:53.031 INFO:teuthology.orchestra.run.smithi044.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T03:01:53.032 INFO:teuthology.orchestra.run.smithi044.stdout:legacy client fscid: 1 2024-08-20T03:01:53.032 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:53.032 INFO:teuthology.orchestra.run.smithi044.stdout:Filesystem 'cephfs' (1) 2024-08-20T03:01:53.032 INFO:teuthology.orchestra.run.smithi044.stdout:fs_name cephfs 2024-08-20T03:01:53.032 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 27 2024-08-20T03:01:53.032 INFO:teuthology.orchestra.run.smithi044.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-08-20T03:01:53.032 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-08-20T03:00:18.935328+0000 2024-08-20T03:01:53.032 INFO:teuthology.orchestra.run.smithi044.stdout:modified 2024-08-20T03:00:30.384102+0000 2024-08-20T03:01:53.032 INFO:teuthology.orchestra.run.smithi044.stdout:tableserver 0 2024-08-20T03:01:53.032 INFO:teuthology.orchestra.run.smithi044.stdout:root 0 2024-08-20T03:01:53.032 INFO:teuthology.orchestra.run.smithi044.stdout:session_timeout 300 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:session_autoclose 300 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:max_file_size 1099511627776 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:max_xattr_size 65536 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:required_client_features {} 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:last_failure 0 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:last_failure_osd_epoch 0 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:max_mds 5 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:in 0,1,2,3,4 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:up {0=24604,1=24596,2=24589,3=24583,4=24569} 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:failed 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:damaged 2024-08-20T03:01:53.033 INFO:teuthology.orchestra.run.smithi044.stdout:stopped 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:data_pools [3] 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:metadata_pool 2 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:inline_data disabled 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:balancer 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:bal_rank_mask -1 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:standby_count_wanted 0 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:qdb_cluster leader: 24604 members: 24604,24589,24596,24583,24569 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.i{0:24604} state up:active seq 6 addr [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.e{0:24557} state up:standby-replay seq 1 addr [v2:172.21.15.99:6834/797787628,v1:172.21.15.99:6835/797787628] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b{1:24596} state up:active seq 6 addr [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.l{1:24550} state up:standby-replay seq 1 addr [v2:172.21.15.165:6832/1910345968,v1:172.21.15.165:6833/1910345968] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.c{2:24589} state up:active seq 10 addr [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.k{2:24521} state up:standby-replay seq 1 addr [v2:172.21.15.99:6832/2092228440,v1:172.21.15.99:6833/2092228440] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T03:01:53.034 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.f{3:24583} state up:active seq 11 addr [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T03:01:53.035 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a{3:14808} state up:standby-replay seq 1 addr [v2:172.21.15.44:6840/370123956,v1:172.21.15.44:6841/370123956] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T03:01:53.035 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.h{4:24569} state up:active seq 10 addr [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T03:01:53.035 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.g{4:14796} state up:standby-replay seq 1 addr [v2:172.21.15.44:6838/335802127,v1:172.21.15.44:6839/335802127] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T03:01:53.035 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:53.035 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:53.035 INFO:teuthology.orchestra.run.smithi044.stdout:Standby daemons: 2024-08-20T03:01:53.035 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:53.035 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.j{-1:14742} state up:standby seq 1 addr [v2:172.21.15.44:6834/1875462475,v1:172.21.15.44:6835/1875462475] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T03:01:53.035 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.d{-1:14754} state up:standby seq 1 addr [v2:172.21.15.44:6836/3289736917,v1:172.21.15.44:6837/3289736917] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T03:01:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-08-20T03:01:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.030+0000 7fb00efaf640 1 -- 172.21.15.44:0/229371886 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faff80b3cb0 msgr2=0x7faff80b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.030+0000 7fb00efaf640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faff80b3cb0 0x7faff80b6170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faffc00baa0 tx=0x7faffc005d90 comp rx=0 tx=0).stop 2024-08-20T03:01:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.030+0000 7fb00efaf640 1 -- 172.21.15.44:0/229371886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb008107d60 msgr2=0x7fb00819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.030+0000 7fb00efaf640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb008107d60 0x7fb00819e850 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb00000cad0 tx=0x7fb00000cf90 comp rx=0 tx=0).stop 2024-08-20T03:01:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.031+0000 7fb00efaf640 1 -- 172.21.15.44:0/229371886 shutdown_connections 2024-08-20T03:01:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.031+0000 7fb00efaf640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7faff80b3cb0 0x7faff80b6170 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:53.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.031+0000 7fb00efaf640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb008107d60 0x7fb00819e850 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:53.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.031+0000 7fb00efaf640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0081073e0 0x7fb0081994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:53.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.031+0000 7fb00efaf640 1 --2- 172.21.15.44:0/229371886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0081019d0 0x7fb008198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:53.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.031+0000 7fb00efaf640 1 -- 172.21.15.44:0/229371886 >> 172.21.15.44:0/229371886 conn(0x7fb0080fd440 msgr2=0x7fb0080ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:53.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.031+0000 7fb00efaf640 1 -- 172.21.15.44:0/229371886 shutdown_connections 2024-08-20T03:01:53.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:53.031+0000 7fb00efaf640 1 -- 172.21.15.44:0/229371886 wait complete. 2024-08-20T03:01:53.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:52 smithi099 ceph-mon[21984]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:53.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:52 smithi165 ceph-mon[22357]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:01:53.786 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- bash -c 'ceph osd dump' 2024-08-20T03:01:54.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:53 smithi044 ceph-mon[18790]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-08-20T03:01:54.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:53 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/229371886' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-20T03:01:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:54.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:53 smithi099 ceph-mon[21984]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-08-20T03:01:54.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:53 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/229371886' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-20T03:01:54.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:53 smithi165 ceph-mon[22357]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-08-20T03:01:54.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:53 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/229371886' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-20T03:01:54.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.536+0000 7f1fb94a6640 1 -- 172.21.15.44:0/2445427833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb4108e20 msgr2=0x7f1fb410f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:54.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.536+0000 7f1fb94a6640 1 --2- 172.21.15.44:0/2445427833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb4108e20 0x7f1fb410f620 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009900 tx=0x7f1fac02a410 comp rx=0 tx=0).stop 2024-08-20T03:01:54.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.536+0000 7f1fb94a6640 1 -- 172.21.15.44:0/2445427833 shutdown_connections 2024-08-20T03:01:54.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.536+0000 7f1fb94a6640 1 --2- 172.21.15.44:0/2445427833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb4108e20 0x7f1fb410f620 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:54.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.536+0000 7f1fb94a6640 1 --2- 172.21.15.44:0/2445427833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb41084a0 0x7f1fb41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:54.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.536+0000 7f1fb94a6640 1 --2- 172.21.15.44:0/2445427833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb4102a90 0x7f1fb4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:54.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.536+0000 7f1fb94a6640 1 -- 172.21.15.44:0/2445427833 >> 172.21.15.44:0/2445427833 conn(0x7f1fb40fe540 msgr2=0x7f1fb4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.536+0000 7f1fb94a6640 1 -- 172.21.15.44:0/2445427833 shutdown_connections 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.536+0000 7f1fb94a6640 1 -- 172.21.15.44:0/2445427833 wait complete. 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.536+0000 7f1fb94a6640 1 Processor -- start 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.536+0000 7f1fb94a6640 1 -- start start 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb94a6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb4102a90 0x7f1fb4199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb94a6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb41084a0 0x7f1fb41995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb94a6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb4108e20 0x7f1fb419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb94a6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb4112b20 con 0x7f1fb4102a90 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb94a6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb41129a0 con 0x7f1fb4108e20 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb94a6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb4112ca0 con 0x7f1fb41084a0 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb4102a90 0x7f1fb4199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb4102a90 0x7f1fb4199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44652/0 (socket says 172.21.15.44:44652) 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb2ffd640 1 -- 172.21.15.44:0/220361226 learned_addr learned my addr 172.21.15.44:0/220361226 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:54.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb37fe640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb4108e20 0x7f1fb419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb27fc640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb41084a0 0x7f1fb41995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb2ffd640 1 -- 172.21.15.44:0/220361226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb41084a0 msgr2=0x7f1fb41995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb2ffd640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb41084a0 0x7f1fb41995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb2ffd640 1 -- 172.21.15.44:0/220361226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb4108e20 msgr2=0x7f1fb419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb2ffd640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb4108e20 0x7f1fb419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb2ffd640 1 -- 172.21.15.44:0/220361226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb419f140 con 0x7f1fb4102a90 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb2ffd640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb4102a90 0x7f1fb4199090 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400ba50 tx=0x7f1fa400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fabfff640 1 -- 172.21.15.44:0/220361226 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4019070 con 0x7f1fb4102a90 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fb94a6640 1 -- 172.21.15.44:0/220361226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb419f430 con 0x7f1fb4102a90 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fabfff640 1 -- 172.21.15.44:0/220361226 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fa400cbe0 con 0x7f1fb4102a90 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.537+0000 7f1fabfff640 1 -- 172.21.15.44:0/220361226 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa40075d0 con 0x7f1fb4102a90 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.538+0000 7f1fb94a6640 1 -- 172.21.15.44:0/220361226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb41a5cc0 con 0x7f1fb4102a90 2024-08-20T03:01:54.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.538+0000 7f1fb94a6640 1 -- 172.21.15.44:0/220361226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb4102ef0 con 0x7f1fb4102a90 2024-08-20T03:01:54.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.540+0000 7f1fabfff640 1 -- 172.21.15.44:0/220361226 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f1fa401c070 con 0x7f1fb4102a90 2024-08-20T03:01:54.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.541+0000 7f1fabfff640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1fa00b3cb0 0x7f1fa00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:54.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.541+0000 7f1fb27fc640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1fa00b3cb0 0x7f1fa00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:54.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.541+0000 7f1fabfff640 1 -- 172.21.15.44:0/220361226 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1fa40e38f0 con 0x7f1fb4102a90 2024-08-20T03:01:54.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.541+0000 7f1fb27fc640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1fa00b3cb0 0x7f1fa00b6170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4107210 tx=0x7f1f9c008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:54.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.544+0000 7f1fabfff640 1 -- 172.21.15.44:0/220361226 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f1fa40aae40 con 0x7f1fb4102a90 2024-08-20T03:01:54.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.730+0000 7f1fb94a6640 1 -- 172.21.15.44:0/220361226 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f1fb419a1c0 con 0x7f1fb4102a90 2024-08-20T03:01:54.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.731+0000 7f1fabfff640 1 -- 172.21.15.44:0/220361226 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v77) ==== 56+0+5630 (secure 0 0 0) 0x7f1fb419a1c0 con 0x7f1fb4102a90 2024-08-20T03:01:54.733 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 77 2024-08-20T03:01:54.733 INFO:teuthology.orchestra.run.smithi044.stdout:fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T03:01:54.733 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-08-20T02:52:10.979264+0000 2024-08-20T03:01:54.734 INFO:teuthology.orchestra.run.smithi044.stdout:modified 2024-08-20T03:00:59.484512+0000 2024-08-20T03:01:54.734 INFO:teuthology.orchestra.run.smithi044.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-08-20T03:01:54.734 INFO:teuthology.orchestra.run.smithi044.stdout:crush_version 26 2024-08-20T03:01:54.734 INFO:teuthology.orchestra.run.smithi044.stdout:full_ratio 0.95 2024-08-20T03:01:54.734 INFO:teuthology.orchestra.run.smithi044.stdout:backfillfull_ratio 0.9 2024-08-20T03:01:54.734 INFO:teuthology.orchestra.run.smithi044.stdout:nearfull_ratio 0.85 2024-08-20T03:01:54.734 INFO:teuthology.orchestra.run.smithi044.stdout:require_min_compat_client luminous 2024-08-20T03:01:54.734 INFO:teuthology.orchestra.run.smithi044.stdout:min_compat_client luminous 2024-08-20T03:01:54.734 INFO:teuthology.orchestra.run.smithi044.stdout:require_osd_release squid 2024-08-20T03:01:54.734 INFO:teuthology.orchestra.run.smithi044.stdout:stretch_mode_enabled false 2024-08-20T03:01:54.734 INFO:teuthology.orchestra.run.smithi044.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-08-20T03:01:54.734 INFO:teuthology.orchestra.run.smithi044.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:max_osd 12 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:osd.0 up in weight 1 up_from 9 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6802/858898316,v1:172.21.15.44:6803/858898316] [v2:172.21.15.44:6804/858898316,v1:172.21.15.44:6805/858898316] exists,up a7708f8c-a799-4b8f-ab1e-78482d73ca1e 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:osd.1 up in weight 1 up_from 14 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/4252911663,v1:172.21.15.99:6801/4252911663] [v2:172.21.15.99:6802/4252911663,v1:172.21.15.99:6803/4252911663] exists,up 12f07986-60bf-498f-a012-cd7ea607b3d3 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:osd.2 up in weight 1 up_from 19 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6800/1492740465,v1:172.21.15.165:6801/1492740465] [v2:172.21.15.165:6802/1492740465,v1:172.21.15.165:6803/1492740465] exists,up 0f189000-ba7f-4412-b0d3-75e8265f6685 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:osd.3 up in weight 1 up_from 26 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6810/3423983702,v1:172.21.15.44:6811/3423983702] [v2:172.21.15.44:6812/3423983702,v1:172.21.15.44:6813/3423983702] exists,up 8f9b042c-d6b5-4eee-aa29-ffc41a48c20a 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:osd.4 up in weight 1 up_from 31 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6808/3099617325,v1:172.21.15.99:6809/3099617325] [v2:172.21.15.99:6810/3099617325,v1:172.21.15.99:6811/3099617325] exists,up 4edebfdb-af10-49ee-b724-2a63b9219221 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:osd.5 up in weight 1 up_from 36 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6808/523245255,v1:172.21.15.165:6809/523245255] [v2:172.21.15.165:6810/523245255,v1:172.21.15.165:6811/523245255] exists,up 6d22c21f-cc43-4ef5-a75c-d4fc1a361c06 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:osd.6 up in weight 1 up_from 41 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6818/1039098986,v1:172.21.15.44:6819/1039098986] [v2:172.21.15.44:6820/1039098986,v1:172.21.15.44:6821/1039098986] exists,up 03df0c2f-1b7c-4242-888c-e047322fd8f8 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:osd.7 up in weight 1 up_from 47 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6816/1438613564,v1:172.21.15.99:6817/1438613564] [v2:172.21.15.99:6818/1438613564,v1:172.21.15.99:6819/1438613564] exists,up 84bbc683-9b13-448e-897c-114458dae33b 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:osd.8 up in weight 1 up_from 52 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6816/632957688,v1:172.21.15.165:6817/632957688] [v2:172.21.15.165:6818/632957688,v1:172.21.15.165:6819/632957688] exists,up 14d7ffd3-db05-4228-a12e-4c69ef65ef8f 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:osd.9 up in weight 1 up_from 58 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6826/2426940093,v1:172.21.15.44:6827/2426940093] [v2:172.21.15.44:6828/2426940093,v1:172.21.15.44:6829/2426940093] exists,up 1ae468a5-3d72-4eb0-a1a6-cd87eae740b5 2024-08-20T03:01:54.735 INFO:teuthology.orchestra.run.smithi044.stdout:osd.10 up in weight 1 up_from 63 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6824/3275793038,v1:172.21.15.99:6825/3275793038] [v2:172.21.15.99:6826/3275793038,v1:172.21.15.99:6827/3275793038] exists,up fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:osd.11 up in weight 1 up_from 69 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.165:6824/107688415,v1:172.21.15.165:6825/107688415] [v2:172.21.15.165:6826/107688415,v1:172.21.15.165:6827/107688415] exists,up 50715410-1fc2-4818-b600-604a4b7cce9a 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 2.0 [7,11] 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 2.14 [2,4] 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 2.1d [2,4,8,11] 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.1 [4,9] 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.10 [5,9] 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.1d [4,8] 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:pg_upmap_items 3.2c [4,9] 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/2597827045 expires 2024-08-21T02:53:58.312659+0000 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/3582612619 expires 2024-08-21T02:53:58.312659+0000 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/1916809746 expires 2024-08-21T02:52:33.983380+0000 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/2919033419 expires 2024-08-21T02:52:33.983380+0000 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6801/1293546305 expires 2024-08-21T02:53:58.312659+0000 2024-08-20T03:01:54.736 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6800/1961932124 expires 2024-08-21T02:52:33.983380+0000 2024-08-20T03:01:54.737 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/2331034359 expires 2024-08-21T02:52:33.983380+0000 2024-08-20T03:01:54.737 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6801/1961932124 expires 2024-08-21T02:52:33.983380+0000 2024-08-20T03:01:54.737 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/2572185146 expires 2024-08-21T02:52:58.451601+0000 2024-08-20T03:01:54.737 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6800/1293546305 expires 2024-08-21T02:53:58.312659+0000 2024-08-20T03:01:54.737 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/4176802819 expires 2024-08-21T02:52:58.451601+0000 2024-08-20T03:01:54.737 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6800/3842608703 expires 2024-08-21T02:52:58.451601+0000 2024-08-20T03:01:54.737 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/3654439062 expires 2024-08-21T02:52:58.451601+0000 2024-08-20T03:01:54.737 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:6801/3842608703 expires 2024-08-21T02:52:58.451601+0000 2024-08-20T03:01:54.737 INFO:teuthology.orchestra.run.smithi044.stdout:blocklist 172.21.15.44:0/2376733275 expires 2024-08-21T02:53:58.312659+0000 2024-08-20T03:01:54.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.733+0000 7f1fb94a6640 1 -- 172.21.15.44:0/220361226 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1fa00b3cb0 msgr2=0x7f1fa00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:54.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.733+0000 7f1fb94a6640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1fa00b3cb0 0x7f1fa00b6170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4107210 tx=0x7f1f9c008040 comp rx=0 tx=0).stop 2024-08-20T03:01:54.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.733+0000 7f1fb94a6640 1 -- 172.21.15.44:0/220361226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb4102a90 msgr2=0x7f1fb4199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:54.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.733+0000 7f1fb94a6640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb4102a90 0x7f1fb4199090 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400ba50 tx=0x7f1fa400bf10 comp rx=0 tx=0).stop 2024-08-20T03:01:54.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.734+0000 7f1fb94a6640 1 -- 172.21.15.44:0/220361226 shutdown_connections 2024-08-20T03:01:54.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.734+0000 7f1fb94a6640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1fa00b3cb0 0x7f1fa00b6170 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:54.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.734+0000 7f1fb94a6640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fb4108e20 0x7f1fb419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:54.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.734+0000 7f1fb94a6640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1fb41084a0 0x7f1fb41995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:54.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.734+0000 7f1fb94a6640 1 --2- 172.21.15.44:0/220361226 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb4102a90 0x7f1fb4199090 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:54.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.734+0000 7f1fb94a6640 1 -- 172.21.15.44:0/220361226 >> 172.21.15.44:0/220361226 conn(0x7f1fb40fe540 msgr2=0x7f1fb4100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:54.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.734+0000 7f1fb94a6640 1 -- 172.21.15.44:0/220361226 shutdown_connections 2024-08-20T03:01:54.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:54.734+0000 7f1fb94a6640 1 -- 172.21.15.44:0/220361226 wait complete. 2024-08-20T03:01:55.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:54 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/220361226' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-20T03:01:55.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:54 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/220361226' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-20T03:01:55.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:54 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/220361226' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-20T03:01:55.502 INFO:teuthology.run_tasks:Running task fs.ready... 2024-08-20T03:01:55.513 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:01:55.773 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:56.156 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:55 smithi044 ceph-mon[18790]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:56.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:55 smithi099 ceph-mon[21984]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.236+0000 7f454110f640 1 -- 172.21.15.44:0/4063481287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c101990 msgr2=0x7f453c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.236+0000 7f454110f640 1 --2- 172.21.15.44:0/4063481287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c101990 0x7f453c101d70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f452c009ca0 tx=0x7f452c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.236+0000 7f454110f640 1 -- 172.21.15.44:0/4063481287 shutdown_connections 2024-08-20T03:01:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.236+0000 7f454110f640 1 --2- 172.21.15.44:0/4063481287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f453c107d20 0x7f453c10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.236+0000 7f454110f640 1 --2- 172.21.15.44:0/4063481287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f453c1073a0 0x7f453c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.236+0000 7f454110f640 1 --2- 172.21.15.44:0/4063481287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c101990 0x7f453c101d70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.236+0000 7f454110f640 1 -- 172.21.15.44:0/4063481287 >> 172.21.15.44:0/4063481287 conn(0x7f453c0fd400 msgr2=0x7f453c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.236+0000 7f454110f640 1 -- 172.21.15.44:0/4063481287 shutdown_connections 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.236+0000 7f454110f640 1 -- 172.21.15.44:0/4063481287 wait complete. 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.237+0000 7f454110f640 1 Processor -- start 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.237+0000 7f454110f640 1 -- start start 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f454110f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f453c101990 0x7f453c199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f454110f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f453c1073a0 0x7f453c199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f454110f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c107d20 0x7f453c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f454110f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f453c112b00 con 0x7f453c107d20 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f454110f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f453c112980 con 0x7f453c1073a0 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f454110f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f453c112c80 con 0x7f453c101990 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f453b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c107d20 0x7f453c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f453b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c107d20 0x7f453c19e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44674/0 (socket says 172.21.15.44:44674) 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f453b59e640 1 -- 172.21.15.44:0/3349059536 learned_addr learned my addr 172.21.15.44:0/3349059536 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f453ad9d640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f453c101990 0x7f453c199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f453a59c640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f453c1073a0 0x7f453c199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f453b59e640 1 -- 172.21.15.44:0/3349059536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f453c101990 msgr2=0x7f453c199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f453b59e640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f453c101990 0x7f453c199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f453b59e640 1 -- 172.21.15.44:0/3349059536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f453c1073a0 msgr2=0x7f453c199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f453b59e640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f453c1073a0 0x7f453c199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f453b59e640 1 -- 172.21.15.44:0/3349059536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453c19f0c0 con 0x7f453c107d20 2024-08-20T03:01:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f453b59e640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c107d20 0x7f453c19e8f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f453400d950 tx=0x7f453400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f4533fff640 1 -- 172.21.15.44:0/3349059536 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534017030 con 0x7f453c107d20 2024-08-20T03:01:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f4533fff640 1 -- 172.21.15.44:0/3349059536 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4534004020 con 0x7f453c107d20 2024-08-20T03:01:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f454110f640 1 -- 172.21.15.44:0/3349059536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453c19f3b0 con 0x7f453c107d20 2024-08-20T03:01:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f4533fff640 1 -- 172.21.15.44:0/3349059536 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534003b50 con 0x7f453c107d20 2024-08-20T03:01:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.238+0000 7f454110f640 1 -- 172.21.15.44:0/3349059536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453c1a5a70 con 0x7f453c107d20 2024-08-20T03:01:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.239+0000 7f454110f640 1 -- 172.21.15.44:0/3349059536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453c101d70 con 0x7f453c107d20 2024-08-20T03:01:56.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.241+0000 7f4533fff640 1 -- 172.21.15.44:0/3349059536 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f453401a070 con 0x7f453c107d20 2024-08-20T03:01:56.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.241+0000 7f4533fff640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f45280b3cb0 0x7f45280b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:56.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.241+0000 7f453ad9d640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f45280b3cb0 0x7f45280b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:56.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.242+0000 7f4533fff640 1 -- 172.21.15.44:0/3349059536 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f45340e3940 con 0x7f453c107d20 2024-08-20T03:01:56.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.242+0000 7f453ad9d640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f45280b3cb0 0x7f45280b6170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f452c009430 tx=0x7f452c009360 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:56.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.244+0000 7f4533fff640 1 -- 172.21.15.44:0/3349059536 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4534021070 con 0x7f453c107d20 2024-08-20T03:01:56.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:55 smithi165 ceph-mon[22357]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:01:56.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.465+0000 7f454110f640 1 -- 172.21.15.44:0/3349059536 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f453c02e340 con 0x7f453c107d20 2024-08-20T03:01:56.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.466+0000 7f4533fff640 1 -- 172.21.15.44:0/3349059536 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11965 (secure 0 0 0) 0x7f45340aad80 con 0x7f453c107d20 2024-08-20T03:01:56.468 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:56.468 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"btime":"2024-08-20T03:00:30:384117+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:00:30.384102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24604,24589,24596,24583,24569]},"id":1}]} 2024-08-20T03:01:56.469 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-08-20T03:01:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.468+0000 7f454110f640 1 -- 172.21.15.44:0/3349059536 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f45280b3cb0 msgr2=0x7f45280b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.468+0000 7f454110f640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f45280b3cb0 0x7f45280b6170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f452c009430 tx=0x7f452c009360 comp rx=0 tx=0).stop 2024-08-20T03:01:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.468+0000 7f454110f640 1 -- 172.21.15.44:0/3349059536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c107d20 msgr2=0x7f453c19e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.468+0000 7f454110f640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c107d20 0x7f453c19e8f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f453400d950 tx=0x7f453400de10 comp rx=0 tx=0).stop 2024-08-20T03:01:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.468+0000 7f454110f640 1 -- 172.21.15.44:0/3349059536 shutdown_connections 2024-08-20T03:01:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.468+0000 7f454110f640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f45280b3cb0 0x7f45280b6170 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.468+0000 7f454110f640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f453c107d20 0x7f453c19e8f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.468+0000 7f454110f640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f453c1073a0 0x7f453c199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.468+0000 7f454110f640 1 --2- 172.21.15.44:0/3349059536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f453c101990 0x7f453c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.468+0000 7f454110f640 1 -- 172.21.15.44:0/3349059536 >> 172.21.15.44:0/3349059536 conn(0x7f453c0fd400 msgr2=0x7f453c0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.468+0000 7f454110f640 1 -- 172.21.15.44:0/3349059536 shutdown_connections 2024-08-20T03:01:56.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:56.469+0000 7f454110f640 1 -- 172.21.15.44:0/3349059536 wait complete. 2024-08-20T03:01:57.104 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:56 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3349059536' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:57.136 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:01:57.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:56 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3349059536' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:57.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:56 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3349059536' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:57.392 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:57.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.895+0000 7f8861118640 1 -- 172.21.15.44:0/2742052790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f885c1039e0 msgr2=0x7f885c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:57.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.895+0000 7f8861118640 1 --2- 172.21.15.44:0/2742052790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f885c1039e0 0x7f885c103dc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f884c009ca0 tx=0x7f884c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:01:57.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.895+0000 7f8861118640 1 -- 172.21.15.44:0/2742052790 shutdown_connections 2024-08-20T03:01:57.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.895+0000 7f8861118640 1 --2- 172.21.15.44:0/2742052790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f885c10ce40 0x7f885c10f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:57.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.895+0000 7f8861118640 1 --2- 172.21.15.44:0/2742052790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f885c104300 0x7f885c10c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:57.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.895+0000 7f8861118640 1 --2- 172.21.15.44:0/2742052790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f885c1039e0 0x7f885c103dc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:57.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.895+0000 7f8861118640 1 -- 172.21.15.44:0/2742052790 >> 172.21.15.44:0/2742052790 conn(0x7f885c0fd440 msgr2=0x7f885c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:57.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.895+0000 7f8861118640 1 -- 172.21.15.44:0/2742052790 shutdown_connections 2024-08-20T03:01:57.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.895+0000 7f8861118640 1 -- 172.21.15.44:0/2742052790 wait complete. 2024-08-20T03:01:57.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.896+0000 7f8861118640 1 Processor -- start 2024-08-20T03:01:57.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.896+0000 7f8861118640 1 -- start start 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f8861118640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f885c1039e0 0x7f885c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f8861118640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f885c104300 0x7f885c19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f8861118640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f885c10ce40 0x7f885c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f8861118640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f885c112ab0 con 0x7f885c1039e0 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f8861118640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f885c112930 con 0x7f885c104300 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f8861118640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f885c112c30 con 0x7f885c10ce40 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f885ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f885c1039e0 0x7f885c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f885ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f885c1039e0 0x7f885c19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53332/0 (socket says 172.21.15.44:53332) 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f885ad9d640 1 -- 172.21.15.44:0/1185585441 learned_addr learned my addr 172.21.15.44:0/1185585441 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f885b59e640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f885c10ce40 0x7f885c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f885a59c640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f885c104300 0x7f885c19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f885ad9d640 1 -- 172.21.15.44:0/1185585441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f885c10ce40 msgr2=0x7f885c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f885ad9d640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f885c10ce40 0x7f885c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:57.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f885ad9d640 1 -- 172.21.15.44:0/1185585441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f885c104300 msgr2=0x7f885c19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f885ad9d640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f885c104300 0x7f885c19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f885ad9d640 1 -- 172.21.15.44:0/1185585441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c1a1330 con 0x7f885c1039e0 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f885ad9d640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f885c1039e0 0x7f885c19b280 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f884c006010 tx=0x7f884c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.897+0000 7f8843fff640 1 -- 172.21.15.44:0/1185585441 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c003bf0 con 0x7f885c1039e0 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.898+0000 7f8843fff640 1 -- 172.21.15.44:0/1185585441 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f884c003d90 con 0x7f885c1039e0 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.898+0000 7f8861118640 1 -- 172.21.15.44:0/1185585441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885c1a15c0 con 0x7f885c1039e0 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.898+0000 7f8843fff640 1 -- 172.21.15.44:0/1185585441 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884c02fc20 con 0x7f885c1039e0 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.898+0000 7f8861118640 1 -- 172.21.15.44:0/1185585441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885c1a7ce0 con 0x7f885c1039e0 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.899+0000 7f8861118640 1 -- 172.21.15.44:0/1185585441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f885c103dc0 con 0x7f885c1039e0 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.901+0000 7f8843fff640 1 -- 172.21.15.44:0/1185585441 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7f884c0046a0 con 0x7f885c1039e0 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.901+0000 7f8843fff640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f88440b3d30 0x7f88440b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.901+0000 7f885a59c640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f88440b3d30 0x7f88440b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.901+0000 7f8843fff640 1 -- 172.21.15.44:0/1185585441 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f884c105390 con 0x7f885c1039e0 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.902+0000 7f885a59c640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f88440b3d30 0x7f88440b61f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f885c10e4c0 tx=0x7f885000b2e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:57.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:57.904+0000 7f8843fff640 1 -- 172.21.15.44:0/1185585441 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f884c0cc8e0 con 0x7f885c1039e0 2024-08-20T03:01:58.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.125+0000 7f8861118640 1 -- 172.21.15.44:0/1185585441 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f885c19c3b0 con 0x7f885c1039e0 2024-08-20T03:01:58.127 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:57 smithi044 ceph-mon[18790]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:58.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.125+0000 7f8843fff640 1 -- 172.21.15.44:0/1185585441 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11965 (secure 0 0 0) 0x7f885c19c3b0 con 0x7f885c1039e0 2024-08-20T03:01:58.128 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:58.128 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":27,"btime":"2024-08-20T03:00:30:384117+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:00:30.384102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24604,24589,24596,24583,24569]},"id":1}]} 2024-08-20T03:01:58.128 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 27 2024-08-20T03:01:58.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.128+0000 7f8861118640 1 -- 172.21.15.44:0/1185585441 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f88440b3d30 msgr2=0x7f88440b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:58.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.128+0000 7f8861118640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f88440b3d30 0x7f88440b61f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f885c10e4c0 tx=0x7f885000b2e0 comp rx=0 tx=0).stop 2024-08-20T03:01:58.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.128+0000 7f8861118640 1 -- 172.21.15.44:0/1185585441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f885c1039e0 msgr2=0x7f885c19b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:58.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.128+0000 7f8861118640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f885c1039e0 0x7f885c19b280 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f884c006010 tx=0x7f884c0319f0 comp rx=0 tx=0).stop 2024-08-20T03:01:58.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.128+0000 7f8861118640 1 -- 172.21.15.44:0/1185585441 shutdown_connections 2024-08-20T03:01:58.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.128+0000 7f8861118640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f88440b3d30 0x7f88440b61f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:58.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.128+0000 7f8861118640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f885c10ce40 0x7f885c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:58.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.128+0000 7f8861118640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f885c104300 0x7f885c19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:58.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.128+0000 7f8861118640 1 --2- 172.21.15.44:0/1185585441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f885c1039e0 0x7f885c19b280 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:58.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.128+0000 7f8861118640 1 -- 172.21.15.44:0/1185585441 >> 172.21.15.44:0/1185585441 conn(0x7f885c0fd440 msgr2=0x7f885c10f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:58.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.129+0000 7f8861118640 1 -- 172.21.15.44:0/1185585441 shutdown_connections 2024-08-20T03:01:58.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:58.129+0000 7f8861118640 1 -- 172.21.15.44:0/1185585441 wait complete. 2024-08-20T03:01:58.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:57 smithi165 ceph-mon[22357]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:58.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:57 smithi099 ceph-mon[21984]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:01:58.945 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T03:01:59.196 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:01:59.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:58 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1185585441' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:59.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:58 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1185585441' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:59.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:58 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1185585441' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:01:59.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.664+0000 7fb785c93640 1 -- 172.21.15.44:0/1396772431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801039b0 msgr2=0x7fb780103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:59.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.664+0000 7fb785c93640 1 --2- 172.21.15.44:0/1396772431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801039b0 0x7fb780103d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb770009900 tx=0x7fb77002a410 comp rx=0 tx=0).stop 2024-08-20T03:01:59.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.664+0000 7fb785c93640 1 -- 172.21.15.44:0/1396772431 shutdown_connections 2024-08-20T03:01:59.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.664+0000 7fb785c93640 1 --2- 172.21.15.44:0/1396772431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb78010ce10 0x7fb78010f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:59.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.664+0000 7fb785c93640 1 --2- 172.21.15.44:0/1396772431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7801042d0 0x7fb78010c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:59.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.664+0000 7fb785c93640 1 --2- 172.21.15.44:0/1396772431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801039b0 0x7fb780103d90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:59.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.664+0000 7fb785c93640 1 -- 172.21.15.44:0/1396772431 >> 172.21.15.44:0/1396772431 conn(0x7fb7800fd3f0 msgr2=0x7fb7800ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:59.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.664+0000 7fb785c93640 1 -- 172.21.15.44:0/1396772431 shutdown_connections 2024-08-20T03:01:59.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.664+0000 7fb785c93640 1 -- 172.21.15.44:0/1396772431 wait complete. 2024-08-20T03:01:59.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.664+0000 7fb785c93640 1 Processor -- start 2024-08-20T03:01:59.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb785c93640 1 -- start start 2024-08-20T03:01:59.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb785c93640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801039b0 0x7fb780198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:59.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb785c93640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7801042d0 0x7fb7801994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:59.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb785c93640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb78010ce10 0x7fb78019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:59.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb785c93640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb780112a80 con 0x7fb7801039b0 2024-08-20T03:01:59.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb785c93640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb780112900 con 0x7fb78010ce10 2024-08-20T03:01:59.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb785c93640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb780112c00 con 0x7fb7801042d0 2024-08-20T03:01:59.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb77f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801039b0 0x7fb780198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:59.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb77f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801039b0 0x7fb780198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53356/0 (socket says 172.21.15.44:53356) 2024-08-20T03:01:59.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb77f7fe640 1 -- 172.21.15.44:0/4077230553 learned_addr learned my addr 172.21.15.44:0/4077230553 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:01:59.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb77effd640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7801042d0 0x7fb7801994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:59.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb77f7fe640 1 -- 172.21.15.44:0/4077230553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7801042d0 msgr2=0x7fb7801994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:59.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb77f7fe640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7801042d0 0x7fb7801994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb77ffff640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb78010ce10 0x7fb78019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb77f7fe640 1 -- 172.21.15.44:0/4077230553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb78010ce10 msgr2=0x7fb78019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb77f7fe640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb78010ce10 0x7fb78019e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.665+0000 7fb77f7fe640 1 -- 172.21.15.44:0/4077230553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78019f020 con 0x7fb7801039b0 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.666+0000 7fb77f7fe640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801039b0 0x7fb780198f70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb770005940 tx=0x7fb7700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.666+0000 7fb77cff9640 1 -- 172.21.15.44:0/4077230553 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77003f030 con 0x7fb7801039b0 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.666+0000 7fb77cff9640 1 -- 172.21.15.44:0/4077230553 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb770003cb0 con 0x7fb7801039b0 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.666+0000 7fb785c93640 1 -- 172.21.15.44:0/4077230553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78019f2b0 con 0x7fb7801039b0 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.666+0000 7fb77cff9640 1 -- 172.21.15.44:0/4077230553 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770035070 con 0x7fb7801039b0 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.666+0000 7fb785c93640 1 -- 172.21.15.44:0/4077230553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7801a59d0 con 0x7fb7801039b0 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.667+0000 7fb785c93640 1 -- 172.21.15.44:0/4077230553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb780103d90 con 0x7fb7801039b0 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.668+0000 7fb77cff9640 1 -- 172.21.15.44:0/4077230553 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 21) ==== 151127+0+0 (secure 0 0 0) 0x7fb770042030 con 0x7fb7801039b0 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.669+0000 7fb77cff9640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb7680b3cb0 0x7fb7680b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:01:59.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.669+0000 7fb77effd640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb7680b3cb0 0x7fb7680b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:01:59.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.669+0000 7fb77cff9640 1 -- 172.21.15.44:0/4077230553 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb770104810 con 0x7fb7801039b0 2024-08-20T03:01:59.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.669+0000 7fb77effd640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb7680b3cb0 0x7fb7680b6170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb774005db0 tx=0x7fb774005d40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:01:59.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.673+0000 7fb77cff9640 1 -- 172.21.15.44:0/4077230553 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb7700cbc50 con 0x7fb7801039b0 2024-08-20T03:01:59.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.857+0000 7fb785c93640 1 -- 172.21.15.44:0/4077230553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7801a5d60 con 0x7fb7801039b0 2024-08-20T03:01:59.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.859+0000 7fb77cff9640 1 -- 172.21.15.44:0/4077230553 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22954 (secure 0 0 0) 0x7fb7700cb550 con 0x7fb7801039b0 2024-08-20T03:01:59.861 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:01:59.861 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":77,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","created":"2024-08-20T02:52:10.979264+0000","modified":"2024-08-20T03:00:59.484512+0000","last_up_change":"2024-08-20T02:59:26.404917+0000","last_in_change":"2024-08-20T02:59:03.707380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T02:55:46.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T03:00:14.354568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T03:00:16.869335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7708f8c-a799-4b8f-ab1e-78482d73ca1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6803","nonce":858898316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6805","nonce":858898316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6809","nonce":858898316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6807","nonce":858898316}]},"public_addr":"172.21.15.44:6803/858898316","cluster_addr":"172.21.15.44:6805/858898316","heartbeat_back_addr":"172.21.15.44:6809/858898316","heartbeat_front_addr":"172.21.15.44:6807/858898316","state":["exists","up"]},{"osd":1,"uuid":"12f07986-60bf-498f-a012-cd7ea607b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6801","nonce":4252911663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6803","nonce":4252911663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6807","nonce":4252911663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6805","nonce":4252911663}]},"public_addr":"172.21.15.99:6801/4252911663","cluster_addr":"172.21.15.99:6803/4252911663","heartbeat_back_addr":"172.21.15.99:6807/4252911663","heartbeat_front_addr":"172.21.15.99:6805/4252911663","state":["exists","up"]},{"osd":2,"uuid":"0f189000-ba7f-4412-b0d3-75e8265f6685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6801","nonce":1492740465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6803","nonce":1492740465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6807","nonce":1492740465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6805","nonce":1492740465}]},"public_addr":"172.21.15.165:6801/1492740465","cluster_addr":"172.21.15.165:6803/1492740465","heartbeat_back_addr":"172.21.15.165:6807/1492740465","heartbeat_front_addr":"172.21.15.165:6805/1492740465","state":["exists","up"]},{"osd":3,"uuid":"8f9b042c-d6b5-4eee-aa29-ffc41a48c20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6811","nonce":3423983702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6813","nonce":3423983702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6817","nonce":3423983702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6815","nonce":3423983702}]},"public_addr":"172.21.15.44:6811/3423983702","cluster_addr":"172.21.15.44:6813/3423983702","heartbeat_back_addr":"172.21.15.44:6817/3423983702","heartbeat_front_addr":"172.21.15.44:6815/3423983702","state":["exists","up"]},{"osd":4,"uuid":"4edebfdb-af10-49ee-b724-2a63b9219221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6809","nonce":3099617325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6811","nonce":3099617325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6815","nonce":3099617325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6813","nonce":3099617325}]},"public_addr":"172.21.15.99:6809/3099617325","cluster_addr":"172.21.15.99:6811/3099617325","heartbeat_back_addr":"172.21.15.99:6815/3099617325","heartbeat_front_addr":"172.21.15.99:6813/3099617325","state":["exists","up"]},{"osd":5,"uuid":"6d22c21f-cc43-4ef5-a75c-d4fc1a361c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6809","nonce":523245255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6811","nonce":523245255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6815","nonce":523245255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6813","nonce":523245255}]},"public_addr":"172.21.15.165:6809/523245255","cluster_addr":"172.21.15.165:6811/523245255","heartbeat_back_addr":"172.21.15.165:6815/523245255","heartbeat_front_addr":"172.21.15.165:6813/523245255","state":["exists","up"]},{"osd":6,"uuid":"03df0c2f-1b7c-4242-888c-e047322fd8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6819","nonce":1039098986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6821","nonce":1039098986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6825","nonce":1039098986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6823","nonce":1039098986}]},"public_addr":"172.21.15.44:6819/1039098986","cluster_addr":"172.21.15.44:6821/1039098986","heartbeat_back_addr":"172.21.15.44:6825/1039098986","heartbeat_front_addr":"172.21.15.44:6823/1039098986","state":["exists","up"]},{"osd":7,"uuid":"84bbc683-9b13-448e-897c-114458dae33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6817","nonce":1438613564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6819","nonce":1438613564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6823","nonce":1438613564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6821","nonce":1438613564}]},"public_addr":"172.21.15.99:6817/1438613564","cluster_addr":"172.21.15.99:6819/1438613564","heartbeat_back_addr":"172.21.15.99:6823/1438613564","heartbeat_front_addr":"172.21.15.99:6821/1438613564","state":["exists","up"]},{"osd":8,"uuid":"14d7ffd3-db05-4228-a12e-4c69ef65ef8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6817","nonce":632957688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6819","nonce":632957688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6823","nonce":632957688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6821","nonce":632957688}]},"public_addr":"172.21.15.165:6817/632957688","cluster_addr":"172.21.15.165:6819/632957688","heartbeat_back_addr":"172.21.15.165:6823/632957688","heartbeat_front_addr":"172.21.15.165:6821/632957688","state":["exists","up"]},{"osd":9,"uuid":"1ae468a5-3d72-4eb0-a1a6-cd87eae740b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6827","nonce":2426940093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6829","nonce":2426940093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6833","nonce":2426940093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6831","nonce":2426940093}]},"public_addr":"172.21.15.44:6827/2426940093","cluster_addr":"172.21.15.44:6829/2426940093","heartbeat_back_addr":"172.21.15.44:6833/2426940093","heartbeat_front_addr":"172.21.15.44:6831/2426940093","state":["exists","up"]},{"osd":10,"uuid":"fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6825","nonce":3275793038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6827","nonce":3275793038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6831","nonce":3275793038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6829","nonce":3275793038}]},"public_addr":"172.21.15.99:6825/3275793038","cluster_addr":"172.21.15.99:6827/3275793038","heartbeat_back_addr":"172.21.15.99:6831/3275793038","heartbeat_front_addr":"172.21.15.99:6829/3275793038","state":["exists","up"]},{"osd":11,"uuid":"50715410-1fc2-4818-b600-604a4b7cce9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6825","nonce":107688415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6827","nonce":107688415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6831","nonce":107688415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6829","nonce":107688415}]},"public_addr":"172.21.15.165:6825/107688415","cluster_addr":"172.21.15.165:6827/107688415","heartbeat_back_addr":"172.21.15.165:6831/107688415","heartbeat_front_addr":"172.21.15.165:6829/107688415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:54:56.697075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:20.486846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:43.353376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:08.333879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:32.186492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:55.891366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:19.681421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:45.475926+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:10.109795+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:34.444169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:59.147646+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:59:25.403071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2597827045":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/3582612619":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/1916809746":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2919033419":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:6800/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2331034359":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2572185146":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/4176802819":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/3654439062":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6801/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/2376733275":"2024-08-21T02:53:58.312659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T03:01:59.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.861+0000 7fb785c93640 1 -- 172.21.15.44:0/4077230553 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb7680b3cb0 msgr2=0x7fb7680b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:59.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.861+0000 7fb785c93640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb7680b3cb0 0x7fb7680b6170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb774005db0 tx=0x7fb774005d40 comp rx=0 tx=0).stop 2024-08-20T03:01:59.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.861+0000 7fb785c93640 1 -- 172.21.15.44:0/4077230553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801039b0 msgr2=0x7fb780198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:01:59.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.861+0000 7fb785c93640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801039b0 0x7fb780198f70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb770005940 tx=0x7fb7700042f0 comp rx=0 tx=0).stop 2024-08-20T03:01:59.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.861+0000 7fb785c93640 1 -- 172.21.15.44:0/4077230553 shutdown_connections 2024-08-20T03:01:59.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.861+0000 7fb785c93640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb7680b3cb0 0x7fb7680b6170 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:59.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.861+0000 7fb785c93640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb78010ce10 0x7fb78019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:59.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.861+0000 7fb785c93640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7801042d0 0x7fb7801994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:59.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.861+0000 7fb785c93640 1 --2- 172.21.15.44:0/4077230553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7801039b0 0x7fb780198f70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:01:59.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.861+0000 7fb785c93640 1 -- 172.21.15.44:0/4077230553 >> 172.21.15.44:0/4077230553 conn(0x7fb7800fd3f0 msgr2=0x7fb7800fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:01:59.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.861+0000 7fb785c93640 1 -- 172.21.15.44:0/4077230553 shutdown_connections 2024-08-20T03:01:59.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:01:59.861+0000 7fb785c93640 1 -- 172.21.15.44:0/4077230553 wait complete. 2024-08-20T03:02:00.249 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:59 smithi044 ceph-mon[18790]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:02:00.250 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:01:59 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/4077230553' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:02:00.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:59 smithi165 ceph-mon[22357]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:02:00.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:01:59 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/4077230553' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:02:00.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:59 smithi099 ceph-mon[21984]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:02:00.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:01:59 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/4077230553' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:02:00.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-20T03:00:18.935328+0000', 'modified': '2024-08-20T03:00:30.384102+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24604, 'mds_1': 24596, 'mds_2': 24589, 'mds_3': 24583, 'mds_4': 24569}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14796': {'gid': 14796, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6839/335802127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 335802127}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 335802127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14808': {'gid': 14808, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6841/370123956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6840', 'nonce': 370123956}, {'type': 'v1', 'addr': '172.21.15.44:6841', 'nonce': 370123956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6833/2092228440', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6832', 'nonce': 2092228440}, {'type': 'v1', 'addr': '172.21.15.99:6833', 'nonce': 2092228440}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24550': {'gid': 24550, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6833/1910345968', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6832', 'nonce': 1910345968}, {'type': 'v1', 'addr': '172.21.15.165:6833', 'nonce': 1910345968}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24557': {'gid': 24557, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/797787628', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 797787628}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 797787628}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.99:6837/728486889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 728486889}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 728486889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24583': {'gid': 24583, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.165:6835/3019941967', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 3019941967}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 3019941967}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24589': {'gid': 24589, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.165:6837/3478633280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 3478633280}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 3478633280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.99:6839/1272675547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 1272675547}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 1272675547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24604': {'gid': 24604, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.165:6839/3029973590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6838', 'nonce': 3029973590}, {'type': 'v1', 'addr': '172.21.15.165:6839', 'nonce': 3029973590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24604, 'qdb_cluster': [24604, 24589, 24596, 24583, 24569]} 2024-08-20T03:02:00.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-08-20T03:02:00.572 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-20T03:02:00.826 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.292+0000 7f7e36155640 1 -- 172.21.15.44:0/2724762739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e301039e0 msgr2=0x7f7e30103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.292+0000 7f7e36155640 1 --2- 172.21.15.44:0/2724762739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e301039e0 0x7f7e30103dc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009900 tx=0x7f7e1802a410 comp rx=0 tx=0).stop 2024-08-20T03:02:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.292+0000 7f7e36155640 1 -- 172.21.15.44:0/2724762739 shutdown_connections 2024-08-20T03:02:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.292+0000 7f7e36155640 1 --2- 172.21.15.44:0/2724762739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e3010ce40 0x7f7e3010f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.292+0000 7f7e36155640 1 --2- 172.21.15.44:0/2724762739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e30104300 0x7f7e3010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.292+0000 7f7e36155640 1 --2- 172.21.15.44:0/2724762739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e301039e0 0x7f7e30103dc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.292+0000 7f7e36155640 1 -- 172.21.15.44:0/2724762739 >> 172.21.15.44:0/2724762739 conn(0x7f7e300fd440 msgr2=0x7f7e300ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:01.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.292+0000 7f7e36155640 1 -- 172.21.15.44:0/2724762739 shutdown_connections 2024-08-20T03:02:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.292+0000 7f7e36155640 1 -- 172.21.15.44:0/2724762739 wait complete. 2024-08-20T03:02:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e36155640 1 Processor -- start 2024-08-20T03:02:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e36155640 1 -- start start 2024-08-20T03:02:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e36155640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e301039e0 0x7f7e30198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e36155640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e30104300 0x7f7e30199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e36155640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e3010ce40 0x7f7e3019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e36155640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7e30112ab0 con 0x7f7e301039e0 2024-08-20T03:02:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e36155640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7e30112930 con 0x7f7e3010ce40 2024-08-20T03:02:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e36155640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7e30112c30 con 0x7f7e30104300 2024-08-20T03:02:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e2f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e301039e0 0x7f7e30198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e2f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e301039e0 0x7f7e30198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53382/0 (socket says 172.21.15.44:53382) 2024-08-20T03:02:01.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e2f7fe640 1 -- 172.21.15.44:0/3760379187 learned_addr learned my addr 172.21.15.44:0/3760379187 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e2effd640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e30104300 0x7f7e30199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.293+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e3010ce40 0x7f7e3019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.294+0000 7f7e2f7fe640 1 -- 172.21.15.44:0/3760379187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e30104300 msgr2=0x7f7e30199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.294+0000 7f7e2f7fe640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e30104300 0x7f7e30199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.294+0000 7f7e2f7fe640 1 -- 172.21.15.44:0/3760379187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e3010ce40 msgr2=0x7f7e3019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.294+0000 7f7e2f7fe640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e3010ce40 0x7f7e3019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.294+0000 7f7e2f7fe640 1 -- 172.21.15.44:0/3760379187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3019f070 con 0x7f7e301039e0 2024-08-20T03:02:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.294+0000 7f7e2f7fe640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e301039e0 0x7f7e30198fc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e18005940 tx=0x7f7e180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.294+0000 7f7e2cff9640 1 -- 172.21.15.44:0/3760379187 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1803f030 con 0x7f7e301039e0 2024-08-20T03:02:01.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.294+0000 7f7e36155640 1 -- 172.21.15.44:0/3760379187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3019f300 con 0x7f7e301039e0 2024-08-20T03:02:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.294+0000 7f7e2cff9640 1 -- 172.21.15.44:0/3760379187 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e18003cb0 con 0x7f7e301039e0 2024-08-20T03:02:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.294+0000 7f7e36155640 1 -- 172.21.15.44:0/3760379187 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e301a5a20 con 0x7f7e301039e0 2024-08-20T03:02:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.294+0000 7f7e2cff9640 1 -- 172.21.15.44:0/3760379187 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18035070 con 0x7f7e301039e0 2024-08-20T03:02:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e3854e740 1 Processor -- start 2024-08-20T03:02:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e3854e740 1 -- start start 2024-08-20T03:02:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e3854e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e3854e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b8510 0x55bf7c2c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e3854e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2c6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e3854e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf7c22e6a0 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e3854e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bf7c21dba0 con 0x55bf7c2b8510 2024-08-20T03:02:01.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e3854e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bf7c221be0 con 0x55bf7c2c44a0 2024-08-20T03:02:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e2ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e2ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2b7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53396/0 (socket says 172.21.15.44:53396) 2024-08-20T03:02:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e2ffff640 1 -- 172.21.15.44:0/3452116399 learned_addr learned my addr 172.21.15.44:0/3452116399 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e2effd640 1 --2- 172.21.15.44:0/3452116399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.297+0000 7f7e2ffff640 1 -- 172.21.15.44:0/3452116399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2c44a0 msgr2=0x55bf7c2c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.298+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/3452116399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.298+0000 7f7e2ffff640 1 -- 172.21.15.44:0/3452116399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b8510 msgr2=0x55bf7c2c3f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:02:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.298+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/3452116399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b8510 0x55bf7c2c3f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.298+0000 7f7e2ffff640 1 -- 172.21.15.44:0/3452116399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf7bf7d540 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.298+0000 7f7e2effd640 1 --2- 172.21.15.44:0/3452116399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.298+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/3452116399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2b7fd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7e28003a20 tx=0x7f7e28003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93db8ea617ae29c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.298+0000 7f7e2cff9640 1 -- 172.21.15.44:0/3760379187 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f7e18042030 con 0x7f7e301039e0 2024-08-20T03:02:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.298+0000 7f7e25ffb640 1 -- 172.21.15.44:0/3452116399 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2801a470 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.298+0000 7f7e25ffb640 1 -- 172.21.15.44:0/3452116399 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e28015070 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.298+0000 7f7e25ffb640 1 -- 172.21.15.44:0/3452116399 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e280107e0 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e2cff9640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7e1c0b3c20 0x7f7e1c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e2f7fe640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7e1c0b3c20 0x7f7e1c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e3854e740 1 -- 172.21.15.44:0/3452116399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 msgr2=0x55bf7c2b7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e3854e740 1 --2- 172.21.15.44:0/3452116399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2b7fd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7e28003a20 tx=0x7f7e28003ee0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e2cff9640 1 -- 172.21.15.44:0/3760379187 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7e18104150 con 0x7f7e301039e0 2024-08-20T03:02:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e3854e740 1 -- 172.21.15.44:0/3452116399 shutdown_connections 2024-08-20T03:02:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e3854e740 1 --2- 172.21.15.44:0/3452116399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e3854e740 1 --2- 172.21.15.44:0/3452116399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b8510 0x55bf7c2c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e3854e740 1 --2- 172.21.15.44:0/3452116399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2b7fd0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e3854e740 1 -- 172.21.15.44:0/3452116399 >> 172.21.15.44:0/3452116399 conn(0x55bf7c2b5a20 msgr2=0x55bf7c2b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e3854e740 1 -- 172.21.15.44:0/3452116399 shutdown_connections 2024-08-20T03:02:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e3854e740 1 -- 172.21.15.44:0/3452116399 wait complete. 2024-08-20T03:02:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e2f7fe640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7e1c0b3c20 0x7f7e1c0b60e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7e180cada0 tx=0x7f7e180facd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.299+0000 7f7e3854e740 1 -- start start 2024-08-20T03:02:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e3854e740 1 Processor -- start 2024-08-20T03:02:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e3854e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2c1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e3854e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b8510 0x55bf7c2c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e3854e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2c1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e3854e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf7c0805f0 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e3854e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bf7c1a46c0 con 0x55bf7c2b8510 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e3854e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bf7c2c7150 con 0x55bf7c2c44a0 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e3854e740 10 client.0 _ll_register_callbacks cb 0x55bf7c1e11e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e2ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e2ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2c1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53406/0 (socket says 172.21.15.44:53406) 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e2ffff640 1 -- 172.21.15.44:0/1308879125 learned_addr learned my addr 172.21.15.44:0/1308879125 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2c1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e2effd640 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b8510 0x55bf7c2c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e2ffff640 1 -- 172.21.15.44:0/1308879125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2c44a0 msgr2=0x55bf7c2c1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2c1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e2ffff640 1 -- 172.21.15.44:0/1308879125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b8510 msgr2=0x55bf7c2c19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b8510 0x55bf7c2c19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e2ffff640 1 -- 172.21.15.44:0/1308879125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf7c2d0590 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.300+0000 7f7e2effd640 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b8510 0x55bf7c2c19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2c1470 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009dd0 tx=0x7f7e28003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee290be1d932a27 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e0effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e0effd640 1 -- 172.21.15.44:0/1308879125 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28011a30 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e0effd640 1 -- 172.21.15.44:0/1308879125 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7e28004a40 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e3854e740 1 -- 172.21.15.44:0/1308879125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e28015070 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e0effd640 1 -- 172.21.15.44:0/1308879125 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e28030050 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e3854e740 10 client.15318 fetch_fsmap learned FSMap version 27 2024-08-20T03:02:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e3854e740 1 -- 172.21.15.44:0/1308879125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf7c2d2f40 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e0effd640 1 -- 172.21.15.44:0/1308879125 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e28015070 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e0effd640 1 -- 172.21.15.44:0/1308879125 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7e28010520 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e3854e740 10 client.15318 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e3854e740 10 client.15318 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e3854e740 20 client.15318 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e3854e740 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55bf7c2d5160 0x55bf7c2d65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e3854e740 4 client.15318 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.301+0000 7f7e3854e740 1 -- 172.21.15.44:0/1308879125 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf7c2305f0 con 0x55bf7c2d5160 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.302+0000 7f7e2effd640 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55bf7c2d5160 0x55bf7c2d65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.302+0000 7f7e2effd640 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55bf7c2d5160 0x55bf7c2d65d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5ee8267dbacf025f server_cookie=45aa9039eb1cacd3 in_seq=0 out_seq=0 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.302+0000 7f7e0effd640 10 client.15318 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e0effd640 1 -- 172.21.15.44:0/1308879125 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf7c2305f0 con 0x55bf7c2d5160 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e0effd640 10 client.15318 handle_command_reply: tid=0 2024-08-20T03:02:01.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e3854e740 1 client.15318 shutdown 2024-08-20T03:02:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e3854e740 1 -- 172.21.15.44:0/1308879125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 msgr2=0x55bf7c2c1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e3854e740 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2c1470 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009dd0 tx=0x7f7e28003fd0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e3854e740 1 -- 172.21.15.44:0/1308879125 shutdown_connections 2024-08-20T03:02:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e3854e740 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2c1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e3854e740 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55bf7c2d5160 0x55bf7c2d65d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e3854e740 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b8510 0x55bf7c2c19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e0effd640 0 client.15318 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:02:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e3854e740 1 --2- 172.21.15.44:0/1308879125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2c1470 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e3854e740 1 -- 172.21.15.44:0/1308879125 >> 172.21.15.44:0/1308879125 conn(0x55bf7c2b5a20 msgr2=0x55bf7c2bb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.303+0000 7f7e3854e740 1 -- 172.21.15.44:0/1308879125 shutdown_connections 2024-08-20T03:02:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.304+0000 7f7e3854e740 1 -- 172.21.15.44:0/1308879125 wait complete. 2024-08-20T03:02:01.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.304+0000 7f7e3854e740 20 client.15318 trim_cache size 0 max 16384 2024-08-20T03:02:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.322+0000 7f7e3854e740 1 Processor -- start 2024-08-20T03:02:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.322+0000 7f7e3854e740 1 -- start start 2024-08-20T03:02:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.322+0000 7f7e3854e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2d65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.322+0000 7f7e3854e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2b8510 0x55bf7c2bbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.322+0000 7f7e3854e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2bfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.322+0000 7f7e3854e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf7c2d4910 con 0x55bf7c2c44a0 2024-08-20T03:02:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.322+0000 7f7e3854e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bf7c2d4fe0 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.322+0000 7f7e3854e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bf7c2d56b0 con 0x55bf7c2b8510 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.322+0000 7f7e2ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2bfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.322+0000 7f7e2ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2bfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53412/0 (socket says 172.21.15.44:53412) 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.322+0000 7f7e2ffff640 1 -- 172.21.15.44:0/3063055225 learned_addr learned my addr 172.21.15.44:0/3063055225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.323+0000 7f7e2effd640 1 --2- 172.21.15.44:0/3063055225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2b8510 0x55bf7c2bbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.323+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/3063055225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2d65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.323+0000 7f7e2ffff640 1 -- 172.21.15.44:0/3063055225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2b8510 msgr2=0x55bf7c2bbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.323+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/3063055225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2b8510 0x55bf7c2bbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.323+0000 7f7e2ffff640 1 -- 172.21.15.44:0/3063055225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b7bf0 msgr2=0x55bf7c2d65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.323+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/3063055225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2d65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.323+0000 7f7e2ffff640 1 -- 172.21.15.44:0/3063055225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf7c2b8340 con 0x55bf7c2c44a0 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.323+0000 7f7e2effd640 1 --2- 172.21.15.44:0/3063055225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2b8510 0x55bf7c2bbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.323+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/3063055225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2bfcf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009d10 tx=0x7f7e28003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcaf370b73d43eae server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:01.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.323+0000 7f7e257fa640 1 -- 172.21.15.44:0/3063055225 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e280044d0 con 0x55bf7c2c44a0 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.323+0000 7f7e257fa640 1 -- 172.21.15.44:0/3063055225 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e28015040 con 0x55bf7c2c44a0 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.323+0000 7f7e257fa640 1 -- 172.21.15.44:0/3063055225 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28011830 con 0x55bf7c2c44a0 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.324+0000 7f7e3854e740 1 -- 172.21.15.44:0/3063055225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 msgr2=0x55bf7c2bfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.324+0000 7f7e3854e740 1 --2- 172.21.15.44:0/3063055225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2bfcf0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009d10 tx=0x7f7e28003ee0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.324+0000 7f7e3854e740 1 -- 172.21.15.44:0/3063055225 shutdown_connections 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.324+0000 7f7e3854e740 1 --2- 172.21.15.44:0/3063055225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 0x55bf7c2bfcf0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.324+0000 7f7e3854e740 1 --2- 172.21.15.44:0/3063055225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2b8510 0x55bf7c2bbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.324+0000 7f7e3854e740 1 --2- 172.21.15.44:0/3063055225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2d65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.324+0000 7f7e3854e740 1 -- 172.21.15.44:0/3063055225 >> 172.21.15.44:0/3063055225 conn(0x55bf7c2b5a20 msgr2=0x55bf7c2b6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.324+0000 7f7e3854e740 1 -- 172.21.15.44:0/3063055225 shutdown_connections 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.324+0000 7f7e3854e740 1 -- 172.21.15.44:0/3063055225 wait complete. 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.324+0000 7f7e3854e740 1 -- start start 2024-08-20T03:02:01.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.324+0000 7f7e3854e740 1 Processor -- start 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e3854e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2d7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e3854e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2b8510 0x55bf7c2d81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e3854e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 0x55bf7c33d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e3854e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bf7c2d6450 con 0x55bf7c2c44a0 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e3854e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bf7c2d5d80 con 0x55bf7c2b7bf0 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e3854e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bf7c2d6b20 con 0x55bf7c2b8510 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e3854e740 10 client.0 _ll_register_callbacks cb 0x7f7e28009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e2ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 0x55bf7c33d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e2ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 0x55bf7c33d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53420/0 (socket says 172.21.15.44:53420) 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e2ffff640 1 -- 172.21.15.44:0/4076069437 learned_addr learned my addr 172.21.15.44:0/4076069437 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e2effd640 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2b8510 0x55bf7c2d81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2d7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e2ffff640 1 -- 172.21.15.44:0/4076069437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2b8510 msgr2=0x55bf7c2d81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2b8510 0x55bf7c2d81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e2ffff640 1 -- 172.21.15.44:0/4076069437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b7bf0 msgr2=0x55bf7c2d7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2d7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e2ffff640 1 -- 172.21.15.44:0/4076069437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf7c33dda0 con 0x55bf7c2c44a0 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e2ffff640 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 0x55bf7c33d860 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7e28003a20 tx=0x7f7e28009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67c3752b03af3e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e0effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e0effd640 1 -- 172.21.15.44:0/4076069437 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28010690 con 0x55bf7c2c44a0 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e0effd640 1 -- 172.21.15.44:0/4076069437 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7e28011830 con 0x55bf7c2c44a0 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.325+0000 7f7e3854e740 1 -- 172.21.15.44:0/4076069437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e28015040 con 0x55bf7c2c44a0 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.326+0000 7f7e0effd640 1 -- 172.21.15.44:0/4076069437 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e28011a30 con 0x55bf7c2c44a0 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.326+0000 7f7e3854e740 10 client.15324 fetch_fsmap learned FSMap version 27 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.326+0000 7f7e3854e740 1 -- 172.21.15.44:0/4076069437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf7c33eff0 con 0x55bf7c2c44a0 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.326+0000 7f7e0effd640 1 -- 172.21.15.44:0/4076069437 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e28015040 con 0x55bf7c2c44a0 2024-08-20T03:02:01.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.326+0000 7f7e0effd640 1 -- 172.21.15.44:0/4076069437 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7e2801a820 con 0x55bf7c2c44a0 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.326+0000 7f7e3854e740 10 client.15324 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.326+0000 7f7e3854e740 10 client.15324 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.326+0000 7f7e3854e740 20 client.15324 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.326+0000 7f7e3854e740 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55bf7c2d5160 0x55bf7c342470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.326+0000 7f7e3854e740 4 client.15324 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.326+0000 7f7e3854e740 1 -- 172.21.15.44:0/4076069437 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "status"}) -- 0x55bf7c22c090 con 0x55bf7c2d5160 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.326+0000 7f7e2effd640 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55bf7c2d5160 0x55bf7c342470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e0effd640 10 client.15324 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e2effd640 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55bf7c2d5160 0x55bf7c342470 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=849e7f97711dbbdf server_cookie=760476f3662abe30 in_seq=0 out_seq=0 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e0effd640 1 -- 172.21.15.44:0/4076069437 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bf7c22c090 con 0x55bf7c2d5160 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e0effd640 10 client.15324 handle_command_reply: tid=0 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e3854e740 1 client.15324 shutdown 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e3854e740 1 -- 172.21.15.44:0/4076069437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 msgr2=0x55bf7c33d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:01.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e3854e740 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 0x55bf7c33d860 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7e28003a20 tx=0x7f7e28009b20 comp rx=0 tx=0).stop 2024-08-20T03:02:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e3854e740 1 -- 172.21.15.44:0/4076069437 shutdown_connections 2024-08-20T03:02:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e3854e740 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bf7c2c44a0 0x55bf7c33d860 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e3854e740 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bf7c2b8510 0x55bf7c2d81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e3854e740 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55bf7c2d5160 0x55bf7c342470 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e3854e740 1 --2- 172.21.15.44:0/4076069437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bf7c2b7bf0 0x55bf7c2d7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e3854e740 1 -- 172.21.15.44:0/4076069437 >> 172.21.15.44:0/4076069437 conn(0x55bf7c2b5a20 msgr2=0x55bf7c2bfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e0effd640 0 client.15324 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:02:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e3854e740 1 -- 172.21.15.44:0/4076069437 shutdown_connections 2024-08-20T03:02:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e3854e740 1 -- 172.21.15.44:0/4076069437 wait complete. 2024-08-20T03:02:01.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.327+0000 7f7e3854e740 20 client.15324 trim_cache size 0 max 16384 2024-08-20T03:02:01.343 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:02:01.344 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T03:02:01.344 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 4, 2024-08-20T03:02:01.344 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24569, 2024-08-20T03:02:01.344 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-08-20T03:02:01.344 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-08-20T03:02:01.344 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-08-20T03:02:01.344 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 91.946981781999995, 2024-08-20T03:02:01.344 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2024-08-20T03:02:01.344 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 77, 2024-08-20T03:02:01.344 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 75, 2024-08-20T03:02:01.344 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 124.64297321700001 2024-08-20T03:02:01.344 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:02:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.328+0000 7f7e36155640 1 -- 172.21.15.44:0/3760379187 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7e1c0b3c20 msgr2=0x7f7e1c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.328+0000 7f7e36155640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7e1c0b3c20 0x7f7e1c0b60e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7e180cada0 tx=0x7f7e180facd0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.328+0000 7f7e36155640 1 -- 172.21.15.44:0/3760379187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e301039e0 msgr2=0x7f7e30198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.328+0000 7f7e36155640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e301039e0 0x7f7e30198fc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e18005940 tx=0x7f7e180042f0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.328+0000 7f7e36155640 1 -- 172.21.15.44:0/3760379187 shutdown_connections 2024-08-20T03:02:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.328+0000 7f7e36155640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7e1c0b3c20 0x7f7e1c0b60e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.328+0000 7f7e36155640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e3010ce40 0x7f7e3019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.328+0000 7f7e36155640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e30104300 0x7f7e30199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.328+0000 7f7e36155640 1 --2- 172.21.15.44:0/3760379187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e301039e0 0x7f7e30198fc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:01.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.328+0000 7f7e36155640 1 -- 172.21.15.44:0/3760379187 >> 172.21.15.44:0/3760379187 conn(0x7f7e300fd440 msgr2=0x7f7e300fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:01.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.328+0000 7f7e36155640 1 -- 172.21.15.44:0/3760379187 shutdown_connections 2024-08-20T03:02:01.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:01.328+0000 7f7e36155640 1 -- 172.21.15.44:0/3760379187 wait complete. 2024-08-20T03:02:02.021 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.f status 2024-08-20T03:02:02.276 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:02.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:01 smithi044 ceph-mon[18790]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-20T03:02:02.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:01 smithi044 ceph-mon[18790]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-08-20T03:02:02.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:01 smithi165 ceph-mon[22357]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-20T03:02:02.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:01 smithi165 ceph-mon[22357]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-08-20T03:02:02.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:01 smithi099 ceph-mon[21984]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-08-20T03:02:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:01 smithi099 ceph-mon[21984]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-08-20T03:02:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.736+0000 7ff312934640 1 -- 172.21.15.44:0/1819812415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30c10a630 msgr2=0x7ff30c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.736+0000 7ff312934640 1 --2- 172.21.15.44:0/1819812415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30c10a630 0x7ff30c10ca20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff300009900 tx=0x7ff30002a410 comp rx=0 tx=0).stop 2024-08-20T03:02:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.736+0000 7ff312934640 1 -- 172.21.15.44:0/1819812415 shutdown_connections 2024-08-20T03:02:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.736+0000 7ff312934640 1 --2- 172.21.15.44:0/1819812415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c10cf60 0x7ff30c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.736+0000 7ff312934640 1 --2- 172.21.15.44:0/1819812415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30c10a630 0x7ff30c10ca20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.736+0000 7ff312934640 1 --2- 172.21.15.44:0/1819812415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff30c101520 0x7ff30c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.736+0000 7ff312934640 1 -- 172.21.15.44:0/1819812415 >> 172.21.15.44:0/1819812415 conn(0x7ff30c076c80 msgr2=0x7ff30c077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.736+0000 7ff312934640 1 -- 172.21.15.44:0/1819812415 shutdown_connections 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.736+0000 7ff312934640 1 -- 172.21.15.44:0/1819812415 wait complete. 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff312934640 1 Processor -- start 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff312934640 1 -- start start 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff312934640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c101520 0x7ff30c1024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff312934640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff30c10a630 0x7ff30c102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff312934640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30c10cf60 0x7ff30c102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff312934640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c112b40 con 0x7ff30c10cf60 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff312934640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1129c0 con 0x7ff30c101520 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff312934640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c112cc0 con 0x7ff30c10a630 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff310ed1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30c10cf60 0x7ff30c102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff310ed1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30c10cf60 0x7ff30c102f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53430/0 (socket says 172.21.15.44:53430) 2024-08-20T03:02:02.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff310ed1640 1 -- 172.21.15.44:0/3563504035 learned_addr learned my addr 172.21.15.44:0/3563504035 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff30c10a630 0x7ff30c102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff30bfff640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c101520 0x7ff30c1024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff310ed1640 1 -- 172.21.15.44:0/3563504035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff30c10a630 msgr2=0x7ff30c102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff310ed1640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff30c10a630 0x7ff30c102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff310ed1640 1 -- 172.21.15.44:0/3563504035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c101520 msgr2=0x7ff30c1024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.737+0000 7ff310ed1640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c101520 0x7ff30c1024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.738+0000 7ff310ed1640 1 -- 172.21.15.44:0/3563504035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c1a7530 con 0x7ff30c10cf60 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.738+0000 7ff30bfff640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c101520 0x7ff30c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.738+0000 7ff310ed1640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30c10cf60 0x7ff30c102f70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff30400b970 tx=0x7ff30400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.738+0000 7ff3097fa640 1 -- 172.21.15.44:0/3563504035 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304019070 con 0x7ff30c10cf60 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.738+0000 7ff3097fa640 1 -- 172.21.15.44:0/3563504035 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff304014440 con 0x7ff30c10cf60 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.738+0000 7ff312934640 1 -- 172.21.15.44:0/3563504035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c1a7820 con 0x7ff30c10cf60 2024-08-20T03:02:02.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.738+0000 7ff3097fa640 1 -- 172.21.15.44:0/3563504035 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3040075d0 con 0x7ff30c10cf60 2024-08-20T03:02:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.738+0000 7ff312934640 1 -- 172.21.15.44:0/3563504035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c1a7d60 con 0x7ff30c10cf60 2024-08-20T03:02:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.740+0000 7ff3097fa640 1 -- 172.21.15.44:0/3563504035 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7ff30401c070 con 0x7ff30c10cf60 2024-08-20T03:02:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff314d2d740 1 Processor -- start 2024-08-20T03:02:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff314d2d740 1 -- start start 2024-08-20T03:02:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff314d2d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f49fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff314d2d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f4a510 0x55d5e3f55f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff314d2d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 0x55d5e3f58890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff314d2d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3ec06a0 con 0x55d5e3f564a0 2024-08-20T03:02:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff314d2d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3eafba0 con 0x55d5e3f4a510 2024-08-20T03:02:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff314d2d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3eb3be0 con 0x55d5e3f49bf0 2024-08-20T03:02:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 0x55d5e3f58890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 0x55d5e3f58890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53432/0 (socket says 172.21.15.44:53432) 2024-08-20T03:02:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff30b7fe640 1 -- 172.21.15.44:0/4061746042 learned_addr learned my addr 172.21.15.44:0/4061746042 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/4061746042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f49fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff30bfff640 1 --2- 172.21.15.44:0/4061746042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f4a510 0x55d5e3f55f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff3097fa640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2f40b3c20 0x7ff2f40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff310ed1640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2f40b3c20 0x7ff2f40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff30b7fe640 1 -- 172.21.15.44:0/4061746042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f49bf0 msgr2=0x55d5e3f49fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/4061746042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f49fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff30b7fe640 1 -- 172.21.15.44:0/4061746042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f4a510 msgr2=0x55d5e3f55f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.741+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/4061746042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f4a510 0x55d5e3f55f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.742+0000 7ff30b7fe640 1 -- 172.21.15.44:0/4061746042 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e3c0f540 con 0x55d5e3f564a0 2024-08-20T03:02:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.742+0000 7ff30bfff640 1 --2- 172.21.15.44:0/4061746042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f4a510 0x55d5e3f55f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:02.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.742+0000 7ff3097fa640 1 -- 172.21.15.44:0/3563504035 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff3040e33c0 con 0x7ff30c10cf60 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.742+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/4061746042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 0x55d5e3f58890 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff30002a930 tx=0x7ff3000058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4292a7724d8c50 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.742+0000 7ff310ed1640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2f40b3c20 0x7ff2f40b60e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff3040e1180 tx=0x7ff3040779b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.742+0000 7ff2f27fc640 1 -- 172.21.15.44:0/4061746042 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300035040 con 0x55d5e3f564a0 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.742+0000 7ff2f27fc640 1 -- 172.21.15.44:0/4061746042 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff300005020 con 0x55d5e3f564a0 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.742+0000 7ff2f27fc640 1 -- 172.21.15.44:0/4061746042 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300004a00 con 0x55d5e3f564a0 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.743+0000 7ff314d2d740 1 -- 172.21.15.44:0/4061746042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 msgr2=0x55d5e3f58890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.743+0000 7ff314d2d740 1 --2- 172.21.15.44:0/4061746042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 0x55d5e3f58890 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff30002a930 tx=0x7ff3000058b0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.743+0000 7ff314d2d740 1 -- 172.21.15.44:0/4061746042 shutdown_connections 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.743+0000 7ff314d2d740 1 --2- 172.21.15.44:0/4061746042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 0x55d5e3f58890 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.743+0000 7ff314d2d740 1 --2- 172.21.15.44:0/4061746042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f4a510 0x55d5e3f55f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.743+0000 7ff314d2d740 1 --2- 172.21.15.44:0/4061746042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f49fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.743+0000 7ff314d2d740 1 -- 172.21.15.44:0/4061746042 >> 172.21.15.44:0/4061746042 conn(0x55d5e3f47a20 msgr2=0x55d5e3f48bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:02.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.743+0000 7ff314d2d740 1 -- 172.21.15.44:0/4061746042 shutdown_connections 2024-08-20T03:02:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.743+0000 7ff314d2d740 1 -- 172.21.15.44:0/4061746042 wait complete. 2024-08-20T03:02:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.743+0000 7ff314d2d740 1 -- start start 2024-08-20T03:02:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.743+0000 7ff314d2d740 1 Processor -- start 2024-08-20T03:02:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff314d2d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff314d2d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f4a510 0x55d5e3f539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff314d2d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 0x55d5e3f53f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff314d2d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3d125f0 con 0x55d5e3f564a0 2024-08-20T03:02:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff314d2d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3e366c0 con 0x55d5e3f49bf0 2024-08-20T03:02:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff314d2d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3f590f0 con 0x55d5e3f4a510 2024-08-20T03:02:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff314d2d740 10 client.0 _ll_register_callbacks cb 0x55d5e3e731e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 0x55d5e3f53f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 0x55d5e3f53f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53444/0 (socket says 172.21.15.44:53444) 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff30b7fe640 1 -- 172.21.15.44:0/3018784619 learned_addr learned my addr 172.21.15.44:0/3018784619 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff30bfff640 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f4a510 0x55d5e3f539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff30b7fe640 1 -- 172.21.15.44:0/3018784619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f4a510 msgr2=0x55d5e3f539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f4a510 0x55d5e3f539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff30b7fe640 1 -- 172.21.15.44:0/3018784619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f49bf0 msgr2=0x55d5e3f534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff30b7fe640 1 -- 172.21.15.44:0/3018784619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e3f62520 con 0x55d5e3f564a0 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff30bfff640 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f4a510 0x55d5e3f539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 0x55d5e3f53f20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff300005c00 tx=0x7ff300005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5d28673c9d136e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff2eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff2eb7fe640 1 -- 172.21.15.44:0/3018784619 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300035030 con 0x55d5e3f564a0 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff2eb7fe640 1 -- 172.21.15.44:0/3018784619 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff3000049b0 con 0x55d5e3f564a0 2024-08-20T03:02:02.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.744+0000 7ff314d2d740 1 -- 172.21.15.44:0/3018784619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff300005020 con 0x55d5e3f564a0 2024-08-20T03:02:02.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.745+0000 7ff2eb7fe640 1 -- 172.21.15.44:0/3018784619 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff300004bb0 con 0x55d5e3f564a0 2024-08-20T03:02:02.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.745+0000 7ff314d2d740 10 client.15336 fetch_fsmap learned FSMap version 27 2024-08-20T03:02:02.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.745+0000 7ff314d2d740 1 -- 172.21.15.44:0/3018784619 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5e3f64eb0 con 0x55d5e3f564a0 2024-08-20T03:02:02.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.745+0000 7ff2eb7fe640 1 -- 172.21.15.44:0/3018784619 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff300005020 con 0x55d5e3f564a0 2024-08-20T03:02:02.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.745+0000 7ff2eb7fe640 1 -- 172.21.15.44:0/3018784619 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff30003a040 con 0x55d5e3f564a0 2024-08-20T03:02:02.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.745+0000 7ff314d2d740 10 client.15336 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:02:02.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.745+0000 7ff314d2d740 10 client.15336 resolve_mds: resolved name 'f' to daemon mds.f 2024-08-20T03:02:02.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.745+0000 7ff314d2d740 20 client.15336 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:02.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.745+0000 7ff314d2d740 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55d5e3f66200 0x55d5e3f68350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.745+0000 7ff314d2d740 4 client.15336 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:02:02.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.745+0000 7ff314d2d740 1 -- 172.21.15.44:0/3018784619 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5e3ec25f0 con 0x55d5e3f66200 2024-08-20T03:02:02.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.745+0000 7ff30bfff640 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55d5e3f66200 0x55d5e3f68350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.746+0000 7ff2eb7fe640 10 client.15336 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.746+0000 7ff30bfff640 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55d5e3f66200 0x55d5e3f68350 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bb483b4c993c588e server_cookie=240e2e6f2ddce15e in_seq=0 out_seq=0 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff2eb7fe640 1 -- 172.21.15.44:0/3018784619 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5e3ec25f0 con 0x55d5e3f66200 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff2eb7fe640 10 client.15336 handle_command_reply: tid=0 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff314d2d740 1 client.15336 shutdown 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff314d2d740 1 -- 172.21.15.44:0/3018784619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 msgr2=0x55d5e3f53f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff314d2d740 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 0x55d5e3f53f20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff300005c00 tx=0x7ff300005b50 comp rx=0 tx=0).stop 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff314d2d740 1 -- 172.21.15.44:0/3018784619 shutdown_connections 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff314d2d740 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55d5e3f66200 0x55d5e3f68350 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff314d2d740 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f564a0 0x55d5e3f53f20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff2eb7fe640 0 client.15336 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff314d2d740 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f4a510 0x55d5e3f539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff314d2d740 1 --2- 172.21.15.44:0/3018784619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff314d2d740 1 -- 172.21.15.44:0/3018784619 >> 172.21.15.44:0/3018784619 conn(0x55d5e3f47a20 msgr2=0x55d5e3f4d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:02.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff314d2d740 1 -- 172.21.15.44:0/3018784619 shutdown_connections 2024-08-20T03:02:02.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff314d2d740 1 -- 172.21.15.44:0/3018784619 wait complete. 2024-08-20T03:02:02.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.747+0000 7ff314d2d740 20 client.15336 trim_cache size 0 max 16384 2024-08-20T03:02:02.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff314d2d740 1 Processor -- start 2024-08-20T03:02:02.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff314d2d740 1 -- start start 2024-08-20T03:02:02.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff314d2d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f68350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff314d2d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f4a510 0x55d5e3f52d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff314d2d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f564a0 0x55d5e3f61a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff314d2d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3f66690 con 0x55d5e3f49bf0 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff314d2d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3f66d60 con 0x55d5e3f4a510 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff314d2d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3f67430 con 0x55d5e3f564a0 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f68350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f68350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53448/0 (socket says 172.21.15.44:53448) 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff30b7fe640 1 -- 172.21.15.44:0/2455161558 learned_addr learned my addr 172.21.15.44:0/2455161558 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff30b7fe640 1 -- 172.21.15.44:0/2455161558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f564a0 msgr2=0x55d5e3f61a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/2455161558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f564a0 0x55d5e3f61a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff30b7fe640 1 -- 172.21.15.44:0/2455161558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f4a510 msgr2=0x55d5e3f52d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/2455161558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f4a510 0x55d5e3f52d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.765+0000 7ff30b7fe640 1 -- 172.21.15.44:0/2455161558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e3f4e1d0 con 0x55d5e3f49bf0 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/2455161558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f68350 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff30002aae0 tx=0x7ff300000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e999b3072d10e8f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff2f1ffb640 1 -- 172.21.15.44:0/2455161558 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300035040 con 0x55d5e3f49bf0 2024-08-20T03:02:02.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff2f1ffb640 1 -- 172.21.15.44:0/2455161558 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3000049b0 con 0x55d5e3f49bf0 2024-08-20T03:02:02.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff2f1ffb640 1 -- 172.21.15.44:0/2455161558 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300004d50 con 0x55d5e3f49bf0 2024-08-20T03:02:02.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff314d2d740 1 -- 172.21.15.44:0/2455161558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 msgr2=0x55d5e3f68350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:02.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff314d2d740 1 --2- 172.21.15.44:0/2455161558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f68350 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff30002aae0 tx=0x7ff300000bc0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff314d2d740 1 -- 172.21.15.44:0/2455161558 shutdown_connections 2024-08-20T03:02:02.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff314d2d740 1 --2- 172.21.15.44:0/2455161558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f564a0 0x55d5e3f61a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff314d2d740 1 --2- 172.21.15.44:0/2455161558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f4a510 0x55d5e3f52d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff314d2d740 1 --2- 172.21.15.44:0/2455161558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f68350 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff314d2d740 1 -- 172.21.15.44:0/2455161558 >> 172.21.15.44:0/2455161558 conn(0x55d5e3f47a20 msgr2=0x55d5e3f61380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:02.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff314d2d740 1 -- 172.21.15.44:0/2455161558 shutdown_connections 2024-08-20T03:02:02.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.766+0000 7ff314d2d740 1 -- 172.21.15.44:0/2455161558 wait complete. 2024-08-20T03:02:02.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.767+0000 7ff314d2d740 1 -- start start 2024-08-20T03:02:02.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.767+0000 7ff314d2d740 1 Processor -- start 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.767+0000 7ff314d2d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f69c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.767+0000 7ff314d2d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f4a510 0x55d5e3f6a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.767+0000 7ff314d2d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f564a0 0x55d5e3f6f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.767+0000 7ff314d2d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3f681d0 con 0x55d5e3f49bf0 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.767+0000 7ff314d2d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3f67b00 con 0x55d5e3f564a0 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.767+0000 7ff314d2d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d5e3f688a0 con 0x55d5e3f4a510 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.767+0000 7ff314d2d740 10 client.0 _ll_register_callbacks cb 0x7ff300032e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.767+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f69c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.767+0000 7ff30b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f69c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53464/0 (socket says 172.21.15.44:53464) 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.767+0000 7ff30b7fe640 1 -- 172.21.15.44:0/1268481660 learned_addr learned my addr 172.21.15.44:0/1268481660 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff30bfff640 1 --2- 172.21.15.44:0/1268481660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f4a510 0x55d5e3f6a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff30b7fe640 1 -- 172.21.15.44:0/1268481660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f4a510 msgr2=0x55d5e3f6a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:02.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/1268481660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f4a510 0x55d5e3f6a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff30b7fe640 1 -- 172.21.15.44:0/1268481660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f564a0 msgr2=0x55d5e3f6f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/1268481660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f564a0 0x55d5e3f6f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff30b7fe640 1 -- 172.21.15.44:0/1268481660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e3f6f910 con 0x55d5e3f49bf0 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff30b7fe640 1 --2- 172.21.15.44:0/1268481660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f69c70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff30002aab0 tx=0x7ff30004af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0235b5fa2276a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff2eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff2eb7fe640 1 -- 172.21.15.44:0/1268481660 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3000495f0 con 0x55d5e3f49bf0 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff2eb7fe640 1 -- 172.21.15.44:0/1268481660 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff3000498d0 con 0x55d5e3f49bf0 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff314d2d740 1 -- 172.21.15.44:0/1268481660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3000049b0 con 0x55d5e3f49bf0 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff2eb7fe640 1 -- 172.21.15.44:0/1268481660 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff300049ad0 con 0x55d5e3f49bf0 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff314d2d740 10 client.15342 fetch_fsmap learned FSMap version 27 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff314d2d740 1 -- 172.21.15.44:0/1268481660 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d5e3f70d70 con 0x55d5e3f49bf0 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.768+0000 7ff2eb7fe640 1 -- 172.21.15.44:0/1268481660 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3000049b0 con 0x55d5e3f49bf0 2024-08-20T03:02:02.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.769+0000 7ff2eb7fe640 1 -- 172.21.15.44:0/1268481660 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff30005e070 con 0x55d5e3f49bf0 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.769+0000 7ff314d2d740 10 client.15342 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.769+0000 7ff314d2d740 10 client.15342 resolve_mds: resolved name 'f' to daemon mds.f 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.769+0000 7ff314d2d740 20 client.15342 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.769+0000 7ff314d2d740 1 --2- 172.21.15.44:0/1268481660 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55d5e3f66200 0x55d5e3fd4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.769+0000 7ff314d2d740 4 client.15342 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.769+0000 7ff314d2d740 1 -- 172.21.15.44:0/1268481660 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "status"}) -- 0x55d5e3ebe090 con 0x55d5e3f66200 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.769+0000 7ff30bfff640 1 --2- 172.21.15.44:0/1268481660 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55d5e3f66200 0x55d5e3fd4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.769+0000 7ff30bfff640 1 --2- 172.21.15.44:0/1268481660 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55d5e3f66200 0x55d5e3fd4290 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc7e9e4b879dd6fc server_cookie=a90b2ea5161ca548 in_seq=0 out_seq=0 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.769+0000 7ff2eb7fe640 10 client.15342 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff2eb7fe640 1 -- 172.21.15.44:0/1268481660 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d5e3ebe090 con 0x55d5e3f66200 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff2eb7fe640 10 client.15342 handle_command_reply: tid=0 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff314d2d740 1 client.15342 shutdown 2024-08-20T03:02:02.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff314d2d740 1 -- 172.21.15.44:0/1268481660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 msgr2=0x55d5e3f69c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:02.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff314d2d740 1 --2- 172.21.15.44:0/1268481660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f69c70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff30002aab0 tx=0x7ff30004af90 comp rx=0 tx=0).stop 2024-08-20T03:02:02.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff314d2d740 1 -- 172.21.15.44:0/1268481660 shutdown_connections 2024-08-20T03:02:02.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff314d2d740 1 --2- 172.21.15.44:0/1268481660 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55d5e3f66200 0x55d5e3fd4290 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff314d2d740 1 --2- 172.21.15.44:0/1268481660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d5e3f564a0 0x55d5e3f6f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff314d2d740 1 --2- 172.21.15.44:0/1268481660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d5e3f4a510 0x55d5e3f6a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff314d2d740 1 --2- 172.21.15.44:0/1268481660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5e3f49bf0 0x55d5e3f69c70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff2eb7fe640 0 client.15342 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:02:02.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff314d2d740 1 -- 172.21.15.44:0/1268481660 >> 172.21.15.44:0/1268481660 conn(0x55d5e3f47a20 msgr2=0x55d5e3f552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:02.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff314d2d740 1 -- 172.21.15.44:0/1268481660 shutdown_connections 2024-08-20T03:02:02.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff314d2d740 1 -- 172.21.15.44:0/1268481660 wait complete. 2024-08-20T03:02:02.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff314d2d740 20 client.15342 trim_cache size 0 max 16384 2024-08-20T03:02:02.787 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:02:02.787 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T03:02:02.787 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 3, 2024-08-20T03:02:02.787 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24583, 2024-08-20T03:02:02.787 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-08-20T03:02:02.787 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-08-20T03:02:02.787 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-08-20T03:02:02.787 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 94.401890355000006, 2024-08-20T03:02:02.787 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2024-08-20T03:02:02.787 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 77, 2024-08-20T03:02:02.787 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 75, 2024-08-20T03:02:02.788 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 131.377518331 2024-08-20T03:02:02.788 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:02:02.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff312934640 1 -- 172.21.15.44:0/3563504035 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2f40b3c20 msgr2=0x7ff2f40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:02.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff312934640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2f40b3c20 0x7ff2f40b60e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff3040e1180 tx=0x7ff3040779b0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff312934640 1 -- 172.21.15.44:0/3563504035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30c10cf60 msgr2=0x7ff30c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:02.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.770+0000 7ff312934640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30c10cf60 0x7ff30c102f70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff30400b970 tx=0x7ff30400be30 comp rx=0 tx=0).stop 2024-08-20T03:02:02.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.771+0000 7ff312934640 1 -- 172.21.15.44:0/3563504035 shutdown_connections 2024-08-20T03:02:02.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.771+0000 7ff312934640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2f40b3c20 0x7ff2f40b60e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.771+0000 7ff312934640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff30c10cf60 0x7ff30c102f70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.771+0000 7ff312934640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff30c10a630 0x7ff30c102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.771+0000 7ff312934640 1 --2- 172.21.15.44:0/3563504035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff30c101520 0x7ff30c1024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.771+0000 7ff312934640 1 -- 172.21.15.44:0/3563504035 >> 172.21.15.44:0/3563504035 conn(0x7ff30c076c80 msgr2=0x7ff30c1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.771+0000 7ff312934640 1 -- 172.21.15.44:0/3563504035 shutdown_connections 2024-08-20T03:02:02.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:02.771+0000 7ff312934640 1 -- 172.21.15.44:0/3563504035 wait complete. 2024-08-20T03:02:03.458 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.c status 2024-08-20T03:02:03.710 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:04.117 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:03 smithi044 ceph-mon[18790]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-08-20T03:02:04.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.196+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4051683898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd0104300 msgr2=0x7fbfd010c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:04.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.196+0000 7fbfd64ee640 1 --2- 172.21.15.44:0/4051683898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd0104300 0x7fbfd010c7f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009900 tx=0x7fbfc402a3f0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.196+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4051683898 shutdown_connections 2024-08-20T03:02:04.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.196+0000 7fbfd64ee640 1 --2- 172.21.15.44:0/4051683898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbfd010ce40 0x7fbfd010f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.196+0000 7fbfd64ee640 1 --2- 172.21.15.44:0/4051683898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd0104300 0x7fbfd010c7f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.196+0000 7fbfd64ee640 1 --2- 172.21.15.44:0/4051683898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd01039e0 0x7fbfd0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.196+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4051683898 >> 172.21.15.44:0/4051683898 conn(0x7fbfd00fd440 msgr2=0x7fbfd00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:04.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.196+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4051683898 shutdown_connections 2024-08-20T03:02:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.196+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4051683898 wait complete. 2024-08-20T03:02:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfd64ee640 1 Processor -- start 2024-08-20T03:02:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfd64ee640 1 -- start start 2024-08-20T03:02:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfd64ee640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbfd01039e0 0x7fbfd01990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfd64ee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd0104300 0x7fbfd01995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfd64ee640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd010ce40 0x7fbfd019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfd64ee640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0112af0 con 0x7fbfd0104300 2024-08-20T03:02:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfd64ee640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0112970 con 0x7fbfd010ce40 2024-08-20T03:02:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfd64ee640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0112c70 con 0x7fbfd01039e0 2024-08-20T03:02:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfcf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd0104300 0x7fbfd01995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfcf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd0104300 0x7fbfd01995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53480/0 (socket says 172.21.15.44:53480) 2024-08-20T03:02:04.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfcf7fe640 1 -- 172.21.15.44:0/4013774934 learned_addr learned my addr 172.21.15.44:0/4013774934 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfd4a8b640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd010ce40 0x7fbfd019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfcffff640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbfd01039e0 0x7fbfd01990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfcf7fe640 1 -- 172.21.15.44:0/4013774934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbfd01039e0 msgr2=0x7fbfd01990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfcf7fe640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbfd01039e0 0x7fbfd01990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfcf7fe640 1 -- 172.21.15.44:0/4013774934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd010ce40 msgr2=0x7fbfd019e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfcf7fe640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd010ce40 0x7fbfd019e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfcf7fe640 1 -- 172.21.15.44:0/4013774934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd019f150 con 0x7fbfd0104300 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.197+0000 7fbfd4a8b640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd010ce40 0x7fbfd019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.198+0000 7fbfcf7fe640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd0104300 0x7fbfd01995e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4004270 tx=0x7fbfc40042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.198+0000 7fbfcd7fa640 1 -- 172.21.15.44:0/4013774934 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc403f030 con 0x7fbfd0104300 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.198+0000 7fbfcd7fa640 1 -- 172.21.15.44:0/4013774934 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc40044c0 con 0x7fbfd0104300 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.198+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4013774934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd019f3e0 con 0x7fbfd0104300 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.198+0000 7fbfcd7fa640 1 -- 172.21.15.44:0/4013774934 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4035070 con 0x7fbfd0104300 2024-08-20T03:02:04.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.198+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4013774934 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd01a5b00 con 0x7fbfd0104300 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.200+0000 7fbfcd7fa640 1 -- 172.21.15.44:0/4013774934 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fbfc4042030 con 0x7fbfd0104300 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.200+0000 7fbfd88e7740 1 Processor -- start 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfd88e7740 1 -- start start 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfd88e7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3e3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfd88e7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 0x55a7bf3eff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfd88e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3f2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfd88e7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a7bf35a6a0 con 0x55a7bf3e4510 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfd88e7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a7bf349ba0 con 0x55a7bf3e3bf0 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfd88e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a7bf34dbe0 con 0x55a7bf3f04a0 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfcffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 0x55a7bf3eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfcffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 0x55a7bf3eff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53496/0 (socket says 172.21.15.44:53496) 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfcffff640 1 -- 172.21.15.44:0/2544480133 learned_addr learned my addr 172.21.15.44:0/2544480133 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:04.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfd4a8b640 1 --2- 172.21.15.44:0/2544480133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3e3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfcd7fa640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbfb80b3c20 0x7fbfb80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfcf7fe640 1 --2- 172.21.15.44:0/2544480133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfd4a8b640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbfb80b3c20 0x7fbfb80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfcffff640 1 -- 172.21.15.44:0/2544480133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3f04a0 msgr2=0x55a7bf3f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfcffff640 1 --2- 172.21.15.44:0/2544480133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfcffff640 1 -- 172.21.15.44:0/2544480133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e3bf0 msgr2=0x55a7bf3e3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfcffff640 1 --2- 172.21.15.44:0/2544480133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3e3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfcffff640 1 -- 172.21.15.44:0/2544480133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7bf0a9540 con 0x55a7bf3e4510 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.201+0000 7fbfcd7fa640 1 -- 172.21.15.44:0/4013774934 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbfc4104310 con 0x7fbfd0104300 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.202+0000 7fbfcffff640 1 --2- 172.21.15.44:0/2544480133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 0x55a7bf3eff60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b850 tx=0x7fbfc000bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc2df915ebdcb57 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.202+0000 7fbfd4a8b640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbfb80b3c20 0x7fbfb80b60e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8012b90 tx=0x7fbfc8004ab0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.202+0000 7fbfb67fc640 1 -- 172.21.15.44:0/2544480133 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000d950 con 0x55a7bf3e4510 2024-08-20T03:02:04.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.202+0000 7fbfb67fc640 1 -- 172.21.15.44:0/2544480133 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc0019070 con 0x55a7bf3e4510 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.202+0000 7fbfb67fc640 1 -- 172.21.15.44:0/2544480133 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0014e40 con 0x55a7bf3e4510 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.202+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2544480133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 msgr2=0x55a7bf3eff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.202+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2544480133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 0x55a7bf3eff60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b850 tx=0x7fbfc000bd10 comp rx=0 tx=0).stop 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.202+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2544480133 shutdown_connections 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.202+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2544480133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.202+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2544480133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 0x55a7bf3eff60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.202+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2544480133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3e3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.202+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2544480133 >> 172.21.15.44:0/2544480133 conn(0x55a7bf3e1a20 msgr2=0x55a7bf3e2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2544480133 shutdown_connections 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2544480133 wait complete. 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfd88e7740 1 -- start start 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfd88e7740 1 Processor -- start 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfd88e7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3efee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfd88e7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 0x55a7bf3ed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfd88e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3edac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfd88e7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a7bf1ac5f0 con 0x55a7bf3e4510 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfd88e7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a7bf2d06c0 con 0x55a7bf3e3bf0 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfd88e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a7bf3f3150 con 0x55a7bf3f04a0 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfd88e7740 10 client.0 _ll_register_callbacks cb 0x55a7bf30d1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfcf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 0x55a7bf3ed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfcf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 0x55a7bf3ed330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53500/0 (socket says 172.21.15.44:53500) 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.203+0000 7fbfcf7fe640 1 -- 172.21.15.44:0/2398847378 learned_addr learned my addr 172.21.15.44:0/2398847378 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfcffff640 1 --2- 172.21.15.44:0/2398847378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfcf7fe640 1 -- 172.21.15.44:0/2398847378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3f04a0 msgr2=0x55a7bf3edac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfcf7fe640 1 --2- 172.21.15.44:0/2398847378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfcf7fe640 1 -- 172.21.15.44:0/2398847378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e3bf0 msgr2=0x55a7bf3efee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfcf7fe640 1 --2- 172.21.15.44:0/2398847378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3efee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfcf7fe640 1 -- 172.21.15.44:0/2398847378 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7bf3fc480 con 0x55a7bf3e4510 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfcf7fe640 1 --2- 172.21.15.44:0/2398847378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 0x55a7bf3ed330 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4006010 tx=0x7fbfc4031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1183f4aa16580ac2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfaf7fe640 1 -- 172.21.15.44:0/2398847378 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4049030 con 0x55a7bf3e4510 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfaf7fe640 1 -- 172.21.15.44:0/2398847378 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbfc40cb410 con 0x55a7bf3e4510 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2398847378 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc0019070 con 0x55a7bf3e4510 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfaf7fe640 1 -- 172.21.15.44:0/2398847378 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfc40cb8b0 con 0x55a7bf3e4510 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfd88e7740 10 client.15354 fetch_fsmap learned FSMap version 27 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2398847378 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7bf3fedb0 con 0x55a7bf3e4510 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfaf7fe640 1 -- 172.21.15.44:0/2398847378 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc0019070 con 0x55a7bf3e4510 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfaf7fe640 1 -- 172.21.15.44:0/2398847378 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbfc40d3dd0 con 0x55a7bf3e4510 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.204+0000 7fbfd88e7740 10 client.15354 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.205+0000 7fbfd88e7740 10 client.15354 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.205+0000 7fbfd88e7740 20 client.15354 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:04.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.205+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2398847378 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55a7bf4013f0 0x55a7bf402860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.205+0000 7fbfd88e7740 4 client.15354 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.205+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2398847378 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7bf35c5f0 con 0x55a7bf4013f0 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.205+0000 7fbfcffff640 1 --2- 172.21.15.44:0/2398847378 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55a7bf4013f0 0x55a7bf402860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.205+0000 7fbfcffff640 1 --2- 172.21.15.44:0/2398847378 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55a7bf4013f0 0x55a7bf402860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a9fbe409089b5c2 server_cookie=92e60580ab50bade in_seq=0 out_seq=0 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.205+0000 7fbfaf7fe640 10 client.15354 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfaf7fe640 1 -- 172.21.15.44:0/2398847378 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7bf35c5f0 con 0x55a7bf4013f0 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfaf7fe640 10 client.15354 handle_command_reply: tid=0 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfd88e7740 1 client.15354 shutdown 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2398847378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 msgr2=0x55a7bf3ed330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2398847378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 0x55a7bf3ed330 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4006010 tx=0x7fbfc4031a10 comp rx=0 tx=0).stop 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2398847378 shutdown_connections 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2398847378 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55a7bf4013f0 0x55a7bf402860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2398847378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3edac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2398847378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e4510 0x55a7bf3ed330 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfaf7fe640 0 client.15354 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:02:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2398847378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3efee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2398847378 >> 172.21.15.44:0/2398847378 conn(0x55a7bf3e1a20 msgr2=0x55a7bf3e75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2398847378 shutdown_connections 2024-08-20T03:02:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2398847378 wait complete. 2024-08-20T03:02:04.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.206+0000 7fbfd88e7740 20 client.15354 trim_cache size 0 max 16384 2024-08-20T03:02:04.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd88e7740 1 Processor -- start 2024-08-20T03:02:04.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd88e7740 1 -- start start 2024-08-20T03:02:04.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd88e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf402860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd88e7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e4510 0x55a7bf3ecbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd88e7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3ea540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd88e7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a7bf400ba0 con 0x55a7bf3f04a0 2024-08-20T03:02:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd88e7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a7bf401270 con 0x55a7bf3e4510 2024-08-20T03:02:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd88e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a7bf401940 con 0x55a7bf3e3bf0 2024-08-20T03:02:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd4a8b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3ea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd4a8b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3ea540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53506/0 (socket says 172.21.15.44:53506) 2024-08-20T03:02:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd4a8b640 1 -- 172.21.15.44:0/4044005480 learned_addr learned my addr 172.21.15.44:0/4044005480 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfcf7fe640 1 --2- 172.21.15.44:0/4044005480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e4510 0x55a7bf3ecbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfcffff640 1 --2- 172.21.15.44:0/4044005480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf402860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd4a8b640 1 -- 172.21.15.44:0/4044005480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3e3bf0 msgr2=0x55a7bf402860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd4a8b640 1 --2- 172.21.15.44:0/4044005480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf402860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd4a8b640 1 -- 172.21.15.44:0/4044005480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e4510 msgr2=0x55a7bf3ecbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd4a8b640 1 --2- 172.21.15.44:0/4044005480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e4510 0x55a7bf3ecbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.230+0000 7fbfd4a8b640 1 -- 172.21.15.44:0/4044005480 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7bf3ede30 con 0x55a7bf3f04a0 2024-08-20T03:02:04.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.231+0000 7fbfd4a8b640 1 --2- 172.21.15.44:0/4044005480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3ea540 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8007ab0 tx=0x7fbfc8007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930e4c8ad34fa391 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:04.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.231+0000 7fbfb5ffb640 1 -- 172.21.15.44:0/4044005480 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8024070 con 0x55a7bf3f04a0 2024-08-20T03:02:04.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.231+0000 7fbfb5ffb640 1 -- 172.21.15.44:0/4044005480 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc8020430 con 0x55a7bf3f04a0 2024-08-20T03:02:04.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.231+0000 7fbfb5ffb640 1 -- 172.21.15.44:0/4044005480 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8008e60 con 0x55a7bf3f04a0 2024-08-20T03:02:04.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.231+0000 7fbfd88e7740 1 -- 172.21.15.44:0/4044005480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3f04a0 msgr2=0x55a7bf3ea540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.231+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/4044005480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3ea540 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8007ab0 tx=0x7fbfc8007590 comp rx=0 tx=0).stop 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 -- 172.21.15.44:0/4044005480 shutdown_connections 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/4044005480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3f04a0 0x55a7bf3ea540 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/4044005480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3e4510 0x55a7bf3ecbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/4044005480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf402860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 -- 172.21.15.44:0/4044005480 >> 172.21.15.44:0/4044005480 conn(0x55a7bf3e1a20 msgr2=0x55a7bf3e2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 -- 172.21.15.44:0/4044005480 shutdown_connections 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 -- 172.21.15.44:0/4044005480 wait complete. 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 -- start start 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 Processor -- start 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3e4510 0x55a7bf3f1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3f04a0 0x55a7bf40ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a7bf4026e0 con 0x55a7bf3e3bf0 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a7bf402010 con 0x55a7bf3f04a0 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a7bf402db0 con 0x55a7bf3e4510 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfd88e7740 10 client.0 _ll_register_callbacks cb 0x7fbfc40ca9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.232+0000 7fbfcffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfcffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3f11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53508/0 (socket says 172.21.15.44:53508) 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfcffff640 1 -- 172.21.15.44:0/2973497652 learned_addr learned my addr 172.21.15.44:0/2973497652 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfcf7fe640 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3e4510 0x55a7bf3f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfd4a8b640 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3f04a0 0x55a7bf40ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfcffff640 1 -- 172.21.15.44:0/2973497652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3e4510 msgr2=0x55a7bf3f1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfcffff640 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3e4510 0x55a7bf3f1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfcffff640 1 -- 172.21.15.44:0/2973497652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3f04a0 msgr2=0x55a7bf40ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:04.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfcffff640 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3f04a0 0x55a7bf40ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfcffff640 1 -- 172.21.15.44:0/2973497652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7bf40b250 con 0x55a7bf3e3bf0 2024-08-20T03:02:04.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfcf7fe640 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3e4510 0x55a7bf3f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:04.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfcffff640 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3f11c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b850 tx=0x7fbfc000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bc59b0ea583a95 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:04.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:04.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfaf7fe640 1 -- 172.21.15.44:0/2973497652 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000dd60 con 0x55a7bf3e3bf0 2024-08-20T03:02:04.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfaf7fe640 1 -- 172.21.15.44:0/2973497652 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbfc0014620 con 0x55a7bf3e3bf0 2024-08-20T03:02:04.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2973497652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc8020430 con 0x55a7bf3e3bf0 2024-08-20T03:02:04.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfaf7fe640 1 -- 172.21.15.44:0/2973497652 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbfc0014820 con 0x55a7bf3e3bf0 2024-08-20T03:02:04.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfd88e7740 10 client.15360 fetch_fsmap learned FSMap version 27 2024-08-20T03:02:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.233+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2973497652 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7bf406640 con 0x55a7bf3e3bf0 2024-08-20T03:02:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.234+0000 7fbfaf7fe640 1 -- 172.21.15.44:0/2973497652 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc8020430 con 0x55a7bf3e3bf0 2024-08-20T03:02:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.234+0000 7fbfaf7fe640 1 -- 172.21.15.44:0/2973497652 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbfc0037020 con 0x55a7bf3e3bf0 2024-08-20T03:02:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.234+0000 7fbfd88e7740 10 client.15360 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:02:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.234+0000 7fbfd88e7740 10 client.15360 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-20T03:02:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.234+0000 7fbfd88e7740 20 client.15360 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.234+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55a7bf474320 0x55a7bf473030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.234+0000 7fbfd88e7740 4 client.15360 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T03:02:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.234+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2973497652 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "status"}) -- 0x55a7bf358090 con 0x55a7bf474320 2024-08-20T03:02:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.234+0000 7fbfcf7fe640 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55a7bf474320 0x55a7bf473030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.234+0000 7fbfaf7fe640 10 client.15360 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:02:04.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.234+0000 7fbfcf7fe640 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55a7bf474320 0x55a7bf473030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=645a8c5257fc34e2 server_cookie=710aac3ea33b81c3 in_seq=0 out_seq=0 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfaf7fe640 1 -- 172.21.15.44:0/2973497652 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a7bf358090 con 0x55a7bf474320 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfaf7fe640 10 client.15360 handle_command_reply: tid=0 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfd88e7740 1 client.15360 shutdown 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2973497652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e3bf0 msgr2=0x55a7bf3f11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3f11c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b850 tx=0x7fbfc000bf80 comp rx=0 tx=0).stop 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2973497652 shutdown_connections 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a7bf3f04a0 0x55a7bf40ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55a7bf474320 0x55a7bf473030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7bf3e4510 0x55a7bf3f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfd88e7740 1 --2- 172.21.15.44:0/2973497652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7bf3e3bf0 0x55a7bf3f11c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2973497652 >> 172.21.15.44:0/2973497652 conn(0x55a7bf3e1a20 msgr2=0x55a7bf3ee960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfaf7fe640 0 client.15360 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:02:04.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2973497652 shutdown_connections 2024-08-20T03:02:04.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfd88e7740 1 -- 172.21.15.44:0/2973497652 wait complete. 2024-08-20T03:02:04.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.235+0000 7fbfd88e7740 20 client.15360 trim_cache size 0 max 16384 2024-08-20T03:02:04.244 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:02:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T03:02:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 2, 2024-08-20T03:02:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24589, 2024-08-20T03:02:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-08-20T03:02:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-08-20T03:02:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-08-20T03:02:04.244 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 96.873734892000002, 2024-08-20T03:02:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2024-08-20T03:02:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 77, 2024-08-20T03:02:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 75, 2024-08-20T03:02:04.245 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 130.25466759099999 2024-08-20T03:02:04.245 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:02:04.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.236+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4013774934 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbfb80b3c20 msgr2=0x7fbfb80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:04.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.236+0000 7fbfd64ee640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbfb80b3c20 0x7fbfb80b60e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8012b90 tx=0x7fbfc8004ab0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.236+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4013774934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd0104300 msgr2=0x7fbfd01995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:04.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.236+0000 7fbfd64ee640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd0104300 0x7fbfd01995e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4004270 tx=0x7fbfc40042a0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.236+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4013774934 shutdown_connections 2024-08-20T03:02:04.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.236+0000 7fbfd64ee640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbfb80b3c20 0x7fbfb80b60e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.236+0000 7fbfd64ee640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd010ce40 0x7fbfd019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.236+0000 7fbfd64ee640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfd0104300 0x7fbfd01995e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.236+0000 7fbfd64ee640 1 --2- 172.21.15.44:0/4013774934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbfd01039e0 0x7fbfd01990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:04.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.236+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4013774934 >> 172.21.15.44:0/4013774934 conn(0x7fbfd00fd440 msgr2=0x7fbfd00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:04.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.236+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4013774934 shutdown_connections 2024-08-20T03:02:04.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:04.236+0000 7fbfd64ee640 1 -- 172.21.15.44:0/4013774934 wait complete. 2024-08-20T03:02:04.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:03 smithi165 ceph-mon[22357]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-08-20T03:02:04.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:03 smithi099 ceph-mon[21984]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 11 op/s 2024-08-20T03:02:05.069 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-20T03:02:05.320 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:05.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.772+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/4025987522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6dc108e60 msgr2=0x7fe6dc10f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:05.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.772+0000 7fe6e0ed4640 1 --2- 172.21.15.44:0/4025987522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6dc108e60 0x7fe6dc10f660 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009ca0 tx=0x7fe6d402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.772+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/4025987522 shutdown_connections 2024-08-20T03:02:05.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.772+0000 7fe6e0ed4640 1 --2- 172.21.15.44:0/4025987522 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6dc108e60 0x7fe6dc10f660 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.772+0000 7fe6e0ed4640 1 --2- 172.21.15.44:0/4025987522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6dc1084e0 0x7fe6dc108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.772+0000 7fe6e0ed4640 1 --2- 172.21.15.44:0/4025987522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6dc102ad0 0x7fe6dc102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.772+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/4025987522 >> 172.21.15.44:0/4025987522 conn(0x7fe6dc0fe540 msgr2=0x7fe6dc100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:05.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.772+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/4025987522 shutdown_connections 2024-08-20T03:02:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.772+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/4025987522 wait complete. 2024-08-20T03:02:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.773+0000 7fe6e0ed4640 1 Processor -- start 2024-08-20T03:02:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.773+0000 7fe6e0ed4640 1 -- start start 2024-08-20T03:02:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.773+0000 7fe6e0ed4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6dc102ad0 0x7fe6dc198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.773+0000 7fe6e0ed4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6dc1084e0 0x7fe6dc1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.773+0000 7fe6e0ed4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6dc108e60 0x7fe6dc19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.773+0000 7fe6e0ed4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc112b20 con 0x7fe6dc102ad0 2024-08-20T03:02:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.773+0000 7fe6e0ed4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc1129a0 con 0x7fe6dc1084e0 2024-08-20T03:02:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.773+0000 7fe6e0ed4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc112ca0 con 0x7fe6dc108e60 2024-08-20T03:02:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.773+0000 7fe6da59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6dc102ad0 0x7fe6dc198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.773+0000 7fe6da59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6dc102ad0 0x7fe6dc198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53522/0 (socket says 172.21.15.44:53522) 2024-08-20T03:02:05.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.773+0000 7fe6da59c640 1 -- 172.21.15.44:0/196205376 learned_addr learned my addr 172.21.15.44:0/196205376 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6dc1084e0 0x7fe6dc1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6da59c640 1 -- 172.21.15.44:0/196205376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6dc108e60 msgr2=0x7fe6dc19e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6dad9d640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6dc108e60 0x7fe6dc19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6da59c640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6dc108e60 0x7fe6dc19e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6da59c640 1 -- 172.21.15.44:0/196205376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6dc1084e0 msgr2=0x7fe6dc1994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6da59c640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6dc1084e0 0x7fe6dc1994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6da59c640 1 -- 172.21.15.44:0/196205376 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc19f040 con 0x7fe6dc102ad0 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6dc1084e0 0x7fe6dc1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6dad9d640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6dc108e60 0x7fe6dc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6da59c640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6dc102ad0 0x7fe6dc198f90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00c890 tx=0x7fe6cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6d37fe640 1 -- 172.21.15.44:0/196205376 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc004220 con 0x7fe6dc102ad0 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/196205376 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc19f330 con 0x7fe6dc102ad0 2024-08-20T03:02:05.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6d37fe640 1 -- 172.21.15.44:0/196205376 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6cc0043c0 con 0x7fe6dc102ad0 2024-08-20T03:02:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6d37fe640 1 -- 172.21.15.44:0/196205376 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc004a00 con 0x7fe6dc102ad0 2024-08-20T03:02:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.774+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/196205376 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc1a5bc0 con 0x7fe6dc102ad0 2024-08-20T03:02:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.777+0000 7fe6e32cd740 1 Processor -- start 2024-08-20T03:02:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.777+0000 7fe6e32cd740 1 -- start start 2024-08-20T03:02:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.777+0000 7fe6d37fe640 1 -- 172.21.15.44:0/196205376 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7fe6cc0079b0 con 0x7fe6dc102ad0 2024-08-20T03:02:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6e32cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 0x564a6ac9dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6e32cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6ac9e510 0x564a6aca9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6e32cd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 0x564a6acac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6e32cd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564a6ac146a0 con 0x564a6acaa4a0 2024-08-20T03:02:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6e32cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564a6ac03ba0 con 0x564a6ac9e510 2024-08-20T03:02:05.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6e32cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564a6ac07be0 con 0x564a6ac9dbf0 2024-08-20T03:02:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6d9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 0x564a6acac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6da59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6ac9e510 0x564a6aca9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6d9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 0x564a6acac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53524/0 (socket says 172.21.15.44:53524) 2024-08-20T03:02:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6d9d9b640 1 -- 172.21.15.44:0/968888633 learned_addr learned my addr 172.21.15.44:0/968888633 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6dad9d640 1 --2- 172.21.15.44:0/968888633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 0x564a6ac9dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6d9d9b640 1 -- 172.21.15.44:0/968888633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 msgr2=0x564a6ac9dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/968888633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 0x564a6ac9dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6d9d9b640 1 -- 172.21.15.44:0/968888633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6ac9e510 msgr2=0x564a6aca9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/968888633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6ac9e510 0x564a6aca9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6d9d9b640 1 -- 172.21.15.44:0/968888633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a6a963540 con 0x564a6acaa4a0 2024-08-20T03:02:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/968888633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 0x564a6acac890 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400aa00 tx=0x7fe6c400aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203cf914851621cb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6d37fe640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe6c80b3c20 0x7fe6c80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6d0ff9640 1 -- 172.21.15.44:0/968888633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4015070 con 0x564a6acaa4a0 2024-08-20T03:02:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.778+0000 7fe6dad9d640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe6c80b3c20 0x7fe6c80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.779+0000 7fe6d37fe640 1 -- 172.21.15.44:0/196205376 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe6cc0e3b30 con 0x7fe6dc102ad0 2024-08-20T03:02:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.779+0000 7fe6d0ff9640 1 -- 172.21.15.44:0/968888633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6c400cc60 con 0x564a6acaa4a0 2024-08-20T03:02:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.779+0000 7fe6d0ff9640 1 -- 172.21.15.44:0/968888633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c40105b0 con 0x564a6acaa4a0 2024-08-20T03:02:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.779+0000 7fe6e32cd740 1 -- 172.21.15.44:0/968888633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 msgr2=0x564a6acac890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.779+0000 7fe6dad9d640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe6c80b3c20 0x7fe6c80b60e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009ca0 tx=0x7fe6d40093f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.779+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/968888633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 0x564a6acac890 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400aa00 tx=0x7fe6c400aec0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.779+0000 7fe6e32cd740 1 -- 172.21.15.44:0/968888633 shutdown_connections 2024-08-20T03:02:05.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.779+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/968888633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 0x564a6acac890 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.779+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/968888633 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6ac9e510 0x564a6aca9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.779+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/968888633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 0x564a6ac9dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.779+0000 7fe6e32cd740 1 -- 172.21.15.44:0/968888633 >> 172.21.15.44:0/968888633 conn(0x564a6ac9ba20 msgr2=0x564a6ac9cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6e32cd740 1 -- 172.21.15.44:0/968888633 shutdown_connections 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6e32cd740 1 -- 172.21.15.44:0/968888633 wait complete. 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6e32cd740 1 -- start start 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6e32cd740 1 Processor -- start 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6e32cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 0x564a6aca9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6e32cd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 0x564a6aca73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6e32cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6acaa4a0 0x564a6aca7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6e32cd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564a6aa665f0 con 0x564a6ac9e510 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6e32cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564a6ab8a6c0 con 0x564a6acaa4a0 2024-08-20T03:02:05.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6e32cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564a6acad150 con 0x564a6ac9dbf0 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6e32cd740 10 client.0 _ll_register_callbacks cb 0x564a6abc71e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6d9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 0x564a6aca73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6d9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 0x564a6aca73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53538/0 (socket says 172.21.15.44:53538) 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6d9d9b640 1 -- 172.21.15.44:0/2956166559 learned_addr learned my addr 172.21.15.44:0/2956166559 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6dad9d640 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6acaa4a0 0x564a6aca7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6da59c640 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 0x564a6aca9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6d9d9b640 1 -- 172.21.15.44:0/2956166559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 msgr2=0x564a6aca9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 0x564a6aca9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6d9d9b640 1 -- 172.21.15.44:0/2956166559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6acaa4a0 msgr2=0x564a6aca7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6acaa4a0 0x564a6aca7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6d9d9b640 1 -- 172.21.15.44:0/2956166559 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a6acb6480 con 0x564a6ac9e510 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.780+0000 7fe6dad9d640 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6acaa4a0 0x564a6aca7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:05.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.781+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 0x564a6aca73f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4008010 tx=0x7fe6c4011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805f0bc53e81852a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.781+0000 7fe6c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.781+0000 7fe6c1ffb640 1 -- 172.21.15.44:0/2956166559 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4020070 con 0x564a6ac9e510 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.781+0000 7fe6c1ffb640 1 -- 172.21.15.44:0/2956166559 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe6c401a450 con 0x564a6ac9e510 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.781+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2956166559 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c400cc60 con 0x564a6ac9e510 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.781+0000 7fe6c1ffb640 1 -- 172.21.15.44:0/2956166559 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe6c4015040 con 0x564a6ac9e510 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.781+0000 7fe6e32cd740 10 client.15372 fetch_fsmap learned FSMap version 27 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.781+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2956166559 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a6acb8e30 con 0x564a6ac9e510 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.781+0000 7fe6c1ffb640 1 -- 172.21.15.44:0/2956166559 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6c400cc60 con 0x564a6ac9e510 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.781+0000 7fe6c1ffb640 1 -- 172.21.15.44:0/2956166559 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe6c4024760 con 0x564a6ac9e510 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.781+0000 7fe6e32cd740 10 client.15372 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.781+0000 7fe6e32cd740 10 client.15372 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.782+0000 7fe6e32cd740 20 client.15372 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.782+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564a6acbb160 0x564a6acbc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.782+0000 7fe6e32cd740 4 client.15372 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:02:05.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.782+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2956166559 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a6ac165f0 con 0x564a6acbb160 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.782+0000 7fe6da59c640 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564a6acbb160 0x564a6acbc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.782+0000 7fe6c1ffb640 10 client.15372 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.782+0000 7fe6da59c640 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564a6acbb160 0x564a6acbc5d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8c995d1b1efc1691 server_cookie=e2440906384c67f6 in_seq=0 out_seq=0 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6c1ffb640 1 -- 172.21.15.44:0/2956166559 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a6ac165f0 con 0x564a6acbb160 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6c1ffb640 10 client.15372 handle_command_reply: tid=0 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6e32cd740 1 client.15372 shutdown 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2956166559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 msgr2=0x564a6aca73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 0x564a6aca73f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4008010 tx=0x7fe6c4011970 comp rx=0 tx=0).stop 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2956166559 shutdown_connections 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564a6acbb160 0x564a6acbc5d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6acaa4a0 0x564a6aca7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 0x564a6aca73f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6c1ffb640 0 client.15372 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:02:05.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/2956166559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 0x564a6aca9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2956166559 >> 172.21.15.44:0/2956166559 conn(0x564a6ac9ba20 msgr2=0x564a6aca15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:05.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2956166559 shutdown_connections 2024-08-20T03:02:05.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2956166559 wait complete. 2024-08-20T03:02:05.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.783+0000 7fe6e32cd740 20 client.15372 trim_cache size 0 max 16384 2024-08-20T03:02:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.801+0000 7fe6e32cd740 1 Processor -- start 2024-08-20T03:02:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.801+0000 7fe6e32cd740 1 -- start start 2024-08-20T03:02:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6e32cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6ac9dbf0 0x564a6ac96710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6e32cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9e510 0x564a6aca90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6e32cd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 0x564a6acc4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6e32cd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564a6acba910 con 0x564a6acaa4a0 2024-08-20T03:02:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6e32cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564a6acbafe0 con 0x564a6ac9dbf0 2024-08-20T03:02:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6e32cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564a6acbb6b0 con 0x564a6ac9e510 2024-08-20T03:02:05.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6dad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 0x564a6acc4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6dad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 0x564a6acc4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53544/0 (socket says 172.21.15.44:53544) 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6dad9d640 1 -- 172.21.15.44:0/2021421792 learned_addr learned my addr 172.21.15.44:0/2021421792 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/2021421792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9e510 0x564a6aca90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6da59c640 1 --2- 172.21.15.44:0/2021421792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6ac9dbf0 0x564a6ac96710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6dad9d640 1 -- 172.21.15.44:0/2021421792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9e510 msgr2=0x564a6aca90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6dad9d640 1 --2- 172.21.15.44:0/2021421792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9e510 0x564a6aca90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6dad9d640 1 -- 172.21.15.44:0/2021421792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6ac9dbf0 msgr2=0x564a6ac96710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6dad9d640 1 --2- 172.21.15.44:0/2021421792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6ac9dbf0 0x564a6ac96710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6dad9d640 1 -- 172.21.15.44:0/2021421792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a6acbbb60 con 0x564a6acaa4a0 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/2021421792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9e510 0x564a6aca90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6dad9d640 1 --2- 172.21.15.44:0/2021421792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 0x564a6acc4ca0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4032fd0 tx=0x7fe6d403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e6d90b75cbf7f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6c3fff640 1 -- 172.21.15.44:0/2021421792 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d40318e0 con 0x564a6acaa4a0 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6c3fff640 1 -- 172.21.15.44:0/2021421792 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d4036040 con 0x564a6acaa4a0 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.802+0000 7fe6c3fff640 1 -- 172.21.15.44:0/2021421792 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4004480 con 0x564a6acaa4a0 2024-08-20T03:02:05.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.803+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2021421792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 msgr2=0x564a6acc4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.803+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/2021421792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 0x564a6acc4ca0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4032fd0 tx=0x7fe6d403f000 comp rx=0 tx=0).stop 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.803+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2021421792 shutdown_connections 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.803+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/2021421792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6acaa4a0 0x564a6acc4ca0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.803+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/2021421792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9e510 0x564a6aca90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.803+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/2021421792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6ac9dbf0 0x564a6ac96710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.803+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2021421792 >> 172.21.15.44:0/2021421792 conn(0x564a6ac9ba20 msgr2=0x564a6aca89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.803+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2021421792 shutdown_connections 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.803+0000 7fe6e32cd740 1 -- 172.21.15.44:0/2021421792 wait complete. 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.803+0000 7fe6e32cd740 1 -- start start 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.803+0000 7fe6e32cd740 1 Processor -- start 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6e32cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 0x564a6acc4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6e32cd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 0x564a6acaaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6e32cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6acaa4a0 0x564a6acab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6e32cd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564a6acbc450 con 0x564a6ac9e510 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6e32cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564a6acbbd80 con 0x564a6acaa4a0 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6e32cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564a6acbcb20 con 0x564a6ac9dbf0 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6e32cd740 10 client.0 _ll_register_callbacks cb 0x7fe6cc0e75c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6d9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 0x564a6acaaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6d9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 0x564a6acaaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53550/0 (socket says 172.21.15.44:53550) 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6d9d9b640 1 -- 172.21.15.44:0/267496792 learned_addr learned my addr 172.21.15.44:0/267496792 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6da59c640 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 0x564a6acc4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6dad9d640 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6acaa4a0 0x564a6acab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6d9d9b640 1 -- 172.21.15.44:0/267496792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 msgr2=0x564a6acc4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 0x564a6acc4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6d9d9b640 1 -- 172.21.15.44:0/267496792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6acaa4a0 msgr2=0x564a6acab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:05.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6acaa4a0 0x564a6acab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6d9d9b640 1 -- 172.21.15.44:0/267496792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a6acab9b0 con 0x564a6ac9e510 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6d9d9b640 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 0x564a6acaaf30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4004810 tx=0x7fe6c400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb189c10ccfcef15 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.804+0000 7fe6c1ffb640 1 -- 172.21.15.44:0/267496792 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c403a060 con 0x564a6ac9e510 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6c1ffb640 1 -- 172.21.15.44:0/267496792 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe6c4024400 con 0x564a6ac9e510 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6e32cd740 1 -- 172.21.15.44:0/267496792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6d4036040 con 0x564a6ac9e510 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6c1ffb640 1 -- 172.21.15.44:0/267496792 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe6c4031050 con 0x564a6ac9e510 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6e32cd740 10 client.15378 fetch_fsmap learned FSMap version 27 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6e32cd740 1 -- 172.21.15.44:0/267496792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564a6ad29470 con 0x564a6ac9e510 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6c1ffb640 1 -- 172.21.15.44:0/267496792 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6d4036040 con 0x564a6ac9e510 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6c1ffb640 1 -- 172.21.15.44:0/267496792 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe6c403d020 con 0x564a6ac9e510 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6e32cd740 10 client.15378 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6e32cd740 10 client.15378 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T03:02:05.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6e32cd740 20 client.15378 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564a6ad2e010 0x564a6ad2cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6e32cd740 4 client.15378 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.805+0000 7fe6e32cd740 1 -- 172.21.15.44:0/267496792 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "status"}) -- 0x564a6ac12090 con 0x564a6ad2e010 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6da59c640 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564a6ad2e010 0x564a6ad2cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6da59c640 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564a6ad2e010 0x564a6ad2cd20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2b96ae30b4e982 server_cookie=73ef9a561f3e5e06 in_seq=0 out_seq=0 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6c1ffb640 10 client.15378 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6c1ffb640 1 -- 172.21.15.44:0/267496792 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564a6ac12090 con 0x564a6ad2e010 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6c1ffb640 10 client.15378 handle_command_reply: tid=0 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6e32cd740 1 client.15378 shutdown 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6e32cd740 1 -- 172.21.15.44:0/267496792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 msgr2=0x564a6acaaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 0x564a6acaaf30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4004810 tx=0x7fe6c400cc60 comp rx=0 tx=0).stop 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6e32cd740 1 -- 172.21.15.44:0/267496792 shutdown_connections 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564a6ad2e010 0x564a6ad2cd20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564a6acaa4a0 0x564a6acab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a6ac9e510 0x564a6acaaf30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6e32cd740 1 --2- 172.21.15.44:0/267496792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564a6ac9dbf0 0x564a6acc4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e32cd740 1 -- 172.21.15.44:0/267496792 >> 172.21.15.44:0/267496792 conn(0x564a6ac9ba20 msgr2=0x564a6acb5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:05.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.806+0000 7fe6c1ffb640 0 client.15378 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:02:05.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e32cd740 1 -- 172.21.15.44:0/267496792 shutdown_connections 2024-08-20T03:02:05.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e32cd740 1 -- 172.21.15.44:0/267496792 wait complete. 2024-08-20T03:02:05.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e32cd740 20 client.15378 trim_cache size 0 max 16384 2024-08-20T03:02:05.816 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:02:05.816 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T03:02:05.816 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 1, 2024-08-20T03:02:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24596, 2024-08-20T03:02:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-08-20T03:02:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-08-20T03:02:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-08-20T03:02:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 99.455759689999994, 2024-08-20T03:02:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2024-08-20T03:02:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 77, 2024-08-20T03:02:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 75, 2024-08-20T03:02:05.817 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 117.68320567799999 2024-08-20T03:02:05.817 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:02:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/196205376 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe6c80b3c20 msgr2=0x7fe6c80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e0ed4640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe6c80b3c20 0x7fe6c80b60e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009ca0 tx=0x7fe6d40093f0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/196205376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6dc102ad0 msgr2=0x7fe6dc198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e0ed4640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6dc102ad0 0x7fe6dc198f90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00c890 tx=0x7fe6cc00cd50 comp rx=0 tx=0).stop 2024-08-20T03:02:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/196205376 shutdown_connections 2024-08-20T03:02:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e0ed4640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe6c80b3c20 0x7fe6c80b60e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e0ed4640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6dc108e60 0x7fe6dc19e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e0ed4640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6dc1084e0 0x7fe6dc1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e0ed4640 1 --2- 172.21.15.44:0/196205376 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6dc102ad0 0x7fe6dc198f90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:05.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/196205376 >> 172.21.15.44:0/196205376 conn(0x7fe6dc0fe540 msgr2=0x7fe6dc100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:05.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.807+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/196205376 shutdown_connections 2024-08-20T03:02:05.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:05.808+0000 7fe6e0ed4640 1 -- 172.21.15.44:0/196205376 wait complete. 2024-08-20T03:02:06.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:05 smithi044 ceph-mon[18790]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-20T03:02:06.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:05 smithi165 ceph-mon[22357]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-20T03:02:06.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:05 smithi099 ceph-mon[21984]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-20T03:02:06.594 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.i status 2024-08-20T03:02:06.844 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.308+0000 7f25ce13f640 1 -- 172.21.15.44:0/3779896118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c81073e0 msgr2=0x7f25c8107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.308+0000 7f25ce13f640 1 --2- 172.21.15.44:0/3779896118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c81073e0 0x7f25c8107820 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009900 tx=0x7f25bc02a410 comp rx=0 tx=0).stop 2024-08-20T03:02:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.308+0000 7f25ce13f640 1 -- 172.21.15.44:0/3779896118 shutdown_connections 2024-08-20T03:02:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.308+0000 7f25ce13f640 1 --2- 172.21.15.44:0/3779896118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c8107d60 0x7f25c810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.308+0000 7f25ce13f640 1 --2- 172.21.15.44:0/3779896118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c81073e0 0x7f25c8107820 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.308+0000 7f25ce13f640 1 --2- 172.21.15.44:0/3779896118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c81019d0 0x7f25c8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.308+0000 7f25ce13f640 1 -- 172.21.15.44:0/3779896118 >> 172.21.15.44:0/3779896118 conn(0x7f25c80fd440 msgr2=0x7f25c80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:07.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.308+0000 7f25ce13f640 1 -- 172.21.15.44:0/3779896118 shutdown_connections 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.308+0000 7f25ce13f640 1 -- 172.21.15.44:0/3779896118 wait complete. 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.309+0000 7f25ce13f640 1 Processor -- start 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.309+0000 7f25ce13f640 1 -- start start 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.309+0000 7f25ce13f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c81019d0 0x7f25c8198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.309+0000 7f25ce13f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c81073e0 0x7f25c81994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.309+0000 7f25ce13f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c8107d60 0x7f25c819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.309+0000 7f25ce13f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8111a20 con 0x7f25c81073e0 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.309+0000 7f25ce13f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f25c81118a0 con 0x7f25c8107d60 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.309+0000 7f25ce13f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8111ba0 con 0x7f25c81019d0 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.309+0000 7f25c6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c81073e0 0x7f25c81994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.309+0000 7f25c6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c81073e0 0x7f25c81994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53558/0 (socket says 172.21.15.44:53558) 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.309+0000 7f25c6ffd640 1 -- 172.21.15.44:0/1752149268 learned_addr learned my addr 172.21.15.44:0/1752149268 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:07.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25c7fff640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c8107d60 0x7f25c819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25c77fe640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c81019d0 0x7f25c8198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25c6ffd640 1 -- 172.21.15.44:0/1752149268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c81019d0 msgr2=0x7f25c8198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25c6ffd640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c81019d0 0x7f25c8198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25c6ffd640 1 -- 172.21.15.44:0/1752149268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c8107d60 msgr2=0x7f25c819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25c6ffd640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c8107d60 0x7f25c819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25c6ffd640 1 -- 172.21.15.44:0/1752149268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c819f050 con 0x7f25c81073e0 2024-08-20T03:02:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25c6ffd640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c81073e0 0x7f25c81994e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005940 tx=0x7f25bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25c4ff9640 1 -- 172.21.15.44:0/1752149268 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc03f030 con 0x7f25c81073e0 2024-08-20T03:02:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25c4ff9640 1 -- 172.21.15.44:0/1752149268 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25bc003aa0 con 0x7f25c81073e0 2024-08-20T03:02:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25ce13f640 1 -- 172.21.15.44:0/1752149268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c819f2e0 con 0x7f25c81073e0 2024-08-20T03:02:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25c4ff9640 1 -- 172.21.15.44:0/1752149268 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc035070 con 0x7f25c81073e0 2024-08-20T03:02:07.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.310+0000 7f25ce13f640 1 -- 172.21.15.44:0/1752149268 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c81a5a00 con 0x7f25c81073e0 2024-08-20T03:02:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.312+0000 7f25c4ff9640 1 -- 172.21.15.44:0/1752149268 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f25bc042030 con 0x7f25c81073e0 2024-08-20T03:02:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25d0538740 1 Processor -- start 2024-08-20T03:02:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25d0538740 1 -- start start 2024-08-20T03:02:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25d0538740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e4fedfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25d0538740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 0x5597e4ff9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25d0538740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4ffa4a0 0x5597e4ffc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25d0538740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5597e4f646a0 con 0x5597e4fee510 2024-08-20T03:02:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25d0538740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5597e4f53ba0 con 0x5597e4fedbf0 2024-08-20T03:02:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25d0538740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5597e4f57be0 con 0x5597e4ffa4a0 2024-08-20T03:02:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25c77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 0x5597e4ff9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25c77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 0x5597e4ff9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53568/0 (socket says 172.21.15.44:53568) 2024-08-20T03:02:07.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25c77fe640 1 -- 172.21.15.44:0/662255839 learned_addr learned my addr 172.21.15.44:0/662255839 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25c4ff9640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f25b00b3c20 0x7f25b00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25c7fff640 1 --2- 172.21.15.44:0/662255839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e4fedfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25c77fe640 1 -- 172.21.15.44:0/662255839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4ffa4a0 msgr2=0x5597e4ffc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25c77fe640 1 --2- 172.21.15.44:0/662255839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4ffa4a0 0x5597e4ffc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25c77fe640 1 -- 172.21.15.44:0/662255839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 msgr2=0x5597e4fedfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25c77fe640 1 --2- 172.21.15.44:0/662255839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e4fedfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.313+0000 7f25c77fe640 1 -- 172.21.15.44:0/662255839 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e4cb3540 con 0x5597e4fee510 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.314+0000 7f25c7fff640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f25b00b3c20 0x7f25b00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.314+0000 7f25c4ff9640 1 -- 172.21.15.44:0/1752149268 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f25bc1043e0 con 0x7f25c81073e0 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.314+0000 7f25c77fe640 1 --2- 172.21.15.44:0/662255839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 0x5597e4ff9f60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009c80 tx=0x7f25b8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4facd0d6770995b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.314+0000 7f25adffb640 1 -- 172.21.15.44:0/662255839 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b801b480 con 0x5597e4fee510 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.314+0000 7f25adffb640 1 -- 172.21.15.44:0/662255839 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b8012900 con 0x5597e4fee510 2024-08-20T03:02:07.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.314+0000 7f25adffb640 1 -- 172.21.15.44:0/662255839 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b80118e0 con 0x5597e4fee510 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25d0538740 1 -- 172.21.15.44:0/662255839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 msgr2=0x5597e4ff9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25c7fff640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f25b00b3c20 0x7f25b00b60e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009370 tx=0x7f25c0004a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25d0538740 1 --2- 172.21.15.44:0/662255839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 0x5597e4ff9f60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009c80 tx=0x7f25b8010740 comp rx=0 tx=0).stop 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25d0538740 1 -- 172.21.15.44:0/662255839 shutdown_connections 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25d0538740 1 --2- 172.21.15.44:0/662255839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4ffa4a0 0x5597e4ffc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25d0538740 1 --2- 172.21.15.44:0/662255839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 0x5597e4ff9f60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25d0538740 1 --2- 172.21.15.44:0/662255839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e4fedfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25d0538740 1 -- 172.21.15.44:0/662255839 >> 172.21.15.44:0/662255839 conn(0x5597e4feba20 msgr2=0x5597e4fecbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25d0538740 1 -- 172.21.15.44:0/662255839 shutdown_connections 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25d0538740 1 -- 172.21.15.44:0/662255839 wait complete. 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25d0538740 1 -- start start 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25d0538740 1 Processor -- start 2024-08-20T03:02:07.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.315+0000 7f25d0538740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e4ff7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25d0538740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4fee510 0x5597e4ff7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25d0538740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4ffa4a0 0x5597e4ff7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25d0538740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5597e4db65f0 con 0x5597e4ffa4a0 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25d0538740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5597e4eda6c0 con 0x5597e4fedbf0 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25d0538740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5597e4ffd150 con 0x5597e4fee510 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25d0538740 10 client.0 _ll_register_callbacks cb 0x5597e4f171e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25c7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4ffa4a0 0x5597e4ff7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25c7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4ffa4a0 0x5597e4ff7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53572/0 (socket says 172.21.15.44:53572) 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25c7fff640 1 -- 172.21.15.44:0/1586075260 learned_addr learned my addr 172.21.15.44:0/1586075260 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25c6ffd640 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4fee510 0x5597e4ff7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25c77fe640 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e4ff7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25c7fff640 1 -- 172.21.15.44:0/1586075260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4fee510 msgr2=0x5597e4ff7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25c7fff640 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4fee510 0x5597e4ff7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25c7fff640 1 -- 172.21.15.44:0/1586075260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 msgr2=0x5597e4ff7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:07.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25c7fff640 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e4ff7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25c7fff640 1 -- 172.21.15.44:0/1586075260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e5006630 con 0x5597e4ffa4a0 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25c7fff640 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4ffa4a0 0x5597e4ff7ff0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f25c00126a0 tx=0x7f25c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c78f7f0c1dcca0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25a6ffd640 1 -- 172.21.15.44:0/1586075260 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c001e9d0 con 0x5597e4ffa4a0 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25a6ffd640 1 -- 172.21.15.44:0/1586075260 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f25c001eb70 con 0x5597e4ffa4a0 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.316+0000 7f25d0538740 1 -- 172.21.15.44:0/1586075260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25b8012900 con 0x5597e4ffa4a0 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.317+0000 7f25a6ffd640 1 -- 172.21.15.44:0/1586075260 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25c001ed70 con 0x5597e4ffa4a0 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.317+0000 7f25d0538740 10 client.15390 fetch_fsmap learned FSMap version 27 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.317+0000 7f25d0538740 1 -- 172.21.15.44:0/1586075260 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597e5009040 con 0x5597e4ffa4a0 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.317+0000 7f25a6ffd640 1 -- 172.21.15.44:0/1586075260 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b8012900 con 0x5597e4ffa4a0 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.317+0000 7f25a6ffd640 1 -- 172.21.15.44:0/1586075260 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f25c00284b0 con 0x5597e4ffa4a0 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.317+0000 7f25d0538740 10 client.15390 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:02:07.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.317+0000 7f25d0538740 10 client.15390 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.317+0000 7f25d0538740 20 client.15390 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.317+0000 7f25d0538740 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5597e500b660 0x5597e500cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.317+0000 7f25d0538740 4 client.15390 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.317+0000 7f25d0538740 1 -- 172.21.15.44:0/1586075260 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597e4f665f0 con 0x5597e500b660 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.317+0000 7f25c77fe640 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5597e500b660 0x5597e500cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.318+0000 7f25a6ffd640 10 client.15390 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.318+0000 7f25c77fe640 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5597e500b660 0x5597e500cad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33877aa738ede313 server_cookie=f8cacaae3059b347 in_seq=0 out_seq=0 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.318+0000 7f25a6ffd640 1 -- 172.21.15.44:0/1586075260 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597e4f665f0 con 0x5597e500b660 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.318+0000 7f25a6ffd640 10 client.15390 handle_command_reply: tid=0 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.318+0000 7f25d0538740 1 client.15390 shutdown 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.319+0000 7f25d0538740 1 -- 172.21.15.44:0/1586075260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4ffa4a0 msgr2=0x5597e4ff7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.319+0000 7f25d0538740 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4ffa4a0 0x5597e4ff7ff0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f25c00126a0 tx=0x7f25c0004ab0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.319+0000 7f25d0538740 1 -- 172.21.15.44:0/1586075260 shutdown_connections 2024-08-20T03:02:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.319+0000 7f25d0538740 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5597e500b660 0x5597e500cad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.319+0000 7f25d0538740 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4ffa4a0 0x5597e4ff7ff0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.319+0000 7f25d0538740 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4fee510 0x5597e4ff7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.319+0000 7f25d0538740 1 --2- 172.21.15.44:0/1586075260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e4ff7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.319+0000 7f25d0538740 1 -- 172.21.15.44:0/1586075260 >> 172.21.15.44:0/1586075260 conn(0x5597e4feba20 msgr2=0x5597e4ff15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.319+0000 7f25a6ffd640 0 client.15390 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.319+0000 7f25d0538740 1 -- 172.21.15.44:0/1586075260 shutdown_connections 2024-08-20T03:02:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.319+0000 7f25d0538740 1 -- 172.21.15.44:0/1586075260 wait complete. 2024-08-20T03:02:07.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.319+0000 7f25d0538740 20 client.15390 trim_cache size 0 max 16384 2024-08-20T03:02:07.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25d0538740 1 Processor -- start 2024-08-20T03:02:07.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25d0538740 1 -- start start 2024-08-20T03:02:07.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25d0538740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e500c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25d0538740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 0x5597e4ff6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25d0538740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4ffa4a0 0x5597e500a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25d0538740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5597e500ae10 con 0x5597e4fee510 2024-08-20T03:02:07.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25d0538740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5597e500b4e0 con 0x5597e4fedbf0 2024-08-20T03:02:07.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25d0538740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5597e500bbb0 con 0x5597e4ffa4a0 2024-08-20T03:02:07.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25c6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 0x5597e4ff6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25c6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 0x5597e4ff6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53580/0 (socket says 172.21.15.44:53580) 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25c6ffd640 1 -- 172.21.15.44:0/2747483252 learned_addr learned my addr 172.21.15.44:0/2747483252 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25c77fe640 1 --2- 172.21.15.44:0/2747483252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e500c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.337+0000 7f25c7fff640 1 --2- 172.21.15.44:0/2747483252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4ffa4a0 0x5597e500a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.338+0000 7f25c6ffd640 1 -- 172.21.15.44:0/2747483252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4ffa4a0 msgr2=0x5597e500a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.338+0000 7f25c6ffd640 1 --2- 172.21.15.44:0/2747483252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4ffa4a0 0x5597e500a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.338+0000 7f25c6ffd640 1 -- 172.21.15.44:0/2747483252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 msgr2=0x5597e500c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.338+0000 7f25c6ffd640 1 --2- 172.21.15.44:0/2747483252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e500c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.338+0000 7f25c6ffd640 1 -- 172.21.15.44:0/2747483252 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597e4fed5d0 con 0x5597e4fee510 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.338+0000 7f25c77fe640 1 --2- 172.21.15.44:0/2747483252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e500c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.338+0000 7f25c7fff640 1 --2- 172.21.15.44:0/2747483252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4ffa4a0 0x5597e500a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.338+0000 7f25c6ffd640 1 --2- 172.21.15.44:0/2747483252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 0x5597e4ff6e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0caf80 tx=0x7f25bc0cabd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34915ed5958e287 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.338+0000 7f25ad7fa640 1 -- 172.21.15.44:0/2747483252 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc049040 con 0x5597e4fee510 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.338+0000 7f25ad7fa640 1 -- 172.21.15.44:0/2747483252 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25bc0cb980 con 0x5597e4fee510 2024-08-20T03:02:07.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.338+0000 7f25ad7fa640 1 -- 172.21.15.44:0/2747483252 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc03bcc0 con 0x5597e4fee510 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 -- 172.21.15.44:0/2747483252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 msgr2=0x5597e4ff6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 --2- 172.21.15.44:0/2747483252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 0x5597e4ff6e00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0caf80 tx=0x7f25bc0cabd0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 -- 172.21.15.44:0/2747483252 shutdown_connections 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 --2- 172.21.15.44:0/2747483252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4ffa4a0 0x5597e500a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 --2- 172.21.15.44:0/2747483252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fee510 0x5597e4ff6e00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 --2- 172.21.15.44:0/2747483252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4fedbf0 0x5597e500c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 -- 172.21.15.44:0/2747483252 >> 172.21.15.44:0/2747483252 conn(0x5597e4feba20 msgr2=0x5597e50053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 -- 172.21.15.44:0/2747483252 shutdown_connections 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 -- 172.21.15.44:0/2747483252 wait complete. 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 -- start start 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 Processor -- start 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fedbf0 0x5597e500e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.339+0000 7f25d0538740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4fee510 0x5597e500e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25d0538740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4ffa4a0 0x5597e5013620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25d0538740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5597e500c950 con 0x5597e4fedbf0 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25d0538740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5597e500c280 con 0x5597e4ffa4a0 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25d0538740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5597e500d020 con 0x5597e4fee510 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25d0538740 10 client.0 _ll_register_callbacks cb 0x7f25c0009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fedbf0 0x5597e500e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fedbf0 0x5597e500e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53590/0 (socket says 172.21.15.44:53590) 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c77fe640 1 -- 172.21.15.44:0/685660911 learned_addr learned my addr 172.21.15.44:0/685660911 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c6ffd640 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4fee510 0x5597e500e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c7fff640 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4ffa4a0 0x5597e5013620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c77fe640 1 -- 172.21.15.44:0/685660911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4fee510 msgr2=0x5597e500e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c77fe640 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4fee510 0x5597e500e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c77fe640 1 -- 172.21.15.44:0/685660911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4ffa4a0 msgr2=0x5597e5013620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:07.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c77fe640 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4ffa4a0 0x5597e5013620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c77fe640 1 -- 172.21.15.44:0/685660911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597e5013d30 con 0x5597e4fedbf0 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c6ffd640 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4fee510 0x5597e500e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c7fff640 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4ffa4a0 0x5597e5013620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25c77fe640 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fedbf0 0x5597e500e000 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f25b8010c40 tx=0x7f25b8009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b986b5e090053fe3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25a6ffd640 1 -- 172.21.15.44:0/685660911 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b80050b0 con 0x5597e4fedbf0 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25a6ffd640 1 -- 172.21.15.44:0/685660911 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f25b8005250 con 0x5597e4fedbf0 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25d0538740 1 -- 172.21.15.44:0/685660911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25bc0cb980 con 0x5597e4fedbf0 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25a6ffd640 1 -- 172.21.15.44:0/685660911 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f25b8005450 con 0x5597e4fedbf0 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.340+0000 7f25d0538740 10 client.15396 fetch_fsmap learned FSMap version 27 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.341+0000 7f25d0538740 1 -- 172.21.15.44:0/685660911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597e5015190 con 0x5597e4fedbf0 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.341+0000 7f25a6ffd640 1 -- 172.21.15.44:0/685660911 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25bc0cb980 con 0x5597e4fedbf0 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.341+0000 7f25a6ffd640 1 -- 172.21.15.44:0/685660911 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f25b800f040 con 0x5597e4fedbf0 2024-08-20T03:02:07.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.341+0000 7f25d0538740 10 client.15396 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.341+0000 7f25d0538740 10 client.15396 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.341+0000 7f25d0538740 20 client.15396 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.341+0000 7f25d0538740 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5597e5079b40 0x5597e5078850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.341+0000 7f25d0538740 4 client.15396 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.341+0000 7f25d0538740 1 -- 172.21.15.44:0/685660911 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "status"}) -- 0x5597e4f62090 con 0x5597e5079b40 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.341+0000 7f25c6ffd640 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5597e5079b40 0x5597e5078850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25c6ffd640 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5597e5079b40 0x5597e5078850 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f12fad851f3e63 server_cookie=aee1f963aff2c1d6 in_seq=0 out_seq=0 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25a6ffd640 10 client.15396 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25a6ffd640 1 -- 172.21.15.44:0/685660911 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5597e4f62090 con 0x5597e5079b40 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25a6ffd640 10 client.15396 handle_command_reply: tid=0 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25d0538740 1 client.15396 shutdown 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25d0538740 1 -- 172.21.15.44:0/685660911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fedbf0 msgr2=0x5597e500e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25d0538740 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fedbf0 0x5597e500e000 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f25b8010c40 tx=0x7f25b8009370 comp rx=0 tx=0).stop 2024-08-20T03:02:07.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25d0538740 1 -- 172.21.15.44:0/685660911 shutdown_connections 2024-08-20T03:02:07.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25d0538740 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5597e5079b40 0x5597e5078850 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25d0538740 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5597e4ffa4a0 0x5597e5013620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25d0538740 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5597e4fee510 0x5597e500e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25d0538740 1 --2- 172.21.15.44:0/685660911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5597e4fedbf0 0x5597e500e000 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25d0538740 1 -- 172.21.15.44:0/685660911 >> 172.21.15.44:0/685660911 conn(0x5597e4feba20 msgr2=0x5597e4ff93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:07.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25a6ffd640 0 client.15396 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:07.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25d0538740 1 -- 172.21.15.44:0/685660911 shutdown_connections 2024-08-20T03:02:07.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25d0538740 1 -- 172.21.15.44:0/685660911 wait complete. 2024-08-20T03:02:07.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.342+0000 7f25d0538740 20 client.15396 trim_cache size 0 max 16384 2024-08-20T03:02:07.358 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:02:07.358 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "0284e16c-5e9f-11ef-bccf-c7b262605968", 2024-08-20T03:02:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-08-20T03:02:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 24604, 2024-08-20T03:02:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-08-20T03:02:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-08-20T03:02:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-08-20T03:02:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 108.10518988, 2024-08-20T03:02:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 27, 2024-08-20T03:02:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 77, 2024-08-20T03:02:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 75, 2024-08-20T03:02:07.359 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 128.01865913899999 2024-08-20T03:02:07.359 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:02:07.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.343+0000 7f25ce13f640 1 -- 172.21.15.44:0/1752149268 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f25b00b3c20 msgr2=0x7f25b00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:07.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.343+0000 7f25ce13f640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f25b00b3c20 0x7f25b00b60e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f25c0009370 tx=0x7f25c0004a40 comp rx=0 tx=0).stop 2024-08-20T03:02:07.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.343+0000 7f25ce13f640 1 -- 172.21.15.44:0/1752149268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c81073e0 msgr2=0x7f25c81994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:07.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.343+0000 7f25ce13f640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c81073e0 0x7f25c81994e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005940 tx=0x7f25bc004270 comp rx=0 tx=0).stop 2024-08-20T03:02:07.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.343+0000 7f25ce13f640 1 -- 172.21.15.44:0/1752149268 shutdown_connections 2024-08-20T03:02:07.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.343+0000 7f25ce13f640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f25b00b3c20 0x7f25b00b60e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.343+0000 7f25ce13f640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c8107d60 0x7f25c819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.343+0000 7f25ce13f640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c81073e0 0x7f25c81994e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.343+0000 7f25ce13f640 1 --2- 172.21.15.44:0/1752149268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c81019d0 0x7f25c8198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:07.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.343+0000 7f25ce13f640 1 -- 172.21.15.44:0/1752149268 >> 172.21.15.44:0/1752149268 conn(0x7f25c80fd440 msgr2=0x7f25c806b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:07.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.343+0000 7f25ce13f640 1 -- 172.21.15.44:0/1752149268 shutdown_connections 2024-08-20T03:02:07.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:07.343+0000 7f25ce13f640 1 -- 172.21.15.44:0/1752149268 wait complete. 2024-08-20T03:02:07.975 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:07 smithi044 ceph-mon[18790]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-08-20T03:02:08.005 INFO:teuthology.run_tasks:Running task sequential... 2024-08-20T03:02:08.015 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-08-20T03:02:08.015 INFO:teuthology.task.sequential:In sequential, running task print... 2024-08-20T03:02:08.015 INFO:teuthology.task.print:Enabling mgr modules 2024-08-20T03:02:08.015 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-08-20T03:02:08.016 INFO:teuthology.task.exec:Executing custom commands... 2024-08-20T03:02:08.016 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-08-20T03:02:08.016 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-08-20T03:02:08.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.118+0000 7f0483388640 1 -- 172.21.15.44:0/328523239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c101920 msgr2=0x7f047c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:08.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.118+0000 7f0483388640 1 --2- 172.21.15.44:0/328523239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c101920 0x7f047c101d00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f046c009ca0 tx=0x7f046c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:02:08.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.118+0000 7f0483388640 1 -- 172.21.15.44:0/328523239 shutdown_connections 2024-08-20T03:02:08.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.118+0000 7f0483388640 1 --2- 172.21.15.44:0/328523239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f047c107c50 0x7f047c10e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:08.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.118+0000 7f0483388640 1 --2- 172.21.15.44:0/328523239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047c1072d0 0x7f047c107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:08.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.118+0000 7f0483388640 1 --2- 172.21.15.44:0/328523239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c101920 0x7f047c101d00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:08.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.118+0000 7f0483388640 1 -- 172.21.15.44:0/328523239 >> 172.21.15.44:0/328523239 conn(0x7f047c078620 msgr2=0x7f047c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:08.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.118+0000 7f0483388640 1 -- 172.21.15.44:0/328523239 shutdown_connections 2024-08-20T03:02:08.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.118+0000 7f0483388640 1 -- 172.21.15.44:0/328523239 wait complete. 2024-08-20T03:02:08.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0483388640 1 Processor -- start 2024-08-20T03:02:08.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0483388640 1 -- start start 2024-08-20T03:02:08.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0483388640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047c101920 0x7f047c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:08.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0483388640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c1072d0 0x7f047c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0483388640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f047c107c50 0x7f047c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0483388640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f047c111960 con 0x7f047c1072d0 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0483388640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1117e0 con 0x7f047c101920 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0483388640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f047c111ae0 con 0x7f047c107c50 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0480923640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c1072d0 0x7f047c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0480923640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c1072d0 0x7f047c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60446/0 (socket says 172.21.15.44:60446) 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0480923640 1 -- 172.21.15.44:0/2881794773 learned_addr learned my addr 172.21.15.44:0/2881794773 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0481124640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047c101920 0x7f047c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0481925640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f047c107c50 0x7f047c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0480923640 1 -- 172.21.15.44:0/2881794773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f047c107c50 msgr2=0x7f047c19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0480923640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f047c107c50 0x7f047c19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0480923640 1 -- 172.21.15.44:0/2881794773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047c101920 msgr2=0x7f047c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:08.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0480923640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047c101920 0x7f047c199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:08.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0480923640 1 -- 172.21.15.44:0/2881794773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c1a51b0 con 0x7f047c1072d0 2024-08-20T03:02:08.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.119+0000 7f0481925640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f047c107c50 0x7f047c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:08.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.120+0000 7f0480923640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c1072d0 0x7f047c19a2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f047000d950 tx=0x7f047000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:08.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.120+0000 7f047a7fc640 1 -- 172.21.15.44:0/2881794773 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470017070 con 0x7f047c1072d0 2024-08-20T03:02:08.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.120+0000 7f047a7fc640 1 -- 172.21.15.44:0/2881794773 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0470009c30 con 0x7f047c1072d0 2024-08-20T03:02:08.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.120+0000 7f0483388640 1 -- 172.21.15.44:0/2881794773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047c1a54a0 con 0x7f047c1072d0 2024-08-20T03:02:08.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.120+0000 7f047a7fc640 1 -- 172.21.15.44:0/2881794773 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047000f620 con 0x7f047c1072d0 2024-08-20T03:02:08.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.120+0000 7f0483388640 1 -- 172.21.15.44:0/2881794773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047c1a59f0 con 0x7f047c1072d0 2024-08-20T03:02:08.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.121+0000 7f0483388640 1 -- 172.21.15.44:0/2881794773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f047c101d00 con 0x7f047c1072d0 2024-08-20T03:02:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.122+0000 7f047a7fc640 1 -- 172.21.15.44:0/2881794773 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 22) ==== 151076+0+0 (secure 0 0 0) 0x7f0470010460 con 0x7f047c1072d0 2024-08-20T03:02:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.123+0000 7f047a7fc640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f04640b3c20 0x7f04640b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.123+0000 7f0481124640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f04640b3c20 0x7f04640b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.123+0000 7f047a7fc640 1 -- 172.21.15.44:0/2881794773 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f04700e39a0 con 0x7f047c1072d0 2024-08-20T03:02:08.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.123+0000 7f0481124640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f04640b3c20 0x7f04640b60e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f046c006010 tx=0x7f046c009820 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:08.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.128+0000 7f047a7fc640 1 -- 172.21.15.44:0/2881794773 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f04700aaee0 con 0x7f047c1072d0 2024-08-20T03:02:08.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.368+0000 7f0483388640 1 -- 172.21.15.44:0/2881794773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f047c02e340 con 0x7f047c1072d0 2024-08-20T03:02:08.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:07 smithi165 ceph-mon[22357]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-08-20T03:02:08.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:07 smithi099 ceph-mon[21984]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-08-20T03:02:08.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.943+0000 7f047a7fc640 1 -- 172.21.15.44:0/2881794773 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 module 'snap_schedule' is already enabled v23) ==== 133+0+0 (secure 0 0 0) 0x7f0470009de0 con 0x7f047c1072d0 2024-08-20T03:02:08.946 INFO:teuthology.orchestra.run.smithi044.stderr:module 'snap_schedule' is already enabled 2024-08-20T03:02:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.946+0000 7f0483388640 1 -- 172.21.15.44:0/2881794773 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f04640b3c20 msgr2=0x7f04640b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:08.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.946+0000 7f0483388640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f04640b3c20 0x7f04640b60e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f046c006010 tx=0x7f046c009820 comp rx=0 tx=0).stop 2024-08-20T03:02:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.946+0000 7f0483388640 1 -- 172.21.15.44:0/2881794773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c1072d0 msgr2=0x7f047c19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.946+0000 7f0483388640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c1072d0 0x7f047c19a2f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f047000d950 tx=0x7f047000de10 comp rx=0 tx=0).stop 2024-08-20T03:02:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.946+0000 7f0483388640 1 -- 172.21.15.44:0/2881794773 shutdown_connections 2024-08-20T03:02:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.946+0000 7f0483388640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f04640b3c20 0x7f04640b60e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.946+0000 7f0483388640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f047c107c50 0x7f047c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.946+0000 7f0483388640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f047c1072d0 0x7f047c19a2f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.946+0000 7f0483388640 1 --2- 172.21.15.44:0/2881794773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047c101920 0x7f047c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.946+0000 7f0483388640 1 -- 172.21.15.44:0/2881794773 >> 172.21.15.44:0/2881794773 conn(0x7f047c078620 msgr2=0x7f047c105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.946+0000 7f0483388640 1 -- 172.21.15.44:0/2881794773 shutdown_connections 2024-08-20T03:02:08.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:08.946+0000 7f0483388640 1 -- 172.21.15.44:0/2881794773 wait complete. 2024-08-20T03:02:08.968 INFO:teuthology.run_tasks:Running task kclient... 2024-08-20T03:02:08.995 INFO:tasks.kclient:Mounting kernel clients... 2024-08-20T03:02:08.995 INFO:tasks.kclient:config is {'client.0': {}} 2024-08-20T03:02:08.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:02:08.996 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-08-20T03:02:09.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:02:09.013 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-08-20T03:02:09.080 INFO:teuthology.orchestra.run.smithi044.stderr:Cannot find device "ceph-brx" 2024-08-20T03:02:09.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T03:02:09.082 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-08-20T03:02:09.082 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-20T03:02:09.083 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-20T03:02:09.083 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs ls 2024-08-20T03:02:09.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:08 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2881794773' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-20T03:02:09.378 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:09.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:08 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2881794773' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-20T03:02:09.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:08 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2881794773' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-20T03:02:09.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.882+0000 7f62a5f78640 1 -- 172.21.15.44:0/3081424344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a01084c0 msgr2=0x7f62a0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:09.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.882+0000 7f62a5f78640 1 --2- 172.21.15.44:0/3081424344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a01084c0 0x7f62a0108900 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6288009900 tx=0x7f628802a410 comp rx=0 tx=0).stop 2024-08-20T03:02:09.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.882+0000 7f62a5f78640 1 -- 172.21.15.44:0/3081424344 shutdown_connections 2024-08-20T03:02:09.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.882+0000 7f62a5f78640 1 --2- 172.21.15.44:0/3081424344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a0108e40 0x7f62a010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.882+0000 7f62a5f78640 1 --2- 172.21.15.44:0/3081424344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a01084c0 0x7f62a0108900 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.882+0000 7f62a5f78640 1 --2- 172.21.15.44:0/3081424344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a0102ab0 0x7f62a0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.882+0000 7f62a5f78640 1 -- 172.21.15.44:0/3081424344 >> 172.21.15.44:0/3081424344 conn(0x7f62a00fe500 msgr2=0x7f62a0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.882+0000 7f62a5f78640 1 -- 172.21.15.44:0/3081424344 shutdown_connections 2024-08-20T03:02:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.882+0000 7f62a5f78640 1 -- 172.21.15.44:0/3081424344 wait complete. 2024-08-20T03:02:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.883+0000 7f62a5f78640 1 Processor -- start 2024-08-20T03:02:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.883+0000 7f62a5f78640 1 -- start start 2024-08-20T03:02:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.883+0000 7f62a5f78640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a0102ab0 0x7f62a0198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.883+0000 7f62a5f78640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a01084c0 0x7f62a01994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.883+0000 7f62a5f78640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a0108e40 0x7f62a019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.883+0000 7f62a5f78640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f62a0112b00 con 0x7f62a0108e40 2024-08-20T03:02:09.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.883+0000 7f62a5f78640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f62a0112980 con 0x7f62a01084c0 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.883+0000 7f62a5f78640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f62a0112c80 con 0x7f62a0102ab0 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.883+0000 7f629ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a0108e40 0x7f62a019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.883+0000 7f629ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a0108e40 0x7f62a019e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60472/0 (socket says 172.21.15.44:60472) 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.883+0000 7f629ffff640 1 -- 172.21.15.44:0/4152501245 learned_addr learned my addr 172.21.15.44:0/4152501245 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.883+0000 7f629f7fe640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a0102ab0 0x7f62a0198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.884+0000 7f6296ffd640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a01084c0 0x7f62a01994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.884+0000 7f629ffff640 1 -- 172.21.15.44:0/4152501245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a0102ab0 msgr2=0x7f62a0198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.884+0000 7f629ffff640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a0102ab0 0x7f62a0198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.884+0000 7f629ffff640 1 -- 172.21.15.44:0/4152501245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a01084c0 msgr2=0x7f62a01994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.884+0000 7f629ffff640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a01084c0 0x7f62a01994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.884+0000 7f629ffff640 1 -- 172.21.15.44:0/4152501245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a019f020 con 0x7f62a0108e40 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.884+0000 7f629ffff640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a0108e40 0x7f62a019e850 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f629800cce0 tx=0x7f629800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:09.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.884+0000 7f629d7fa640 1 -- 172.21.15.44:0/4152501245 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629800f450 con 0x7f62a0108e40 2024-08-20T03:02:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.884+0000 7f62a5f78640 1 -- 172.21.15.44:0/4152501245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a019f310 con 0x7f62a0108e40 2024-08-20T03:02:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.884+0000 7f629d7fa640 1 -- 172.21.15.44:0/4152501245 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629800faa0 con 0x7f62a0108e40 2024-08-20T03:02:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.884+0000 7f629d7fa640 1 -- 172.21.15.44:0/4152501245 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298018a80 con 0x7f62a0108e40 2024-08-20T03:02:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.884+0000 7f62a5f78640 1 -- 172.21.15.44:0/4152501245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a01a5a50 con 0x7f62a0108e40 2024-08-20T03:02:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.885+0000 7f62a5f78640 1 -- 172.21.15.44:0/4152501245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a0102e90 con 0x7f62a0108e40 2024-08-20T03:02:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.886+0000 7f629d7fa640 1 -- 172.21.15.44:0/4152501245 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f629800fc50 con 0x7f62a0108e40 2024-08-20T03:02:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.887+0000 7f629d7fa640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f628c0b3c20 0x7f628c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.887+0000 7f629f7fe640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f628c0b3c20 0x7f628c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.887+0000 7f629d7fa640 1 -- 172.21.15.44:0/4152501245 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f62980e41c0 con 0x7f62a0108e40 2024-08-20T03:02:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.888+0000 7f629f7fe640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f628c0b3c20 0x7f628c0b60e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6290009990 tx=0x7f6290009110 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:09.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:09.891+0000 7f629d7fa640 1 -- 172.21.15.44:0/4152501245 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f6298014030 con 0x7f62a0108e40 2024-08-20T03:02:10.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.111+0000 7f62a5f78640 1 -- 172.21.15.44:0/4152501245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f62a010a690 con 0x7f62a0108e40 2024-08-20T03:02:10.115 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:09 smithi044 ceph-mon[18790]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-20T03:02:10.115 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:09 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2881794773' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-20T03:02:10.115 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:09 smithi044 ceph-mon[18790]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-08-20T03:02:10.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.111+0000 7f629d7fa640 1 -- 172.21.15.44:0/4152501245 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f62980ab760 con 0x7f62a0108e40 2024-08-20T03:02:10.115 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-20T03:02:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.113+0000 7f62a5f78640 1 -- 172.21.15.44:0/4152501245 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f628c0b3c20 msgr2=0x7f628c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.113+0000 7f62a5f78640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f628c0b3c20 0x7f628c0b60e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6290009990 tx=0x7f6290009110 comp rx=0 tx=0).stop 2024-08-20T03:02:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.113+0000 7f62a5f78640 1 -- 172.21.15.44:0/4152501245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a0108e40 msgr2=0x7f62a019e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.113+0000 7f62a5f78640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a0108e40 0x7f62a019e850 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f629800cce0 tx=0x7f629800a620 comp rx=0 tx=0).stop 2024-08-20T03:02:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.114+0000 7f62a5f78640 1 -- 172.21.15.44:0/4152501245 shutdown_connections 2024-08-20T03:02:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.114+0000 7f62a5f78640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f628c0b3c20 0x7f628c0b60e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.114+0000 7f62a5f78640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f62a0108e40 0x7f62a019e850 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.114+0000 7f62a5f78640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62a01084c0 0x7f62a01994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:10.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.114+0000 7f62a5f78640 1 --2- 172.21.15.44:0/4152501245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62a0102ab0 0x7f62a0198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.114+0000 7f62a5f78640 1 -- 172.21.15.44:0/4152501245 >> 172.21.15.44:0/4152501245 conn(0x7f62a00fe500 msgr2=0x7f62a01008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.114+0000 7f62a5f78640 1 -- 172.21.15.44:0/4152501245 shutdown_connections 2024-08-20T03:02:10.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:10.114+0000 7f62a5f78640 1 -- 172.21.15.44:0/4152501245 wait complete. 2024-08-20T03:02:10.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:09 smithi165 ceph-mon[22357]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-20T03:02:10.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:09 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2881794773' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-20T03:02:10.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:09 smithi165 ceph-mon[22357]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-08-20T03:02:10.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:09 smithi099 ceph-mon[21984]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-20T03:02:10.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:09 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2881794773' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-20T03:02:10.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:09 smithi099 ceph-mon[21984]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-08-20T03:02:10.906 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-20T03:02:10.907 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-20T03:02:10.907 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-08-20T03:02:10.907 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-20T03:02:10.907 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-20T03:02:10.907 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-20T03:02:10.907 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-20T03:02:10.907 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-20T03:02:10.907 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-20T03:02:10.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:02:10.907 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-08-20T03:02:10.924 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-20T03:02:10.924 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-20T03:02:10.924 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-08-20T03:02:10.924 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-08-20T03:02:10.925 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-08-20T03:02:10.925 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-08-20T03:02:10.925 INFO:teuthology.orchestra.run.smithi044.stdout:2: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-20T03:02:10.925 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-08-20T03:02:10.925 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-08-20T03:02:10.925 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-08-20T03:02:10.925 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42168sec preferred_lft 42168sec 2024-08-20T03:02:10.925 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::858d:6445:9466:81d1/64 scope link noprefixroute 2024-08-20T03:02:10.925 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-08-20T03:02:10.925 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-20T03:02:10.925 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-08-20T03:02:10.926 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-08-20T03:02:10.926 INFO:teuthology.orchestra.run.smithi044.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-20T03:02:10.926 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-08-20T03:02:10.926 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-08-20T03:02:10.926 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-20T03:02:10.926 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-08-20T03:02:10.926 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-08-20T03:02:10.926 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-20T03:02:10.927 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T03:02:10.927 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-08-20T03:02:10.927 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2024-08-20T03:02:10.927 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2024-08-20T03:02:10.927 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2024-08-20T03:02:10.927 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-20T03:02:10.927 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-08-20T03:02:11.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T03:02:11.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T03:02:11.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:02:11.128 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-20T03:02:11.144 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:10 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/4152501245' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-20T03:02:11.152 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-08-20T03:02:11.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:02:11.154 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-08-20T03:02:11.220 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-08-20T03:02:11.220 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-20T03:02:11.220 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-08-20T03:02:11.220 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-08-20T03:02:11.220 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-20T03:02:11.220 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T03:02:11.221 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-08-20T03:02:11.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-08-20T03:02:11.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-08-20T03:02:11.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-08-20T03:02:11.221 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-08-20T03:02:11.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T03:02:11.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:10 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/4152501245' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-20T03:02:11.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:10 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/4152501245' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-20T03:02:11.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T03:02:11.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:02:11.503 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-08-20T03:02:11.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:02:11.519 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-08-20T03:02:11.574 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T03:02:11.574 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-08-20T03:02:11.574 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-20T03:02:11.574 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-20T03:02:11.574 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-08-20T03:02:11.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T03:02:11.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T03:02:11.690 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-20T03:02:11.690 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T03:02:11.690 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-08-20T03:02:11.691 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-20T03:02:11.691 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-20T03:02:11.691 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-20T03:02:11.691 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-20T03:02:11.691 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-20T03:02:11.691 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-08-20T03:02:11.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T03:02:11.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T03:02:11.817 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T03:02:11.817 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-08-20T03:02:11.817 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-08-20T03:02:11.817 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2024-08-20T03:02:11.817 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-08-20T03:02:11.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T03:02:11.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T03:02:11.893 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-20T03:02:11.893 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-20T03:02:11.908 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-20T03:02:11.908 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-20T03:02:11.908 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-20T03:02:11.963 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-08-20T03:02:11.963 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-08-20T03:02:11.964 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2 2024-08-20T03:02:11.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:02:11.964 DEBUG:teuthology.orchestra.run.smithi044:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-08-20T03:02:12.063 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-08-20T03:02:12.063 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-08-20T03:02:12.199 INFO:teuthology.orchestra.run.smithi044.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-08-20T03:02:12.199 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-08-20T03:02:12.199 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: resolved to: "172.21.15.44:6789,172.21.15.99:6789,172.21.15.165:6789" 2024-08-20T03:02:12.199 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: trying mount with new device syntax: 0@0284e16c-5e9f-11ef-bccf-c7b262605968.cephfs=/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2 2024-08-20T03:02:12.200 INFO:teuthology.orchestra.run.smithi044.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.44:6789/172.21.15.99:6789/172.21.15.165:6789" will pass to kernel 2024-08-20T03:02:12.200 INFO:teuthology.orchestra.run.smithi044.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-08-20T03:02:12.200 INFO:teuthology.orchestra.run.smithi044.stdout: You just mounted a file system that supports labels which does not 2024-08-20T03:02:12.201 INFO:teuthology.orchestra.run.smithi044.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-20T03:02:12.201 INFO:teuthology.orchestra.run.smithi044.stdout: applications will generate AVC messages and not be allowed access to 2024-08-20T03:02:12.201 INFO:teuthology.orchestra.run.smithi044.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-20T03:02:12.202 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-20T03:02:12.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:02:12.202 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-20T03:02:12.224 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:11 smithi044 ceph-mon[18790]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-20T03:02:12.229 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-20T03:02:12.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-20T03:02:12.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T03:02:12.326 INFO:teuthology.orchestra.run.smithi044.stdout:client15411 2024-08-20T03:02:12.327 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T03:02:12.327 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-20T03:02:12.343 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-20T03:02:12.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:11 smithi165 ceph-mon[22357]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-20T03:02:12.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:11 smithi099 ceph-mon[21984]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-08-20T03:02:12.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-20T03:02:12.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T03:02:12.439 INFO:teuthology.orchestra.run.smithi044.stdout:client15411 2024-08-20T03:02:12.440 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-20T03:02:12.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-20T03:02:12.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T03:02:12.496 INFO:teuthology.orchestra.run.smithi044.stdout:client15411 2024-08-20T03:02:12.496 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968.client15411/status') 2024-08-20T03:02:12.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T03:02:12.555 INFO:teuthology.orchestra.run.smithi044.stdout:instance: client.15411 (3)192.168.0.1:0/59020579 2024-08-20T03:02:12.555 INFO:teuthology.orchestra.run.smithi044.stdout:blocklisted: false 2024-08-20T03:02:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records in 2024-08-20T03:02:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:0+1 records out 2024-08-20T03:02:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:68 bytes copied, 2.4214e-05 s, 2.8 MB/s 2024-08-20T03:02:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T03:02:12.563 INFO:teuthology.run_tasks:Running task exec... 2024-08-20T03:02:12.571 INFO:teuthology.task.exec:Executing custom commands... 2024-08-20T03:02:12.572 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-08-20T03:02:12.572 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-08-20T03:02:12.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.671+0000 7f86f67a2640 1 -- 172.21.15.44:0/4203457653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f0107c50 msgr2=0x7f86f010e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:12.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.671+0000 7f86f67a2640 1 --2- 172.21.15.44:0/4203457653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f0107c50 0x7f86f010e450 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009ca0 tx=0x7f86e802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:02:12.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.671+0000 7f86f67a2640 1 -- 172.21.15.44:0/4203457653 shutdown_connections 2024-08-20T03:02:12.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.671+0000 7f86f67a2640 1 --2- 172.21.15.44:0/4203457653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f0107c50 0x7f86f010e450 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:12.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.671+0000 7f86f67a2640 1 --2- 172.21.15.44:0/4203457653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86f01072d0 0x7f86f0107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:12.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.671+0000 7f86f67a2640 1 --2- 172.21.15.44:0/4203457653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86f0101920 0x7f86f0101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:12.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.671+0000 7f86f67a2640 1 -- 172.21.15.44:0/4203457653 >> 172.21.15.44:0/4203457653 conn(0x7f86f0078620 msgr2=0x7f86f00ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:12.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.671+0000 7f86f67a2640 1 -- 172.21.15.44:0/4203457653 shutdown_connections 2024-08-20T03:02:12.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.671+0000 7f86f67a2640 1 -- 172.21.15.44:0/4203457653 wait complete. 2024-08-20T03:02:12.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.671+0000 7f86f67a2640 1 Processor -- start 2024-08-20T03:02:12.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f67a2640 1 -- start start 2024-08-20T03:02:12.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f67a2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86f0101920 0x7f86f0199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:12.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f67a2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86f01072d0 0x7f86f019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:12.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f67a2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f0107c50 0x7f86f019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:12.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f67a2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01118d0 con 0x7f86f0107c50 2024-08-20T03:02:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f67a2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0111750 con 0x7f86f0101920 2024-08-20T03:02:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f67a2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0111a50 con 0x7f86f01072d0 2024-08-20T03:02:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f4d3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f0107c50 0x7f86f019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f4d3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f0107c50 0x7f86f019a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60478/0 (socket says 172.21.15.44:60478) 2024-08-20T03:02:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f4d3f640 1 -- 172.21.15.44:0/3542851249 learned_addr learned my addr 172.21.15.44:0/3542851249 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86effff640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86f0101920 0x7f86f0199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86ef7fe640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86f01072d0 0x7f86f019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f4d3f640 1 -- 172.21.15.44:0/3542851249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86f01072d0 msgr2=0x7f86f019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f4d3f640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86f01072d0 0x7f86f019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f4d3f640 1 -- 172.21.15.44:0/3542851249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86f0101920 msgr2=0x7f86f0199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:12.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f4d3f640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86f0101920 0x7f86f0199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.672+0000 7f86f4d3f640 1 -- 172.21.15.44:0/3542851249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f01a5100 con 0x7f86f0107c50 2024-08-20T03:02:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.673+0000 7f86f4d3f640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f0107c50 0x7f86f019a7c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005cc0 tx=0x7f86e8031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.673+0000 7f86ed7fa640 1 -- 172.21.15.44:0/3542851249 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8003b70 con 0x7f86f0107c50 2024-08-20T03:02:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.673+0000 7f86ed7fa640 1 -- 172.21.15.44:0/3542851249 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e80318f0 con 0x7f86f0107c50 2024-08-20T03:02:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.673+0000 7f86f67a2640 1 -- 172.21.15.44:0/3542851249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f01a5390 con 0x7f86f0107c50 2024-08-20T03:02:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.673+0000 7f86ed7fa640 1 -- 172.21.15.44:0/3542851249 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e802f950 con 0x7f86f0107c50 2024-08-20T03:02:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.673+0000 7f86f67a2640 1 -- 172.21.15.44:0/3542851249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a58e0 con 0x7f86f0107c50 2024-08-20T03:02:12.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.674+0000 7f86f67a2640 1 -- 172.21.15.44:0/3542851249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f0101d00 con 0x7f86f0107c50 2024-08-20T03:02:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.675+0000 7f86ed7fa640 1 -- 172.21.15.44:0/3542851249 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f86e802fb50 con 0x7f86f0107c50 2024-08-20T03:02:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.676+0000 7f86ed7fa640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86d80b3c20 0x7f86d80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.676+0000 7f86effff640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86d80b3c20 0x7f86d80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.676+0000 7f86ed7fa640 1 -- 172.21.15.44:0/3542851249 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f86e8105070 con 0x7f86f0107c50 2024-08-20T03:02:12.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.677+0000 7f86effff640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86d80b3c20 0x7f86d80b60e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009bd0 tx=0x7f86e0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:12.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.680+0000 7f86ed7fa640 1 -- 172.21.15.44:0/3542851249 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f86e80cc610 con 0x7f86f0107c50 2024-08-20T03:02:12.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.904+0000 7f86f67a2640 1 -- 172.21.15.44:0/3542851249 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f86f002e340 con 0x7f86f0107c50 2024-08-20T03:02:12.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.957+0000 7f86ed7fa640 1 -- 172.21.15.44:0/3542851249 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f86e80cbf10 con 0x7f86f0107c50 2024-08-20T03:02:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.960+0000 7f86f67a2640 1 -- 172.21.15.44:0/3542851249 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86d80b3c20 msgr2=0x7f86d80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.960+0000 7f86f67a2640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86d80b3c20 0x7f86d80b60e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009bd0 tx=0x7f86e0005f90 comp rx=0 tx=0).stop 2024-08-20T03:02:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.960+0000 7f86f67a2640 1 -- 172.21.15.44:0/3542851249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f0107c50 msgr2=0x7f86f019a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:12.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.960+0000 7f86f67a2640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f0107c50 0x7f86f019a7c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005cc0 tx=0x7f86e8031700 comp rx=0 tx=0).stop 2024-08-20T03:02:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.960+0000 7f86f67a2640 1 -- 172.21.15.44:0/3542851249 shutdown_connections 2024-08-20T03:02:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.960+0000 7f86f67a2640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86d80b3c20 0x7f86d80b60e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.960+0000 7f86f67a2640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86f0107c50 0x7f86f019a7c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.960+0000 7f86f67a2640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86f01072d0 0x7f86f019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.960+0000 7f86f67a2640 1 --2- 172.21.15.44:0/3542851249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86f0101920 0x7f86f0199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.960+0000 7f86f67a2640 1 -- 172.21.15.44:0/3542851249 >> 172.21.15.44:0/3542851249 conn(0x7f86f0078620 msgr2=0x7f86f0105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.960+0000 7f86f67a2640 1 -- 172.21.15.44:0/3542851249 shutdown_connections 2024-08-20T03:02:12.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:12.960+0000 7f86f67a2640 1 -- 172.21.15.44:0/3542851249 wait complete. 2024-08-20T03:02:12.974 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-08-20T03:02:13.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.075+0000 7fe9e958f640 1 -- 172.21.15.44:0/415402673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9e41083e0 msgr2=0x7fe9e4108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.075+0000 7fe9e958f640 1 --2- 172.21.15.44:0/415402673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9e41083e0 0x7fe9e4108820 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009900 tx=0x7fe9cc02a410 comp rx=0 tx=0).stop 2024-08-20T03:02:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.075+0000 7fe9e958f640 1 -- 172.21.15.44:0/415402673 shutdown_connections 2024-08-20T03:02:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.075+0000 7fe9e958f640 1 --2- 172.21.15.44:0/415402673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9e4108d60 0x7fe9e410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.075+0000 7fe9e958f640 1 --2- 172.21.15.44:0/415402673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9e41083e0 0x7fe9e4108820 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.075+0000 7fe9e958f640 1 --2- 172.21.15.44:0/415402673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9e41029e0 0x7fe9e4102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.075+0000 7fe9e958f640 1 -- 172.21.15.44:0/415402673 >> 172.21.15.44:0/415402673 conn(0x7fe9e40fe420 msgr2=0x7fe9e4100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.075+0000 7fe9e958f640 1 -- 172.21.15.44:0/415402673 shutdown_connections 2024-08-20T03:02:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.075+0000 7fe9e958f640 1 -- 172.21.15.44:0/415402673 wait complete. 2024-08-20T03:02:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.076+0000 7fe9e958f640 1 Processor -- start 2024-08-20T03:02:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.076+0000 7fe9e958f640 1 -- start start 2024-08-20T03:02:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.076+0000 7fe9e958f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9e41029e0 0x7fe9e4198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.076+0000 7fe9e958f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9e41083e0 0x7fe9e4199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:13.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.076+0000 7fe9e958f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9e4108d60 0x7fe9e419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.076+0000 7fe9e958f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e4112a20 con 0x7fe9e41083e0 2024-08-20T03:02:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.076+0000 7fe9e958f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e41128a0 con 0x7fe9e41029e0 2024-08-20T03:02:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.076+0000 7fe9e958f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e4112ba0 con 0x7fe9e4108d60 2024-08-20T03:02:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.076+0000 7fe9e27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9e41083e0 0x7fe9e4199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.076+0000 7fe9e27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9e41083e0 0x7fe9e4199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60498/0 (socket says 172.21.15.44:60498) 2024-08-20T03:02:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9e27fc640 1 -- 172.21.15.44:0/1610556570 learned_addr learned my addr 172.21.15.44:0/1610556570 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9e2ffd640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9e41029e0 0x7fe9e4198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9e37fe640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9e4108d60 0x7fe9e419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9e27fc640 1 -- 172.21.15.44:0/1610556570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9e4108d60 msgr2=0x7fe9e419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9e27fc640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9e4108d60 0x7fe9e419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9e27fc640 1 -- 172.21.15.44:0/1610556570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9e41029e0 msgr2=0x7fe9e4198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9e27fc640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9e41029e0 0x7fe9e4198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9e27fc640 1 -- 172.21.15.44:0/1610556570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e419efe0 con 0x7fe9e41083e0 2024-08-20T03:02:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9e27fc640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9e41083e0 0x7fe9e4199480 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc005940 tx=0x7fe9cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9dbfff640 1 -- 172.21.15.44:0/1610556570 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc03f030 con 0x7fe9e41083e0 2024-08-20T03:02:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9dbfff640 1 -- 172.21.15.44:0/1610556570 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9cc003cb0 con 0x7fe9e41083e0 2024-08-20T03:02:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9e958f640 1 -- 172.21.15.44:0/1610556570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e419f270 con 0x7fe9e41083e0 2024-08-20T03:02:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9dbfff640 1 -- 172.21.15.44:0/1610556570 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc035070 con 0x7fe9e41083e0 2024-08-20T03:02:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.077+0000 7fe9e958f640 1 -- 172.21.15.44:0/1610556570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e41a5990 con 0x7fe9e41083e0 2024-08-20T03:02:13.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.078+0000 7fe9e958f640 1 -- 172.21.15.44:0/1610556570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e4102dc0 con 0x7fe9e41083e0 2024-08-20T03:02:13.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.080+0000 7fe9dbfff640 1 -- 172.21.15.44:0/1610556570 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7fe9cc042030 con 0x7fe9e41083e0 2024-08-20T03:02:13.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.080+0000 7fe9dbfff640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe9d00b3c20 0x7fe9d00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:13.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.080+0000 7fe9e2ffd640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe9d00b3c20 0x7fe9d00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:13.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.080+0000 7fe9dbfff640 1 -- 172.21.15.44:0/1610556570 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe9cc104790 con 0x7fe9e41083e0 2024-08-20T03:02:13.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.081+0000 7fe9e2ffd640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe9d00b3c20 0x7fe9d00b60e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4005fd0 tx=0x7fe9d4009680 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:13.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.084+0000 7fe9dbfff640 1 -- 172.21.15.44:0/1610556570 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe9cc0cb6e0 con 0x7fe9e41083e0 2024-08-20T03:02:13.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:12 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3542851249' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-08-20T03:02:13.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.296+0000 7fe9e958f640 1 -- 172.21.15.44:0/1610556570 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fe9e40632c0 con 0x7fe9d00b3c20 2024-08-20T03:02:13.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.321+0000 7fe9dbfff640 1 -- 172.21.15.44:0/1610556570 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fe9e40632c0 con 0x7fe9d00b3c20 2024-08-20T03:02:13.325 INFO:teuthology.orchestra.run.smithi044.stdout:{} 2024-08-20T03:02:13.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.324+0000 7fe9e958f640 1 -- 172.21.15.44:0/1610556570 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe9d00b3c20 msgr2=0x7fe9d00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.324+0000 7fe9e958f640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe9d00b3c20 0x7fe9d00b60e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4005fd0 tx=0x7fe9d4009680 comp rx=0 tx=0).stop 2024-08-20T03:02:13.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.324+0000 7fe9e958f640 1 -- 172.21.15.44:0/1610556570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9e41083e0 msgr2=0x7fe9e4199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.324+0000 7fe9e958f640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9e41083e0 0x7fe9e4199480 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc005940 tx=0x7fe9cc0042f0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.324+0000 7fe9e958f640 1 -- 172.21.15.44:0/1610556570 shutdown_connections 2024-08-20T03:02:13.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.324+0000 7fe9e958f640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe9d00b3c20 0x7fe9d00b60e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.324+0000 7fe9e958f640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9e4108d60 0x7fe9e419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.324+0000 7fe9e958f640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe9e41083e0 0x7fe9e4199480 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.324+0000 7fe9e958f640 1 --2- 172.21.15.44:0/1610556570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9e41029e0 0x7fe9e4198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.324+0000 7fe9e958f640 1 -- 172.21.15.44:0/1610556570 >> 172.21.15.44:0/1610556570 conn(0x7fe9e40fe420 msgr2=0x7fe9e4100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:13.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.324+0000 7fe9e958f640 1 -- 172.21.15.44:0/1610556570 shutdown_connections 2024-08-20T03:02:13.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.324+0000 7fe9e958f640 1 -- 172.21.15.44:0/1610556570 wait complete. 2024-08-20T03:02:13.338 INFO:teuthology.run_tasks:Running task check-counter... 2024-08-20T03:02:13.349 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-08-20T03:02:13.359 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-20T03:02:13.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:12 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3542851249' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-08-20T03:02:13.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:12 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3542851249' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-08-20T03:02:13.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.457+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2759785439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44104a70 msgr2=0x7fbb4410e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.457+0000 7fbb48e8d640 1 --2- 172.21.15.44:0/2759785439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44104a70 0x7fbb4410e470 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009ca0 tx=0x7fbb3c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2759785439 shutdown_connections 2024-08-20T03:02:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 --2- 172.21.15.44:0/2759785439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb44104a70 0x7fbb4410e470 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 --2- 172.21.15.44:0/2759785439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb441040d0 0x7fbb44104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 --2- 172.21.15.44:0/2759785439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb44101cb0 0x7fbb44102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2759785439 >> 172.21.15.44:0/2759785439 conn(0x7fbb440773c0 msgr2=0x7fbb440777d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2759785439 shutdown_connections 2024-08-20T03:02:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2759785439 wait complete. 2024-08-20T03:02:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 Processor -- start 2024-08-20T03:02:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 -- start start 2024-08-20T03:02:13.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb44101cb0 0x7fbb44198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb441040d0 0x7fbb441993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb44104a70 0x7fbb4419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbb44111930 con 0x7fbb441040d0 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb441117b0 con 0x7fbb44104a70 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.458+0000 7fbb48e8d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbb44111ab0 con 0x7fbb44101cb0 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb41d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb441040d0 0x7fbb441993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb41d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb441040d0 0x7fbb441993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60530/0 (socket says 172.21.15.44:60530) 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb41d9b640 1 -- 172.21.15.44:0/2724518940 learned_addr learned my addr 172.21.15.44:0/2724518940 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb4259c640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb44101cb0 0x7fbb44198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb42d9d640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb44104a70 0x7fbb4419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb41d9b640 1 -- 172.21.15.44:0/2724518940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb44101cb0 msgr2=0x7fbb44198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb41d9b640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb44101cb0 0x7fbb44198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb41d9b640 1 -- 172.21.15.44:0/2724518940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb44104a70 msgr2=0x7fbb4419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb41d9b640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb44104a70 0x7fbb4419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb41d9b640 1 -- 172.21.15.44:0/2724518940 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4419ef40 con 0x7fbb441040d0 2024-08-20T03:02:13.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb41d9b640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb441040d0 0x7fbb441993e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800d950 tx=0x7fbb3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:13.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb337fe640 1 -- 172.21.15.44:0/2724518940 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38017070 con 0x7fbb441040d0 2024-08-20T03:02:13.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb337fe640 1 -- 172.21.15.44:0/2724518940 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb38009c30 con 0x7fbb441040d0 2024-08-20T03:02:13.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2724518940 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4419f230 con 0x7fbb441040d0 2024-08-20T03:02:13.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb337fe640 1 -- 172.21.15.44:0/2724518940 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3800f620 con 0x7fbb441040d0 2024-08-20T03:02:13.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.459+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2724518940 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb441a58f0 con 0x7fbb441040d0 2024-08-20T03:02:13.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.460+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2724518940 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb44102090 con 0x7fbb441040d0 2024-08-20T03:02:13.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.462+0000 7fbb337fe640 1 -- 172.21.15.44:0/2724518940 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7fbb38010460 con 0x7fbb441040d0 2024-08-20T03:02:13.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.462+0000 7fbb337fe640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbb2c0b3c20 0x7fbb2c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:13.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.463+0000 7fbb4259c640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbb2c0b3c20 0x7fbb2c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:13.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.463+0000 7fbb337fe640 1 -- 172.21.15.44:0/2724518940 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbb380e3990 con 0x7fbb441040d0 2024-08-20T03:02:13.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.463+0000 7fbb4259c640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbb2c0b3c20 0x7fbb2c0b60e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbb340058c0 tx=0x7fbb3400a250 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:13.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.465+0000 7fbb337fe640 1 -- 172.21.15.44:0/2724518940 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fbb380aaed0 con 0x7fbb441040d0 2024-08-20T03:02:13.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.652+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2724518940 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb441a5c70 con 0x7fbb441040d0 2024-08-20T03:02:13.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.654+0000 7fbb337fe640 1 -- 172.21.15.44:0/2724518940 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22954 (secure 0 0 0) 0x7fbb38009de0 con 0x7fbb441040d0 2024-08-20T03:02:13.657 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:02:13.657 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":77,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","created":"2024-08-20T02:52:10.979264+0000","modified":"2024-08-20T03:00:59.484512+0000","last_up_change":"2024-08-20T02:59:26.404917+0000","last_in_change":"2024-08-20T02:59:03.707380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T02:55:46.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T03:00:14.354568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T03:00:16.869335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7708f8c-a799-4b8f-ab1e-78482d73ca1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6803","nonce":858898316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6805","nonce":858898316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6809","nonce":858898316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6807","nonce":858898316}]},"public_addr":"172.21.15.44:6803/858898316","cluster_addr":"172.21.15.44:6805/858898316","heartbeat_back_addr":"172.21.15.44:6809/858898316","heartbeat_front_addr":"172.21.15.44:6807/858898316","state":["exists","up"]},{"osd":1,"uuid":"12f07986-60bf-498f-a012-cd7ea607b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6801","nonce":4252911663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6803","nonce":4252911663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6807","nonce":4252911663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6805","nonce":4252911663}]},"public_addr":"172.21.15.99:6801/4252911663","cluster_addr":"172.21.15.99:6803/4252911663","heartbeat_back_addr":"172.21.15.99:6807/4252911663","heartbeat_front_addr":"172.21.15.99:6805/4252911663","state":["exists","up"]},{"osd":2,"uuid":"0f189000-ba7f-4412-b0d3-75e8265f6685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6801","nonce":1492740465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6803","nonce":1492740465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6807","nonce":1492740465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6805","nonce":1492740465}]},"public_addr":"172.21.15.165:6801/1492740465","cluster_addr":"172.21.15.165:6803/1492740465","heartbeat_back_addr":"172.21.15.165:6807/1492740465","heartbeat_front_addr":"172.21.15.165:6805/1492740465","state":["exists","up"]},{"osd":3,"uuid":"8f9b042c-d6b5-4eee-aa29-ffc41a48c20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6811","nonce":3423983702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6813","nonce":3423983702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6817","nonce":3423983702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6815","nonce":3423983702}]},"public_addr":"172.21.15.44:6811/3423983702","cluster_addr":"172.21.15.44:6813/3423983702","heartbeat_back_addr":"172.21.15.44:6817/3423983702","heartbeat_front_addr":"172.21.15.44:6815/3423983702","state":["exists","up"]},{"osd":4,"uuid":"4edebfdb-af10-49ee-b724-2a63b9219221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6809","nonce":3099617325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6811","nonce":3099617325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6815","nonce":3099617325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6813","nonce":3099617325}]},"public_addr":"172.21.15.99:6809/3099617325","cluster_addr":"172.21.15.99:6811/3099617325","heartbeat_back_addr":"172.21.15.99:6815/3099617325","heartbeat_front_addr":"172.21.15.99:6813/3099617325","state":["exists","up"]},{"osd":5,"uuid":"6d22c21f-cc43-4ef5-a75c-d4fc1a361c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6809","nonce":523245255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6811","nonce":523245255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6815","nonce":523245255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6813","nonce":523245255}]},"public_addr":"172.21.15.165:6809/523245255","cluster_addr":"172.21.15.165:6811/523245255","heartbeat_back_addr":"172.21.15.165:6815/523245255","heartbeat_front_addr":"172.21.15.165:6813/523245255","state":["exists","up"]},{"osd":6,"uuid":"03df0c2f-1b7c-4242-888c-e047322fd8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6819","nonce":1039098986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6821","nonce":1039098986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6825","nonce":1039098986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6823","nonce":1039098986}]},"public_addr":"172.21.15.44:6819/1039098986","cluster_addr":"172.21.15.44:6821/1039098986","heartbeat_back_addr":"172.21.15.44:6825/1039098986","heartbeat_front_addr":"172.21.15.44:6823/1039098986","state":["exists","up"]},{"osd":7,"uuid":"84bbc683-9b13-448e-897c-114458dae33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6817","nonce":1438613564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6819","nonce":1438613564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6823","nonce":1438613564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6821","nonce":1438613564}]},"public_addr":"172.21.15.99:6817/1438613564","cluster_addr":"172.21.15.99:6819/1438613564","heartbeat_back_addr":"172.21.15.99:6823/1438613564","heartbeat_front_addr":"172.21.15.99:6821/1438613564","state":["exists","up"]},{"osd":8,"uuid":"14d7ffd3-db05-4228-a12e-4c69ef65ef8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6817","nonce":632957688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6819","nonce":632957688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6823","nonce":632957688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6821","nonce":632957688}]},"public_addr":"172.21.15.165:6817/632957688","cluster_addr":"172.21.15.165:6819/632957688","heartbeat_back_addr":"172.21.15.165:6823/632957688","heartbeat_front_addr":"172.21.15.165:6821/632957688","state":["exists","up"]},{"osd":9,"uuid":"1ae468a5-3d72-4eb0-a1a6-cd87eae740b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6827","nonce":2426940093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6829","nonce":2426940093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6833","nonce":2426940093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6831","nonce":2426940093}]},"public_addr":"172.21.15.44:6827/2426940093","cluster_addr":"172.21.15.44:6829/2426940093","heartbeat_back_addr":"172.21.15.44:6833/2426940093","heartbeat_front_addr":"172.21.15.44:6831/2426940093","state":["exists","up"]},{"osd":10,"uuid":"fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6825","nonce":3275793038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6827","nonce":3275793038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6831","nonce":3275793038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6829","nonce":3275793038}]},"public_addr":"172.21.15.99:6825/3275793038","cluster_addr":"172.21.15.99:6827/3275793038","heartbeat_back_addr":"172.21.15.99:6831/3275793038","heartbeat_front_addr":"172.21.15.99:6829/3275793038","state":["exists","up"]},{"osd":11,"uuid":"50715410-1fc2-4818-b600-604a4b7cce9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6825","nonce":107688415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6827","nonce":107688415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6831","nonce":107688415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6829","nonce":107688415}]},"public_addr":"172.21.15.165:6825/107688415","cluster_addr":"172.21.15.165:6827/107688415","heartbeat_back_addr":"172.21.15.165:6831/107688415","heartbeat_front_addr":"172.21.15.165:6829/107688415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:54:56.697075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:20.486846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:43.353376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:08.333879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:32.186492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:55.891366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:19.681421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:45.475926+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:10.109795+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:34.444169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:59.147646+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:59:25.403071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2597827045":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/3582612619":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/1916809746":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2919033419":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:6800/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2331034359":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2572185146":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/4176802819":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/3654439062":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6801/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/2376733275":"2024-08-21T02:53:58.312659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T03:02:13.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.656+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2724518940 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbb2c0b3c20 msgr2=0x7fbb2c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.656+0000 7fbb48e8d640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbb2c0b3c20 0x7fbb2c0b60e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbb340058c0 tx=0x7fbb3400a250 comp rx=0 tx=0).stop 2024-08-20T03:02:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.656+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2724518940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb441040d0 msgr2=0x7fbb441993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.656+0000 7fbb48e8d640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb441040d0 0x7fbb441993e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800d950 tx=0x7fbb3800de10 comp rx=0 tx=0).stop 2024-08-20T03:02:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.656+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2724518940 shutdown_connections 2024-08-20T03:02:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.656+0000 7fbb48e8d640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbb2c0b3c20 0x7fbb2c0b60e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.656+0000 7fbb48e8d640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb44104a70 0x7fbb4419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.656+0000 7fbb48e8d640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb441040d0 0x7fbb441993e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.656+0000 7fbb48e8d640 1 --2- 172.21.15.44:0/2724518940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb44101cb0 0x7fbb44198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.656+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2724518940 >> 172.21.15.44:0/2724518940 conn(0x7fbb440773c0 msgr2=0x7fbb4406a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.656+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2724518940 shutdown_connections 2024-08-20T03:02:13.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.656+0000 7fbb48e8d640 1 -- 172.21.15.44:0/2724518940 wait complete. 2024-08-20T03:02:13.669 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:02:13 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:02:13.301+0000 7efc74f7a640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:02:13.670 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:02:13 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:02:13.301+0000 7efc74f7a640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:02:13.670 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:02:13 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:02:13.301+0000 7efc74f7a640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:02:13.670 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:02:13 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:02:13.301+0000 7efc74f7a640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:02:13.670 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:02:13 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:02:13.301+0000 7efc74f7a640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:02:13.671 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-20T02:55:46.770770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-20T03:00:14.354568+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-20T03:00:16.869335+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-20T03:02:13.672 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-20T03:02:13.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.772+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/659180950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4107c90 msgr2=0x7f0ec410e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.772+0000 7f0ec8ae5640 1 --2- 172.21.15.44:0/659180950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4107c90 0x7f0ec410e490 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009ca0 tx=0x7f0ebc02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.772+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/659180950 shutdown_connections 2024-08-20T03:02:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.772+0000 7f0ec8ae5640 1 --2- 172.21.15.44:0/659180950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4107c90 0x7f0ec410e490 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.772+0000 7f0ec8ae5640 1 --2- 172.21.15.44:0/659180950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec4107310 0x7f0ec4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.772+0000 7f0ec8ae5640 1 --2- 172.21.15.44:0/659180950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ec4101910 0x7f0ec4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.772+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/659180950 >> 172.21.15.44:0/659180950 conn(0x7f0ec40fd330 msgr2=0x7f0ec40ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.772+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/659180950 shutdown_connections 2024-08-20T03:02:13.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.772+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/659180950 wait complete. 2024-08-20T03:02:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.773+0000 7f0ec8ae5640 1 Processor -- start 2024-08-20T03:02:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.773+0000 7f0ec8ae5640 1 -- start start 2024-08-20T03:02:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.773+0000 7f0ec8ae5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ec4101910 0x7f0ec40fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.773+0000 7f0ec8ae5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec4107310 0x7f0ec40fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.773+0000 7f0ec8ae5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4107c90 0x7f0ec40fe540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.773+0000 7f0ec8ae5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec4111910 con 0x7f0ec4107c90 2024-08-20T03:02:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.773+0000 7f0ec8ae5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec4111790 con 0x7f0ec4101910 2024-08-20T03:02:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.773+0000 7f0ec8ae5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec4111a90 con 0x7f0ec4107310 2024-08-20T03:02:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.773+0000 7f0ec2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4107c90 0x7f0ec40fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.773+0000 7f0ec2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4107c90 0x7f0ec40fe540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60548/0 (socket says 172.21.15.44:60548) 2024-08-20T03:02:13.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.773+0000 7f0ec2d9d640 1 -- 172.21.15.44:0/1809817885 learned_addr learned my addr 172.21.15.44:0/1809817885 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0ec259c640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ec4101910 0x7f0ec40fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0ec1d9b640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec4107310 0x7f0ec40fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0ec2d9d640 1 -- 172.21.15.44:0/1809817885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec4107310 msgr2=0x7f0ec40fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0ec2d9d640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec4107310 0x7f0ec40fe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0ec2d9d640 1 -- 172.21.15.44:0/1809817885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ec4101910 msgr2=0x7f0ec40fdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0ec2d9d640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ec4101910 0x7f0ec40fdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0ec2d9d640 1 -- 172.21.15.44:0/1809817885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec41a7370 con 0x7f0ec4107c90 2024-08-20T03:02:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0ec2d9d640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4107c90 0x7f0ec40fe540 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc005cc0 tx=0x7f0ebc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:13.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0eb37fe640 1 -- 172.21.15.44:0/1809817885 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc003bf0 con 0x7f0ec4107c90 2024-08-20T03:02:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0eb37fe640 1 -- 172.21.15.44:0/1809817885 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ebc003d90 con 0x7f0ec4107c90 2024-08-20T03:02:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/1809817885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec41a7600 con 0x7f0ec4107c90 2024-08-20T03:02:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0eb37fe640 1 -- 172.21.15.44:0/1809817885 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc02fbc0 con 0x7f0ec4107c90 2024-08-20T03:02:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.774+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/1809817885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec41a7b50 con 0x7f0ec4107c90 2024-08-20T03:02:13.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.775+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/1809817885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec4101cf0 con 0x7f0ec4107c90 2024-08-20T03:02:13.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.777+0000 7f0eb37fe640 1 -- 172.21.15.44:0/1809817885 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f0ebc02fd60 con 0x7f0ec4107c90 2024-08-20T03:02:13.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.777+0000 7f0eb37fe640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0eac0b3c20 0x7f0eac0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:13.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.777+0000 7f0ec259c640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0eac0b3c20 0x7f0eac0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:13.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.777+0000 7f0eb37fe640 1 -- 172.21.15.44:0/1809817885 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0ebc1055a0 con 0x7f0ec4107c90 2024-08-20T03:02:13.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.778+0000 7f0ec259c640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0eac0b3c20 0x7f0eac0b60e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400bbe0 tx=0x7f0eb400b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:13.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.780+0000 7f0eb37fe640 1 -- 172.21.15.44:0/1809817885 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0ebc0ccb40 con 0x7f0ec4107c90 2024-08-20T03:02:13.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.965+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/1809817885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ec410aae0 con 0x7f0ec4107c90 2024-08-20T03:02:13.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.966+0000 7f0eb37fe640 1 -- 172.21.15.44:0/1809817885 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f0ebc0098b0 con 0x7f0ec4107c90 2024-08-20T03:02:13.969 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-08-20T03:02:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.968+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/1809817885 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0eac0b3c20 msgr2=0x7f0eac0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.968+0000 7f0ec8ae5640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0eac0b3c20 0x7f0eac0b60e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400bbe0 tx=0x7f0eb400b390 comp rx=0 tx=0).stop 2024-08-20T03:02:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.968+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/1809817885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4107c90 msgr2=0x7f0ec40fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.968+0000 7f0ec8ae5640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4107c90 0x7f0ec40fe540 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc005cc0 tx=0x7f0ebc02f380 comp rx=0 tx=0).stop 2024-08-20T03:02:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.968+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/1809817885 shutdown_connections 2024-08-20T03:02:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.968+0000 7f0ec8ae5640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0eac0b3c20 0x7f0eac0b60e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.968+0000 7f0ec8ae5640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4107c90 0x7f0ec40fe540 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.968+0000 7f0ec8ae5640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec4107310 0x7f0ec40fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.968+0000 7f0ec8ae5640 1 --2- 172.21.15.44:0/1809817885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ec4101910 0x7f0ec40fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:13.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.968+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/1809817885 >> 172.21.15.44:0/1809817885 conn(0x7f0ec40fd330 msgr2=0x7f0ec406b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:13.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.968+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/1809817885 shutdown_connections 2024-08-20T03:02:13.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:13.968+0000 7f0ec8ae5640 1 -- 172.21.15.44:0/1809817885 wait complete. 2024-08-20T03:02:13.982 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-20T03:02:14.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.083+0000 7f0afbb0b640 1 -- 172.21.15.44:0/2979566513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4105190 msgr2=0x7f0af41055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:14.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.083+0000 7f0afbb0b640 1 --2- 172.21.15.44:0/2979566513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4105190 0x7f0af41055f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009900 tx=0x7f0ae802a410 comp rx=0 tx=0).stop 2024-08-20T03:02:14.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.083+0000 7f0afbb0b640 1 -- 172.21.15.44:0/2979566513 shutdown_connections 2024-08-20T03:02:14.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.083+0000 7f0afbb0b640 1 --2- 172.21.15.44:0/2979566513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0af4105b30 0x7f0af410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.083+0000 7f0afbb0b640 1 --2- 172.21.15.44:0/2979566513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4105190 0x7f0af41055f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.083+0000 7f0afbb0b640 1 --2- 172.21.15.44:0/2979566513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0af406a2f0 0x7f0af406a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.083+0000 7f0afbb0b640 1 -- 172.21.15.44:0/2979566513 >> 172.21.15.44:0/2979566513 conn(0x7f0af4077370 msgr2=0x7f0af4077780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:14.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.083+0000 7f0afbb0b640 1 -- 172.21.15.44:0/2979566513 shutdown_connections 2024-08-20T03:02:14.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.083+0000 7f0afbb0b640 1 -- 172.21.15.44:0/2979566513 wait complete. 2024-08-20T03:02:14.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.083+0000 7f0afbb0b640 1 Processor -- start 2024-08-20T03:02:14.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.083+0000 7f0afbb0b640 1 -- start start 2024-08-20T03:02:14.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afbb0b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0af406a2f0 0x7f0af4198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:14.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afbb0b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0af4105190 0x7f0af4199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:14.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afbb0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4105b30 0x7f0af419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:14.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afbb0b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0af41129c0 con 0x7f0af4105b30 2024-08-20T03:02:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afbb0b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0af4112840 con 0x7f0af4105190 2024-08-20T03:02:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afbb0b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0af4112b40 con 0x7f0af406a2f0 2024-08-20T03:02:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afa0a8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4105b30 0x7f0af419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0af90a6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0af4105190 0x7f0af4199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afa0a8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4105b30 0x7f0af419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60572/0 (socket says 172.21.15.44:60572) 2024-08-20T03:02:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0af90a6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0af4105190 0x7f0af4199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:36732/0 (socket says 172.21.15.44:36732) 2024-08-20T03:02:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afa0a8640 1 -- 172.21.15.44:0/4224554546 learned_addr learned my addr 172.21.15.44:0/4224554546 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0af98a7640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0af406a2f0 0x7f0af4198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afa0a8640 1 -- 172.21.15.44:0/4224554546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0af406a2f0 msgr2=0x7f0af4198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:14.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afa0a8640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0af406a2f0 0x7f0af4198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afa0a8640 1 -- 172.21.15.44:0/4224554546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0af4105190 msgr2=0x7f0af4199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afa0a8640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0af4105190 0x7f0af4199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afa0a8640 1 -- 172.21.15.44:0/4224554546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af419efc0 con 0x7f0af4105b30 2024-08-20T03:02:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0afa0a8640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4105b30 0x7f0af419e7f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009fd0 tx=0x7f0ae400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.084+0000 7f0af2ffd640 1 -- 172.21.15.44:0/4224554546 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4017070 con 0x7f0af4105b30 2024-08-20T03:02:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.085+0000 7f0af2ffd640 1 -- 172.21.15.44:0/4224554546 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ae4003e20 con 0x7f0af4105b30 2024-08-20T03:02:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.085+0000 7f0afbb0b640 1 -- 172.21.15.44:0/4224554546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af419f2b0 con 0x7f0af4105b30 2024-08-20T03:02:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.085+0000 7f0af2ffd640 1 -- 172.21.15.44:0/4224554546 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4010460 con 0x7f0af4105b30 2024-08-20T03:02:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.085+0000 7f0afbb0b640 1 -- 172.21.15.44:0/4224554546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af41a5970 con 0x7f0af4105b30 2024-08-20T03:02:14.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.085+0000 7f0afbb0b640 1 -- 172.21.15.44:0/4224554546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af406a6d0 con 0x7f0af4105b30 2024-08-20T03:02:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.087+0000 7f0af2ffd640 1 -- 172.21.15.44:0/4224554546 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f0ae4003970 con 0x7f0af4105b30 2024-08-20T03:02:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.087+0000 7f0af2ffd640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0adc0b3ca0 0x7f0adc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.088+0000 7f0af98a7640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0adc0b3ca0 0x7f0adc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.088+0000 7f0af2ffd640 1 -- 172.21.15.44:0/4224554546 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0ae40e39a0 con 0x7f0af4105b30 2024-08-20T03:02:14.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.088+0000 7f0af98a7640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0adc0b3ca0 0x7f0adc0b6160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009bd0 tx=0x7f0aec005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:14.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.090+0000 7f0af2ffd640 1 -- 172.21.15.44:0/4224554546 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0ae40aa970 con 0x7f0af4105b30 2024-08-20T03:02:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.276+0000 7f0afbb0b640 1 -- 172.21.15.44:0/4224554546 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0af402e340 con 0x7f0af4105b30 2024-08-20T03:02:14.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.276+0000 7f0af2ffd640 1 -- 172.21.15.44:0/4224554546 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f0ae40b3290 con 0x7f0af4105b30 2024-08-20T03:02:14.280 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-08-20T03:02:14.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.279+0000 7f0afbb0b640 1 -- 172.21.15.44:0/4224554546 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0adc0b3ca0 msgr2=0x7f0adc0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:14.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.279+0000 7f0afbb0b640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0adc0b3ca0 0x7f0adc0b6160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009bd0 tx=0x7f0aec005f90 comp rx=0 tx=0).stop 2024-08-20T03:02:14.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.279+0000 7f0afbb0b640 1 -- 172.21.15.44:0/4224554546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4105b30 msgr2=0x7f0af419e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:14.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.279+0000 7f0afbb0b640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4105b30 0x7f0af419e7f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009fd0 tx=0x7f0ae400b260 comp rx=0 tx=0).stop 2024-08-20T03:02:14.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.279+0000 7f0afbb0b640 1 -- 172.21.15.44:0/4224554546 shutdown_connections 2024-08-20T03:02:14.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.279+0000 7f0afbb0b640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0adc0b3ca0 0x7f0adc0b6160 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.279+0000 7f0afbb0b640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0af4105b30 0x7f0af419e7f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.279+0000 7f0afbb0b640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0af4105190 0x7f0af4199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.279+0000 7f0afbb0b640 1 --2- 172.21.15.44:0/4224554546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0af406a2f0 0x7f0af4198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.279+0000 7f0afbb0b640 1 -- 172.21.15.44:0/4224554546 >> 172.21.15.44:0/4224554546 conn(0x7f0af4077370 msgr2=0x7f0af41063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:14.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.279+0000 7f0afbb0b640 1 -- 172.21.15.44:0/4224554546 shutdown_connections 2024-08-20T03:02:14.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.279+0000 7f0afbb0b640 1 -- 172.21.15.44:0/4224554546 wait complete. 2024-08-20T03:02:14.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:13 smithi044 ceph-mon[18790]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-08-20T03:02:14.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:13 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3542851249' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-20T03:02:14.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:13 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:02:14.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:13 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2724518940' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:02:14.293 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-20T03:02:14.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:13 smithi165 ceph-mon[22357]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-08-20T03:02:14.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:13 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3542851249' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-20T03:02:14.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:13 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:02:14.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:13 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2724518940' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:02:14.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.394+0000 7f5db418f640 1 -- 172.21.15.44:0/3938864031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dac075b10 msgr2=0x7f5dac10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:14.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.394+0000 7f5db418f640 1 --2- 172.21.15.44:0/3938864031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dac075b10 0x7f5dac10eaa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009900 tx=0x7f5d9c02a410 comp rx=0 tx=0).stop 2024-08-20T03:02:14.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.394+0000 7f5db418f640 1 -- 172.21.15.44:0/3938864031 shutdown_connections 2024-08-20T03:02:14.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.394+0000 7f5db418f640 1 --2- 172.21.15.44:0/3938864031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dac10efe0 0x7f5dac1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.394+0000 7f5db418f640 1 --2- 172.21.15.44:0/3938864031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dac075b10 0x7f5dac10eaa0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.394+0000 7f5db418f640 1 --2- 172.21.15.44:0/3938864031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dac0751f0 0x7f5dac0755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.394+0000 7f5db418f640 1 -- 172.21.15.44:0/3938864031 >> 172.21.15.44:0/3938864031 conn(0x7f5dac0fe440 msgr2=0x7f5dac100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.394+0000 7f5db418f640 1 -- 172.21.15.44:0/3938864031 shutdown_connections 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.394+0000 7f5db418f640 1 -- 172.21.15.44:0/3938864031 wait complete. 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.395+0000 7f5db418f640 1 Processor -- start 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.395+0000 7f5db418f640 1 -- start start 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.395+0000 7f5db418f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dac0751f0 0x7f5dac19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.395+0000 7f5db418f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dac075b10 0x7f5dac19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.395+0000 7f5db418f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dac10efe0 0x7f5dac1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.395+0000 7f5db418f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac114ca0 con 0x7f5dac0751f0 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.395+0000 7f5db418f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac114b20 con 0x7f5dac10efe0 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.395+0000 7f5db418f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac114e20 con 0x7f5dac075b10 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.395+0000 7f5db1f2b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dac0751f0 0x7f5dac19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.395+0000 7f5db1f2b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dac0751f0 0x7f5dac19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60582/0 (socket says 172.21.15.44:60582) 2024-08-20T03:02:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.395+0000 7f5db1f2b640 1 -- 172.21.15.44:0/4063259215 learned_addr learned my addr 172.21.15.44:0/4063259215 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5db272c640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dac10efe0 0x7f5dac1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5db172a640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dac075b10 0x7f5dac19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5db1f2b640 1 -- 172.21.15.44:0/4063259215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dac075b10 msgr2=0x7f5dac19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5db1f2b640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dac075b10 0x7f5dac19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5db1f2b640 1 -- 172.21.15.44:0/4063259215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dac10efe0 msgr2=0x7f5dac1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5db1f2b640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dac10efe0 0x7f5dac1a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5db1f2b640 1 -- 172.21.15.44:0/4063259215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac1a1320 con 0x7f5dac0751f0 2024-08-20T03:02:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5db172a640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dac075b10 0x7f5dac19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5db272c640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dac10efe0 0x7f5dac1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5db1f2b640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dac0751f0 0x7f5dac19b280 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5da800ba50 tx=0x7f5da800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5da2ffd640 1 -- 172.21.15.44:0/4063259215 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da8019070 con 0x7f5dac0751f0 2024-08-20T03:02:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5db418f640 1 -- 172.21.15.44:0/4063259215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dac1a1610 con 0x7f5dac0751f0 2024-08-20T03:02:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5da2ffd640 1 -- 172.21.15.44:0/4063259215 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5da800cbe0 con 0x7f5dac0751f0 2024-08-20T03:02:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5da2ffd640 1 -- 172.21.15.44:0/4063259215 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da80075d0 con 0x7f5dac0751f0 2024-08-20T03:02:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.396+0000 7f5db418f640 1 -- 172.21.15.44:0/4063259215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dac1a7eb0 con 0x7f5dac0751f0 2024-08-20T03:02:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.397+0000 7f5db418f640 1 -- 172.21.15.44:0/4063259215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dac075650 con 0x7f5dac0751f0 2024-08-20T03:02:14.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.398+0000 7f5da2ffd640 1 -- 172.21.15.44:0/4063259215 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 23) ==== 151076+0+0 (secure 0 0 0) 0x7f5da801c070 con 0x7f5dac0751f0 2024-08-20T03:02:14.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.399+0000 7f5da2ffd640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d980b3c20 0x7f5d980b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:14.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.399+0000 7f5da2ffd640 1 -- 172.21.15.44:0/4063259215 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5da80e38a0 con 0x7f5dac0751f0 2024-08-20T03:02:14.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.399+0000 7f5db172a640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d980b3c20 0x7f5d980b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:14.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.400+0000 7f5db172a640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d980b3c20 0x7f5d980b60e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c000c00 tx=0x7f5d9c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:14.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.402+0000 7f5da2ffd640 1 -- 172.21.15.44:0/4063259215 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5da80aade0 con 0x7f5dac0751f0 2024-08-20T03:02:14.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:13 smithi099 ceph-mon[21984]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 12 op/s 2024-08-20T03:02:14.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:13 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3542851249' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-20T03:02:14.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:13 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:02:14.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:13 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2724518940' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:02:14.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.587+0000 7f5db418f640 1 -- 172.21.15.44:0/4063259215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5dac1a8200 con 0x7f5dac0751f0 2024-08-20T03:02:14.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.588+0000 7f5da2ffd640 1 -- 172.21.15.44:0/4063259215 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f5da80b30b0 con 0x7f5dac0751f0 2024-08-20T03:02:14.591 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-08-20T03:02:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.590+0000 7f5db418f640 1 -- 172.21.15.44:0/4063259215 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d980b3c20 msgr2=0x7f5d980b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.590+0000 7f5db418f640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d980b3c20 0x7f5d980b60e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c000c00 tx=0x7f5d9c031400 comp rx=0 tx=0).stop 2024-08-20T03:02:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.590+0000 7f5db418f640 1 -- 172.21.15.44:0/4063259215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dac0751f0 msgr2=0x7f5dac19b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.590+0000 7f5db418f640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dac0751f0 0x7f5dac19b280 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5da800ba50 tx=0x7f5da800bf10 comp rx=0 tx=0).stop 2024-08-20T03:02:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.590+0000 7f5db418f640 1 -- 172.21.15.44:0/4063259215 shutdown_connections 2024-08-20T03:02:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.590+0000 7f5db418f640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d980b3c20 0x7f5d980b60e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.590+0000 7f5db418f640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dac10efe0 0x7f5dac1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.590+0000 7f5db418f640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dac075b10 0x7f5dac19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.590+0000 7f5db418f640 1 --2- 172.21.15.44:0/4063259215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dac0751f0 0x7f5dac19b280 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.590+0000 7f5db418f640 1 -- 172.21.15.44:0/4063259215 >> 172.21.15.44:0/4063259215 conn(0x7f5dac0fe440 msgr2=0x7f5dac0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:14.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.590+0000 7f5db418f640 1 -- 172.21.15.44:0/4063259215 shutdown_connections 2024-08-20T03:02:14.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:14.590+0000 7f5db418f640 1 -- 172.21.15.44:0/4063259215 wait complete. 2024-08-20T03:02:14.604 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-08-20T03:02:14.605 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:02:14.858 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:15.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:14 smithi044 ceph-mon[18790]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:15.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:14 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1809817885' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T03:02:15.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:14 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/4224554546' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T03:02:15.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:14 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/4063259215' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T03:02:15.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.281+0000 7fe731520640 1 -- 172.21.15.44:0/1459069559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c101990 msgr2=0x7fe72c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:15.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.281+0000 7fe731520640 1 --2- 172.21.15.44:0/1459069559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c101990 0x7fe72c101d70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009ca0 tx=0x7fe71c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:02:15.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.281+0000 7fe731520640 1 -- 172.21.15.44:0/1459069559 shutdown_connections 2024-08-20T03:02:15.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.281+0000 7fe731520640 1 --2- 172.21.15.44:0/1459069559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe72c107d20 0x7fe72c10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:15.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.281+0000 7fe731520640 1 --2- 172.21.15.44:0/1459069559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe72c1073a0 0x7fe72c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:15.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.281+0000 7fe731520640 1 --2- 172.21.15.44:0/1459069559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c101990 0x7fe72c101d70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:15.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.281+0000 7fe731520640 1 -- 172.21.15.44:0/1459069559 >> 172.21.15.44:0/1459069559 conn(0x7fe72c0fd400 msgr2=0x7fe72c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.281+0000 7fe731520640 1 -- 172.21.15.44:0/1459069559 shutdown_connections 2024-08-20T03:02:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.281+0000 7fe731520640 1 -- 172.21.15.44:0/1459069559 wait complete. 2024-08-20T03:02:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.282+0000 7fe731520640 1 Processor -- start 2024-08-20T03:02:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.282+0000 7fe731520640 1 -- start start 2024-08-20T03:02:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.282+0000 7fe731520640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe72c101990 0x7fe72c199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.282+0000 7fe731520640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe72c1073a0 0x7fe72c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.282+0000 7fe731520640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c107d20 0x7fe72c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.282+0000 7fe731520640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c1119e0 con 0x7fe72c107d20 2024-08-20T03:02:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.282+0000 7fe731520640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c111860 con 0x7fe72c1073a0 2024-08-20T03:02:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.282+0000 7fe731520640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c111b60 con 0x7fe72c101990 2024-08-20T03:02:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe72b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c107d20 0x7fe72c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:15.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe72b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c107d20 0x7fe72c19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60600/0 (socket says 172.21.15.44:60600) 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe72b7fe640 1 -- 172.21.15.44:0/3553410133 learned_addr learned my addr 172.21.15.44:0/3553410133 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe72a7fc640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe72c1073a0 0x7fe72c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe72affd640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe72c101990 0x7fe72c199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe72b7fe640 1 -- 172.21.15.44:0/3553410133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe72c101990 msgr2=0x7fe72c199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe72b7fe640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe72c101990 0x7fe72c199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe72b7fe640 1 -- 172.21.15.44:0/3553410133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe72c1073a0 msgr2=0x7fe72c19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe72b7fe640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe72c1073a0 0x7fe72c19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe72b7fe640 1 -- 172.21.15.44:0/3553410133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c1a5170 con 0x7fe72c107d20 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe72a7fc640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe72c1073a0 0x7fe72c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe72b7fe640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c107d20 0x7fe72c19a830 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c970 tx=0x7fe72400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe713fff640 1 -- 172.21.15.44:0/3553410133 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724007bb0 con 0x7fe72c107d20 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe731520640 1 -- 172.21.15.44:0/3553410133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72c197be0 con 0x7fe72c107d20 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe713fff640 1 -- 172.21.15.44:0/3553410133 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe724007d50 con 0x7fe72c107d20 2024-08-20T03:02:15.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe713fff640 1 -- 172.21.15.44:0/3553410133 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724018770 con 0x7fe72c107d20 2024-08-20T03:02:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.283+0000 7fe731520640 1 -- 172.21.15.44:0/3553410133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72c198120 con 0x7fe72c107d20 2024-08-20T03:02:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.285+0000 7fe731520640 1 -- 172.21.15.44:0/3553410133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72c101d70 con 0x7fe72c107d20 2024-08-20T03:02:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.286+0000 7fe713fff640 1 -- 172.21.15.44:0/3553410133 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7fe72400f990 con 0x7fe72c107d20 2024-08-20T03:02:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.287+0000 7fe713fff640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe7140b3cb0 0x7fe7140b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.287+0000 7fe72affd640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe7140b3cb0 0x7fe7140b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.287+0000 7fe713fff640 1 -- 172.21.15.44:0/3553410133 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe7240e4020 con 0x7fe72c107d20 2024-08-20T03:02:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.287+0000 7fe72affd640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe7140b3cb0 0x7fe7140b6170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe71c000f80 tx=0x7fe71c005ef0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:15.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.291+0000 7fe713fff640 1 -- 172.21.15.44:0/3553410133 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe724014030 con 0x7fe72c107d20 2024-08-20T03:02:15.316 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:14 smithi099 ceph-mon[21984]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:15.316 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:14 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1809817885' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T03:02:15.316 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:14 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/4224554546' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T03:02:15.316 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:14 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/4063259215' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T03:02:15.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:14 smithi165 ceph-mon[22357]: from='client.15423 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:15.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:14 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1809817885' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T03:02:15.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:14 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/4224554546' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T03:02:15.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:14 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/4063259215' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T03:02:15.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.512+0000 7fe731520640 1 -- 172.21.15.44:0/3553410133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe72c02e340 con 0x7fe72c107d20 2024-08-20T03:02:15.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.513+0000 7fe713fff640 1 -- 172.21.15.44:0/3553410133 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7fe7240ab460 con 0x7fe72c107d20 2024-08-20T03:02:15.516 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:02:15.516 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":28,"btime":"2024-08-20T03:02:12:953534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:02:12.906757+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24569,24583,24596,24589,24604]},"id":1}]} 2024-08-20T03:02:15.516 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 28 2024-08-20T03:02:15.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.515+0000 7fe731520640 1 -- 172.21.15.44:0/3553410133 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe7140b3cb0 msgr2=0x7fe7140b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:15.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.515+0000 7fe731520640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe7140b3cb0 0x7fe7140b6170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe71c000f80 tx=0x7fe71c005ef0 comp rx=0 tx=0).stop 2024-08-20T03:02:15.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.515+0000 7fe731520640 1 -- 172.21.15.44:0/3553410133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c107d20 msgr2=0x7fe72c19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:15.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.515+0000 7fe731520640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c107d20 0x7fe72c19a830 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c970 tx=0x7fe72400ce30 comp rx=0 tx=0).stop 2024-08-20T03:02:15.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.515+0000 7fe731520640 1 -- 172.21.15.44:0/3553410133 shutdown_connections 2024-08-20T03:02:15.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.515+0000 7fe731520640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe7140b3cb0 0x7fe7140b6170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:15.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.515+0000 7fe731520640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe72c107d20 0x7fe72c19a830 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:15.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.515+0000 7fe731520640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe72c1073a0 0x7fe72c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:15.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.515+0000 7fe731520640 1 --2- 172.21.15.44:0/3553410133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe72c101990 0x7fe72c199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:15.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.515+0000 7fe731520640 1 -- 172.21.15.44:0/3553410133 >> 172.21.15.44:0/3553410133 conn(0x7fe72c0fd400 msgr2=0x7fe72c105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:15.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.515+0000 7fe731520640 1 -- 172.21.15.44:0/3553410133 shutdown_connections 2024-08-20T03:02:15.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:15.515+0000 7fe731520640 1 -- 172.21.15.44:0/3553410133 wait complete. 2024-08-20T03:02:16.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:15 smithi044 ceph-mon[18790]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-08-20T03:02:16.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:15 smithi044 ceph-mon[18790]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-08-20T03:02:16.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:15 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3553410133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:02:16.381 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-08-20T03:02:16.381 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-08-20T03:02:16.381 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-20T03:02:16.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:15 smithi165 ceph-mon[22357]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-08-20T03:02:16.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:15 smithi165 ceph-mon[22357]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-08-20T03:02:16.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:15 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3553410133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:02:16.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:15 smithi099 ceph-mon[21984]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-08-20T03:02:16.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:15 smithi099 ceph-mon[21984]: mgrmap e24: x(active, since 8m), standbys: z, y 2024-08-20T03:02:16.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:15 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3553410133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:02:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.486+0000 7f99d6e95640 1 -- 172.21.15.44:0/332788504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99d01040d0 msgr2=0x7f99d0104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.486+0000 7f99d6e95640 1 --2- 172.21.15.44:0/332788504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99d01040d0 0x7f99d0104530 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009900 tx=0x7f99b802a410 comp rx=0 tx=0).stop 2024-08-20T03:02:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.486+0000 7f99d6e95640 1 -- 172.21.15.44:0/332788504 shutdown_connections 2024-08-20T03:02:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.486+0000 7f99d6e95640 1 --2- 172.21.15.44:0/332788504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99d0104a70 0x7f99d010e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.486+0000 7f99d6e95640 1 --2- 172.21.15.44:0/332788504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99d01040d0 0x7f99d0104530 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.486+0000 7f99d6e95640 1 --2- 172.21.15.44:0/332788504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99d0101cb0 0x7f99d0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.486+0000 7f99d6e95640 1 -- 172.21.15.44:0/332788504 >> 172.21.15.44:0/332788504 conn(0x7f99d00773c0 msgr2=0x7f99d00777d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.486+0000 7f99d6e95640 1 -- 172.21.15.44:0/332788504 shutdown_connections 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.486+0000 7f99d6e95640 1 -- 172.21.15.44:0/332788504 wait complete. 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.486+0000 7f99d6e95640 1 Processor -- start 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99d6e95640 1 -- start start 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99d6e95640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99d0101cb0 0x7f99d0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99d6e95640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99d01040d0 0x7f99d01983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99d6e95640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99d0104a70 0x7f99d019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99d6e95640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0111980 con 0x7f99d01040d0 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99d6e95640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0111800 con 0x7f99d0101cb0 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99d6e95640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0111b00 con 0x7f99d0104a70 2024-08-20T03:02:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99cffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99d01040d0 0x7f99d01983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99cffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99d01040d0 0x7f99d01983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60626/0 (socket says 172.21.15.44:60626) 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99cffff640 1 -- 172.21.15.44:0/593805988 learned_addr learned my addr 172.21.15.44:0/593805988 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99d4c31640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99d0101cb0 0x7f99d0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99d5432640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99d0104a70 0x7f99d019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99cffff640 1 -- 172.21.15.44:0/593805988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99d0104a70 msgr2=0x7f99d019d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99cffff640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99d0104a70 0x7f99d019d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99cffff640 1 -- 172.21.15.44:0/593805988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99d0101cb0 msgr2=0x7f99d0197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99cffff640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99d0101cb0 0x7f99d0197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99cffff640 1 -- 172.21.15.44:0/593805988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d019df40 con 0x7f99d01040d0 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99d5432640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99d0104a70 0x7f99d019d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.487+0000 7f99cffff640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99d01040d0 0x7f99d01983e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f99b8005ab0 tx=0x7f99b8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.488+0000 7f99cdffb640 1 -- 172.21.15.44:0/593805988 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b803f070 con 0x7f99d01040d0 2024-08-20T03:02:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.488+0000 7f99cdffb640 1 -- 172.21.15.44:0/593805988 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b8031d60 con 0x7f99d01040d0 2024-08-20T03:02:16.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.488+0000 7f99d6e95640 1 -- 172.21.15.44:0/593805988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d019e1d0 con 0x7f99d01040d0 2024-08-20T03:02:16.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.488+0000 7f99cdffb640 1 -- 172.21.15.44:0/593805988 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8035030 con 0x7f99d01040d0 2024-08-20T03:02:16.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.488+0000 7f99d6e95640 1 -- 172.21.15.44:0/593805988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d01a48f0 con 0x7f99d01040d0 2024-08-20T03:02:16.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.489+0000 7f99d6e95640 1 -- 172.21.15.44:0/593805988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d0102090 con 0x7f99d01040d0 2024-08-20T03:02:16.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.490+0000 7f99cdffb640 1 -- 172.21.15.44:0/593805988 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f99b8042070 con 0x7f99d01040d0 2024-08-20T03:02:16.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.491+0000 7f99cdffb640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f99bc0b3cb0 0x7f99bc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:16.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.491+0000 7f99d4c31640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f99bc0b3cb0 0x7f99bc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:16.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.492+0000 7f99cdffb640 1 -- 172.21.15.44:0/593805988 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f99b8104e30 con 0x7f99d01040d0 2024-08-20T03:02:16.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.492+0000 7f99d4c31640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f99bc0b3cb0 0x7f99bc0b6170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009990 tx=0x7f99c0009110 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:16.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.495+0000 7f99cdffb640 1 -- 172.21.15.44:0/593805988 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f99b80cc270 con 0x7f99d01040d0 2024-08-20T03:02:16.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.678+0000 7f99d6e95640 1 -- 172.21.15.44:0/593805988 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99d00632c0 con 0x7f99bc0b3cb0 2024-08-20T03:02:16.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.682+0000 7f99cdffb640 1 -- 172.21.15.44:0/593805988 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259165 (secure 0 0 0) 0x7f99d00632c0 con 0x7f99bc0b3cb0 2024-08-20T03:02:16.688 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:02:16.689 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-08-20T03:02:16.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.685+0000 7f99d6e95640 1 -- 172.21.15.44:0/593805988 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f99bc0b3cb0 msgr2=0x7f99bc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:16.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.685+0000 7f99d6e95640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f99bc0b3cb0 0x7f99bc0b6170 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009990 tx=0x7f99c0009110 comp rx=0 tx=0).stop 2024-08-20T03:02:16.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.685+0000 7f99d6e95640 1 -- 172.21.15.44:0/593805988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99d01040d0 msgr2=0x7f99d01983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:16.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.685+0000 7f99d6e95640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99d01040d0 0x7f99d01983e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f99b8005ab0 tx=0x7f99b8003cd0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.685+0000 7f99d6e95640 1 -- 172.21.15.44:0/593805988 shutdown_connections 2024-08-20T03:02:16.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.685+0000 7f99d6e95640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f99bc0b3cb0 0x7f99bc0b6170 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.685+0000 7f99d6e95640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99d0104a70 0x7f99d019d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.685+0000 7f99d6e95640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f99d01040d0 0x7f99d01983e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.685+0000 7f99d6e95640 1 --2- 172.21.15.44:0/593805988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99d0101cb0 0x7f99d0197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.685+0000 7f99d6e95640 1 -- 172.21.15.44:0/593805988 >> 172.21.15.44:0/593805988 conn(0x7f99d00773c0 msgr2=0x7f99d01052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:16.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.685+0000 7f99d6e95640 1 -- 172.21.15.44:0/593805988 shutdown_connections 2024-08-20T03:02:16.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.685+0000 7f99d6e95640 1 -- 172.21.15.44:0/593805988 wait complete. 2024-08-20T03:02:16.700 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-08-20T03:02:16.416583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668172,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1168,"num_read_kb":697,"num_write":390,"num_write_kb":2310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":3011602,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":822,"ondisk_log_size":822,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363840,"kb_used_data":10432,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577984,"statfs":{"total":1151940427776,"available":1151567855616,"internally_reserved":0,"allocated":10682368,"data_stored":6991551,"data_compressed":186921,"data_compressed_allocated":245760,"data_compressed_original":2023424,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10073,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":79,"num_write":11,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003014"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809342+0000","last_change":"2024-08-20T03:00:18.238595+0000","last_active":"2024-08-20T03:01:58.809342+0000","last_peered":"2024-08-20T03:01:58.809342+0000","last_clean":"2024-08-20T03:01:58.809342+0000","last_became_active":"2024-08-20T03:00:18.238306+0000","last_became_peered":"2024-08-20T03:00:18.238306+0000","last_unstale":"2024-08-20T03:01:58.809342+0000","last_undegraded":"2024-08-20T03:01:58.809342+0000","last_fullsized":"2024-08-20T03:01:58.809342+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:33:04.681614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805183+0000","last_change":"2024-08-20T03:00:18.239722+0000","last_active":"2024-08-20T03:01:58.805183+0000","last_peered":"2024-08-20T03:01:58.805183+0000","last_clean":"2024-08-20T03:01:58.805183+0000","last_became_active":"2024-08-20T03:00:18.239391+0000","last_became_peered":"2024-08-20T03:00:18.239391+0000","last_unstale":"2024-08-20T03:01:58.805183+0000","last_undegraded":"2024-08-20T03:01:58.805183+0000","last_fullsized":"2024-08-20T03:01:58.805183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:08:34.657173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812333+0000","last_change":"2024-08-20T03:00:18.242389+0000","last_active":"2024-08-20T03:01:58.812333+0000","last_peered":"2024-08-20T03:01:58.812333+0000","last_clean":"2024-08-20T03:01:58.812333+0000","last_became_active":"2024-08-20T03:00:18.242080+0000","last_became_peered":"2024-08-20T03:00:18.242080+0000","last_unstale":"2024-08-20T03:01:58.812333+0000","last_undegraded":"2024-08-20T03:01:58.812333+0000","last_fullsized":"2024-08-20T03:01:58.812333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:37.042894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.798552+0000","last_change":"2024-08-20T03:00:18.233709+0000","last_active":"2024-08-20T03:01:58.798552+0000","last_peered":"2024-08-20T03:01:58.798552+0000","last_clean":"2024-08-20T03:01:58.798552+0000","last_became_active":"2024-08-20T03:00:18.233402+0000","last_became_peered":"2024-08-20T03:00:18.233402+0000","last_unstale":"2024-08-20T03:01:58.798552+0000","last_undegraded":"2024-08-20T03:01:58.798552+0000","last_fullsized":"2024-08-20T03:01:58.798552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:30:01.308181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.776928+0000","last_change":"2024-08-20T03:00:18.237770+0000","last_active":"2024-08-20T03:01:58.776928+0000","last_peered":"2024-08-20T03:01:58.776928+0000","last_clean":"2024-08-20T03:01:58.776928+0000","last_became_active":"2024-08-20T03:00:18.237475+0000","last_became_peered":"2024-08-20T03:00:18.237475+0000","last_unstale":"2024-08-20T03:01:58.776928+0000","last_undegraded":"2024-08-20T03:01:58.776928+0000","last_fullsized":"2024-08-20T03:01:58.776928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:37:12.707935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.797162+0000","last_change":"2024-08-20T03:00:18.245400+0000","last_active":"2024-08-20T03:01:58.797162+0000","last_peered":"2024-08-20T03:01:58.797162+0000","last_clean":"2024-08-20T03:01:58.797162+0000","last_became_active":"2024-08-20T03:00:18.244735+0000","last_became_peered":"2024-08-20T03:00:18.244735+0000","last_unstale":"2024-08-20T03:01:58.797162+0000","last_undegraded":"2024-08-20T03:01:58.797162+0000","last_fullsized":"2024-08-20T03:01:58.797162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:23:25.459441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805943+0000","last_change":"2024-08-20T03:00:18.247366+0000","last_active":"2024-08-20T03:01:58.805943+0000","last_peered":"2024-08-20T03:01:58.805943+0000","last_clean":"2024-08-20T03:01:58.805943+0000","last_became_active":"2024-08-20T03:00:18.247032+0000","last_became_peered":"2024-08-20T03:00:18.247032+0000","last_unstale":"2024-08-20T03:01:58.805943+0000","last_undegraded":"2024-08-20T03:01:58.805943+0000","last_fullsized":"2024-08-20T03:01:58.805943+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:02:18.070393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805505+0000","last_change":"2024-08-20T03:00:18.240845+0000","last_active":"2024-08-20T03:01:58.805505+0000","last_peered":"2024-08-20T03:01:58.805505+0000","last_clean":"2024-08-20T03:01:58.805505+0000","last_became_active":"2024-08-20T03:00:18.240531+0000","last_became_peered":"2024-08-20T03:00:18.240531+0000","last_unstale":"2024-08-20T03:01:58.805505+0000","last_undegraded":"2024-08-20T03:01:58.805505+0000","last_fullsized":"2024-08-20T03:01:58.805505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:15:18.975804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.814341+0000","last_change":"2024-08-20T03:00:18.240817+0000","last_active":"2024-08-20T03:01:58.814341+0000","last_peered":"2024-08-20T03:01:58.814341+0000","last_clean":"2024-08-20T03:01:58.814341+0000","last_became_active":"2024-08-20T03:00:18.240262+0000","last_became_peered":"2024-08-20T03:00:18.240262+0000","last_unstale":"2024-08-20T03:01:58.814341+0000","last_undegraded":"2024-08-20T03:01:58.814341+0000","last_fullsized":"2024-08-20T03:01:58.814341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:30.512597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815785+0000","last_change":"2024-08-20T03:00:18.242139+0000","last_active":"2024-08-20T03:01:58.815785+0000","last_peered":"2024-08-20T03:01:58.815785+0000","last_clean":"2024-08-20T03:01:58.815785+0000","last_became_active":"2024-08-20T03:00:18.241817+0000","last_became_peered":"2024-08-20T03:00:18.241817+0000","last_unstale":"2024-08-20T03:01:58.815785+0000","last_undegraded":"2024-08-20T03:01:58.815785+0000","last_fullsized":"2024-08-20T03:01:58.815785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:25.121306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805745+0000","last_change":"2024-08-20T03:00:18.243205+0000","last_active":"2024-08-20T03:01:58.805745+0000","last_peered":"2024-08-20T03:01:58.805745+0000","last_clean":"2024-08-20T03:01:58.805745+0000","last_became_active":"2024-08-20T03:00:18.242296+0000","last_became_peered":"2024-08-20T03:00:18.242296+0000","last_unstale":"2024-08-20T03:01:58.805745+0000","last_undegraded":"2024-08-20T03:01:58.805745+0000","last_fullsized":"2024-08-20T03:01:58.805745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:50.490934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.798339+0000","last_change":"2024-08-20T03:00:18.237205+0000","last_active":"2024-08-20T03:01:58.798339+0000","last_peered":"2024-08-20T03:01:58.798339+0000","last_clean":"2024-08-20T03:01:58.798339+0000","last_became_active":"2024-08-20T03:00:18.236631+0000","last_became_peered":"2024-08-20T03:00:18.236631+0000","last_unstale":"2024-08-20T03:01:58.798339+0000","last_undegraded":"2024-08-20T03:01:58.798339+0000","last_fullsized":"2024-08-20T03:01:58.798339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:13.327598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811110+0000","last_change":"2024-08-20T03:00:18.235768+0000","last_active":"2024-08-20T03:01:58.811110+0000","last_peered":"2024-08-20T03:01:58.811110+0000","last_clean":"2024-08-20T03:01:58.811110+0000","last_became_active":"2024-08-20T03:00:18.235461+0000","last_became_peered":"2024-08-20T03:00:18.235461+0000","last_unstale":"2024-08-20T03:01:58.811110+0000","last_undegraded":"2024-08-20T03:01:58.811110+0000","last_fullsized":"2024-08-20T03:01:58.811110+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:08:22.238020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809538+0000","last_change":"2024-08-20T03:00:18.246560+0000","last_active":"2024-08-20T03:01:58.809538+0000","last_peered":"2024-08-20T03:01:58.809538+0000","last_clean":"2024-08-20T03:01:58.809538+0000","last_became_active":"2024-08-20T03:00:18.246270+0000","last_became_peered":"2024-08-20T03:00:18.246270+0000","last_unstale":"2024-08-20T03:01:58.809538+0000","last_undegraded":"2024-08-20T03:01:58.809538+0000","last_fullsized":"2024-08-20T03:01:58.809538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:47:13.402635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815437+0000","last_change":"2024-08-20T03:00:18.245907+0000","last_active":"2024-08-20T03:01:58.815437+0000","last_peered":"2024-08-20T03:01:58.815437+0000","last_clean":"2024-08-20T03:01:58.815437+0000","last_became_active":"2024-08-20T03:00:18.245429+0000","last_became_peered":"2024-08-20T03:00:18.245429+0000","last_unstale":"2024-08-20T03:01:58.815437+0000","last_undegraded":"2024-08-20T03:01:58.815437+0000","last_fullsized":"2024-08-20T03:01:58.815437+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:46.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811598+0000","last_change":"2024-08-20T03:00:18.242561+0000","last_active":"2024-08-20T03:01:58.811598+0000","last_peered":"2024-08-20T03:01:58.811598+0000","last_clean":"2024-08-20T03:01:58.811598+0000","last_became_active":"2024-08-20T03:00:18.242048+0000","last_became_peered":"2024-08-20T03:00:18.242048+0000","last_unstale":"2024-08-20T03:01:58.811598+0000","last_undegraded":"2024-08-20T03:01:58.811598+0000","last_fullsized":"2024-08-20T03:01:58.811598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:27.784131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.814235+0000","last_change":"2024-08-20T03:00:18.238547+0000","last_active":"2024-08-20T03:01:58.814235+0000","last_peered":"2024-08-20T03:01:58.814235+0000","last_clean":"2024-08-20T03:01:58.814235+0000","last_became_active":"2024-08-20T03:00:18.238268+0000","last_became_peered":"2024-08-20T03:00:18.238268+0000","last_unstale":"2024-08-20T03:01:58.814235+0000","last_undegraded":"2024-08-20T03:01:58.814235+0000","last_fullsized":"2024-08-20T03:01:58.814235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:15:11.923674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812359+0000","last_change":"2024-08-20T03:00:18.241995+0000","last_active":"2024-08-20T03:01:58.812359+0000","last_peered":"2024-08-20T03:01:58.812359+0000","last_clean":"2024-08-20T03:01:58.812359+0000","last_became_active":"2024-08-20T03:00:18.241705+0000","last_became_peered":"2024-08-20T03:00:18.241705+0000","last_unstale":"2024-08-20T03:01:58.812359+0000","last_undegraded":"2024-08-20T03:01:58.812359+0000","last_fullsized":"2024-08-20T03:01:58.812359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:33:02.183712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815557+0000","last_change":"2024-08-20T03:00:18.236800+0000","last_active":"2024-08-20T03:01:58.815557+0000","last_peered":"2024-08-20T03:01:58.815557+0000","last_clean":"2024-08-20T03:01:58.815557+0000","last_became_active":"2024-08-20T03:00:18.235909+0000","last_became_peered":"2024-08-20T03:00:18.235909+0000","last_unstale":"2024-08-20T03:01:58.815557+0000","last_undegraded":"2024-08-20T03:01:58.815557+0000","last_fullsized":"2024-08-20T03:01:58.815557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:12.427760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.808784+0000","last_change":"2024-08-20T03:00:59.544187+0000","last_active":"2024-08-20T03:01:58.808784+0000","last_peered":"2024-08-20T03:01:58.808784+0000","last_clean":"2024-08-20T03:01:58.808784+0000","last_became_active":"2024-08-20T03:00:59.543807+0000","last_became_peered":"2024-08-20T03:00:59.543807+0000","last_unstale":"2024-08-20T03:01:58.808784+0000","last_undegraded":"2024-08-20T03:01:58.808784+0000","last_fullsized":"2024-08-20T03:01:58.808784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:16:12.425646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.807112+0000","last_change":"2024-08-20T03:00:18.243425+0000","last_active":"2024-08-20T03:01:58.807112+0000","last_peered":"2024-08-20T03:01:58.807112+0000","last_clean":"2024-08-20T03:01:58.807112+0000","last_became_active":"2024-08-20T03:00:18.243082+0000","last_became_peered":"2024-08-20T03:00:18.243082+0000","last_unstale":"2024-08-20T03:01:58.807112+0000","last_undegraded":"2024-08-20T03:01:58.807112+0000","last_fullsized":"2024-08-20T03:01:58.807112+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:50.008655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.813507+0000","last_change":"2024-08-20T03:00:18.242765+0000","last_active":"2024-08-20T03:01:58.813507+0000","last_peered":"2024-08-20T03:01:58.813507+0000","last_clean":"2024-08-20T03:01:58.813507+0000","last_became_active":"2024-08-20T03:00:18.242459+0000","last_became_peered":"2024-08-20T03:00:18.242459+0000","last_unstale":"2024-08-20T03:01:58.813507+0000","last_undegraded":"2024-08-20T03:01:58.813507+0000","last_fullsized":"2024-08-20T03:01:58.813507+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:16.467068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815444+0000","last_change":"2024-08-20T03:00:18.236518+0000","last_active":"2024-08-20T03:01:58.815444+0000","last_peered":"2024-08-20T03:01:58.815444+0000","last_clean":"2024-08-20T03:01:58.815444+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:01:58.815444+0000","last_undegraded":"2024-08-20T03:01:58.815444+0000","last_fullsized":"2024-08-20T03:01:58.815444+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:55:55.387134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811758+0000","last_change":"2024-08-20T03:00:18.236730+0000","last_active":"2024-08-20T03:01:58.811758+0000","last_peered":"2024-08-20T03:01:58.811758+0000","last_clean":"2024-08-20T03:01:58.811758+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:01:58.811758+0000","last_undegraded":"2024-08-20T03:01:58.811758+0000","last_fullsized":"2024-08-20T03:01:58.811758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:18.196161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811526+0000","last_change":"2024-08-20T03:00:18.245390+0000","last_active":"2024-08-20T03:01:58.811526+0000","last_peered":"2024-08-20T03:01:58.811526+0000","last_clean":"2024-08-20T03:01:58.811526+0000","last_became_active":"2024-08-20T03:00:18.241175+0000","last_became_peered":"2024-08-20T03:00:18.241175+0000","last_unstale":"2024-08-20T03:01:58.811526+0000","last_undegraded":"2024-08-20T03:01:58.811526+0000","last_fullsized":"2024-08-20T03:01:58.811526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:51:45.792846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815324+0000","last_change":"2024-08-20T03:00:18.242700+0000","last_active":"2024-08-20T03:01:58.815324+0000","last_peered":"2024-08-20T03:01:58.815324+0000","last_clean":"2024-08-20T03:01:58.815324+0000","last_became_active":"2024-08-20T03:00:18.242407+0000","last_became_peered":"2024-08-20T03:00:18.242407+0000","last_unstale":"2024-08-20T03:01:58.815324+0000","last_undegraded":"2024-08-20T03:01:58.815324+0000","last_fullsized":"2024-08-20T03:01:58.815324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:31:28.061514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809942+0000","last_change":"2024-08-20T03:00:18.233894+0000","last_active":"2024-08-20T03:01:58.809942+0000","last_peered":"2024-08-20T03:01:58.809942+0000","last_clean":"2024-08-20T03:01:58.809942+0000","last_became_active":"2024-08-20T03:00:18.233584+0000","last_became_peered":"2024-08-20T03:00:18.233584+0000","last_unstale":"2024-08-20T03:01:58.809942+0000","last_undegraded":"2024-08-20T03:01:58.809942+0000","last_fullsized":"2024-08-20T03:01:58.809942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:36:53.156215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809607+0000","last_change":"2024-08-20T03:00:18.239256+0000","last_active":"2024-08-20T03:01:58.809607+0000","last_peered":"2024-08-20T03:01:58.809607+0000","last_clean":"2024-08-20T03:01:58.809607+0000","last_became_active":"2024-08-20T03:00:18.238931+0000","last_became_peered":"2024-08-20T03:00:18.238931+0000","last_unstale":"2024-08-20T03:01:58.809607+0000","last_undegraded":"2024-08-20T03:01:58.809607+0000","last_fullsized":"2024-08-20T03:01:58.809607+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:38:42.536488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.807629+0000","last_change":"2024-08-20T03:00:18.245187+0000","last_active":"2024-08-20T03:01:58.807629+0000","last_peered":"2024-08-20T03:01:58.807629+0000","last_clean":"2024-08-20T03:01:58.807629+0000","last_became_active":"2024-08-20T03:00:18.244894+0000","last_became_peered":"2024-08-20T03:00:18.244894+0000","last_unstale":"2024-08-20T03:01:58.807629+0000","last_undegraded":"2024-08-20T03:01:58.807629+0000","last_fullsized":"2024-08-20T03:01:58.807629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:16:58.002014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811573+0000","last_change":"2024-08-20T03:00:18.241634+0000","last_active":"2024-08-20T03:01:58.811573+0000","last_peered":"2024-08-20T03:01:58.811573+0000","last_clean":"2024-08-20T03:01:58.811573+0000","last_became_active":"2024-08-20T03:00:18.241341+0000","last_became_peered":"2024-08-20T03:00:18.241341+0000","last_unstale":"2024-08-20T03:01:58.811573+0000","last_undegraded":"2024-08-20T03:01:58.811573+0000","last_fullsized":"2024-08-20T03:01:58.811573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:37:16.185544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809305+0000","last_change":"2024-08-20T03:00:18.239116+0000","last_active":"2024-08-20T03:01:58.809305+0000","last_peered":"2024-08-20T03:01:58.809305+0000","last_clean":"2024-08-20T03:01:58.809305+0000","last_became_active":"2024-08-20T03:00:18.236125+0000","last_became_peered":"2024-08-20T03:00:18.236125+0000","last_unstale":"2024-08-20T03:01:58.809305+0000","last_undegraded":"2024-08-20T03:01:58.809305+0000","last_fullsized":"2024-08-20T03:01:58.809305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:57:00.332483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.813414+0000","last_change":"2024-08-20T03:00:18.235996+0000","last_active":"2024-08-20T03:01:58.813414+0000","last_peered":"2024-08-20T03:01:58.813414+0000","last_clean":"2024-08-20T03:01:58.813414+0000","last_became_active":"2024-08-20T03:00:18.235670+0000","last_became_peered":"2024-08-20T03:00:18.235670+0000","last_unstale":"2024-08-20T03:01:58.813414+0000","last_undegraded":"2024-08-20T03:01:58.813414+0000","last_fullsized":"2024-08-20T03:01:58.813414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:59.992914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805791+0000","last_change":"2024-08-20T03:00:18.239679+0000","last_active":"2024-08-20T03:01:58.805791+0000","last_peered":"2024-08-20T03:01:58.805791+0000","last_clean":"2024-08-20T03:01:58.805791+0000","last_became_active":"2024-08-20T03:00:18.239361+0000","last_became_peered":"2024-08-20T03:00:18.239361+0000","last_unstale":"2024-08-20T03:01:58.805791+0000","last_undegraded":"2024-08-20T03:01:58.805791+0000","last_fullsized":"2024-08-20T03:01:58.805791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:28:09.430535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'8","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:13.532735+0000","last_change":"2024-08-20T03:00:16.221221+0000","last_active":"2024-08-20T03:02:13.532735+0000","last_peered":"2024-08-20T03:02:13.532735+0000","last_clean":"2024-08-20T03:02:13.532735+0000","last_became_active":"2024-08-20T03:00:16.220186+0000","last_became_peered":"2024-08-20T03:00:16.220186+0000","last_unstale":"2024-08-20T03:02:13.532735+0000","last_undegraded":"2024-08-20T03:02:13.532735+0000","last_fullsized":"2024-08-20T03:02:13.532735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:51.045235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.808630+0000","last_change":"2024-08-20T03:00:18.234809+0000","last_active":"2024-08-20T03:01:58.808630+0000","last_peered":"2024-08-20T03:01:58.808630+0000","last_clean":"2024-08-20T03:01:58.808630+0000","last_became_active":"2024-08-20T03:00:18.234486+0000","last_became_peered":"2024-08-20T03:00:18.234486+0000","last_unstale":"2024-08-20T03:01:58.808630+0000","last_undegraded":"2024-08-20T03:01:58.808630+0000","last_fullsized":"2024-08-20T03:01:58.808630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:11:52.527224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491303+0000","last_change":"2024-08-20T03:00:16.216876+0000","last_active":"2024-08-20T03:00:59.491303+0000","last_peered":"2024-08-20T03:00:59.491303+0000","last_clean":"2024-08-20T03:00:59.491303+0000","last_became_active":"2024-08-20T03:00:16.216522+0000","last_became_peered":"2024-08-20T03:00:16.216522+0000","last_unstale":"2024-08-20T03:00:59.491303+0000","last_undegraded":"2024-08-20T03:00:59.491303+0000","last_fullsized":"2024-08-20T03:00:59.491303+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:22.363331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.489899+0000","last_change":"2024-08-20T03:00:16.213751+0000","last_active":"2024-08-20T03:00:59.489899+0000","last_peered":"2024-08-20T03:00:59.489899+0000","last_clean":"2024-08-20T03:00:59.489899+0000","last_became_active":"2024-08-20T03:00:16.213348+0000","last_became_peered":"2024-08-20T03:00:16.213348+0000","last_unstale":"2024-08-20T03:00:59.489899+0000","last_undegraded":"2024-08-20T03:00:59.489899+0000","last_fullsized":"2024-08-20T03:00:59.489899+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:01:08.637151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809027+0000","last_change":"2024-08-20T03:00:18.237972+0000","last_active":"2024-08-20T03:01:58.809027+0000","last_peered":"2024-08-20T03:01:58.809027+0000","last_clean":"2024-08-20T03:01:58.809027+0000","last_became_active":"2024-08-20T03:00:18.236769+0000","last_became_peered":"2024-08-20T03:00:18.236769+0000","last_unstale":"2024-08-20T03:01:58.809027+0000","last_undegraded":"2024-08-20T03:01:58.809027+0000","last_fullsized":"2024-08-20T03:01:58.809027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:35:21.001045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492196+0000","last_change":"2024-08-20T03:00:16.221399+0000","last_active":"2024-08-20T03:00:59.492196+0000","last_peered":"2024-08-20T03:00:59.492196+0000","last_clean":"2024-08-20T03:00:59.492196+0000","last_became_active":"2024-08-20T03:00:16.220285+0000","last_became_peered":"2024-08-20T03:00:16.220285+0000","last_unstale":"2024-08-20T03:00:59.492196+0000","last_undegraded":"2024-08-20T03:00:59.492196+0000","last_fullsized":"2024-08-20T03:00:59.492196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:17.034657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.802020+0000","last_change":"2024-08-20T03:00:18.236722+0000","last_active":"2024-08-20T03:01:58.802020+0000","last_peered":"2024-08-20T03:01:58.802020+0000","last_clean":"2024-08-20T03:01:58.802020+0000","last_became_active":"2024-08-20T03:00:18.236365+0000","last_became_peered":"2024-08-20T03:00:18.236365+0000","last_unstale":"2024-08-20T03:01:58.802020+0000","last_undegraded":"2024-08-20T03:01:58.802020+0000","last_fullsized":"2024-08-20T03:01:58.802020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:17.344161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":241,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:15.535498+0000","last_change":"2024-08-20T03:00:16.222331+0000","last_active":"2024-08-20T03:02:15.535498+0000","last_peered":"2024-08-20T03:02:15.535498+0000","last_clean":"2024-08-20T03:02:15.535498+0000","last_became_active":"2024-08-20T03:00:16.222019+0000","last_became_peered":"2024-08-20T03:00:16.222019+0000","last_unstale":"2024-08-20T03:02:15.535498+0000","last_undegraded":"2024-08-20T03:02:15.535498+0000","last_fullsized":"2024-08-20T03:02:15.535498+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:46:55.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":212,"num_read_kb":212,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812391+0000","last_change":"2024-08-20T03:00:18.239020+0000","last_active":"2024-08-20T03:01:58.812391+0000","last_peered":"2024-08-20T03:01:58.812391+0000","last_clean":"2024-08-20T03:01:58.812391+0000","last_became_active":"2024-08-20T03:00:18.238620+0000","last_became_peered":"2024-08-20T03:00:18.238620+0000","last_unstale":"2024-08-20T03:01:58.812391+0000","last_undegraded":"2024-08-20T03:01:58.812391+0000","last_fullsized":"2024-08-20T03:01:58.812391+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:17:12.632357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491087+0000","last_change":"2024-08-20T03:00:16.214907+0000","last_active":"2024-08-20T03:00:59.491087+0000","last_peered":"2024-08-20T03:00:59.491087+0000","last_clean":"2024-08-20T03:00:59.491087+0000","last_became_active":"2024-08-20T03:00:16.214573+0000","last_became_peered":"2024-08-20T03:00:16.214573+0000","last_unstale":"2024-08-20T03:00:59.491087+0000","last_undegraded":"2024-08-20T03:00:59.491087+0000","last_fullsized":"2024-08-20T03:00:59.491087+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:21:34.084149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.807582+0000","last_change":"2024-08-20T03:00:18.245524+0000","last_active":"2024-08-20T03:01:58.807582+0000","last_peered":"2024-08-20T03:01:58.807582+0000","last_clean":"2024-08-20T03:01:58.807582+0000","last_became_active":"2024-08-20T03:00:18.244966+0000","last_became_peered":"2024-08-20T03:00:18.244966+0000","last_unstale":"2024-08-20T03:01:58.807582+0000","last_undegraded":"2024-08-20T03:01:58.807582+0000","last_fullsized":"2024-08-20T03:01:58.807582+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:03:21.584182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491504+0000","last_change":"2024-08-20T03:00:16.470353+0000","last_active":"2024-08-20T03:00:59.491504+0000","last_peered":"2024-08-20T03:00:59.491504+0000","last_clean":"2024-08-20T03:00:59.491504+0000","last_became_active":"2024-08-20T03:00:16.469976+0000","last_became_peered":"2024-08-20T03:00:16.469976+0000","last_unstale":"2024-08-20T03:00:59.491504+0000","last_undegraded":"2024-08-20T03:00:59.491504+0000","last_fullsized":"2024-08-20T03:00:59.491504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:25:03.595784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.814962+0000","last_change":"2024-08-20T03:00:18.245244+0000","last_active":"2024-08-20T03:01:58.814962+0000","last_peered":"2024-08-20T03:01:58.814962+0000","last_clean":"2024-08-20T03:01:58.814962+0000","last_became_active":"2024-08-20T03:00:18.244934+0000","last_became_peered":"2024-08-20T03:00:18.244934+0000","last_unstale":"2024-08-20T03:01:58.814962+0000","last_undegraded":"2024-08-20T03:01:58.814962+0000","last_fullsized":"2024-08-20T03:01:58.814962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:33:35.035039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'37","reported_seq":305,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:15.626540+0000","last_change":"2024-08-20T03:00:59.539537+0000","last_active":"2024-08-20T03:02:15.626540+0000","last_peered":"2024-08-20T03:02:15.626540+0000","last_clean":"2024-08-20T03:02:15.626540+0000","last_became_active":"2024-08-20T03:00:59.518110+0000","last_became_peered":"2024-08-20T03:00:59.518110+0000","last_unstale":"2024-08-20T03:02:15.626540+0000","last_undegraded":"2024-08-20T03:02:15.626540+0000","last_fullsized":"2024-08-20T03:02:15.626540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:10:58.722741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68603,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":218,"num_read_kb":176,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59762,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.802994+0000","last_change":"2024-08-20T03:00:18.242931+0000","last_active":"2024-08-20T03:01:58.802994+0000","last_peered":"2024-08-20T03:01:58.802994+0000","last_clean":"2024-08-20T03:01:58.802994+0000","last_became_active":"2024-08-20T03:00:18.242619+0000","last_became_peered":"2024-08-20T03:00:18.242619+0000","last_unstale":"2024-08-20T03:01:58.802994+0000","last_undegraded":"2024-08-20T03:01:58.802994+0000","last_fullsized":"2024-08-20T03:01:58.802994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:43:02.296990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492488+0000","last_change":"2024-08-20T03:00:16.214688+0000","last_active":"2024-08-20T03:00:59.492488+0000","last_peered":"2024-08-20T03:00:59.492488+0000","last_clean":"2024-08-20T03:00:59.492488+0000","last_became_active":"2024-08-20T03:00:16.214343+0000","last_became_peered":"2024-08-20T03:00:16.214343+0000","last_unstale":"2024-08-20T03:00:59.492488+0000","last_undegraded":"2024-08-20T03:00:59.492488+0000","last_fullsized":"2024-08-20T03:00:59.492488+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:10:56.587164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.806082+0000","last_change":"2024-08-20T03:00:18.244158+0000","last_active":"2024-08-20T03:01:58.806082+0000","last_peered":"2024-08-20T03:01:58.806082+0000","last_clean":"2024-08-20T03:01:58.806082+0000","last_became_active":"2024-08-20T03:00:18.243864+0000","last_became_peered":"2024-08-20T03:00:18.243864+0000","last_unstale":"2024-08-20T03:01:58.806082+0000","last_undegraded":"2024-08-20T03:01:58.806082+0000","last_fullsized":"2024-08-20T03:01:58.806082+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:10:31.373759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491090+0000","last_change":"2024-08-20T03:00:16.220846+0000","last_active":"2024-08-20T03:00:59.491090+0000","last_peered":"2024-08-20T03:00:59.491090+0000","last_clean":"2024-08-20T03:00:59.491090+0000","last_became_active":"2024-08-20T03:00:16.220519+0000","last_became_peered":"2024-08-20T03:00:16.220519+0000","last_unstale":"2024-08-20T03:00:59.491090+0000","last_undegraded":"2024-08-20T03:00:59.491090+0000","last_fullsized":"2024-08-20T03:00:59.491090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:43:41.721522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.808946+0000","last_change":"2024-08-20T03:00:18.246555+0000","last_active":"2024-08-20T03:01:58.808946+0000","last_peered":"2024-08-20T03:01:58.808946+0000","last_clean":"2024-08-20T03:01:58.808946+0000","last_became_active":"2024-08-20T03:00:18.246182+0000","last_became_peered":"2024-08-20T03:00:18.246182+0000","last_unstale":"2024-08-20T03:01:58.808946+0000","last_undegraded":"2024-08-20T03:01:58.808946+0000","last_fullsized":"2024-08-20T03:01:58.808946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:55.827649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'6","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:13.527275+0000","last_change":"2024-08-20T03:00:16.216521+0000","last_active":"2024-08-20T03:02:13.527275+0000","last_peered":"2024-08-20T03:02:13.527275+0000","last_clean":"2024-08-20T03:02:13.527275+0000","last_became_active":"2024-08-20T03:00:16.216187+0000","last_became_peered":"2024-08-20T03:00:16.216187+0000","last_unstale":"2024-08-20T03:02:13.527275+0000","last_undegraded":"2024-08-20T03:02:13.527275+0000","last_fullsized":"2024-08-20T03:02:13.527275+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:41:13.639218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.786505+0000","last_change":"2024-08-20T03:00:59.544335+0000","last_active":"2024-08-20T03:01:58.786505+0000","last_peered":"2024-08-20T03:01:58.786505+0000","last_clean":"2024-08-20T03:01:58.786505+0000","last_became_active":"2024-08-20T03:00:59.544063+0000","last_became_peered":"2024-08-20T03:00:59.544063+0000","last_unstale":"2024-08-20T03:01:58.786505+0000","last_undegraded":"2024-08-20T03:01:58.786505+0000","last_fullsized":"2024-08-20T03:01:58.786505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:48:36.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490874+0000","last_change":"2024-08-20T03:00:16.470129+0000","last_active":"2024-08-20T03:00:59.490874+0000","last_peered":"2024-08-20T03:00:59.490874+0000","last_clean":"2024-08-20T03:00:59.490874+0000","last_became_active":"2024-08-20T03:00:16.469697+0000","last_became_peered":"2024-08-20T03:00:16.469697+0000","last_unstale":"2024-08-20T03:00:59.490874+0000","last_undegraded":"2024-08-20T03:00:59.490874+0000","last_fullsized":"2024-08-20T03:00:59.490874+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:42:22.623141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.804109+0000","last_change":"2024-08-20T03:00:18.245090+0000","last_active":"2024-08-20T03:01:58.804109+0000","last_peered":"2024-08-20T03:01:58.804109+0000","last_clean":"2024-08-20T03:01:58.804109+0000","last_became_active":"2024-08-20T03:00:18.244524+0000","last_became_peered":"2024-08-20T03:00:18.244524+0000","last_unstale":"2024-08-20T03:01:58.804109+0000","last_undegraded":"2024-08-20T03:01:58.804109+0000","last_fullsized":"2024-08-20T03:01:58.804109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:14.561183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492233+0000","last_change":"2024-08-20T03:00:16.220042+0000","last_active":"2024-08-20T03:00:59.492233+0000","last_peered":"2024-08-20T03:00:59.492233+0000","last_clean":"2024-08-20T03:00:59.492233+0000","last_became_active":"2024-08-20T03:00:16.219567+0000","last_became_peered":"2024-08-20T03:00:16.219567+0000","last_unstale":"2024-08-20T03:00:59.492233+0000","last_undegraded":"2024-08-20T03:00:59.492233+0000","last_fullsized":"2024-08-20T03:00:59.492233+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:04.834039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812386+0000","last_change":"2024-08-20T03:00:18.237497+0000","last_active":"2024-08-20T03:01:58.812386+0000","last_peered":"2024-08-20T03:01:58.812386+0000","last_clean":"2024-08-20T03:01:58.812386+0000","last_became_active":"2024-08-20T03:00:18.237170+0000","last_became_peered":"2024-08-20T03:00:18.237170+0000","last_unstale":"2024-08-20T03:01:58.812386+0000","last_undegraded":"2024-08-20T03:01:58.812386+0000","last_fullsized":"2024-08-20T03:01:58.812386+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:52.387152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490226+0000","last_change":"2024-08-20T03:00:16.214861+0000","last_active":"2024-08-20T03:00:59.490226+0000","last_peered":"2024-08-20T03:00:59.490226+0000","last_clean":"2024-08-20T03:00:59.490226+0000","last_became_active":"2024-08-20T03:00:16.214487+0000","last_became_peered":"2024-08-20T03:00:16.214487+0000","last_unstale":"2024-08-20T03:00:59.490226+0000","last_undegraded":"2024-08-20T03:00:59.490226+0000","last_fullsized":"2024-08-20T03:00:59.490226+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:50:09.933304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811230+0000","last_change":"2024-08-20T03:00:18.244685+0000","last_active":"2024-08-20T03:01:58.811230+0000","last_peered":"2024-08-20T03:01:58.811230+0000","last_clean":"2024-08-20T03:01:58.811230+0000","last_became_active":"2024-08-20T03:00:18.244372+0000","last_became_peered":"2024-08-20T03:00:18.244372+0000","last_unstale":"2024-08-20T03:01:58.811230+0000","last_undegraded":"2024-08-20T03:01:58.811230+0000","last_fullsized":"2024-08-20T03:01:58.811230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:21.821712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491625+0000","last_change":"2024-08-20T03:00:16.222287+0000","last_active":"2024-08-20T03:00:59.491625+0000","last_peered":"2024-08-20T03:00:59.491625+0000","last_clean":"2024-08-20T03:00:59.491625+0000","last_became_active":"2024-08-20T03:00:16.221989+0000","last_became_peered":"2024-08-20T03:00:16.221989+0000","last_unstale":"2024-08-20T03:00:59.491625+0000","last_undegraded":"2024-08-20T03:00:59.491625+0000","last_fullsized":"2024-08-20T03:00:59.491625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:32:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.807215+0000","last_change":"2024-08-20T03:00:18.250488+0000","last_active":"2024-08-20T03:01:58.807215+0000","last_peered":"2024-08-20T03:01:58.807215+0000","last_clean":"2024-08-20T03:01:58.807215+0000","last_became_active":"2024-08-20T03:00:18.250094+0000","last_became_peered":"2024-08-20T03:00:18.250094+0000","last_unstale":"2024-08-20T03:01:58.807215+0000","last_undegraded":"2024-08-20T03:01:58.807215+0000","last_fullsized":"2024-08-20T03:01:58.807215+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:00:55.755920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'7","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:14.534593+0000","last_change":"2024-08-20T03:00:16.470223+0000","last_active":"2024-08-20T03:02:14.534593+0000","last_peered":"2024-08-20T03:02:14.534593+0000","last_clean":"2024-08-20T03:02:14.534593+0000","last_became_active":"2024-08-20T03:00:16.469871+0000","last_became_peered":"2024-08-20T03:00:16.469871+0000","last_unstale":"2024-08-20T03:02:14.534593+0000","last_undegraded":"2024-08-20T03:02:14.534593+0000","last_fullsized":"2024-08-20T03:02:14.534593+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:05:11.779164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.810540+0000","last_change":"2024-08-20T03:00:18.242755+0000","last_active":"2024-08-20T03:01:58.810540+0000","last_peered":"2024-08-20T03:01:58.810540+0000","last_clean":"2024-08-20T03:01:58.810540+0000","last_became_active":"2024-08-20T03:00:18.242447+0000","last_became_peered":"2024-08-20T03:00:18.242447+0000","last_unstale":"2024-08-20T03:01:58.810540+0000","last_undegraded":"2024-08-20T03:01:58.810540+0000","last_fullsized":"2024-08-20T03:01:58.810540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:55:40.107933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":97,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490628+0000","last_change":"2024-08-20T02:59:02.774484+0000","last_active":"2024-08-20T03:00:59.490628+0000","last_peered":"2024-08-20T03:00:59.490628+0000","last_clean":"2024-08-20T03:00:59.490628+0000","last_became_active":"2024-08-20T02:59:02.769208+0000","last_became_peered":"2024-08-20T02:59:02.769208+0000","last_unstale":"2024-08-20T03:00:59.490628+0000","last_undegraded":"2024-08-20T03:00:59.490628+0000","last_fullsized":"2024-08-20T03:00:59.490628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_clean_scrub_stamp":"2024-08-20T02:55:46.827249+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:36:27.931786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.516703+0000","last_change":"2024-08-20T03:00:16.213869+0000","last_active":"2024-08-20T03:00:59.516703+0000","last_peered":"2024-08-20T03:00:59.516703+0000","last_clean":"2024-08-20T03:00:59.516703+0000","last_became_active":"2024-08-20T03:00:16.213478+0000","last_became_peered":"2024-08-20T03:00:16.213478+0000","last_unstale":"2024-08-20T03:00:59.516703+0000","last_undegraded":"2024-08-20T03:00:59.516703+0000","last_fullsized":"2024-08-20T03:00:59.516703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:27.830497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.804627+0000","last_change":"2024-08-20T03:00:18.242283+0000","last_active":"2024-08-20T03:01:58.804627+0000","last_peered":"2024-08-20T03:01:58.804627+0000","last_clean":"2024-08-20T03:01:58.804627+0000","last_became_active":"2024-08-20T03:00:18.241943+0000","last_became_peered":"2024-08-20T03:00:18.241943+0000","last_unstale":"2024-08-20T03:01:58.804627+0000","last_undegraded":"2024-08-20T03:01:58.804627+0000","last_fullsized":"2024-08-20T03:01:58.804627+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:22:25.776964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.498057+0000","last_change":"2024-08-20T03:00:59.497552+0000","last_active":"2024-08-20T03:00:59.498057+0000","last_peered":"2024-08-20T03:00:59.498057+0000","last_clean":"2024-08-20T03:00:59.498057+0000","last_became_active":"2024-08-20T03:00:59.493978+0000","last_became_peered":"2024-08-20T03:00:59.493978+0000","last_unstale":"2024-08-20T03:00:59.498057+0000","last_undegraded":"2024-08-20T03:00:59.498057+0000","last_fullsized":"2024-08-20T03:00:59.498057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:38:12.908159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812189+0000","last_change":"2024-08-20T03:00:59.544696+0000","last_active":"2024-08-20T03:01:58.812189+0000","last_peered":"2024-08-20T03:01:58.812189+0000","last_clean":"2024-08-20T03:01:58.812189+0000","last_became_active":"2024-08-20T03:00:59.544303+0000","last_became_peered":"2024-08-20T03:00:59.544303+0000","last_unstale":"2024-08-20T03:01:58.812189+0000","last_undegraded":"2024-08-20T03:01:58.812189+0000","last_fullsized":"2024-08-20T03:01:58.812189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:01:04.814695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":131,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:10.522455+0000","last_change":"2024-08-20T03:00:16.468073+0000","last_active":"2024-08-20T03:02:10.522455+0000","last_peered":"2024-08-20T03:02:10.522455+0000","last_clean":"2024-08-20T03:02:10.522455+0000","last_became_active":"2024-08-20T03:00:16.467560+0000","last_became_peered":"2024-08-20T03:00:16.467560+0000","last_unstale":"2024-08-20T03:02:10.522455+0000","last_undegraded":"2024-08-20T03:02:10.522455+0000","last_fullsized":"2024-08-20T03:02:10.522455+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:42:24.499777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812418+0000","last_change":"2024-08-20T03:00:18.232537+0000","last_active":"2024-08-20T03:01:58.812418+0000","last_peered":"2024-08-20T03:01:58.812418+0000","last_clean":"2024-08-20T03:01:58.812418+0000","last_became_active":"2024-08-20T03:00:18.232273+0000","last_became_peered":"2024-08-20T03:00:18.232273+0000","last_unstale":"2024-08-20T03:01:58.812418+0000","last_undegraded":"2024-08-20T03:01:58.812418+0000","last_fullsized":"2024-08-20T03:01:58.812418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:36:04.981869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'7","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:11.524410+0000","last_change":"2024-08-20T03:00:16.214807+0000","last_active":"2024-08-20T03:02:11.524410+0000","last_peered":"2024-08-20T03:02:11.524410+0000","last_clean":"2024-08-20T03:02:11.524410+0000","last_became_active":"2024-08-20T03:00:16.214439+0000","last_became_peered":"2024-08-20T03:00:16.214439+0000","last_unstale":"2024-08-20T03:02:11.524410+0000","last_undegraded":"2024-08-20T03:02:11.524410+0000","last_fullsized":"2024-08-20T03:02:11.524410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:07.519151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.795094+0000","last_change":"2024-08-20T03:00:18.247220+0000","last_active":"2024-08-20T03:01:58.795094+0000","last_peered":"2024-08-20T03:01:58.795094+0000","last_clean":"2024-08-20T03:01:58.795094+0000","last_became_active":"2024-08-20T03:00:18.246896+0000","last_became_peered":"2024-08-20T03:00:18.246896+0000","last_unstale":"2024-08-20T03:01:58.795094+0000","last_undegraded":"2024-08-20T03:01:58.795094+0000","last_fullsized":"2024-08-20T03:01:58.795094+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:43:16.690590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543240+0000","last_change":"2024-08-20T03:00:16.468280+0000","last_active":"2024-08-20T03:00:59.543240+0000","last_peered":"2024-08-20T03:00:59.543240+0000","last_clean":"2024-08-20T03:00:59.543240+0000","last_became_active":"2024-08-20T03:00:16.467930+0000","last_became_peered":"2024-08-20T03:00:16.467930+0000","last_unstale":"2024-08-20T03:00:59.543240+0000","last_undegraded":"2024-08-20T03:00:59.543240+0000","last_fullsized":"2024-08-20T03:00:59.543240+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:30.423563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.797891+0000","last_change":"2024-08-20T03:00:18.240322+0000","last_active":"2024-08-20T03:01:58.797891+0000","last_peered":"2024-08-20T03:01:58.797891+0000","last_clean":"2024-08-20T03:01:58.797891+0000","last_became_active":"2024-08-20T03:00:18.239973+0000","last_became_peered":"2024-08-20T03:00:18.239973+0000","last_unstale":"2024-08-20T03:01:58.797891+0000","last_undegraded":"2024-08-20T03:01:58.797891+0000","last_fullsized":"2024-08-20T03:01:58.797891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:41:28.259762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.493654+0000","last_change":"2024-08-20T03:00:16.468307+0000","last_active":"2024-08-20T03:00:59.493654+0000","last_peered":"2024-08-20T03:00:59.493654+0000","last_clean":"2024-08-20T03:00:59.493654+0000","last_became_active":"2024-08-20T03:00:16.467993+0000","last_became_peered":"2024-08-20T03:00:16.467993+0000","last_unstale":"2024-08-20T03:00:59.493654+0000","last_undegraded":"2024-08-20T03:00:59.493654+0000","last_fullsized":"2024-08-20T03:00:59.493654+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:34.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812353+0000","last_change":"2024-08-20T03:00:18.234009+0000","last_active":"2024-08-20T03:01:58.812353+0000","last_peered":"2024-08-20T03:01:58.812353+0000","last_clean":"2024-08-20T03:01:58.812353+0000","last_became_active":"2024-08-20T03:00:18.233311+0000","last_became_peered":"2024-08-20T03:00:18.233311+0000","last_unstale":"2024-08-20T03:01:58.812353+0000","last_undegraded":"2024-08-20T03:01:58.812353+0000","last_fullsized":"2024-08-20T03:01:58.812353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:53:10.759339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'17","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543424+0000","last_change":"2024-08-20T03:00:16.214932+0000","last_active":"2024-08-20T03:00:59.543424+0000","last_peered":"2024-08-20T03:00:59.543424+0000","last_clean":"2024-08-20T03:00:59.543424+0000","last_became_active":"2024-08-20T03:00:16.214584+0000","last_became_peered":"2024-08-20T03:00:16.214584+0000","last_unstale":"2024-08-20T03:00:59.543424+0000","last_undegraded":"2024-08-20T03:00:59.543424+0000","last_fullsized":"2024-08-20T03:00:59.543424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:47.270685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809771+0000","last_change":"2024-08-20T03:00:18.247892+0000","last_active":"2024-08-20T03:01:58.809771+0000","last_peered":"2024-08-20T03:01:58.809771+0000","last_clean":"2024-08-20T03:01:58.809771+0000","last_became_active":"2024-08-20T03:00:18.245903+0000","last_became_peered":"2024-08-20T03:00:18.245903+0000","last_unstale":"2024-08-20T03:01:58.809771+0000","last_undegraded":"2024-08-20T03:01:58.809771+0000","last_fullsized":"2024-08-20T03:01:58.809771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:08:50.909638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.516720+0000","last_change":"2024-08-20T03:00:16.372503+0000","last_active":"2024-08-20T03:00:59.516720+0000","last_peered":"2024-08-20T03:00:59.516720+0000","last_clean":"2024-08-20T03:00:59.516720+0000","last_became_active":"2024-08-20T03:00:16.372218+0000","last_became_peered":"2024-08-20T03:00:16.372218+0000","last_unstale":"2024-08-20T03:00:59.516720+0000","last_undegraded":"2024-08-20T03:00:59.516720+0000","last_fullsized":"2024-08-20T03:00:59.516720+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:13.341900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805821+0000","last_change":"2024-08-20T03:00:18.244738+0000","last_active":"2024-08-20T03:01:58.805821+0000","last_peered":"2024-08-20T03:01:58.805821+0000","last_clean":"2024-08-20T03:01:58.805821+0000","last_became_active":"2024-08-20T03:00:18.244398+0000","last_became_peered":"2024-08-20T03:00:18.244398+0000","last_unstale":"2024-08-20T03:01:58.805821+0000","last_undegraded":"2024-08-20T03:01:58.805821+0000","last_fullsized":"2024-08-20T03:01:58.805821+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:45:14.540125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494339+0000","last_change":"2024-08-20T03:00:16.468251+0000","last_active":"2024-08-20T03:00:59.494339+0000","last_peered":"2024-08-20T03:00:59.494339+0000","last_clean":"2024-08-20T03:00:59.494339+0000","last_became_active":"2024-08-20T03:00:16.467912+0000","last_became_peered":"2024-08-20T03:00:16.467912+0000","last_unstale":"2024-08-20T03:00:59.494339+0000","last_undegraded":"2024-08-20T03:00:59.494339+0000","last_fullsized":"2024-08-20T03:00:59.494339+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:43.799747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.770324+0000","last_change":"2024-08-20T03:00:18.234624+0000","last_active":"2024-08-20T03:01:58.770324+0000","last_peered":"2024-08-20T03:01:58.770324+0000","last_clean":"2024-08-20T03:01:58.770324+0000","last_became_active":"2024-08-20T03:00:18.234331+0000","last_became_peered":"2024-08-20T03:00:18.234331+0000","last_unstale":"2024-08-20T03:01:58.770324+0000","last_undegraded":"2024-08-20T03:01:58.770324+0000","last_fullsized":"2024-08-20T03:01:58.770324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:05:09.665645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487438+0000","last_change":"2024-08-20T03:00:16.216658+0000","last_active":"2024-08-20T03:00:59.487438+0000","last_peered":"2024-08-20T03:00:59.487438+0000","last_clean":"2024-08-20T03:00:59.487438+0000","last_became_active":"2024-08-20T03:00:16.216352+0000","last_became_peered":"2024-08-20T03:00:16.216352+0000","last_unstale":"2024-08-20T03:00:59.487438+0000","last_undegraded":"2024-08-20T03:00:59.487438+0000","last_fullsized":"2024-08-20T03:00:59.487438+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:20.763280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811335+0000","last_change":"2024-08-20T03:00:18.240984+0000","last_active":"2024-08-20T03:01:58.811335+0000","last_peered":"2024-08-20T03:01:58.811335+0000","last_clean":"2024-08-20T03:01:58.811335+0000","last_became_active":"2024-08-20T03:00:18.240640+0000","last_became_peered":"2024-08-20T03:00:18.240640+0000","last_unstale":"2024-08-20T03:01:58.811335+0000","last_undegraded":"2024-08-20T03:01:58.811335+0000","last_fullsized":"2024-08-20T03:01:58.811335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:05.589186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492391+0000","last_change":"2024-08-20T03:00:16.214818+0000","last_active":"2024-08-20T03:00:59.492391+0000","last_peered":"2024-08-20T03:00:59.492391+0000","last_clean":"2024-08-20T03:00:59.492391+0000","last_became_active":"2024-08-20T03:00:16.214493+0000","last_became_peered":"2024-08-20T03:00:16.214493+0000","last_unstale":"2024-08-20T03:00:59.492391+0000","last_undegraded":"2024-08-20T03:00:59.492391+0000","last_fullsized":"2024-08-20T03:00:59.492391+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:56:12.895142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.806590+0000","last_change":"2024-08-20T03:00:18.240350+0000","last_active":"2024-08-20T03:01:58.806590+0000","last_peered":"2024-08-20T03:01:58.806590+0000","last_clean":"2024-08-20T03:01:58.806590+0000","last_became_active":"2024-08-20T03:00:18.239464+0000","last_became_peered":"2024-08-20T03:00:18.239464+0000","last_unstale":"2024-08-20T03:01:58.806590+0000","last_undegraded":"2024-08-20T03:01:58.806590+0000","last_fullsized":"2024-08-20T03:01:58.806590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:48.715130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491431+0000","last_change":"2024-08-20T03:00:16.216144+0000","last_active":"2024-08-20T03:00:59.491431+0000","last_peered":"2024-08-20T03:00:59.491431+0000","last_clean":"2024-08-20T03:00:59.491431+0000","last_became_active":"2024-08-20T03:00:16.215792+0000","last_became_peered":"2024-08-20T03:00:16.215792+0000","last_unstale":"2024-08-20T03:00:59.491431+0000","last_undegraded":"2024-08-20T03:00:59.491431+0000","last_fullsized":"2024-08-20T03:00:59.491431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:25:01.500878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809127+0000","last_change":"2024-08-20T03:00:18.245286+0000","last_active":"2024-08-20T03:01:58.809127+0000","last_peered":"2024-08-20T03:01:58.809127+0000","last_clean":"2024-08-20T03:01:58.809127+0000","last_became_active":"2024-08-20T03:00:18.244760+0000","last_became_peered":"2024-08-20T03:00:18.244760+0000","last_unstale":"2024-08-20T03:01:58.809127+0000","last_undegraded":"2024-08-20T03:01:58.809127+0000","last_fullsized":"2024-08-20T03:01:58.809127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:09.415794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.814003+0000","last_change":"2024-08-20T03:00:59.495948+0000","last_active":"2024-08-20T03:01:58.814003+0000","last_peered":"2024-08-20T03:01:58.814003+0000","last_clean":"2024-08-20T03:01:58.814003+0000","last_became_active":"2024-08-20T03:00:59.495543+0000","last_became_peered":"2024-08-20T03:00:59.495543+0000","last_unstale":"2024-08-20T03:01:58.814003+0000","last_undegraded":"2024-08-20T03:01:58.814003+0000","last_fullsized":"2024-08-20T03:01:58.814003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:38:30.671235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492392+0000","last_change":"2024-08-20T03:00:16.220019+0000","last_active":"2024-08-20T03:00:59.492392+0000","last_peered":"2024-08-20T03:00:59.492392+0000","last_clean":"2024-08-20T03:00:59.492392+0000","last_became_active":"2024-08-20T03:00:16.219624+0000","last_became_peered":"2024-08-20T03:00:16.219624+0000","last_unstale":"2024-08-20T03:00:59.492392+0000","last_undegraded":"2024-08-20T03:00:59.492392+0000","last_fullsized":"2024-08-20T03:00:59.492392+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:33:24.926695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.813429+0000","last_change":"2024-08-20T03:00:18.242156+0000","last_active":"2024-08-20T03:01:58.813429+0000","last_peered":"2024-08-20T03:01:58.813429+0000","last_clean":"2024-08-20T03:01:58.813429+0000","last_became_active":"2024-08-20T03:00:18.241811+0000","last_became_peered":"2024-08-20T03:00:18.241811+0000","last_unstale":"2024-08-20T03:01:58.813429+0000","last_undegraded":"2024-08-20T03:01:58.813429+0000","last_fullsized":"2024-08-20T03:01:58.813429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:58.230658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.534283+0000","last_change":"2024-08-20T03:00:59.533906+0000","last_active":"2024-08-20T03:00:59.534283+0000","last_peered":"2024-08-20T03:00:59.534283+0000","last_clean":"2024-08-20T03:00:59.534283+0000","last_became_active":"2024-08-20T03:00:59.517824+0000","last_became_peered":"2024-08-20T03:00:59.517824+0000","last_unstale":"2024-08-20T03:00:59.534283+0000","last_undegraded":"2024-08-20T03:00:59.534283+0000","last_fullsized":"2024-08-20T03:00:59.534283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:25:28.219430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.813189+0000","last_change":"2024-08-20T03:00:18.245256+0000","last_active":"2024-08-20T03:01:58.813189+0000","last_peered":"2024-08-20T03:01:58.813189+0000","last_clean":"2024-08-20T03:01:58.813189+0000","last_became_active":"2024-08-20T03:00:18.244339+0000","last_became_peered":"2024-08-20T03:00:18.244339+0000","last_unstale":"2024-08-20T03:01:58.813189+0000","last_undegraded":"2024-08-20T03:01:58.813189+0000","last_fullsized":"2024-08-20T03:01:58.813189+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:00.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487797+0000","last_change":"2024-08-20T03:00:16.470004+0000","last_active":"2024-08-20T03:00:59.487797+0000","last_peered":"2024-08-20T03:00:59.487797+0000","last_clean":"2024-08-20T03:00:59.487797+0000","last_became_active":"2024-08-20T03:00:16.469646+0000","last_became_peered":"2024-08-20T03:00:16.469646+0000","last_unstale":"2024-08-20T03:00:59.487797+0000","last_undegraded":"2024-08-20T03:00:59.487797+0000","last_fullsized":"2024-08-20T03:00:59.487797+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:35.244683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811833+0000","last_change":"2024-08-20T03:00:18.238053+0000","last_active":"2024-08-20T03:01:58.811833+0000","last_peered":"2024-08-20T03:01:58.811833+0000","last_clean":"2024-08-20T03:01:58.811833+0000","last_became_active":"2024-08-20T03:00:18.237186+0000","last_became_peered":"2024-08-20T03:00:18.237186+0000","last_unstale":"2024-08-20T03:01:58.811833+0000","last_undegraded":"2024-08-20T03:01:58.811833+0000","last_fullsized":"2024-08-20T03:01:58.811833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:14.425688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'14","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:10.522436+0000","last_change":"2024-08-20T03:00:16.214584+0000","last_active":"2024-08-20T03:02:10.522436+0000","last_peered":"2024-08-20T03:02:10.522436+0000","last_clean":"2024-08-20T03:02:10.522436+0000","last_became_active":"2024-08-20T03:00:16.214255+0000","last_became_peered":"2024-08-20T03:00:16.214255+0000","last_unstale":"2024-08-20T03:02:10.522436+0000","last_undegraded":"2024-08-20T03:02:10.522436+0000","last_fullsized":"2024-08-20T03:02:10.522436+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:16:30.983745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77570,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1087,"num_read_kb":628,"num_write":265,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":59762,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":226580,"data_compressed":9639,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281428,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":417802,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939690,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1077825,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":417732,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299443,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":418110,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":1011301,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":417981,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1010838,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":483784,"data_compressed":15466,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":417274,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378706,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":420204,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542230,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":479213,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":419487,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":67174,"data_compressed":9639,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T03:02:16.704 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-20T03:02:16.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.838+0000 7f1f59640640 1 -- 172.21.15.44:0/1524190332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54101cb0 msgr2=0x7f1f54102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:16.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.838+0000 7f1f59640640 1 --2- 172.21.15.44:0/1524190332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54101cb0 0x7f1f54102090 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009900 tx=0x7f1f4402a410 comp rx=0 tx=0).stop 2024-08-20T03:02:16.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.838+0000 7f1f59640640 1 -- 172.21.15.44:0/1524190332 shutdown_connections 2024-08-20T03:02:16.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.838+0000 7f1f59640640 1 --2- 172.21.15.44:0/1524190332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f54104a70 0x7f1f5410e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.838+0000 7f1f59640640 1 --2- 172.21.15.44:0/1524190332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f541040d0 0x7f1f54104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.838+0000 7f1f59640640 1 --2- 172.21.15.44:0/1524190332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54101cb0 0x7f1f54102090 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.838+0000 7f1f59640640 1 -- 172.21.15.44:0/1524190332 >> 172.21.15.44:0/1524190332 conn(0x7f1f540773c0 msgr2=0x7f1f540777d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.838+0000 7f1f59640640 1 -- 172.21.15.44:0/1524190332 shutdown_connections 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.838+0000 7f1f59640640 1 -- 172.21.15.44:0/1524190332 wait complete. 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.838+0000 7f1f59640640 1 Processor -- start 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.838+0000 7f1f59640640 1 -- start start 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f59640640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54101cb0 0x7f1f54198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f59640640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f541040d0 0x7f1f54199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f59640640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f54104a70 0x7f1f5419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f59640640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54112a50 con 0x7f1f54101cb0 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f59640640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541128d0 con 0x7f1f541040d0 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f59640640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54112bd0 con 0x7f1f54104a70 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54101cb0 0x7f1f54198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54101cb0 0x7f1f54198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60632/0 (socket says 172.21.15.44:60632) 2024-08-20T03:02:16.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f52ffd640 1 -- 172.21.15.44:0/304533247 learned_addr learned my addr 172.21.15.44:0/304533247 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f527fc640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f541040d0 0x7f1f54199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f537fe640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f54104a70 0x7f1f5419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f52ffd640 1 -- 172.21.15.44:0/304533247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f54104a70 msgr2=0x7f1f5419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f52ffd640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f54104a70 0x7f1f5419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f52ffd640 1 -- 172.21.15.44:0/304533247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f541040d0 msgr2=0x7f1f54199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f52ffd640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f541040d0 0x7f1f54199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f52ffd640 1 -- 172.21.15.44:0/304533247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5419eff0 con 0x7f1f54101cb0 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f52ffd640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54101cb0 0x7f1f54198f50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1f44005940 tx=0x7f1f440040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f3bfff640 1 -- 172.21.15.44:0/304533247 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4403f030 con 0x7f1f54101cb0 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f3bfff640 1 -- 172.21.15.44:0/304533247 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f440044c0 con 0x7f1f54101cb0 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f59640640 1 -- 172.21.15.44:0/304533247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5419f280 con 0x7f1f54101cb0 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.839+0000 7f1f3bfff640 1 -- 172.21.15.44:0/304533247 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44035070 con 0x7f1f54101cb0 2024-08-20T03:02:16.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.840+0000 7f1f59640640 1 -- 172.21.15.44:0/304533247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f541a59a0 con 0x7f1f54101cb0 2024-08-20T03:02:16.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.840+0000 7f1f59640640 1 -- 172.21.15.44:0/304533247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f54102090 con 0x7f1f54101cb0 2024-08-20T03:02:16.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.842+0000 7f1f3bfff640 1 -- 172.21.15.44:0/304533247 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f1f44042030 con 0x7f1f54101cb0 2024-08-20T03:02:16.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.842+0000 7f1f3bfff640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f3c0b3cb0 0x7f1f3c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.843+0000 7f1f527fc640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f3c0b3cb0 0x7f1f3c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.843+0000 7f1f3bfff640 1 -- 172.21.15.44:0/304533247 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1f441044d0 con 0x7f1f54101cb0 2024-08-20T03:02:16.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.843+0000 7f1f527fc640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f3c0b3cb0 0x7f1f3c0b6170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f54106950 tx=0x7f1f48005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:16.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:16.845+0000 7f1f3bfff640 1 -- 172.21.15.44:0/304533247 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f1f440cb910 con 0x7f1f54101cb0 2024-08-20T03:02:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.033+0000 7f1f59640640 1 -- 172.21.15.44:0/304533247 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1f54105930 con 0x7f1f3c0b3cb0 2024-08-20T03:02:17.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.038+0000 7f1f3bfff640 1 -- 172.21.15.44:0/304533247 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259165 (secure 0 0 0) 0x7f1f54105930 con 0x7f1f3c0b3cb0 2024-08-20T03:02:17.043 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:02:17.044 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-08-20T03:02:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.041+0000 7f1f59640640 1 -- 172.21.15.44:0/304533247 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f3c0b3cb0 msgr2=0x7f1f3c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.041+0000 7f1f59640640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f3c0b3cb0 0x7f1f3c0b6170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f54106950 tx=0x7f1f48005f90 comp rx=0 tx=0).stop 2024-08-20T03:02:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.041+0000 7f1f59640640 1 -- 172.21.15.44:0/304533247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54101cb0 msgr2=0x7f1f54198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.041+0000 7f1f59640640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54101cb0 0x7f1f54198f50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1f44005940 tx=0x7f1f440040c0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.041+0000 7f1f59640640 1 -- 172.21.15.44:0/304533247 shutdown_connections 2024-08-20T03:02:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.041+0000 7f1f59640640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f3c0b3cb0 0x7f1f3c0b6170 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.041+0000 7f1f59640640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f54104a70 0x7f1f5419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.041+0000 7f1f59640640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f541040d0 0x7f1f54199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.041+0000 7f1f59640640 1 --2- 172.21.15.44:0/304533247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f54101cb0 0x7f1f54198f50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.041+0000 7f1f59640640 1 -- 172.21.15.44:0/304533247 >> 172.21.15.44:0/304533247 conn(0x7f1f540773c0 msgr2=0x7f1f54105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:17.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.041+0000 7f1f59640640 1 -- 172.21.15.44:0/304533247 shutdown_connections 2024-08-20T03:02:17.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.041+0000 7f1f59640640 1 -- 172.21.15.44:0/304533247 wait complete. 2024-08-20T03:02:17.055 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-08-20T03:02:16.416583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668172,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1168,"num_read_kb":697,"num_write":390,"num_write_kb":2310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":3011602,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":822,"ondisk_log_size":822,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363840,"kb_used_data":10432,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577984,"statfs":{"total":1151940427776,"available":1151567855616,"internally_reserved":0,"allocated":10682368,"data_stored":6991551,"data_compressed":186921,"data_compressed_allocated":245760,"data_compressed_original":2023424,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10073,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":79,"num_write":11,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003014"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809342+0000","last_change":"2024-08-20T03:00:18.238595+0000","last_active":"2024-08-20T03:01:58.809342+0000","last_peered":"2024-08-20T03:01:58.809342+0000","last_clean":"2024-08-20T03:01:58.809342+0000","last_became_active":"2024-08-20T03:00:18.238306+0000","last_became_peered":"2024-08-20T03:00:18.238306+0000","last_unstale":"2024-08-20T03:01:58.809342+0000","last_undegraded":"2024-08-20T03:01:58.809342+0000","last_fullsized":"2024-08-20T03:01:58.809342+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:33:04.681614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805183+0000","last_change":"2024-08-20T03:00:18.239722+0000","last_active":"2024-08-20T03:01:58.805183+0000","last_peered":"2024-08-20T03:01:58.805183+0000","last_clean":"2024-08-20T03:01:58.805183+0000","last_became_active":"2024-08-20T03:00:18.239391+0000","last_became_peered":"2024-08-20T03:00:18.239391+0000","last_unstale":"2024-08-20T03:01:58.805183+0000","last_undegraded":"2024-08-20T03:01:58.805183+0000","last_fullsized":"2024-08-20T03:01:58.805183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:08:34.657173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812333+0000","last_change":"2024-08-20T03:00:18.242389+0000","last_active":"2024-08-20T03:01:58.812333+0000","last_peered":"2024-08-20T03:01:58.812333+0000","last_clean":"2024-08-20T03:01:58.812333+0000","last_became_active":"2024-08-20T03:00:18.242080+0000","last_became_peered":"2024-08-20T03:00:18.242080+0000","last_unstale":"2024-08-20T03:01:58.812333+0000","last_undegraded":"2024-08-20T03:01:58.812333+0000","last_fullsized":"2024-08-20T03:01:58.812333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:37.042894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.798552+0000","last_change":"2024-08-20T03:00:18.233709+0000","last_active":"2024-08-20T03:01:58.798552+0000","last_peered":"2024-08-20T03:01:58.798552+0000","last_clean":"2024-08-20T03:01:58.798552+0000","last_became_active":"2024-08-20T03:00:18.233402+0000","last_became_peered":"2024-08-20T03:00:18.233402+0000","last_unstale":"2024-08-20T03:01:58.798552+0000","last_undegraded":"2024-08-20T03:01:58.798552+0000","last_fullsized":"2024-08-20T03:01:58.798552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:30:01.308181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.776928+0000","last_change":"2024-08-20T03:00:18.237770+0000","last_active":"2024-08-20T03:01:58.776928+0000","last_peered":"2024-08-20T03:01:58.776928+0000","last_clean":"2024-08-20T03:01:58.776928+0000","last_became_active":"2024-08-20T03:00:18.237475+0000","last_became_peered":"2024-08-20T03:00:18.237475+0000","last_unstale":"2024-08-20T03:01:58.776928+0000","last_undegraded":"2024-08-20T03:01:58.776928+0000","last_fullsized":"2024-08-20T03:01:58.776928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:37:12.707935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.797162+0000","last_change":"2024-08-20T03:00:18.245400+0000","last_active":"2024-08-20T03:01:58.797162+0000","last_peered":"2024-08-20T03:01:58.797162+0000","last_clean":"2024-08-20T03:01:58.797162+0000","last_became_active":"2024-08-20T03:00:18.244735+0000","last_became_peered":"2024-08-20T03:00:18.244735+0000","last_unstale":"2024-08-20T03:01:58.797162+0000","last_undegraded":"2024-08-20T03:01:58.797162+0000","last_fullsized":"2024-08-20T03:01:58.797162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:23:25.459441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805943+0000","last_change":"2024-08-20T03:00:18.247366+0000","last_active":"2024-08-20T03:01:58.805943+0000","last_peered":"2024-08-20T03:01:58.805943+0000","last_clean":"2024-08-20T03:01:58.805943+0000","last_became_active":"2024-08-20T03:00:18.247032+0000","last_became_peered":"2024-08-20T03:00:18.247032+0000","last_unstale":"2024-08-20T03:01:58.805943+0000","last_undegraded":"2024-08-20T03:01:58.805943+0000","last_fullsized":"2024-08-20T03:01:58.805943+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:02:18.070393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805505+0000","last_change":"2024-08-20T03:00:18.240845+0000","last_active":"2024-08-20T03:01:58.805505+0000","last_peered":"2024-08-20T03:01:58.805505+0000","last_clean":"2024-08-20T03:01:58.805505+0000","last_became_active":"2024-08-20T03:00:18.240531+0000","last_became_peered":"2024-08-20T03:00:18.240531+0000","last_unstale":"2024-08-20T03:01:58.805505+0000","last_undegraded":"2024-08-20T03:01:58.805505+0000","last_fullsized":"2024-08-20T03:01:58.805505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:15:18.975804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.814341+0000","last_change":"2024-08-20T03:00:18.240817+0000","last_active":"2024-08-20T03:01:58.814341+0000","last_peered":"2024-08-20T03:01:58.814341+0000","last_clean":"2024-08-20T03:01:58.814341+0000","last_became_active":"2024-08-20T03:00:18.240262+0000","last_became_peered":"2024-08-20T03:00:18.240262+0000","last_unstale":"2024-08-20T03:01:58.814341+0000","last_undegraded":"2024-08-20T03:01:58.814341+0000","last_fullsized":"2024-08-20T03:01:58.814341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:30.512597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815785+0000","last_change":"2024-08-20T03:00:18.242139+0000","last_active":"2024-08-20T03:01:58.815785+0000","last_peered":"2024-08-20T03:01:58.815785+0000","last_clean":"2024-08-20T03:01:58.815785+0000","last_became_active":"2024-08-20T03:00:18.241817+0000","last_became_peered":"2024-08-20T03:00:18.241817+0000","last_unstale":"2024-08-20T03:01:58.815785+0000","last_undegraded":"2024-08-20T03:01:58.815785+0000","last_fullsized":"2024-08-20T03:01:58.815785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:25.121306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805745+0000","last_change":"2024-08-20T03:00:18.243205+0000","last_active":"2024-08-20T03:01:58.805745+0000","last_peered":"2024-08-20T03:01:58.805745+0000","last_clean":"2024-08-20T03:01:58.805745+0000","last_became_active":"2024-08-20T03:00:18.242296+0000","last_became_peered":"2024-08-20T03:00:18.242296+0000","last_unstale":"2024-08-20T03:01:58.805745+0000","last_undegraded":"2024-08-20T03:01:58.805745+0000","last_fullsized":"2024-08-20T03:01:58.805745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:50.490934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.798339+0000","last_change":"2024-08-20T03:00:18.237205+0000","last_active":"2024-08-20T03:01:58.798339+0000","last_peered":"2024-08-20T03:01:58.798339+0000","last_clean":"2024-08-20T03:01:58.798339+0000","last_became_active":"2024-08-20T03:00:18.236631+0000","last_became_peered":"2024-08-20T03:00:18.236631+0000","last_unstale":"2024-08-20T03:01:58.798339+0000","last_undegraded":"2024-08-20T03:01:58.798339+0000","last_fullsized":"2024-08-20T03:01:58.798339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:13.327598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811110+0000","last_change":"2024-08-20T03:00:18.235768+0000","last_active":"2024-08-20T03:01:58.811110+0000","last_peered":"2024-08-20T03:01:58.811110+0000","last_clean":"2024-08-20T03:01:58.811110+0000","last_became_active":"2024-08-20T03:00:18.235461+0000","last_became_peered":"2024-08-20T03:00:18.235461+0000","last_unstale":"2024-08-20T03:01:58.811110+0000","last_undegraded":"2024-08-20T03:01:58.811110+0000","last_fullsized":"2024-08-20T03:01:58.811110+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:08:22.238020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809538+0000","last_change":"2024-08-20T03:00:18.246560+0000","last_active":"2024-08-20T03:01:58.809538+0000","last_peered":"2024-08-20T03:01:58.809538+0000","last_clean":"2024-08-20T03:01:58.809538+0000","last_became_active":"2024-08-20T03:00:18.246270+0000","last_became_peered":"2024-08-20T03:00:18.246270+0000","last_unstale":"2024-08-20T03:01:58.809538+0000","last_undegraded":"2024-08-20T03:01:58.809538+0000","last_fullsized":"2024-08-20T03:01:58.809538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:47:13.402635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815437+0000","last_change":"2024-08-20T03:00:18.245907+0000","last_active":"2024-08-20T03:01:58.815437+0000","last_peered":"2024-08-20T03:01:58.815437+0000","last_clean":"2024-08-20T03:01:58.815437+0000","last_became_active":"2024-08-20T03:00:18.245429+0000","last_became_peered":"2024-08-20T03:00:18.245429+0000","last_unstale":"2024-08-20T03:01:58.815437+0000","last_undegraded":"2024-08-20T03:01:58.815437+0000","last_fullsized":"2024-08-20T03:01:58.815437+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:46.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811598+0000","last_change":"2024-08-20T03:00:18.242561+0000","last_active":"2024-08-20T03:01:58.811598+0000","last_peered":"2024-08-20T03:01:58.811598+0000","last_clean":"2024-08-20T03:01:58.811598+0000","last_became_active":"2024-08-20T03:00:18.242048+0000","last_became_peered":"2024-08-20T03:00:18.242048+0000","last_unstale":"2024-08-20T03:01:58.811598+0000","last_undegraded":"2024-08-20T03:01:58.811598+0000","last_fullsized":"2024-08-20T03:01:58.811598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:27.784131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.814235+0000","last_change":"2024-08-20T03:00:18.238547+0000","last_active":"2024-08-20T03:01:58.814235+0000","last_peered":"2024-08-20T03:01:58.814235+0000","last_clean":"2024-08-20T03:01:58.814235+0000","last_became_active":"2024-08-20T03:00:18.238268+0000","last_became_peered":"2024-08-20T03:00:18.238268+0000","last_unstale":"2024-08-20T03:01:58.814235+0000","last_undegraded":"2024-08-20T03:01:58.814235+0000","last_fullsized":"2024-08-20T03:01:58.814235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:15:11.923674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812359+0000","last_change":"2024-08-20T03:00:18.241995+0000","last_active":"2024-08-20T03:01:58.812359+0000","last_peered":"2024-08-20T03:01:58.812359+0000","last_clean":"2024-08-20T03:01:58.812359+0000","last_became_active":"2024-08-20T03:00:18.241705+0000","last_became_peered":"2024-08-20T03:00:18.241705+0000","last_unstale":"2024-08-20T03:01:58.812359+0000","last_undegraded":"2024-08-20T03:01:58.812359+0000","last_fullsized":"2024-08-20T03:01:58.812359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:33:02.183712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815557+0000","last_change":"2024-08-20T03:00:18.236800+0000","last_active":"2024-08-20T03:01:58.815557+0000","last_peered":"2024-08-20T03:01:58.815557+0000","last_clean":"2024-08-20T03:01:58.815557+0000","last_became_active":"2024-08-20T03:00:18.235909+0000","last_became_peered":"2024-08-20T03:00:18.235909+0000","last_unstale":"2024-08-20T03:01:58.815557+0000","last_undegraded":"2024-08-20T03:01:58.815557+0000","last_fullsized":"2024-08-20T03:01:58.815557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:12.427760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.808784+0000","last_change":"2024-08-20T03:00:59.544187+0000","last_active":"2024-08-20T03:01:58.808784+0000","last_peered":"2024-08-20T03:01:58.808784+0000","last_clean":"2024-08-20T03:01:58.808784+0000","last_became_active":"2024-08-20T03:00:59.543807+0000","last_became_peered":"2024-08-20T03:00:59.543807+0000","last_unstale":"2024-08-20T03:01:58.808784+0000","last_undegraded":"2024-08-20T03:01:58.808784+0000","last_fullsized":"2024-08-20T03:01:58.808784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:16:12.425646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.807112+0000","last_change":"2024-08-20T03:00:18.243425+0000","last_active":"2024-08-20T03:01:58.807112+0000","last_peered":"2024-08-20T03:01:58.807112+0000","last_clean":"2024-08-20T03:01:58.807112+0000","last_became_active":"2024-08-20T03:00:18.243082+0000","last_became_peered":"2024-08-20T03:00:18.243082+0000","last_unstale":"2024-08-20T03:01:58.807112+0000","last_undegraded":"2024-08-20T03:01:58.807112+0000","last_fullsized":"2024-08-20T03:01:58.807112+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:50.008655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.813507+0000","last_change":"2024-08-20T03:00:18.242765+0000","last_active":"2024-08-20T03:01:58.813507+0000","last_peered":"2024-08-20T03:01:58.813507+0000","last_clean":"2024-08-20T03:01:58.813507+0000","last_became_active":"2024-08-20T03:00:18.242459+0000","last_became_peered":"2024-08-20T03:00:18.242459+0000","last_unstale":"2024-08-20T03:01:58.813507+0000","last_undegraded":"2024-08-20T03:01:58.813507+0000","last_fullsized":"2024-08-20T03:01:58.813507+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:16.467068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815444+0000","last_change":"2024-08-20T03:00:18.236518+0000","last_active":"2024-08-20T03:01:58.815444+0000","last_peered":"2024-08-20T03:01:58.815444+0000","last_clean":"2024-08-20T03:01:58.815444+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:01:58.815444+0000","last_undegraded":"2024-08-20T03:01:58.815444+0000","last_fullsized":"2024-08-20T03:01:58.815444+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:55:55.387134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811758+0000","last_change":"2024-08-20T03:00:18.236730+0000","last_active":"2024-08-20T03:01:58.811758+0000","last_peered":"2024-08-20T03:01:58.811758+0000","last_clean":"2024-08-20T03:01:58.811758+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:01:58.811758+0000","last_undegraded":"2024-08-20T03:01:58.811758+0000","last_fullsized":"2024-08-20T03:01:58.811758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:18.196161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811526+0000","last_change":"2024-08-20T03:00:18.245390+0000","last_active":"2024-08-20T03:01:58.811526+0000","last_peered":"2024-08-20T03:01:58.811526+0000","last_clean":"2024-08-20T03:01:58.811526+0000","last_became_active":"2024-08-20T03:00:18.241175+0000","last_became_peered":"2024-08-20T03:00:18.241175+0000","last_unstale":"2024-08-20T03:01:58.811526+0000","last_undegraded":"2024-08-20T03:01:58.811526+0000","last_fullsized":"2024-08-20T03:01:58.811526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:51:45.792846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815324+0000","last_change":"2024-08-20T03:00:18.242700+0000","last_active":"2024-08-20T03:01:58.815324+0000","last_peered":"2024-08-20T03:01:58.815324+0000","last_clean":"2024-08-20T03:01:58.815324+0000","last_became_active":"2024-08-20T03:00:18.242407+0000","last_became_peered":"2024-08-20T03:00:18.242407+0000","last_unstale":"2024-08-20T03:01:58.815324+0000","last_undegraded":"2024-08-20T03:01:58.815324+0000","last_fullsized":"2024-08-20T03:01:58.815324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:31:28.061514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809942+0000","last_change":"2024-08-20T03:00:18.233894+0000","last_active":"2024-08-20T03:01:58.809942+0000","last_peered":"2024-08-20T03:01:58.809942+0000","last_clean":"2024-08-20T03:01:58.809942+0000","last_became_active":"2024-08-20T03:00:18.233584+0000","last_became_peered":"2024-08-20T03:00:18.233584+0000","last_unstale":"2024-08-20T03:01:58.809942+0000","last_undegraded":"2024-08-20T03:01:58.809942+0000","last_fullsized":"2024-08-20T03:01:58.809942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:36:53.156215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809607+0000","last_change":"2024-08-20T03:00:18.239256+0000","last_active":"2024-08-20T03:01:58.809607+0000","last_peered":"2024-08-20T03:01:58.809607+0000","last_clean":"2024-08-20T03:01:58.809607+0000","last_became_active":"2024-08-20T03:00:18.238931+0000","last_became_peered":"2024-08-20T03:00:18.238931+0000","last_unstale":"2024-08-20T03:01:58.809607+0000","last_undegraded":"2024-08-20T03:01:58.809607+0000","last_fullsized":"2024-08-20T03:01:58.809607+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:38:42.536488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.807629+0000","last_change":"2024-08-20T03:00:18.245187+0000","last_active":"2024-08-20T03:01:58.807629+0000","last_peered":"2024-08-20T03:01:58.807629+0000","last_clean":"2024-08-20T03:01:58.807629+0000","last_became_active":"2024-08-20T03:00:18.244894+0000","last_became_peered":"2024-08-20T03:00:18.244894+0000","last_unstale":"2024-08-20T03:01:58.807629+0000","last_undegraded":"2024-08-20T03:01:58.807629+0000","last_fullsized":"2024-08-20T03:01:58.807629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:16:58.002014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811573+0000","last_change":"2024-08-20T03:00:18.241634+0000","last_active":"2024-08-20T03:01:58.811573+0000","last_peered":"2024-08-20T03:01:58.811573+0000","last_clean":"2024-08-20T03:01:58.811573+0000","last_became_active":"2024-08-20T03:00:18.241341+0000","last_became_peered":"2024-08-20T03:00:18.241341+0000","last_unstale":"2024-08-20T03:01:58.811573+0000","last_undegraded":"2024-08-20T03:01:58.811573+0000","last_fullsized":"2024-08-20T03:01:58.811573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:37:16.185544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809305+0000","last_change":"2024-08-20T03:00:18.239116+0000","last_active":"2024-08-20T03:01:58.809305+0000","last_peered":"2024-08-20T03:01:58.809305+0000","last_clean":"2024-08-20T03:01:58.809305+0000","last_became_active":"2024-08-20T03:00:18.236125+0000","last_became_peered":"2024-08-20T03:00:18.236125+0000","last_unstale":"2024-08-20T03:01:58.809305+0000","last_undegraded":"2024-08-20T03:01:58.809305+0000","last_fullsized":"2024-08-20T03:01:58.809305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:57:00.332483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.813414+0000","last_change":"2024-08-20T03:00:18.235996+0000","last_active":"2024-08-20T03:01:58.813414+0000","last_peered":"2024-08-20T03:01:58.813414+0000","last_clean":"2024-08-20T03:01:58.813414+0000","last_became_active":"2024-08-20T03:00:18.235670+0000","last_became_peered":"2024-08-20T03:00:18.235670+0000","last_unstale":"2024-08-20T03:01:58.813414+0000","last_undegraded":"2024-08-20T03:01:58.813414+0000","last_fullsized":"2024-08-20T03:01:58.813414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:59.992914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805791+0000","last_change":"2024-08-20T03:00:18.239679+0000","last_active":"2024-08-20T03:01:58.805791+0000","last_peered":"2024-08-20T03:01:58.805791+0000","last_clean":"2024-08-20T03:01:58.805791+0000","last_became_active":"2024-08-20T03:00:18.239361+0000","last_became_peered":"2024-08-20T03:00:18.239361+0000","last_unstale":"2024-08-20T03:01:58.805791+0000","last_undegraded":"2024-08-20T03:01:58.805791+0000","last_fullsized":"2024-08-20T03:01:58.805791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:28:09.430535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'8","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:13.532735+0000","last_change":"2024-08-20T03:00:16.221221+0000","last_active":"2024-08-20T03:02:13.532735+0000","last_peered":"2024-08-20T03:02:13.532735+0000","last_clean":"2024-08-20T03:02:13.532735+0000","last_became_active":"2024-08-20T03:00:16.220186+0000","last_became_peered":"2024-08-20T03:00:16.220186+0000","last_unstale":"2024-08-20T03:02:13.532735+0000","last_undegraded":"2024-08-20T03:02:13.532735+0000","last_fullsized":"2024-08-20T03:02:13.532735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:51.045235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.808630+0000","last_change":"2024-08-20T03:00:18.234809+0000","last_active":"2024-08-20T03:01:58.808630+0000","last_peered":"2024-08-20T03:01:58.808630+0000","last_clean":"2024-08-20T03:01:58.808630+0000","last_became_active":"2024-08-20T03:00:18.234486+0000","last_became_peered":"2024-08-20T03:00:18.234486+0000","last_unstale":"2024-08-20T03:01:58.808630+0000","last_undegraded":"2024-08-20T03:01:58.808630+0000","last_fullsized":"2024-08-20T03:01:58.808630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:11:52.527224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491303+0000","last_change":"2024-08-20T03:00:16.216876+0000","last_active":"2024-08-20T03:00:59.491303+0000","last_peered":"2024-08-20T03:00:59.491303+0000","last_clean":"2024-08-20T03:00:59.491303+0000","last_became_active":"2024-08-20T03:00:16.216522+0000","last_became_peered":"2024-08-20T03:00:16.216522+0000","last_unstale":"2024-08-20T03:00:59.491303+0000","last_undegraded":"2024-08-20T03:00:59.491303+0000","last_fullsized":"2024-08-20T03:00:59.491303+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:22.363331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.489899+0000","last_change":"2024-08-20T03:00:16.213751+0000","last_active":"2024-08-20T03:00:59.489899+0000","last_peered":"2024-08-20T03:00:59.489899+0000","last_clean":"2024-08-20T03:00:59.489899+0000","last_became_active":"2024-08-20T03:00:16.213348+0000","last_became_peered":"2024-08-20T03:00:16.213348+0000","last_unstale":"2024-08-20T03:00:59.489899+0000","last_undegraded":"2024-08-20T03:00:59.489899+0000","last_fullsized":"2024-08-20T03:00:59.489899+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:01:08.637151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809027+0000","last_change":"2024-08-20T03:00:18.237972+0000","last_active":"2024-08-20T03:01:58.809027+0000","last_peered":"2024-08-20T03:01:58.809027+0000","last_clean":"2024-08-20T03:01:58.809027+0000","last_became_active":"2024-08-20T03:00:18.236769+0000","last_became_peered":"2024-08-20T03:00:18.236769+0000","last_unstale":"2024-08-20T03:01:58.809027+0000","last_undegraded":"2024-08-20T03:01:58.809027+0000","last_fullsized":"2024-08-20T03:01:58.809027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:35:21.001045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492196+0000","last_change":"2024-08-20T03:00:16.221399+0000","last_active":"2024-08-20T03:00:59.492196+0000","last_peered":"2024-08-20T03:00:59.492196+0000","last_clean":"2024-08-20T03:00:59.492196+0000","last_became_active":"2024-08-20T03:00:16.220285+0000","last_became_peered":"2024-08-20T03:00:16.220285+0000","last_unstale":"2024-08-20T03:00:59.492196+0000","last_undegraded":"2024-08-20T03:00:59.492196+0000","last_fullsized":"2024-08-20T03:00:59.492196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:17.034657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.802020+0000","last_change":"2024-08-20T03:00:18.236722+0000","last_active":"2024-08-20T03:01:58.802020+0000","last_peered":"2024-08-20T03:01:58.802020+0000","last_clean":"2024-08-20T03:01:58.802020+0000","last_became_active":"2024-08-20T03:00:18.236365+0000","last_became_peered":"2024-08-20T03:00:18.236365+0000","last_unstale":"2024-08-20T03:01:58.802020+0000","last_undegraded":"2024-08-20T03:01:58.802020+0000","last_fullsized":"2024-08-20T03:01:58.802020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:17.344161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":241,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:15.535498+0000","last_change":"2024-08-20T03:00:16.222331+0000","last_active":"2024-08-20T03:02:15.535498+0000","last_peered":"2024-08-20T03:02:15.535498+0000","last_clean":"2024-08-20T03:02:15.535498+0000","last_became_active":"2024-08-20T03:00:16.222019+0000","last_became_peered":"2024-08-20T03:00:16.222019+0000","last_unstale":"2024-08-20T03:02:15.535498+0000","last_undegraded":"2024-08-20T03:02:15.535498+0000","last_fullsized":"2024-08-20T03:02:15.535498+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:46:55.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":212,"num_read_kb":212,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812391+0000","last_change":"2024-08-20T03:00:18.239020+0000","last_active":"2024-08-20T03:01:58.812391+0000","last_peered":"2024-08-20T03:01:58.812391+0000","last_clean":"2024-08-20T03:01:58.812391+0000","last_became_active":"2024-08-20T03:00:18.238620+0000","last_became_peered":"2024-08-20T03:00:18.238620+0000","last_unstale":"2024-08-20T03:01:58.812391+0000","last_undegraded":"2024-08-20T03:01:58.812391+0000","last_fullsized":"2024-08-20T03:01:58.812391+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:17:12.632357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491087+0000","last_change":"2024-08-20T03:00:16.214907+0000","last_active":"2024-08-20T03:00:59.491087+0000","last_peered":"2024-08-20T03:00:59.491087+0000","last_clean":"2024-08-20T03:00:59.491087+0000","last_became_active":"2024-08-20T03:00:16.214573+0000","last_became_peered":"2024-08-20T03:00:16.214573+0000","last_unstale":"2024-08-20T03:00:59.491087+0000","last_undegraded":"2024-08-20T03:00:59.491087+0000","last_fullsized":"2024-08-20T03:00:59.491087+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:21:34.084149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.807582+0000","last_change":"2024-08-20T03:00:18.245524+0000","last_active":"2024-08-20T03:01:58.807582+0000","last_peered":"2024-08-20T03:01:58.807582+0000","last_clean":"2024-08-20T03:01:58.807582+0000","last_became_active":"2024-08-20T03:00:18.244966+0000","last_became_peered":"2024-08-20T03:00:18.244966+0000","last_unstale":"2024-08-20T03:01:58.807582+0000","last_undegraded":"2024-08-20T03:01:58.807582+0000","last_fullsized":"2024-08-20T03:01:58.807582+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:03:21.584182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491504+0000","last_change":"2024-08-20T03:00:16.470353+0000","last_active":"2024-08-20T03:00:59.491504+0000","last_peered":"2024-08-20T03:00:59.491504+0000","last_clean":"2024-08-20T03:00:59.491504+0000","last_became_active":"2024-08-20T03:00:16.469976+0000","last_became_peered":"2024-08-20T03:00:16.469976+0000","last_unstale":"2024-08-20T03:00:59.491504+0000","last_undegraded":"2024-08-20T03:00:59.491504+0000","last_fullsized":"2024-08-20T03:00:59.491504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:25:03.595784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.814962+0000","last_change":"2024-08-20T03:00:18.245244+0000","last_active":"2024-08-20T03:01:58.814962+0000","last_peered":"2024-08-20T03:01:58.814962+0000","last_clean":"2024-08-20T03:01:58.814962+0000","last_became_active":"2024-08-20T03:00:18.244934+0000","last_became_peered":"2024-08-20T03:00:18.244934+0000","last_unstale":"2024-08-20T03:01:58.814962+0000","last_undegraded":"2024-08-20T03:01:58.814962+0000","last_fullsized":"2024-08-20T03:01:58.814962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:33:35.035039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'37","reported_seq":305,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:15.626540+0000","last_change":"2024-08-20T03:00:59.539537+0000","last_active":"2024-08-20T03:02:15.626540+0000","last_peered":"2024-08-20T03:02:15.626540+0000","last_clean":"2024-08-20T03:02:15.626540+0000","last_became_active":"2024-08-20T03:00:59.518110+0000","last_became_peered":"2024-08-20T03:00:59.518110+0000","last_unstale":"2024-08-20T03:02:15.626540+0000","last_undegraded":"2024-08-20T03:02:15.626540+0000","last_fullsized":"2024-08-20T03:02:15.626540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:10:58.722741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68603,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":218,"num_read_kb":176,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59762,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.802994+0000","last_change":"2024-08-20T03:00:18.242931+0000","last_active":"2024-08-20T03:01:58.802994+0000","last_peered":"2024-08-20T03:01:58.802994+0000","last_clean":"2024-08-20T03:01:58.802994+0000","last_became_active":"2024-08-20T03:00:18.242619+0000","last_became_peered":"2024-08-20T03:00:18.242619+0000","last_unstale":"2024-08-20T03:01:58.802994+0000","last_undegraded":"2024-08-20T03:01:58.802994+0000","last_fullsized":"2024-08-20T03:01:58.802994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:43:02.296990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492488+0000","last_change":"2024-08-20T03:00:16.214688+0000","last_active":"2024-08-20T03:00:59.492488+0000","last_peered":"2024-08-20T03:00:59.492488+0000","last_clean":"2024-08-20T03:00:59.492488+0000","last_became_active":"2024-08-20T03:00:16.214343+0000","last_became_peered":"2024-08-20T03:00:16.214343+0000","last_unstale":"2024-08-20T03:00:59.492488+0000","last_undegraded":"2024-08-20T03:00:59.492488+0000","last_fullsized":"2024-08-20T03:00:59.492488+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:10:56.587164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.806082+0000","last_change":"2024-08-20T03:00:18.244158+0000","last_active":"2024-08-20T03:01:58.806082+0000","last_peered":"2024-08-20T03:01:58.806082+0000","last_clean":"2024-08-20T03:01:58.806082+0000","last_became_active":"2024-08-20T03:00:18.243864+0000","last_became_peered":"2024-08-20T03:00:18.243864+0000","last_unstale":"2024-08-20T03:01:58.806082+0000","last_undegraded":"2024-08-20T03:01:58.806082+0000","last_fullsized":"2024-08-20T03:01:58.806082+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:10:31.373759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491090+0000","last_change":"2024-08-20T03:00:16.220846+0000","last_active":"2024-08-20T03:00:59.491090+0000","last_peered":"2024-08-20T03:00:59.491090+0000","last_clean":"2024-08-20T03:00:59.491090+0000","last_became_active":"2024-08-20T03:00:16.220519+0000","last_became_peered":"2024-08-20T03:00:16.220519+0000","last_unstale":"2024-08-20T03:00:59.491090+0000","last_undegraded":"2024-08-20T03:00:59.491090+0000","last_fullsized":"2024-08-20T03:00:59.491090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:43:41.721522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.808946+0000","last_change":"2024-08-20T03:00:18.246555+0000","last_active":"2024-08-20T03:01:58.808946+0000","last_peered":"2024-08-20T03:01:58.808946+0000","last_clean":"2024-08-20T03:01:58.808946+0000","last_became_active":"2024-08-20T03:00:18.246182+0000","last_became_peered":"2024-08-20T03:00:18.246182+0000","last_unstale":"2024-08-20T03:01:58.808946+0000","last_undegraded":"2024-08-20T03:01:58.808946+0000","last_fullsized":"2024-08-20T03:01:58.808946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:55.827649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'6","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:13.527275+0000","last_change":"2024-08-20T03:00:16.216521+0000","last_active":"2024-08-20T03:02:13.527275+0000","last_peered":"2024-08-20T03:02:13.527275+0000","last_clean":"2024-08-20T03:02:13.527275+0000","last_became_active":"2024-08-20T03:00:16.216187+0000","last_became_peered":"2024-08-20T03:00:16.216187+0000","last_unstale":"2024-08-20T03:02:13.527275+0000","last_undegraded":"2024-08-20T03:02:13.527275+0000","last_fullsized":"2024-08-20T03:02:13.527275+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:41:13.639218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.786505+0000","last_change":"2024-08-20T03:00:59.544335+0000","last_active":"2024-08-20T03:01:58.786505+0000","last_peered":"2024-08-20T03:01:58.786505+0000","last_clean":"2024-08-20T03:01:58.786505+0000","last_became_active":"2024-08-20T03:00:59.544063+0000","last_became_peered":"2024-08-20T03:00:59.544063+0000","last_unstale":"2024-08-20T03:01:58.786505+0000","last_undegraded":"2024-08-20T03:01:58.786505+0000","last_fullsized":"2024-08-20T03:01:58.786505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:48:36.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490874+0000","last_change":"2024-08-20T03:00:16.470129+0000","last_active":"2024-08-20T03:00:59.490874+0000","last_peered":"2024-08-20T03:00:59.490874+0000","last_clean":"2024-08-20T03:00:59.490874+0000","last_became_active":"2024-08-20T03:00:16.469697+0000","last_became_peered":"2024-08-20T03:00:16.469697+0000","last_unstale":"2024-08-20T03:00:59.490874+0000","last_undegraded":"2024-08-20T03:00:59.490874+0000","last_fullsized":"2024-08-20T03:00:59.490874+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:42:22.623141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.804109+0000","last_change":"2024-08-20T03:00:18.245090+0000","last_active":"2024-08-20T03:01:58.804109+0000","last_peered":"2024-08-20T03:01:58.804109+0000","last_clean":"2024-08-20T03:01:58.804109+0000","last_became_active":"2024-08-20T03:00:18.244524+0000","last_became_peered":"2024-08-20T03:00:18.244524+0000","last_unstale":"2024-08-20T03:01:58.804109+0000","last_undegraded":"2024-08-20T03:01:58.804109+0000","last_fullsized":"2024-08-20T03:01:58.804109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:14.561183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492233+0000","last_change":"2024-08-20T03:00:16.220042+0000","last_active":"2024-08-20T03:00:59.492233+0000","last_peered":"2024-08-20T03:00:59.492233+0000","last_clean":"2024-08-20T03:00:59.492233+0000","last_became_active":"2024-08-20T03:00:16.219567+0000","last_became_peered":"2024-08-20T03:00:16.219567+0000","last_unstale":"2024-08-20T03:00:59.492233+0000","last_undegraded":"2024-08-20T03:00:59.492233+0000","last_fullsized":"2024-08-20T03:00:59.492233+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:04.834039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812386+0000","last_change":"2024-08-20T03:00:18.237497+0000","last_active":"2024-08-20T03:01:58.812386+0000","last_peered":"2024-08-20T03:01:58.812386+0000","last_clean":"2024-08-20T03:01:58.812386+0000","last_became_active":"2024-08-20T03:00:18.237170+0000","last_became_peered":"2024-08-20T03:00:18.237170+0000","last_unstale":"2024-08-20T03:01:58.812386+0000","last_undegraded":"2024-08-20T03:01:58.812386+0000","last_fullsized":"2024-08-20T03:01:58.812386+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:52.387152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490226+0000","last_change":"2024-08-20T03:00:16.214861+0000","last_active":"2024-08-20T03:00:59.490226+0000","last_peered":"2024-08-20T03:00:59.490226+0000","last_clean":"2024-08-20T03:00:59.490226+0000","last_became_active":"2024-08-20T03:00:16.214487+0000","last_became_peered":"2024-08-20T03:00:16.214487+0000","last_unstale":"2024-08-20T03:00:59.490226+0000","last_undegraded":"2024-08-20T03:00:59.490226+0000","last_fullsized":"2024-08-20T03:00:59.490226+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:50:09.933304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811230+0000","last_change":"2024-08-20T03:00:18.244685+0000","last_active":"2024-08-20T03:01:58.811230+0000","last_peered":"2024-08-20T03:01:58.811230+0000","last_clean":"2024-08-20T03:01:58.811230+0000","last_became_active":"2024-08-20T03:00:18.244372+0000","last_became_peered":"2024-08-20T03:00:18.244372+0000","last_unstale":"2024-08-20T03:01:58.811230+0000","last_undegraded":"2024-08-20T03:01:58.811230+0000","last_fullsized":"2024-08-20T03:01:58.811230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:21.821712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491625+0000","last_change":"2024-08-20T03:00:16.222287+0000","last_active":"2024-08-20T03:00:59.491625+0000","last_peered":"2024-08-20T03:00:59.491625+0000","last_clean":"2024-08-20T03:00:59.491625+0000","last_became_active":"2024-08-20T03:00:16.221989+0000","last_became_peered":"2024-08-20T03:00:16.221989+0000","last_unstale":"2024-08-20T03:00:59.491625+0000","last_undegraded":"2024-08-20T03:00:59.491625+0000","last_fullsized":"2024-08-20T03:00:59.491625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:32:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.807215+0000","last_change":"2024-08-20T03:00:18.250488+0000","last_active":"2024-08-20T03:01:58.807215+0000","last_peered":"2024-08-20T03:01:58.807215+0000","last_clean":"2024-08-20T03:01:58.807215+0000","last_became_active":"2024-08-20T03:00:18.250094+0000","last_became_peered":"2024-08-20T03:00:18.250094+0000","last_unstale":"2024-08-20T03:01:58.807215+0000","last_undegraded":"2024-08-20T03:01:58.807215+0000","last_fullsized":"2024-08-20T03:01:58.807215+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:00:55.755920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'7","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:14.534593+0000","last_change":"2024-08-20T03:00:16.470223+0000","last_active":"2024-08-20T03:02:14.534593+0000","last_peered":"2024-08-20T03:02:14.534593+0000","last_clean":"2024-08-20T03:02:14.534593+0000","last_became_active":"2024-08-20T03:00:16.469871+0000","last_became_peered":"2024-08-20T03:00:16.469871+0000","last_unstale":"2024-08-20T03:02:14.534593+0000","last_undegraded":"2024-08-20T03:02:14.534593+0000","last_fullsized":"2024-08-20T03:02:14.534593+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:05:11.779164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.810540+0000","last_change":"2024-08-20T03:00:18.242755+0000","last_active":"2024-08-20T03:01:58.810540+0000","last_peered":"2024-08-20T03:01:58.810540+0000","last_clean":"2024-08-20T03:01:58.810540+0000","last_became_active":"2024-08-20T03:00:18.242447+0000","last_became_peered":"2024-08-20T03:00:18.242447+0000","last_unstale":"2024-08-20T03:01:58.810540+0000","last_undegraded":"2024-08-20T03:01:58.810540+0000","last_fullsized":"2024-08-20T03:01:58.810540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:55:40.107933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":97,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490628+0000","last_change":"2024-08-20T02:59:02.774484+0000","last_active":"2024-08-20T03:00:59.490628+0000","last_peered":"2024-08-20T03:00:59.490628+0000","last_clean":"2024-08-20T03:00:59.490628+0000","last_became_active":"2024-08-20T02:59:02.769208+0000","last_became_peered":"2024-08-20T02:59:02.769208+0000","last_unstale":"2024-08-20T03:00:59.490628+0000","last_undegraded":"2024-08-20T03:00:59.490628+0000","last_fullsized":"2024-08-20T03:00:59.490628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_clean_scrub_stamp":"2024-08-20T02:55:46.827249+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:36:27.931786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.516703+0000","last_change":"2024-08-20T03:00:16.213869+0000","last_active":"2024-08-20T03:00:59.516703+0000","last_peered":"2024-08-20T03:00:59.516703+0000","last_clean":"2024-08-20T03:00:59.516703+0000","last_became_active":"2024-08-20T03:00:16.213478+0000","last_became_peered":"2024-08-20T03:00:16.213478+0000","last_unstale":"2024-08-20T03:00:59.516703+0000","last_undegraded":"2024-08-20T03:00:59.516703+0000","last_fullsized":"2024-08-20T03:00:59.516703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:27.830497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.804627+0000","last_change":"2024-08-20T03:00:18.242283+0000","last_active":"2024-08-20T03:01:58.804627+0000","last_peered":"2024-08-20T03:01:58.804627+0000","last_clean":"2024-08-20T03:01:58.804627+0000","last_became_active":"2024-08-20T03:00:18.241943+0000","last_became_peered":"2024-08-20T03:00:18.241943+0000","last_unstale":"2024-08-20T03:01:58.804627+0000","last_undegraded":"2024-08-20T03:01:58.804627+0000","last_fullsized":"2024-08-20T03:01:58.804627+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:22:25.776964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.498057+0000","last_change":"2024-08-20T03:00:59.497552+0000","last_active":"2024-08-20T03:00:59.498057+0000","last_peered":"2024-08-20T03:00:59.498057+0000","last_clean":"2024-08-20T03:00:59.498057+0000","last_became_active":"2024-08-20T03:00:59.493978+0000","last_became_peered":"2024-08-20T03:00:59.493978+0000","last_unstale":"2024-08-20T03:00:59.498057+0000","last_undegraded":"2024-08-20T03:00:59.498057+0000","last_fullsized":"2024-08-20T03:00:59.498057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:38:12.908159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812189+0000","last_change":"2024-08-20T03:00:59.544696+0000","last_active":"2024-08-20T03:01:58.812189+0000","last_peered":"2024-08-20T03:01:58.812189+0000","last_clean":"2024-08-20T03:01:58.812189+0000","last_became_active":"2024-08-20T03:00:59.544303+0000","last_became_peered":"2024-08-20T03:00:59.544303+0000","last_unstale":"2024-08-20T03:01:58.812189+0000","last_undegraded":"2024-08-20T03:01:58.812189+0000","last_fullsized":"2024-08-20T03:01:58.812189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:01:04.814695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":131,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:10.522455+0000","last_change":"2024-08-20T03:00:16.468073+0000","last_active":"2024-08-20T03:02:10.522455+0000","last_peered":"2024-08-20T03:02:10.522455+0000","last_clean":"2024-08-20T03:02:10.522455+0000","last_became_active":"2024-08-20T03:00:16.467560+0000","last_became_peered":"2024-08-20T03:00:16.467560+0000","last_unstale":"2024-08-20T03:02:10.522455+0000","last_undegraded":"2024-08-20T03:02:10.522455+0000","last_fullsized":"2024-08-20T03:02:10.522455+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:42:24.499777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812418+0000","last_change":"2024-08-20T03:00:18.232537+0000","last_active":"2024-08-20T03:01:58.812418+0000","last_peered":"2024-08-20T03:01:58.812418+0000","last_clean":"2024-08-20T03:01:58.812418+0000","last_became_active":"2024-08-20T03:00:18.232273+0000","last_became_peered":"2024-08-20T03:00:18.232273+0000","last_unstale":"2024-08-20T03:01:58.812418+0000","last_undegraded":"2024-08-20T03:01:58.812418+0000","last_fullsized":"2024-08-20T03:01:58.812418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:36:04.981869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'7","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:11.524410+0000","last_change":"2024-08-20T03:00:16.214807+0000","last_active":"2024-08-20T03:02:11.524410+0000","last_peered":"2024-08-20T03:02:11.524410+0000","last_clean":"2024-08-20T03:02:11.524410+0000","last_became_active":"2024-08-20T03:00:16.214439+0000","last_became_peered":"2024-08-20T03:00:16.214439+0000","last_unstale":"2024-08-20T03:02:11.524410+0000","last_undegraded":"2024-08-20T03:02:11.524410+0000","last_fullsized":"2024-08-20T03:02:11.524410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:07.519151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.795094+0000","last_change":"2024-08-20T03:00:18.247220+0000","last_active":"2024-08-20T03:01:58.795094+0000","last_peered":"2024-08-20T03:01:58.795094+0000","last_clean":"2024-08-20T03:01:58.795094+0000","last_became_active":"2024-08-20T03:00:18.246896+0000","last_became_peered":"2024-08-20T03:00:18.246896+0000","last_unstale":"2024-08-20T03:01:58.795094+0000","last_undegraded":"2024-08-20T03:01:58.795094+0000","last_fullsized":"2024-08-20T03:01:58.795094+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:43:16.690590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543240+0000","last_change":"2024-08-20T03:00:16.468280+0000","last_active":"2024-08-20T03:00:59.543240+0000","last_peered":"2024-08-20T03:00:59.543240+0000","last_clean":"2024-08-20T03:00:59.543240+0000","last_became_active":"2024-08-20T03:00:16.467930+0000","last_became_peered":"2024-08-20T03:00:16.467930+0000","last_unstale":"2024-08-20T03:00:59.543240+0000","last_undegraded":"2024-08-20T03:00:59.543240+0000","last_fullsized":"2024-08-20T03:00:59.543240+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:30.423563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.797891+0000","last_change":"2024-08-20T03:00:18.240322+0000","last_active":"2024-08-20T03:01:58.797891+0000","last_peered":"2024-08-20T03:01:58.797891+0000","last_clean":"2024-08-20T03:01:58.797891+0000","last_became_active":"2024-08-20T03:00:18.239973+0000","last_became_peered":"2024-08-20T03:00:18.239973+0000","last_unstale":"2024-08-20T03:01:58.797891+0000","last_undegraded":"2024-08-20T03:01:58.797891+0000","last_fullsized":"2024-08-20T03:01:58.797891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:41:28.259762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.493654+0000","last_change":"2024-08-20T03:00:16.468307+0000","last_active":"2024-08-20T03:00:59.493654+0000","last_peered":"2024-08-20T03:00:59.493654+0000","last_clean":"2024-08-20T03:00:59.493654+0000","last_became_active":"2024-08-20T03:00:16.467993+0000","last_became_peered":"2024-08-20T03:00:16.467993+0000","last_unstale":"2024-08-20T03:00:59.493654+0000","last_undegraded":"2024-08-20T03:00:59.493654+0000","last_fullsized":"2024-08-20T03:00:59.493654+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:34.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812353+0000","last_change":"2024-08-20T03:00:18.234009+0000","last_active":"2024-08-20T03:01:58.812353+0000","last_peered":"2024-08-20T03:01:58.812353+0000","last_clean":"2024-08-20T03:01:58.812353+0000","last_became_active":"2024-08-20T03:00:18.233311+0000","last_became_peered":"2024-08-20T03:00:18.233311+0000","last_unstale":"2024-08-20T03:01:58.812353+0000","last_undegraded":"2024-08-20T03:01:58.812353+0000","last_fullsized":"2024-08-20T03:01:58.812353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:53:10.759339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'17","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543424+0000","last_change":"2024-08-20T03:00:16.214932+0000","last_active":"2024-08-20T03:00:59.543424+0000","last_peered":"2024-08-20T03:00:59.543424+0000","last_clean":"2024-08-20T03:00:59.543424+0000","last_became_active":"2024-08-20T03:00:16.214584+0000","last_became_peered":"2024-08-20T03:00:16.214584+0000","last_unstale":"2024-08-20T03:00:59.543424+0000","last_undegraded":"2024-08-20T03:00:59.543424+0000","last_fullsized":"2024-08-20T03:00:59.543424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:47.270685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809771+0000","last_change":"2024-08-20T03:00:18.247892+0000","last_active":"2024-08-20T03:01:58.809771+0000","last_peered":"2024-08-20T03:01:58.809771+0000","last_clean":"2024-08-20T03:01:58.809771+0000","last_became_active":"2024-08-20T03:00:18.245903+0000","last_became_peered":"2024-08-20T03:00:18.245903+0000","last_unstale":"2024-08-20T03:01:58.809771+0000","last_undegraded":"2024-08-20T03:01:58.809771+0000","last_fullsized":"2024-08-20T03:01:58.809771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:08:50.909638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.516720+0000","last_change":"2024-08-20T03:00:16.372503+0000","last_active":"2024-08-20T03:00:59.516720+0000","last_peered":"2024-08-20T03:00:59.516720+0000","last_clean":"2024-08-20T03:00:59.516720+0000","last_became_active":"2024-08-20T03:00:16.372218+0000","last_became_peered":"2024-08-20T03:00:16.372218+0000","last_unstale":"2024-08-20T03:00:59.516720+0000","last_undegraded":"2024-08-20T03:00:59.516720+0000","last_fullsized":"2024-08-20T03:00:59.516720+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:13.341900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805821+0000","last_change":"2024-08-20T03:00:18.244738+0000","last_active":"2024-08-20T03:01:58.805821+0000","last_peered":"2024-08-20T03:01:58.805821+0000","last_clean":"2024-08-20T03:01:58.805821+0000","last_became_active":"2024-08-20T03:00:18.244398+0000","last_became_peered":"2024-08-20T03:00:18.244398+0000","last_unstale":"2024-08-20T03:01:58.805821+0000","last_undegraded":"2024-08-20T03:01:58.805821+0000","last_fullsized":"2024-08-20T03:01:58.805821+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:45:14.540125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494339+0000","last_change":"2024-08-20T03:00:16.468251+0000","last_active":"2024-08-20T03:00:59.494339+0000","last_peered":"2024-08-20T03:00:59.494339+0000","last_clean":"2024-08-20T03:00:59.494339+0000","last_became_active":"2024-08-20T03:00:16.467912+0000","last_became_peered":"2024-08-20T03:00:16.467912+0000","last_unstale":"2024-08-20T03:00:59.494339+0000","last_undegraded":"2024-08-20T03:00:59.494339+0000","last_fullsized":"2024-08-20T03:00:59.494339+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:43.799747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.770324+0000","last_change":"2024-08-20T03:00:18.234624+0000","last_active":"2024-08-20T03:01:58.770324+0000","last_peered":"2024-08-20T03:01:58.770324+0000","last_clean":"2024-08-20T03:01:58.770324+0000","last_became_active":"2024-08-20T03:00:18.234331+0000","last_became_peered":"2024-08-20T03:00:18.234331+0000","last_unstale":"2024-08-20T03:01:58.770324+0000","last_undegraded":"2024-08-20T03:01:58.770324+0000","last_fullsized":"2024-08-20T03:01:58.770324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:05:09.665645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487438+0000","last_change":"2024-08-20T03:00:16.216658+0000","last_active":"2024-08-20T03:00:59.487438+0000","last_peered":"2024-08-20T03:00:59.487438+0000","last_clean":"2024-08-20T03:00:59.487438+0000","last_became_active":"2024-08-20T03:00:16.216352+0000","last_became_peered":"2024-08-20T03:00:16.216352+0000","last_unstale":"2024-08-20T03:00:59.487438+0000","last_undegraded":"2024-08-20T03:00:59.487438+0000","last_fullsized":"2024-08-20T03:00:59.487438+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:20.763280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811335+0000","last_change":"2024-08-20T03:00:18.240984+0000","last_active":"2024-08-20T03:01:58.811335+0000","last_peered":"2024-08-20T03:01:58.811335+0000","last_clean":"2024-08-20T03:01:58.811335+0000","last_became_active":"2024-08-20T03:00:18.240640+0000","last_became_peered":"2024-08-20T03:00:18.240640+0000","last_unstale":"2024-08-20T03:01:58.811335+0000","last_undegraded":"2024-08-20T03:01:58.811335+0000","last_fullsized":"2024-08-20T03:01:58.811335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:05.589186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492391+0000","last_change":"2024-08-20T03:00:16.214818+0000","last_active":"2024-08-20T03:00:59.492391+0000","last_peered":"2024-08-20T03:00:59.492391+0000","last_clean":"2024-08-20T03:00:59.492391+0000","last_became_active":"2024-08-20T03:00:16.214493+0000","last_became_peered":"2024-08-20T03:00:16.214493+0000","last_unstale":"2024-08-20T03:00:59.492391+0000","last_undegraded":"2024-08-20T03:00:59.492391+0000","last_fullsized":"2024-08-20T03:00:59.492391+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:56:12.895142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.806590+0000","last_change":"2024-08-20T03:00:18.240350+0000","last_active":"2024-08-20T03:01:58.806590+0000","last_peered":"2024-08-20T03:01:58.806590+0000","last_clean":"2024-08-20T03:01:58.806590+0000","last_became_active":"2024-08-20T03:00:18.239464+0000","last_became_peered":"2024-08-20T03:00:18.239464+0000","last_unstale":"2024-08-20T03:01:58.806590+0000","last_undegraded":"2024-08-20T03:01:58.806590+0000","last_fullsized":"2024-08-20T03:01:58.806590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:48.715130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491431+0000","last_change":"2024-08-20T03:00:16.216144+0000","last_active":"2024-08-20T03:00:59.491431+0000","last_peered":"2024-08-20T03:00:59.491431+0000","last_clean":"2024-08-20T03:00:59.491431+0000","last_became_active":"2024-08-20T03:00:16.215792+0000","last_became_peered":"2024-08-20T03:00:16.215792+0000","last_unstale":"2024-08-20T03:00:59.491431+0000","last_undegraded":"2024-08-20T03:00:59.491431+0000","last_fullsized":"2024-08-20T03:00:59.491431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:25:01.500878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809127+0000","last_change":"2024-08-20T03:00:18.245286+0000","last_active":"2024-08-20T03:01:58.809127+0000","last_peered":"2024-08-20T03:01:58.809127+0000","last_clean":"2024-08-20T03:01:58.809127+0000","last_became_active":"2024-08-20T03:00:18.244760+0000","last_became_peered":"2024-08-20T03:00:18.244760+0000","last_unstale":"2024-08-20T03:01:58.809127+0000","last_undegraded":"2024-08-20T03:01:58.809127+0000","last_fullsized":"2024-08-20T03:01:58.809127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:09.415794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.814003+0000","last_change":"2024-08-20T03:00:59.495948+0000","last_active":"2024-08-20T03:01:58.814003+0000","last_peered":"2024-08-20T03:01:58.814003+0000","last_clean":"2024-08-20T03:01:58.814003+0000","last_became_active":"2024-08-20T03:00:59.495543+0000","last_became_peered":"2024-08-20T03:00:59.495543+0000","last_unstale":"2024-08-20T03:01:58.814003+0000","last_undegraded":"2024-08-20T03:01:58.814003+0000","last_fullsized":"2024-08-20T03:01:58.814003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:38:30.671235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492392+0000","last_change":"2024-08-20T03:00:16.220019+0000","last_active":"2024-08-20T03:00:59.492392+0000","last_peered":"2024-08-20T03:00:59.492392+0000","last_clean":"2024-08-20T03:00:59.492392+0000","last_became_active":"2024-08-20T03:00:16.219624+0000","last_became_peered":"2024-08-20T03:00:16.219624+0000","last_unstale":"2024-08-20T03:00:59.492392+0000","last_undegraded":"2024-08-20T03:00:59.492392+0000","last_fullsized":"2024-08-20T03:00:59.492392+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:33:24.926695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.813429+0000","last_change":"2024-08-20T03:00:18.242156+0000","last_active":"2024-08-20T03:01:58.813429+0000","last_peered":"2024-08-20T03:01:58.813429+0000","last_clean":"2024-08-20T03:01:58.813429+0000","last_became_active":"2024-08-20T03:00:18.241811+0000","last_became_peered":"2024-08-20T03:00:18.241811+0000","last_unstale":"2024-08-20T03:01:58.813429+0000","last_undegraded":"2024-08-20T03:01:58.813429+0000","last_fullsized":"2024-08-20T03:01:58.813429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:58.230658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.534283+0000","last_change":"2024-08-20T03:00:59.533906+0000","last_active":"2024-08-20T03:00:59.534283+0000","last_peered":"2024-08-20T03:00:59.534283+0000","last_clean":"2024-08-20T03:00:59.534283+0000","last_became_active":"2024-08-20T03:00:59.517824+0000","last_became_peered":"2024-08-20T03:00:59.517824+0000","last_unstale":"2024-08-20T03:00:59.534283+0000","last_undegraded":"2024-08-20T03:00:59.534283+0000","last_fullsized":"2024-08-20T03:00:59.534283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:25:28.219430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.813189+0000","last_change":"2024-08-20T03:00:18.245256+0000","last_active":"2024-08-20T03:01:58.813189+0000","last_peered":"2024-08-20T03:01:58.813189+0000","last_clean":"2024-08-20T03:01:58.813189+0000","last_became_active":"2024-08-20T03:00:18.244339+0000","last_became_peered":"2024-08-20T03:00:18.244339+0000","last_unstale":"2024-08-20T03:01:58.813189+0000","last_undegraded":"2024-08-20T03:01:58.813189+0000","last_fullsized":"2024-08-20T03:01:58.813189+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:00.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487797+0000","last_change":"2024-08-20T03:00:16.470004+0000","last_active":"2024-08-20T03:00:59.487797+0000","last_peered":"2024-08-20T03:00:59.487797+0000","last_clean":"2024-08-20T03:00:59.487797+0000","last_became_active":"2024-08-20T03:00:16.469646+0000","last_became_peered":"2024-08-20T03:00:16.469646+0000","last_unstale":"2024-08-20T03:00:59.487797+0000","last_undegraded":"2024-08-20T03:00:59.487797+0000","last_fullsized":"2024-08-20T03:00:59.487797+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:35.244683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811833+0000","last_change":"2024-08-20T03:00:18.238053+0000","last_active":"2024-08-20T03:01:58.811833+0000","last_peered":"2024-08-20T03:01:58.811833+0000","last_clean":"2024-08-20T03:01:58.811833+0000","last_became_active":"2024-08-20T03:00:18.237186+0000","last_became_peered":"2024-08-20T03:00:18.237186+0000","last_unstale":"2024-08-20T03:01:58.811833+0000","last_undegraded":"2024-08-20T03:01:58.811833+0000","last_fullsized":"2024-08-20T03:01:58.811833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:14.425688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'14","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:10.522436+0000","last_change":"2024-08-20T03:00:16.214584+0000","last_active":"2024-08-20T03:02:10.522436+0000","last_peered":"2024-08-20T03:02:10.522436+0000","last_clean":"2024-08-20T03:02:10.522436+0000","last_became_active":"2024-08-20T03:00:16.214255+0000","last_became_peered":"2024-08-20T03:00:16.214255+0000","last_unstale":"2024-08-20T03:02:10.522436+0000","last_undegraded":"2024-08-20T03:02:10.522436+0000","last_fullsized":"2024-08-20T03:02:10.522436+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:16:30.983745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77570,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1087,"num_read_kb":628,"num_write":265,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":59762,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":226580,"data_compressed":9639,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281428,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":417802,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939690,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1077825,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":417732,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299443,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":418110,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":1011301,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":417981,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1010838,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":483784,"data_compressed":15466,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":417274,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378706,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":420204,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542230,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":479213,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":419487,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":67174,"data_compressed":9639,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T03:02:17.059 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-08-20T03:02:17.059 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-20T03:02:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.192+0000 7f43d7c3b640 1 -- 172.21.15.44:0/3550163881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d01014b0 msgr2=0x7f43d0101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.192+0000 7f43d7c3b640 1 --2- 172.21.15.44:0/3550163881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d01014b0 0x7f43d0101890 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009900 tx=0x7f43cc02a480 comp rx=0 tx=0).stop 2024-08-20T03:02:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.192+0000 7f43d7c3b640 1 -- 172.21.15.44:0/3550163881 shutdown_connections 2024-08-20T03:02:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.192+0000 7f43d7c3b640 1 --2- 172.21.15.44:0/3550163881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d010f060 0x7f43d0111450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.192+0000 7f43d7c3b640 1 --2- 172.21.15.44:0/3550163881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d0101dd0 0x7f43d010eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.192+0000 7f43d7c3b640 1 --2- 172.21.15.44:0/3550163881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d01014b0 0x7f43d0101890 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.192+0000 7f43d7c3b640 1 -- 172.21.15.44:0/3550163881 >> 172.21.15.44:0/3550163881 conn(0x7f43d00fd360 msgr2=0x7f43d00ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.192+0000 7f43d7c3b640 1 -- 172.21.15.44:0/3550163881 shutdown_connections 2024-08-20T03:02:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.192+0000 7f43d7c3b640 1 -- 172.21.15.44:0/3550163881 wait complete. 2024-08-20T03:02:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.193+0000 7f43d7c3b640 1 Processor -- start 2024-08-20T03:02:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.193+0000 7f43d7c3b640 1 -- start start 2024-08-20T03:02:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.193+0000 7f43d7c3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d01014b0 0x7f43d019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.193+0000 7f43d7c3b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0101dd0 0x7f43d019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.193+0000 7f43d7c3b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d010f060 0x7f43d01a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.193+0000 7f43d7c3b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0114c10 con 0x7f43d0101dd0 2024-08-20T03:02:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.193+0000 7f43d7c3b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0114a90 con 0x7f43d01014b0 2024-08-20T03:02:17.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.193+0000 7f43d7c3b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0114d90 con 0x7f43d010f060 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.193+0000 7f43d51d6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0101dd0 0x7f43d019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.193+0000 7f43d51d6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0101dd0 0x7f43d019b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60648/0 (socket says 172.21.15.44:60648) 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.193+0000 7f43d51d6640 1 -- 172.21.15.44:0/1379829907 learned_addr learned my addr 172.21.15.44:0/1379829907 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43d61d8640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d010f060 0x7f43d01a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43d59d7640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d01014b0 0x7f43d019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43d51d6640 1 -- 172.21.15.44:0/1379829907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d010f060 msgr2=0x7f43d01a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43d51d6640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d010f060 0x7f43d01a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43d51d6640 1 -- 172.21.15.44:0/1379829907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d01014b0 msgr2=0x7f43d019b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43d51d6640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d01014b0 0x7f43d019b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43d51d6640 1 -- 172.21.15.44:0/1379829907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d01a11d0 con 0x7f43d0101dd0 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43d59d7640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d01014b0 0x7f43d019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43d51d6640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0101dd0 0x7f43d019b670 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c890 tx=0x7f43c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43c6ffd640 1 -- 172.21.15.44:0/1379829907 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0004220 con 0x7f43d0101dd0 2024-08-20T03:02:17.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43c6ffd640 1 -- 172.21.15.44:0/1379829907 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43c00043c0 con 0x7f43d0101dd0 2024-08-20T03:02:17.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43d7c3b640 1 -- 172.21.15.44:0/1379829907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d01a14c0 con 0x7f43d0101dd0 2024-08-20T03:02:17.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43c6ffd640 1 -- 172.21.15.44:0/1379829907 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0004a00 con 0x7f43d0101dd0 2024-08-20T03:02:17.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.194+0000 7f43d7c3b640 1 -- 172.21.15.44:0/1379829907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d01a7b80 con 0x7f43d0101dd0 2024-08-20T03:02:17.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.195+0000 7f43d7c3b640 1 -- 172.21.15.44:0/1379829907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d0101890 con 0x7f43d0101dd0 2024-08-20T03:02:17.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.197+0000 7f43c6ffd640 1 -- 172.21.15.44:0/1379829907 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f43c00079b0 con 0x7f43d0101dd0 2024-08-20T03:02:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.198+0000 7f43c6ffd640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f43bc0b3cb0 0x7f43bc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.198+0000 7f43d59d7640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f43bc0b3cb0 0x7f43bc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.198+0000 7f43c6ffd640 1 -- 172.21.15.44:0/1379829907 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f43c00e4110 con 0x7f43d0101dd0 2024-08-20T03:02:17.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.199+0000 7f43d59d7640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f43bc0b3cb0 0x7f43bc0b6170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43cc000c00 tx=0x7f43cc031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:17.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.201+0000 7f43c6ffd640 1 -- 172.21.15.44:0/1379829907 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f43c0014030 con 0x7f43d0101dd0 2024-08-20T03:02:17.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.382+0000 7f43d7c3b640 1 -- 172.21.15.44:0/1379829907 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43d0110420 con 0x7f43bc0b3cb0 2024-08-20T03:02:17.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.386+0000 7f43c6ffd640 1 -- 172.21.15.44:0/1379829907 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259165 (secure 0 0 0) 0x7f43d0110420 con 0x7f43bc0b3cb0 2024-08-20T03:02:17.392 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:02:17.393 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-08-20T03:02:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.389+0000 7f43d7c3b640 1 -- 172.21.15.44:0/1379829907 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f43bc0b3cb0 msgr2=0x7f43bc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.389+0000 7f43d7c3b640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f43bc0b3cb0 0x7f43bc0b6170 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43cc000c00 tx=0x7f43cc031400 comp rx=0 tx=0).stop 2024-08-20T03:02:17.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.389+0000 7f43d7c3b640 1 -- 172.21.15.44:0/1379829907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0101dd0 msgr2=0x7f43d019b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.389+0000 7f43d7c3b640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0101dd0 0x7f43d019b670 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c890 tx=0x7f43c000cd50 comp rx=0 tx=0).stop 2024-08-20T03:02:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.389+0000 7f43d7c3b640 1 -- 172.21.15.44:0/1379829907 shutdown_connections 2024-08-20T03:02:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.389+0000 7f43d7c3b640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f43bc0b3cb0 0x7f43bc0b6170 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.389+0000 7f43d7c3b640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43d010f060 0x7f43d01a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.389+0000 7f43d7c3b640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43d0101dd0 0x7f43d019b670 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.389+0000 7f43d7c3b640 1 --2- 172.21.15.44:0/1379829907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43d01014b0 0x7f43d019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.389+0000 7f43d7c3b640 1 -- 172.21.15.44:0/1379829907 >> 172.21.15.44:0/1379829907 conn(0x7f43d00fd360 msgr2=0x7f43d00ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.389+0000 7f43d7c3b640 1 -- 172.21.15.44:0/1379829907 shutdown_connections 2024-08-20T03:02:17.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:17.389+0000 7f43d7c3b640 1 -- 172.21.15.44:0/1379829907 wait complete. 2024-08-20T03:02:17.403 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-08-20T03:02:16.416583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668172,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1168,"num_read_kb":697,"num_write":390,"num_write_kb":2310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":3011602,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":822,"ondisk_log_size":822,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363840,"kb_used_data":10432,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577984,"statfs":{"total":1151940427776,"available":1151567855616,"internally_reserved":0,"allocated":10682368,"data_stored":6991551,"data_compressed":186921,"data_compressed_allocated":245760,"data_compressed_original":2023424,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10073,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":79,"num_write":11,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003014"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809342+0000","last_change":"2024-08-20T03:00:18.238595+0000","last_active":"2024-08-20T03:01:58.809342+0000","last_peered":"2024-08-20T03:01:58.809342+0000","last_clean":"2024-08-20T03:01:58.809342+0000","last_became_active":"2024-08-20T03:00:18.238306+0000","last_became_peered":"2024-08-20T03:00:18.238306+0000","last_unstale":"2024-08-20T03:01:58.809342+0000","last_undegraded":"2024-08-20T03:01:58.809342+0000","last_fullsized":"2024-08-20T03:01:58.809342+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:33:04.681614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805183+0000","last_change":"2024-08-20T03:00:18.239722+0000","last_active":"2024-08-20T03:01:58.805183+0000","last_peered":"2024-08-20T03:01:58.805183+0000","last_clean":"2024-08-20T03:01:58.805183+0000","last_became_active":"2024-08-20T03:00:18.239391+0000","last_became_peered":"2024-08-20T03:00:18.239391+0000","last_unstale":"2024-08-20T03:01:58.805183+0000","last_undegraded":"2024-08-20T03:01:58.805183+0000","last_fullsized":"2024-08-20T03:01:58.805183+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:08:34.657173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812333+0000","last_change":"2024-08-20T03:00:18.242389+0000","last_active":"2024-08-20T03:01:58.812333+0000","last_peered":"2024-08-20T03:01:58.812333+0000","last_clean":"2024-08-20T03:01:58.812333+0000","last_became_active":"2024-08-20T03:00:18.242080+0000","last_became_peered":"2024-08-20T03:00:18.242080+0000","last_unstale":"2024-08-20T03:01:58.812333+0000","last_undegraded":"2024-08-20T03:01:58.812333+0000","last_fullsized":"2024-08-20T03:01:58.812333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:37.042894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.798552+0000","last_change":"2024-08-20T03:00:18.233709+0000","last_active":"2024-08-20T03:01:58.798552+0000","last_peered":"2024-08-20T03:01:58.798552+0000","last_clean":"2024-08-20T03:01:58.798552+0000","last_became_active":"2024-08-20T03:00:18.233402+0000","last_became_peered":"2024-08-20T03:00:18.233402+0000","last_unstale":"2024-08-20T03:01:58.798552+0000","last_undegraded":"2024-08-20T03:01:58.798552+0000","last_fullsized":"2024-08-20T03:01:58.798552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:30:01.308181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.776928+0000","last_change":"2024-08-20T03:00:18.237770+0000","last_active":"2024-08-20T03:01:58.776928+0000","last_peered":"2024-08-20T03:01:58.776928+0000","last_clean":"2024-08-20T03:01:58.776928+0000","last_became_active":"2024-08-20T03:00:18.237475+0000","last_became_peered":"2024-08-20T03:00:18.237475+0000","last_unstale":"2024-08-20T03:01:58.776928+0000","last_undegraded":"2024-08-20T03:01:58.776928+0000","last_fullsized":"2024-08-20T03:01:58.776928+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:37:12.707935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.797162+0000","last_change":"2024-08-20T03:00:18.245400+0000","last_active":"2024-08-20T03:01:58.797162+0000","last_peered":"2024-08-20T03:01:58.797162+0000","last_clean":"2024-08-20T03:01:58.797162+0000","last_became_active":"2024-08-20T03:00:18.244735+0000","last_became_peered":"2024-08-20T03:00:18.244735+0000","last_unstale":"2024-08-20T03:01:58.797162+0000","last_undegraded":"2024-08-20T03:01:58.797162+0000","last_fullsized":"2024-08-20T03:01:58.797162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:23:25.459441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805943+0000","last_change":"2024-08-20T03:00:18.247366+0000","last_active":"2024-08-20T03:01:58.805943+0000","last_peered":"2024-08-20T03:01:58.805943+0000","last_clean":"2024-08-20T03:01:58.805943+0000","last_became_active":"2024-08-20T03:00:18.247032+0000","last_became_peered":"2024-08-20T03:00:18.247032+0000","last_unstale":"2024-08-20T03:01:58.805943+0000","last_undegraded":"2024-08-20T03:01:58.805943+0000","last_fullsized":"2024-08-20T03:01:58.805943+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:02:18.070393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805505+0000","last_change":"2024-08-20T03:00:18.240845+0000","last_active":"2024-08-20T03:01:58.805505+0000","last_peered":"2024-08-20T03:01:58.805505+0000","last_clean":"2024-08-20T03:01:58.805505+0000","last_became_active":"2024-08-20T03:00:18.240531+0000","last_became_peered":"2024-08-20T03:00:18.240531+0000","last_unstale":"2024-08-20T03:01:58.805505+0000","last_undegraded":"2024-08-20T03:01:58.805505+0000","last_fullsized":"2024-08-20T03:01:58.805505+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:15:18.975804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.814341+0000","last_change":"2024-08-20T03:00:18.240817+0000","last_active":"2024-08-20T03:01:58.814341+0000","last_peered":"2024-08-20T03:01:58.814341+0000","last_clean":"2024-08-20T03:01:58.814341+0000","last_became_active":"2024-08-20T03:00:18.240262+0000","last_became_peered":"2024-08-20T03:00:18.240262+0000","last_unstale":"2024-08-20T03:01:58.814341+0000","last_undegraded":"2024-08-20T03:01:58.814341+0000","last_fullsized":"2024-08-20T03:01:58.814341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:30.512597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815785+0000","last_change":"2024-08-20T03:00:18.242139+0000","last_active":"2024-08-20T03:01:58.815785+0000","last_peered":"2024-08-20T03:01:58.815785+0000","last_clean":"2024-08-20T03:01:58.815785+0000","last_became_active":"2024-08-20T03:00:18.241817+0000","last_became_peered":"2024-08-20T03:00:18.241817+0000","last_unstale":"2024-08-20T03:01:58.815785+0000","last_undegraded":"2024-08-20T03:01:58.815785+0000","last_fullsized":"2024-08-20T03:01:58.815785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:25.121306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805745+0000","last_change":"2024-08-20T03:00:18.243205+0000","last_active":"2024-08-20T03:01:58.805745+0000","last_peered":"2024-08-20T03:01:58.805745+0000","last_clean":"2024-08-20T03:01:58.805745+0000","last_became_active":"2024-08-20T03:00:18.242296+0000","last_became_peered":"2024-08-20T03:00:18.242296+0000","last_unstale":"2024-08-20T03:01:58.805745+0000","last_undegraded":"2024-08-20T03:01:58.805745+0000","last_fullsized":"2024-08-20T03:01:58.805745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:50.490934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.798339+0000","last_change":"2024-08-20T03:00:18.237205+0000","last_active":"2024-08-20T03:01:58.798339+0000","last_peered":"2024-08-20T03:01:58.798339+0000","last_clean":"2024-08-20T03:01:58.798339+0000","last_became_active":"2024-08-20T03:00:18.236631+0000","last_became_peered":"2024-08-20T03:00:18.236631+0000","last_unstale":"2024-08-20T03:01:58.798339+0000","last_undegraded":"2024-08-20T03:01:58.798339+0000","last_fullsized":"2024-08-20T03:01:58.798339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:13.327598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811110+0000","last_change":"2024-08-20T03:00:18.235768+0000","last_active":"2024-08-20T03:01:58.811110+0000","last_peered":"2024-08-20T03:01:58.811110+0000","last_clean":"2024-08-20T03:01:58.811110+0000","last_became_active":"2024-08-20T03:00:18.235461+0000","last_became_peered":"2024-08-20T03:00:18.235461+0000","last_unstale":"2024-08-20T03:01:58.811110+0000","last_undegraded":"2024-08-20T03:01:58.811110+0000","last_fullsized":"2024-08-20T03:01:58.811110+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:08:22.238020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809538+0000","last_change":"2024-08-20T03:00:18.246560+0000","last_active":"2024-08-20T03:01:58.809538+0000","last_peered":"2024-08-20T03:01:58.809538+0000","last_clean":"2024-08-20T03:01:58.809538+0000","last_became_active":"2024-08-20T03:00:18.246270+0000","last_became_peered":"2024-08-20T03:00:18.246270+0000","last_unstale":"2024-08-20T03:01:58.809538+0000","last_undegraded":"2024-08-20T03:01:58.809538+0000","last_fullsized":"2024-08-20T03:01:58.809538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:47:13.402635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815437+0000","last_change":"2024-08-20T03:00:18.245907+0000","last_active":"2024-08-20T03:01:58.815437+0000","last_peered":"2024-08-20T03:01:58.815437+0000","last_clean":"2024-08-20T03:01:58.815437+0000","last_became_active":"2024-08-20T03:00:18.245429+0000","last_became_peered":"2024-08-20T03:00:18.245429+0000","last_unstale":"2024-08-20T03:01:58.815437+0000","last_undegraded":"2024-08-20T03:01:58.815437+0000","last_fullsized":"2024-08-20T03:01:58.815437+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:46.002217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811598+0000","last_change":"2024-08-20T03:00:18.242561+0000","last_active":"2024-08-20T03:01:58.811598+0000","last_peered":"2024-08-20T03:01:58.811598+0000","last_clean":"2024-08-20T03:01:58.811598+0000","last_became_active":"2024-08-20T03:00:18.242048+0000","last_became_peered":"2024-08-20T03:00:18.242048+0000","last_unstale":"2024-08-20T03:01:58.811598+0000","last_undegraded":"2024-08-20T03:01:58.811598+0000","last_fullsized":"2024-08-20T03:01:58.811598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:27.784131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.814235+0000","last_change":"2024-08-20T03:00:18.238547+0000","last_active":"2024-08-20T03:01:58.814235+0000","last_peered":"2024-08-20T03:01:58.814235+0000","last_clean":"2024-08-20T03:01:58.814235+0000","last_became_active":"2024-08-20T03:00:18.238268+0000","last_became_peered":"2024-08-20T03:00:18.238268+0000","last_unstale":"2024-08-20T03:01:58.814235+0000","last_undegraded":"2024-08-20T03:01:58.814235+0000","last_fullsized":"2024-08-20T03:01:58.814235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:15:11.923674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812359+0000","last_change":"2024-08-20T03:00:18.241995+0000","last_active":"2024-08-20T03:01:58.812359+0000","last_peered":"2024-08-20T03:01:58.812359+0000","last_clean":"2024-08-20T03:01:58.812359+0000","last_became_active":"2024-08-20T03:00:18.241705+0000","last_became_peered":"2024-08-20T03:00:18.241705+0000","last_unstale":"2024-08-20T03:01:58.812359+0000","last_undegraded":"2024-08-20T03:01:58.812359+0000","last_fullsized":"2024-08-20T03:01:58.812359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:33:02.183712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815557+0000","last_change":"2024-08-20T03:00:18.236800+0000","last_active":"2024-08-20T03:01:58.815557+0000","last_peered":"2024-08-20T03:01:58.815557+0000","last_clean":"2024-08-20T03:01:58.815557+0000","last_became_active":"2024-08-20T03:00:18.235909+0000","last_became_peered":"2024-08-20T03:00:18.235909+0000","last_unstale":"2024-08-20T03:01:58.815557+0000","last_undegraded":"2024-08-20T03:01:58.815557+0000","last_fullsized":"2024-08-20T03:01:58.815557+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:12.427760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.808784+0000","last_change":"2024-08-20T03:00:59.544187+0000","last_active":"2024-08-20T03:01:58.808784+0000","last_peered":"2024-08-20T03:01:58.808784+0000","last_clean":"2024-08-20T03:01:58.808784+0000","last_became_active":"2024-08-20T03:00:59.543807+0000","last_became_peered":"2024-08-20T03:00:59.543807+0000","last_unstale":"2024-08-20T03:01:58.808784+0000","last_undegraded":"2024-08-20T03:01:58.808784+0000","last_fullsized":"2024-08-20T03:01:58.808784+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:16:12.425646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.807112+0000","last_change":"2024-08-20T03:00:18.243425+0000","last_active":"2024-08-20T03:01:58.807112+0000","last_peered":"2024-08-20T03:01:58.807112+0000","last_clean":"2024-08-20T03:01:58.807112+0000","last_became_active":"2024-08-20T03:00:18.243082+0000","last_became_peered":"2024-08-20T03:00:18.243082+0000","last_unstale":"2024-08-20T03:01:58.807112+0000","last_undegraded":"2024-08-20T03:01:58.807112+0000","last_fullsized":"2024-08-20T03:01:58.807112+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:50.008655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.813507+0000","last_change":"2024-08-20T03:00:18.242765+0000","last_active":"2024-08-20T03:01:58.813507+0000","last_peered":"2024-08-20T03:01:58.813507+0000","last_clean":"2024-08-20T03:01:58.813507+0000","last_became_active":"2024-08-20T03:00:18.242459+0000","last_became_peered":"2024-08-20T03:00:18.242459+0000","last_unstale":"2024-08-20T03:01:58.813507+0000","last_undegraded":"2024-08-20T03:01:58.813507+0000","last_fullsized":"2024-08-20T03:01:58.813507+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:16.467068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815444+0000","last_change":"2024-08-20T03:00:18.236518+0000","last_active":"2024-08-20T03:01:58.815444+0000","last_peered":"2024-08-20T03:01:58.815444+0000","last_clean":"2024-08-20T03:01:58.815444+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:01:58.815444+0000","last_undegraded":"2024-08-20T03:01:58.815444+0000","last_fullsized":"2024-08-20T03:01:58.815444+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:55:55.387134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811758+0000","last_change":"2024-08-20T03:00:18.236730+0000","last_active":"2024-08-20T03:01:58.811758+0000","last_peered":"2024-08-20T03:01:58.811758+0000","last_clean":"2024-08-20T03:01:58.811758+0000","last_became_active":"2024-08-20T03:00:18.236210+0000","last_became_peered":"2024-08-20T03:00:18.236210+0000","last_unstale":"2024-08-20T03:01:58.811758+0000","last_undegraded":"2024-08-20T03:01:58.811758+0000","last_fullsized":"2024-08-20T03:01:58.811758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:18.196161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811526+0000","last_change":"2024-08-20T03:00:18.245390+0000","last_active":"2024-08-20T03:01:58.811526+0000","last_peered":"2024-08-20T03:01:58.811526+0000","last_clean":"2024-08-20T03:01:58.811526+0000","last_became_active":"2024-08-20T03:00:18.241175+0000","last_became_peered":"2024-08-20T03:00:18.241175+0000","last_unstale":"2024-08-20T03:01:58.811526+0000","last_undegraded":"2024-08-20T03:01:58.811526+0000","last_fullsized":"2024-08-20T03:01:58.811526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:51:45.792846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.815324+0000","last_change":"2024-08-20T03:00:18.242700+0000","last_active":"2024-08-20T03:01:58.815324+0000","last_peered":"2024-08-20T03:01:58.815324+0000","last_clean":"2024-08-20T03:01:58.815324+0000","last_became_active":"2024-08-20T03:00:18.242407+0000","last_became_peered":"2024-08-20T03:00:18.242407+0000","last_unstale":"2024-08-20T03:01:58.815324+0000","last_undegraded":"2024-08-20T03:01:58.815324+0000","last_fullsized":"2024-08-20T03:01:58.815324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:31:28.061514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809942+0000","last_change":"2024-08-20T03:00:18.233894+0000","last_active":"2024-08-20T03:01:58.809942+0000","last_peered":"2024-08-20T03:01:58.809942+0000","last_clean":"2024-08-20T03:01:58.809942+0000","last_became_active":"2024-08-20T03:00:18.233584+0000","last_became_peered":"2024-08-20T03:00:18.233584+0000","last_unstale":"2024-08-20T03:01:58.809942+0000","last_undegraded":"2024-08-20T03:01:58.809942+0000","last_fullsized":"2024-08-20T03:01:58.809942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:36:53.156215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809607+0000","last_change":"2024-08-20T03:00:18.239256+0000","last_active":"2024-08-20T03:01:58.809607+0000","last_peered":"2024-08-20T03:01:58.809607+0000","last_clean":"2024-08-20T03:01:58.809607+0000","last_became_active":"2024-08-20T03:00:18.238931+0000","last_became_peered":"2024-08-20T03:00:18.238931+0000","last_unstale":"2024-08-20T03:01:58.809607+0000","last_undegraded":"2024-08-20T03:01:58.809607+0000","last_fullsized":"2024-08-20T03:01:58.809607+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:38:42.536488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.807629+0000","last_change":"2024-08-20T03:00:18.245187+0000","last_active":"2024-08-20T03:01:58.807629+0000","last_peered":"2024-08-20T03:01:58.807629+0000","last_clean":"2024-08-20T03:01:58.807629+0000","last_became_active":"2024-08-20T03:00:18.244894+0000","last_became_peered":"2024-08-20T03:00:18.244894+0000","last_unstale":"2024-08-20T03:01:58.807629+0000","last_undegraded":"2024-08-20T03:01:58.807629+0000","last_fullsized":"2024-08-20T03:01:58.807629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:16:58.002014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811573+0000","last_change":"2024-08-20T03:00:18.241634+0000","last_active":"2024-08-20T03:01:58.811573+0000","last_peered":"2024-08-20T03:01:58.811573+0000","last_clean":"2024-08-20T03:01:58.811573+0000","last_became_active":"2024-08-20T03:00:18.241341+0000","last_became_peered":"2024-08-20T03:00:18.241341+0000","last_unstale":"2024-08-20T03:01:58.811573+0000","last_undegraded":"2024-08-20T03:01:58.811573+0000","last_fullsized":"2024-08-20T03:01:58.811573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:37:16.185544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809305+0000","last_change":"2024-08-20T03:00:18.239116+0000","last_active":"2024-08-20T03:01:58.809305+0000","last_peered":"2024-08-20T03:01:58.809305+0000","last_clean":"2024-08-20T03:01:58.809305+0000","last_became_active":"2024-08-20T03:00:18.236125+0000","last_became_peered":"2024-08-20T03:00:18.236125+0000","last_unstale":"2024-08-20T03:01:58.809305+0000","last_undegraded":"2024-08-20T03:01:58.809305+0000","last_fullsized":"2024-08-20T03:01:58.809305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:57:00.332483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.813414+0000","last_change":"2024-08-20T03:00:18.235996+0000","last_active":"2024-08-20T03:01:58.813414+0000","last_peered":"2024-08-20T03:01:58.813414+0000","last_clean":"2024-08-20T03:01:58.813414+0000","last_became_active":"2024-08-20T03:00:18.235670+0000","last_became_peered":"2024-08-20T03:00:18.235670+0000","last_unstale":"2024-08-20T03:01:58.813414+0000","last_undegraded":"2024-08-20T03:01:58.813414+0000","last_fullsized":"2024-08-20T03:01:58.813414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:59.992914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805791+0000","last_change":"2024-08-20T03:00:18.239679+0000","last_active":"2024-08-20T03:01:58.805791+0000","last_peered":"2024-08-20T03:01:58.805791+0000","last_clean":"2024-08-20T03:01:58.805791+0000","last_became_active":"2024-08-20T03:00:18.239361+0000","last_became_peered":"2024-08-20T03:00:18.239361+0000","last_unstale":"2024-08-20T03:01:58.805791+0000","last_undegraded":"2024-08-20T03:01:58.805791+0000","last_fullsized":"2024-08-20T03:01:58.805791+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:28:09.430535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'8","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:13.532735+0000","last_change":"2024-08-20T03:00:16.221221+0000","last_active":"2024-08-20T03:02:13.532735+0000","last_peered":"2024-08-20T03:02:13.532735+0000","last_clean":"2024-08-20T03:02:13.532735+0000","last_became_active":"2024-08-20T03:00:16.220186+0000","last_became_peered":"2024-08-20T03:00:16.220186+0000","last_unstale":"2024-08-20T03:02:13.532735+0000","last_undegraded":"2024-08-20T03:02:13.532735+0000","last_fullsized":"2024-08-20T03:02:13.532735+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:51.045235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.808630+0000","last_change":"2024-08-20T03:00:18.234809+0000","last_active":"2024-08-20T03:01:58.808630+0000","last_peered":"2024-08-20T03:01:58.808630+0000","last_clean":"2024-08-20T03:01:58.808630+0000","last_became_active":"2024-08-20T03:00:18.234486+0000","last_became_peered":"2024-08-20T03:00:18.234486+0000","last_unstale":"2024-08-20T03:01:58.808630+0000","last_undegraded":"2024-08-20T03:01:58.808630+0000","last_fullsized":"2024-08-20T03:01:58.808630+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:11:52.527224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491303+0000","last_change":"2024-08-20T03:00:16.216876+0000","last_active":"2024-08-20T03:00:59.491303+0000","last_peered":"2024-08-20T03:00:59.491303+0000","last_clean":"2024-08-20T03:00:59.491303+0000","last_became_active":"2024-08-20T03:00:16.216522+0000","last_became_peered":"2024-08-20T03:00:16.216522+0000","last_unstale":"2024-08-20T03:00:59.491303+0000","last_undegraded":"2024-08-20T03:00:59.491303+0000","last_fullsized":"2024-08-20T03:00:59.491303+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:15:22.363331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.489899+0000","last_change":"2024-08-20T03:00:16.213751+0000","last_active":"2024-08-20T03:00:59.489899+0000","last_peered":"2024-08-20T03:00:59.489899+0000","last_clean":"2024-08-20T03:00:59.489899+0000","last_became_active":"2024-08-20T03:00:16.213348+0000","last_became_peered":"2024-08-20T03:00:16.213348+0000","last_unstale":"2024-08-20T03:00:59.489899+0000","last_undegraded":"2024-08-20T03:00:59.489899+0000","last_fullsized":"2024-08-20T03:00:59.489899+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:01:08.637151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809027+0000","last_change":"2024-08-20T03:00:18.237972+0000","last_active":"2024-08-20T03:01:58.809027+0000","last_peered":"2024-08-20T03:01:58.809027+0000","last_clean":"2024-08-20T03:01:58.809027+0000","last_became_active":"2024-08-20T03:00:18.236769+0000","last_became_peered":"2024-08-20T03:00:18.236769+0000","last_unstale":"2024-08-20T03:01:58.809027+0000","last_undegraded":"2024-08-20T03:01:58.809027+0000","last_fullsized":"2024-08-20T03:01:58.809027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:35:21.001045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492196+0000","last_change":"2024-08-20T03:00:16.221399+0000","last_active":"2024-08-20T03:00:59.492196+0000","last_peered":"2024-08-20T03:00:59.492196+0000","last_clean":"2024-08-20T03:00:59.492196+0000","last_became_active":"2024-08-20T03:00:16.220285+0000","last_became_peered":"2024-08-20T03:00:16.220285+0000","last_unstale":"2024-08-20T03:00:59.492196+0000","last_undegraded":"2024-08-20T03:00:59.492196+0000","last_fullsized":"2024-08-20T03:00:59.492196+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:30:17.034657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.802020+0000","last_change":"2024-08-20T03:00:18.236722+0000","last_active":"2024-08-20T03:01:58.802020+0000","last_peered":"2024-08-20T03:01:58.802020+0000","last_clean":"2024-08-20T03:01:58.802020+0000","last_became_active":"2024-08-20T03:00:18.236365+0000","last_became_peered":"2024-08-20T03:00:18.236365+0000","last_unstale":"2024-08-20T03:01:58.802020+0000","last_undegraded":"2024-08-20T03:01:58.802020+0000","last_fullsized":"2024-08-20T03:01:58.802020+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:13:17.344161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":241,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:15.535498+0000","last_change":"2024-08-20T03:00:16.222331+0000","last_active":"2024-08-20T03:02:15.535498+0000","last_peered":"2024-08-20T03:02:15.535498+0000","last_clean":"2024-08-20T03:02:15.535498+0000","last_became_active":"2024-08-20T03:00:16.222019+0000","last_became_peered":"2024-08-20T03:00:16.222019+0000","last_unstale":"2024-08-20T03:02:15.535498+0000","last_undegraded":"2024-08-20T03:02:15.535498+0000","last_fullsized":"2024-08-20T03:02:15.535498+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:46:55.897145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":212,"num_read_kb":212,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812391+0000","last_change":"2024-08-20T03:00:18.239020+0000","last_active":"2024-08-20T03:01:58.812391+0000","last_peered":"2024-08-20T03:01:58.812391+0000","last_clean":"2024-08-20T03:01:58.812391+0000","last_became_active":"2024-08-20T03:00:18.238620+0000","last_became_peered":"2024-08-20T03:00:18.238620+0000","last_unstale":"2024-08-20T03:01:58.812391+0000","last_undegraded":"2024-08-20T03:01:58.812391+0000","last_fullsized":"2024-08-20T03:01:58.812391+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:17:12.632357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491087+0000","last_change":"2024-08-20T03:00:16.214907+0000","last_active":"2024-08-20T03:00:59.491087+0000","last_peered":"2024-08-20T03:00:59.491087+0000","last_clean":"2024-08-20T03:00:59.491087+0000","last_became_active":"2024-08-20T03:00:16.214573+0000","last_became_peered":"2024-08-20T03:00:16.214573+0000","last_unstale":"2024-08-20T03:00:59.491087+0000","last_undegraded":"2024-08-20T03:00:59.491087+0000","last_fullsized":"2024-08-20T03:00:59.491087+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:21:34.084149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.807582+0000","last_change":"2024-08-20T03:00:18.245524+0000","last_active":"2024-08-20T03:01:58.807582+0000","last_peered":"2024-08-20T03:01:58.807582+0000","last_clean":"2024-08-20T03:01:58.807582+0000","last_became_active":"2024-08-20T03:00:18.244966+0000","last_became_peered":"2024-08-20T03:00:18.244966+0000","last_unstale":"2024-08-20T03:01:58.807582+0000","last_undegraded":"2024-08-20T03:01:58.807582+0000","last_fullsized":"2024-08-20T03:01:58.807582+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:03:21.584182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491504+0000","last_change":"2024-08-20T03:00:16.470353+0000","last_active":"2024-08-20T03:00:59.491504+0000","last_peered":"2024-08-20T03:00:59.491504+0000","last_clean":"2024-08-20T03:00:59.491504+0000","last_became_active":"2024-08-20T03:00:16.469976+0000","last_became_peered":"2024-08-20T03:00:16.469976+0000","last_unstale":"2024-08-20T03:00:59.491504+0000","last_undegraded":"2024-08-20T03:00:59.491504+0000","last_fullsized":"2024-08-20T03:00:59.491504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:25:03.595784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.814962+0000","last_change":"2024-08-20T03:00:18.245244+0000","last_active":"2024-08-20T03:01:58.814962+0000","last_peered":"2024-08-20T03:01:58.814962+0000","last_clean":"2024-08-20T03:01:58.814962+0000","last_became_active":"2024-08-20T03:00:18.244934+0000","last_became_peered":"2024-08-20T03:00:18.244934+0000","last_unstale":"2024-08-20T03:01:58.814962+0000","last_undegraded":"2024-08-20T03:01:58.814962+0000","last_fullsized":"2024-08-20T03:01:58.814962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:33:35.035039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'37","reported_seq":305,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:15.626540+0000","last_change":"2024-08-20T03:00:59.539537+0000","last_active":"2024-08-20T03:02:15.626540+0000","last_peered":"2024-08-20T03:02:15.626540+0000","last_clean":"2024-08-20T03:02:15.626540+0000","last_became_active":"2024-08-20T03:00:59.518110+0000","last_became_peered":"2024-08-20T03:00:59.518110+0000","last_unstale":"2024-08-20T03:02:15.626540+0000","last_undegraded":"2024-08-20T03:02:15.626540+0000","last_fullsized":"2024-08-20T03:02:15.626540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:10:58.722741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68603,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":218,"num_read_kb":176,"num_write":39,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":59762,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.802994+0000","last_change":"2024-08-20T03:00:18.242931+0000","last_active":"2024-08-20T03:01:58.802994+0000","last_peered":"2024-08-20T03:01:58.802994+0000","last_clean":"2024-08-20T03:01:58.802994+0000","last_became_active":"2024-08-20T03:00:18.242619+0000","last_became_peered":"2024-08-20T03:00:18.242619+0000","last_unstale":"2024-08-20T03:01:58.802994+0000","last_undegraded":"2024-08-20T03:01:58.802994+0000","last_fullsized":"2024-08-20T03:01:58.802994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:43:02.296990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492488+0000","last_change":"2024-08-20T03:00:16.214688+0000","last_active":"2024-08-20T03:00:59.492488+0000","last_peered":"2024-08-20T03:00:59.492488+0000","last_clean":"2024-08-20T03:00:59.492488+0000","last_became_active":"2024-08-20T03:00:16.214343+0000","last_became_peered":"2024-08-20T03:00:16.214343+0000","last_unstale":"2024-08-20T03:00:59.492488+0000","last_undegraded":"2024-08-20T03:00:59.492488+0000","last_fullsized":"2024-08-20T03:00:59.492488+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:10:56.587164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.806082+0000","last_change":"2024-08-20T03:00:18.244158+0000","last_active":"2024-08-20T03:01:58.806082+0000","last_peered":"2024-08-20T03:01:58.806082+0000","last_clean":"2024-08-20T03:01:58.806082+0000","last_became_active":"2024-08-20T03:00:18.243864+0000","last_became_peered":"2024-08-20T03:00:18.243864+0000","last_unstale":"2024-08-20T03:01:58.806082+0000","last_undegraded":"2024-08-20T03:01:58.806082+0000","last_fullsized":"2024-08-20T03:01:58.806082+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:10:31.373759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491090+0000","last_change":"2024-08-20T03:00:16.220846+0000","last_active":"2024-08-20T03:00:59.491090+0000","last_peered":"2024-08-20T03:00:59.491090+0000","last_clean":"2024-08-20T03:00:59.491090+0000","last_became_active":"2024-08-20T03:00:16.220519+0000","last_became_peered":"2024-08-20T03:00:16.220519+0000","last_unstale":"2024-08-20T03:00:59.491090+0000","last_undegraded":"2024-08-20T03:00:59.491090+0000","last_fullsized":"2024-08-20T03:00:59.491090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:43:41.721522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.808946+0000","last_change":"2024-08-20T03:00:18.246555+0000","last_active":"2024-08-20T03:01:58.808946+0000","last_peered":"2024-08-20T03:01:58.808946+0000","last_clean":"2024-08-20T03:01:58.808946+0000","last_became_active":"2024-08-20T03:00:18.246182+0000","last_became_peered":"2024-08-20T03:00:18.246182+0000","last_unstale":"2024-08-20T03:01:58.808946+0000","last_undegraded":"2024-08-20T03:01:58.808946+0000","last_fullsized":"2024-08-20T03:01:58.808946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:55.827649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'6","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:13.527275+0000","last_change":"2024-08-20T03:00:16.216521+0000","last_active":"2024-08-20T03:02:13.527275+0000","last_peered":"2024-08-20T03:02:13.527275+0000","last_clean":"2024-08-20T03:02:13.527275+0000","last_became_active":"2024-08-20T03:00:16.216187+0000","last_became_peered":"2024-08-20T03:00:16.216187+0000","last_unstale":"2024-08-20T03:02:13.527275+0000","last_undegraded":"2024-08-20T03:02:13.527275+0000","last_fullsized":"2024-08-20T03:02:13.527275+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:41:13.639218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.786505+0000","last_change":"2024-08-20T03:00:59.544335+0000","last_active":"2024-08-20T03:01:58.786505+0000","last_peered":"2024-08-20T03:01:58.786505+0000","last_clean":"2024-08-20T03:01:58.786505+0000","last_became_active":"2024-08-20T03:00:59.544063+0000","last_became_peered":"2024-08-20T03:00:59.544063+0000","last_unstale":"2024-08-20T03:01:58.786505+0000","last_undegraded":"2024-08-20T03:01:58.786505+0000","last_fullsized":"2024-08-20T03:01:58.786505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:48:36.945035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490874+0000","last_change":"2024-08-20T03:00:16.470129+0000","last_active":"2024-08-20T03:00:59.490874+0000","last_peered":"2024-08-20T03:00:59.490874+0000","last_clean":"2024-08-20T03:00:59.490874+0000","last_became_active":"2024-08-20T03:00:16.469697+0000","last_became_peered":"2024-08-20T03:00:16.469697+0000","last_unstale":"2024-08-20T03:00:59.490874+0000","last_undegraded":"2024-08-20T03:00:59.490874+0000","last_fullsized":"2024-08-20T03:00:59.490874+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:42:22.623141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.804109+0000","last_change":"2024-08-20T03:00:18.245090+0000","last_active":"2024-08-20T03:01:58.804109+0000","last_peered":"2024-08-20T03:01:58.804109+0000","last_clean":"2024-08-20T03:01:58.804109+0000","last_became_active":"2024-08-20T03:00:18.244524+0000","last_became_peered":"2024-08-20T03:00:18.244524+0000","last_unstale":"2024-08-20T03:01:58.804109+0000","last_undegraded":"2024-08-20T03:01:58.804109+0000","last_fullsized":"2024-08-20T03:01:58.804109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:14.561183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492233+0000","last_change":"2024-08-20T03:00:16.220042+0000","last_active":"2024-08-20T03:00:59.492233+0000","last_peered":"2024-08-20T03:00:59.492233+0000","last_clean":"2024-08-20T03:00:59.492233+0000","last_became_active":"2024-08-20T03:00:16.219567+0000","last_became_peered":"2024-08-20T03:00:16.219567+0000","last_unstale":"2024-08-20T03:00:59.492233+0000","last_undegraded":"2024-08-20T03:00:59.492233+0000","last_fullsized":"2024-08-20T03:00:59.492233+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:04.834039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812386+0000","last_change":"2024-08-20T03:00:18.237497+0000","last_active":"2024-08-20T03:01:58.812386+0000","last_peered":"2024-08-20T03:01:58.812386+0000","last_clean":"2024-08-20T03:01:58.812386+0000","last_became_active":"2024-08-20T03:00:18.237170+0000","last_became_peered":"2024-08-20T03:00:18.237170+0000","last_unstale":"2024-08-20T03:01:58.812386+0000","last_undegraded":"2024-08-20T03:01:58.812386+0000","last_fullsized":"2024-08-20T03:01:58.812386+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:52.387152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490226+0000","last_change":"2024-08-20T03:00:16.214861+0000","last_active":"2024-08-20T03:00:59.490226+0000","last_peered":"2024-08-20T03:00:59.490226+0000","last_clean":"2024-08-20T03:00:59.490226+0000","last_became_active":"2024-08-20T03:00:16.214487+0000","last_became_peered":"2024-08-20T03:00:16.214487+0000","last_unstale":"2024-08-20T03:00:59.490226+0000","last_undegraded":"2024-08-20T03:00:59.490226+0000","last_fullsized":"2024-08-20T03:00:59.490226+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:50:09.933304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811230+0000","last_change":"2024-08-20T03:00:18.244685+0000","last_active":"2024-08-20T03:01:58.811230+0000","last_peered":"2024-08-20T03:01:58.811230+0000","last_clean":"2024-08-20T03:01:58.811230+0000","last_became_active":"2024-08-20T03:00:18.244372+0000","last_became_peered":"2024-08-20T03:00:18.244372+0000","last_unstale":"2024-08-20T03:01:58.811230+0000","last_undegraded":"2024-08-20T03:01:58.811230+0000","last_fullsized":"2024-08-20T03:01:58.811230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:21.821712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491625+0000","last_change":"2024-08-20T03:00:16.222287+0000","last_active":"2024-08-20T03:00:59.491625+0000","last_peered":"2024-08-20T03:00:59.491625+0000","last_clean":"2024-08-20T03:00:59.491625+0000","last_became_active":"2024-08-20T03:00:16.221989+0000","last_became_peered":"2024-08-20T03:00:16.221989+0000","last_unstale":"2024-08-20T03:00:59.491625+0000","last_undegraded":"2024-08-20T03:00:59.491625+0000","last_fullsized":"2024-08-20T03:00:59.491625+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:32:07.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.807215+0000","last_change":"2024-08-20T03:00:18.250488+0000","last_active":"2024-08-20T03:01:58.807215+0000","last_peered":"2024-08-20T03:01:58.807215+0000","last_clean":"2024-08-20T03:01:58.807215+0000","last_became_active":"2024-08-20T03:00:18.250094+0000","last_became_peered":"2024-08-20T03:00:18.250094+0000","last_unstale":"2024-08-20T03:01:58.807215+0000","last_undegraded":"2024-08-20T03:01:58.807215+0000","last_fullsized":"2024-08-20T03:01:58.807215+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:00:55.755920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'7","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:14.534593+0000","last_change":"2024-08-20T03:00:16.470223+0000","last_active":"2024-08-20T03:02:14.534593+0000","last_peered":"2024-08-20T03:02:14.534593+0000","last_clean":"2024-08-20T03:02:14.534593+0000","last_became_active":"2024-08-20T03:00:16.469871+0000","last_became_peered":"2024-08-20T03:00:16.469871+0000","last_unstale":"2024-08-20T03:02:14.534593+0000","last_undegraded":"2024-08-20T03:02:14.534593+0000","last_fullsized":"2024-08-20T03:02:14.534593+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:05:11.779164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.810540+0000","last_change":"2024-08-20T03:00:18.242755+0000","last_active":"2024-08-20T03:01:58.810540+0000","last_peered":"2024-08-20T03:01:58.810540+0000","last_clean":"2024-08-20T03:01:58.810540+0000","last_became_active":"2024-08-20T03:00:18.242447+0000","last_became_peered":"2024-08-20T03:00:18.242447+0000","last_unstale":"2024-08-20T03:01:58.810540+0000","last_undegraded":"2024-08-20T03:01:58.810540+0000","last_fullsized":"2024-08-20T03:01:58.810540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:55:40.107933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":97,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.490628+0000","last_change":"2024-08-20T02:59:02.774484+0000","last_active":"2024-08-20T03:00:59.490628+0000","last_peered":"2024-08-20T03:00:59.490628+0000","last_clean":"2024-08-20T03:00:59.490628+0000","last_became_active":"2024-08-20T02:59:02.769208+0000","last_became_peered":"2024-08-20T02:59:02.769208+0000","last_unstale":"2024-08-20T03:00:59.490628+0000","last_undegraded":"2024-08-20T03:00:59.490628+0000","last_fullsized":"2024-08-20T03:00:59.490628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T02:55:46.827249+0000","last_clean_scrub_stamp":"2024-08-20T02:55:46.827249+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:36:27.931786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.516703+0000","last_change":"2024-08-20T03:00:16.213869+0000","last_active":"2024-08-20T03:00:59.516703+0000","last_peered":"2024-08-20T03:00:59.516703+0000","last_clean":"2024-08-20T03:00:59.516703+0000","last_became_active":"2024-08-20T03:00:16.213478+0000","last_became_peered":"2024-08-20T03:00:16.213478+0000","last_unstale":"2024-08-20T03:00:59.516703+0000","last_undegraded":"2024-08-20T03:00:59.516703+0000","last_fullsized":"2024-08-20T03:00:59.516703+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:27.830497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.804627+0000","last_change":"2024-08-20T03:00:18.242283+0000","last_active":"2024-08-20T03:01:58.804627+0000","last_peered":"2024-08-20T03:01:58.804627+0000","last_clean":"2024-08-20T03:01:58.804627+0000","last_became_active":"2024-08-20T03:00:18.241943+0000","last_became_peered":"2024-08-20T03:00:18.241943+0000","last_unstale":"2024-08-20T03:01:58.804627+0000","last_undegraded":"2024-08-20T03:01:58.804627+0000","last_fullsized":"2024-08-20T03:01:58.804627+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:22:25.776964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.498057+0000","last_change":"2024-08-20T03:00:59.497552+0000","last_active":"2024-08-20T03:00:59.498057+0000","last_peered":"2024-08-20T03:00:59.498057+0000","last_clean":"2024-08-20T03:00:59.498057+0000","last_became_active":"2024-08-20T03:00:59.493978+0000","last_became_peered":"2024-08-20T03:00:59.493978+0000","last_unstale":"2024-08-20T03:00:59.498057+0000","last_undegraded":"2024-08-20T03:00:59.498057+0000","last_fullsized":"2024-08-20T03:00:59.498057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:38:12.908159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812189+0000","last_change":"2024-08-20T03:00:59.544696+0000","last_active":"2024-08-20T03:01:58.812189+0000","last_peered":"2024-08-20T03:01:58.812189+0000","last_clean":"2024-08-20T03:01:58.812189+0000","last_became_active":"2024-08-20T03:00:59.544303+0000","last_became_peered":"2024-08-20T03:00:59.544303+0000","last_unstale":"2024-08-20T03:01:58.812189+0000","last_undegraded":"2024-08-20T03:01:58.812189+0000","last_fullsized":"2024-08-20T03:01:58.812189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:01:04.814695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":131,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:10.522455+0000","last_change":"2024-08-20T03:00:16.468073+0000","last_active":"2024-08-20T03:02:10.522455+0000","last_peered":"2024-08-20T03:02:10.522455+0000","last_clean":"2024-08-20T03:02:10.522455+0000","last_became_active":"2024-08-20T03:00:16.467560+0000","last_became_peered":"2024-08-20T03:00:16.467560+0000","last_unstale":"2024-08-20T03:02:10.522455+0000","last_undegraded":"2024-08-20T03:02:10.522455+0000","last_fullsized":"2024-08-20T03:02:10.522455+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:42:24.499777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812418+0000","last_change":"2024-08-20T03:00:18.232537+0000","last_active":"2024-08-20T03:01:58.812418+0000","last_peered":"2024-08-20T03:01:58.812418+0000","last_clean":"2024-08-20T03:01:58.812418+0000","last_became_active":"2024-08-20T03:00:18.232273+0000","last_became_peered":"2024-08-20T03:00:18.232273+0000","last_unstale":"2024-08-20T03:01:58.812418+0000","last_undegraded":"2024-08-20T03:01:58.812418+0000","last_fullsized":"2024-08-20T03:01:58.812418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:36:04.981869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'7","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:11.524410+0000","last_change":"2024-08-20T03:00:16.214807+0000","last_active":"2024-08-20T03:02:11.524410+0000","last_peered":"2024-08-20T03:02:11.524410+0000","last_clean":"2024-08-20T03:02:11.524410+0000","last_became_active":"2024-08-20T03:00:16.214439+0000","last_became_peered":"2024-08-20T03:00:16.214439+0000","last_unstale":"2024-08-20T03:02:11.524410+0000","last_undegraded":"2024-08-20T03:02:11.524410+0000","last_fullsized":"2024-08-20T03:02:11.524410+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:07.519151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.795094+0000","last_change":"2024-08-20T03:00:18.247220+0000","last_active":"2024-08-20T03:01:58.795094+0000","last_peered":"2024-08-20T03:01:58.795094+0000","last_clean":"2024-08-20T03:01:58.795094+0000","last_became_active":"2024-08-20T03:00:18.246896+0000","last_became_peered":"2024-08-20T03:00:18.246896+0000","last_unstale":"2024-08-20T03:01:58.795094+0000","last_undegraded":"2024-08-20T03:01:58.795094+0000","last_fullsized":"2024-08-20T03:01:58.795094+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:43:16.690590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543240+0000","last_change":"2024-08-20T03:00:16.468280+0000","last_active":"2024-08-20T03:00:59.543240+0000","last_peered":"2024-08-20T03:00:59.543240+0000","last_clean":"2024-08-20T03:00:59.543240+0000","last_became_active":"2024-08-20T03:00:16.467930+0000","last_became_peered":"2024-08-20T03:00:16.467930+0000","last_unstale":"2024-08-20T03:00:59.543240+0000","last_undegraded":"2024-08-20T03:00:59.543240+0000","last_fullsized":"2024-08-20T03:00:59.543240+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:30.423563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.797891+0000","last_change":"2024-08-20T03:00:18.240322+0000","last_active":"2024-08-20T03:01:58.797891+0000","last_peered":"2024-08-20T03:01:58.797891+0000","last_clean":"2024-08-20T03:01:58.797891+0000","last_became_active":"2024-08-20T03:00:18.239973+0000","last_became_peered":"2024-08-20T03:00:18.239973+0000","last_unstale":"2024-08-20T03:01:58.797891+0000","last_undegraded":"2024-08-20T03:01:58.797891+0000","last_fullsized":"2024-08-20T03:01:58.797891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:41:28.259762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.493654+0000","last_change":"2024-08-20T03:00:16.468307+0000","last_active":"2024-08-20T03:00:59.493654+0000","last_peered":"2024-08-20T03:00:59.493654+0000","last_clean":"2024-08-20T03:00:59.493654+0000","last_became_active":"2024-08-20T03:00:16.467993+0000","last_became_peered":"2024-08-20T03:00:16.467993+0000","last_unstale":"2024-08-20T03:00:59.493654+0000","last_undegraded":"2024-08-20T03:00:59.493654+0000","last_fullsized":"2024-08-20T03:00:59.493654+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:34.464135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.812353+0000","last_change":"2024-08-20T03:00:18.234009+0000","last_active":"2024-08-20T03:01:58.812353+0000","last_peered":"2024-08-20T03:01:58.812353+0000","last_clean":"2024-08-20T03:01:58.812353+0000","last_became_active":"2024-08-20T03:00:18.233311+0000","last_became_peered":"2024-08-20T03:00:18.233311+0000","last_unstale":"2024-08-20T03:01:58.812353+0000","last_undegraded":"2024-08-20T03:01:58.812353+0000","last_fullsized":"2024-08-20T03:01:58.812353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:53:10.759339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'17","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.543424+0000","last_change":"2024-08-20T03:00:16.214932+0000","last_active":"2024-08-20T03:00:59.543424+0000","last_peered":"2024-08-20T03:00:59.543424+0000","last_clean":"2024-08-20T03:00:59.543424+0000","last_became_active":"2024-08-20T03:00:16.214584+0000","last_became_peered":"2024-08-20T03:00:16.214584+0000","last_unstale":"2024-08-20T03:00:59.543424+0000","last_undegraded":"2024-08-20T03:00:59.543424+0000","last_fullsized":"2024-08-20T03:00:59.543424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:47.270685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809771+0000","last_change":"2024-08-20T03:00:18.247892+0000","last_active":"2024-08-20T03:01:58.809771+0000","last_peered":"2024-08-20T03:01:58.809771+0000","last_clean":"2024-08-20T03:01:58.809771+0000","last_became_active":"2024-08-20T03:00:18.245903+0000","last_became_peered":"2024-08-20T03:00:18.245903+0000","last_unstale":"2024-08-20T03:01:58.809771+0000","last_undegraded":"2024-08-20T03:01:58.809771+0000","last_fullsized":"2024-08-20T03:01:58.809771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:08:50.909638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.516720+0000","last_change":"2024-08-20T03:00:16.372503+0000","last_active":"2024-08-20T03:00:59.516720+0000","last_peered":"2024-08-20T03:00:59.516720+0000","last_clean":"2024-08-20T03:00:59.516720+0000","last_became_active":"2024-08-20T03:00:16.372218+0000","last_became_peered":"2024-08-20T03:00:16.372218+0000","last_unstale":"2024-08-20T03:00:59.516720+0000","last_undegraded":"2024-08-20T03:00:59.516720+0000","last_fullsized":"2024-08-20T03:00:59.516720+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:13.341900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.805821+0000","last_change":"2024-08-20T03:00:18.244738+0000","last_active":"2024-08-20T03:01:58.805821+0000","last_peered":"2024-08-20T03:01:58.805821+0000","last_clean":"2024-08-20T03:01:58.805821+0000","last_became_active":"2024-08-20T03:00:18.244398+0000","last_became_peered":"2024-08-20T03:00:18.244398+0000","last_unstale":"2024-08-20T03:01:58.805821+0000","last_undegraded":"2024-08-20T03:01:58.805821+0000","last_fullsized":"2024-08-20T03:01:58.805821+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:45:14.540125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.494339+0000","last_change":"2024-08-20T03:00:16.468251+0000","last_active":"2024-08-20T03:00:59.494339+0000","last_peered":"2024-08-20T03:00:59.494339+0000","last_clean":"2024-08-20T03:00:59.494339+0000","last_became_active":"2024-08-20T03:00:16.467912+0000","last_became_peered":"2024-08-20T03:00:16.467912+0000","last_unstale":"2024-08-20T03:00:59.494339+0000","last_undegraded":"2024-08-20T03:00:59.494339+0000","last_fullsized":"2024-08-20T03:00:59.494339+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:43.799747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.770324+0000","last_change":"2024-08-20T03:00:18.234624+0000","last_active":"2024-08-20T03:01:58.770324+0000","last_peered":"2024-08-20T03:01:58.770324+0000","last_clean":"2024-08-20T03:01:58.770324+0000","last_became_active":"2024-08-20T03:00:18.234331+0000","last_became_peered":"2024-08-20T03:00:18.234331+0000","last_unstale":"2024-08-20T03:01:58.770324+0000","last_undegraded":"2024-08-20T03:01:58.770324+0000","last_fullsized":"2024-08-20T03:01:58.770324+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:05:09.665645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487438+0000","last_change":"2024-08-20T03:00:16.216658+0000","last_active":"2024-08-20T03:00:59.487438+0000","last_peered":"2024-08-20T03:00:59.487438+0000","last_clean":"2024-08-20T03:00:59.487438+0000","last_became_active":"2024-08-20T03:00:16.216352+0000","last_became_peered":"2024-08-20T03:00:16.216352+0000","last_unstale":"2024-08-20T03:00:59.487438+0000","last_undegraded":"2024-08-20T03:00:59.487438+0000","last_fullsized":"2024-08-20T03:00:59.487438+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:31:20.763280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811335+0000","last_change":"2024-08-20T03:00:18.240984+0000","last_active":"2024-08-20T03:01:58.811335+0000","last_peered":"2024-08-20T03:01:58.811335+0000","last_clean":"2024-08-20T03:01:58.811335+0000","last_became_active":"2024-08-20T03:00:18.240640+0000","last_became_peered":"2024-08-20T03:00:18.240640+0000","last_unstale":"2024-08-20T03:01:58.811335+0000","last_undegraded":"2024-08-20T03:01:58.811335+0000","last_fullsized":"2024-08-20T03:01:58.811335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:05.589186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"77'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492391+0000","last_change":"2024-08-20T03:00:16.214818+0000","last_active":"2024-08-20T03:00:59.492391+0000","last_peered":"2024-08-20T03:00:59.492391+0000","last_clean":"2024-08-20T03:00:59.492391+0000","last_became_active":"2024-08-20T03:00:16.214493+0000","last_became_peered":"2024-08-20T03:00:16.214493+0000","last_unstale":"2024-08-20T03:00:59.492391+0000","last_undegraded":"2024-08-20T03:00:59.492391+0000","last_fullsized":"2024-08-20T03:00:59.492391+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:56:12.895142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.806590+0000","last_change":"2024-08-20T03:00:18.240350+0000","last_active":"2024-08-20T03:01:58.806590+0000","last_peered":"2024-08-20T03:01:58.806590+0000","last_clean":"2024-08-20T03:01:58.806590+0000","last_became_active":"2024-08-20T03:00:18.239464+0000","last_became_peered":"2024-08-20T03:00:18.239464+0000","last_unstale":"2024-08-20T03:01:58.806590+0000","last_undegraded":"2024-08-20T03:01:58.806590+0000","last_fullsized":"2024-08-20T03:01:58.806590+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:48.715130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.491431+0000","last_change":"2024-08-20T03:00:16.216144+0000","last_active":"2024-08-20T03:00:59.491431+0000","last_peered":"2024-08-20T03:00:59.491431+0000","last_clean":"2024-08-20T03:00:59.491431+0000","last_became_active":"2024-08-20T03:00:16.215792+0000","last_became_peered":"2024-08-20T03:00:16.215792+0000","last_unstale":"2024-08-20T03:00:59.491431+0000","last_undegraded":"2024-08-20T03:00:59.491431+0000","last_fullsized":"2024-08-20T03:00:59.491431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:25:01.500878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.809127+0000","last_change":"2024-08-20T03:00:18.245286+0000","last_active":"2024-08-20T03:01:58.809127+0000","last_peered":"2024-08-20T03:01:58.809127+0000","last_clean":"2024-08-20T03:01:58.809127+0000","last_became_active":"2024-08-20T03:00:18.244760+0000","last_became_peered":"2024-08-20T03:00:18.244760+0000","last_unstale":"2024-08-20T03:01:58.809127+0000","last_undegraded":"2024-08-20T03:01:58.809127+0000","last_fullsized":"2024-08-20T03:01:58.809127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:09.415794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.814003+0000","last_change":"2024-08-20T03:00:59.495948+0000","last_active":"2024-08-20T03:01:58.814003+0000","last_peered":"2024-08-20T03:01:58.814003+0000","last_clean":"2024-08-20T03:01:58.814003+0000","last_became_active":"2024-08-20T03:00:59.495543+0000","last_became_peered":"2024-08-20T03:00:59.495543+0000","last_unstale":"2024-08-20T03:01:58.814003+0000","last_undegraded":"2024-08-20T03:01:58.814003+0000","last_fullsized":"2024-08-20T03:01:58.814003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:38:30.671235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.492392+0000","last_change":"2024-08-20T03:00:16.220019+0000","last_active":"2024-08-20T03:00:59.492392+0000","last_peered":"2024-08-20T03:00:59.492392+0000","last_clean":"2024-08-20T03:00:59.492392+0000","last_became_active":"2024-08-20T03:00:16.219624+0000","last_became_peered":"2024-08-20T03:00:16.219624+0000","last_unstale":"2024-08-20T03:00:59.492392+0000","last_undegraded":"2024-08-20T03:00:59.492392+0000","last_fullsized":"2024-08-20T03:00:59.492392+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:33:24.926695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.813429+0000","last_change":"2024-08-20T03:00:18.242156+0000","last_active":"2024-08-20T03:01:58.813429+0000","last_peered":"2024-08-20T03:01:58.813429+0000","last_clean":"2024-08-20T03:01:58.813429+0000","last_became_active":"2024-08-20T03:00:18.241811+0000","last_became_peered":"2024-08-20T03:00:18.241811+0000","last_unstale":"2024-08-20T03:01:58.813429+0000","last_undegraded":"2024-08-20T03:01:58.813429+0000","last_fullsized":"2024-08-20T03:01:58.813429+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:58.230658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.534283+0000","last_change":"2024-08-20T03:00:59.533906+0000","last_active":"2024-08-20T03:00:59.534283+0000","last_peered":"2024-08-20T03:00:59.534283+0000","last_clean":"2024-08-20T03:00:59.534283+0000","last_became_active":"2024-08-20T03:00:59.517824+0000","last_became_peered":"2024-08-20T03:00:59.517824+0000","last_unstale":"2024-08-20T03:00:59.534283+0000","last_undegraded":"2024-08-20T03:00:59.534283+0000","last_fullsized":"2024-08-20T03:00:59.534283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:25:28.219430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.813189+0000","last_change":"2024-08-20T03:00:18.245256+0000","last_active":"2024-08-20T03:01:58.813189+0000","last_peered":"2024-08-20T03:01:58.813189+0000","last_clean":"2024-08-20T03:01:58.813189+0000","last_became_active":"2024-08-20T03:00:18.244339+0000","last_became_peered":"2024-08-20T03:00:18.244339+0000","last_unstale":"2024-08-20T03:01:58.813189+0000","last_undegraded":"2024-08-20T03:01:58.813189+0000","last_fullsized":"2024-08-20T03:01:58.813189+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:00.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:00:59.487797+0000","last_change":"2024-08-20T03:00:16.470004+0000","last_active":"2024-08-20T03:00:59.487797+0000","last_peered":"2024-08-20T03:00:59.487797+0000","last_clean":"2024-08-20T03:00:59.487797+0000","last_became_active":"2024-08-20T03:00:16.469646+0000","last_became_peered":"2024-08-20T03:00:16.469646+0000","last_unstale":"2024-08-20T03:00:59.487797+0000","last_undegraded":"2024-08-20T03:00:59.487797+0000","last_fullsized":"2024-08-20T03:00:59.487797+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:35.244683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:01:58.811833+0000","last_change":"2024-08-20T03:00:18.238053+0000","last_active":"2024-08-20T03:01:58.811833+0000","last_peered":"2024-08-20T03:01:58.811833+0000","last_clean":"2024-08-20T03:01:58.811833+0000","last_became_active":"2024-08-20T03:00:18.237186+0000","last_became_peered":"2024-08-20T03:00:18.237186+0000","last_unstale":"2024-08-20T03:01:58.811833+0000","last_undegraded":"2024-08-20T03:01:58.811833+0000","last_fullsized":"2024-08-20T03:01:58.811833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:17.213131+0000","last_clean_scrub_stamp":"2024-08-20T03:00:17.213131+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:14.425688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'14","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-08-20T03:02:10.522436+0000","last_change":"2024-08-20T03:00:16.214584+0000","last_active":"2024-08-20T03:02:10.522436+0000","last_peered":"2024-08-20T03:02:10.522436+0000","last_clean":"2024-08-20T03:02:10.522436+0000","last_became_active":"2024-08-20T03:00:16.214255+0000","last_became_peered":"2024-08-20T03:00:16.214255+0000","last_unstale":"2024-08-20T03:02:10.522436+0000","last_undegraded":"2024-08-20T03:02:10.522436+0000","last_fullsized":"2024-08-20T03:02:10.522436+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T03:00:15.194259+0000","last_clean_scrub_stamp":"2024-08-20T03:00:15.194259+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:16:30.983745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77570,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1087,"num_read_kb":628,"num_write":265,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":59762,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":226580,"data_compressed":9639,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281428,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":417802,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939690,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":1077825,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":417732,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299443,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":864256,"data_stored":418110,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":1011301,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713136,"statfs":{"total":95995035648,"available":95962251264,"internally_reserved":0,"allocated":851968,"data_stored":417981,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":1010838,"data_compressed":41613,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":483784,"data_compressed":15466,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":868352,"data_stored":417274,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378706,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":420204,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542230,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":479213,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":419487,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34166,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35786,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":68758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":67174,"data_compressed":9639,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T03:02:17.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:02:17.696 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:18.095 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:18 smithi044 ceph-mon[18790]: pgmap v314: 97 pgs: 97 active+clean; 653 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-08-20T03:02:18.095 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:18 smithi044 ceph-mon[18790]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:02:18.095 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:18 smithi044 ceph-mon[18790]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:02:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.173+0000 7f14db865640 1 -- 172.21.15.44:0/3697577638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14d4101e90 msgr2=0x7f14d410ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.173+0000 7f14db865640 1 --2- 172.21.15.44:0/3697577638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14d4101e90 0x7f14d410ebe0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009900 tx=0x7f14c402a480 comp rx=0 tx=0).stop 2024-08-20T03:02:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.173+0000 7f14db865640 1 -- 172.21.15.44:0/3697577638 shutdown_connections 2024-08-20T03:02:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.173+0000 7f14db865640 1 --2- 172.21.15.44:0/3697577638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d410f120 0x7f14d4111510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.173+0000 7f14db865640 1 --2- 172.21.15.44:0/3697577638 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14d4101e90 0x7f14d410ebe0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.173+0000 7f14db865640 1 --2- 172.21.15.44:0/3697577638 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d4101570 0x7f14d4101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.173+0000 7f14db865640 1 -- 172.21.15.44:0/3697577638 >> 172.21.15.44:0/3697577638 conn(0x7f14d40fd440 msgr2=0x7f14d40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.173+0000 7f14db865640 1 -- 172.21.15.44:0/3697577638 shutdown_connections 2024-08-20T03:02:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.173+0000 7f14db865640 1 -- 172.21.15.44:0/3697577638 wait complete. 2024-08-20T03:02:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.174+0000 7f14db865640 1 Processor -- start 2024-08-20T03:02:18.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.174+0000 7f14db865640 1 -- start start 2024-08-20T03:02:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.174+0000 7f14db865640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d4101570 0x7f14d419b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.174+0000 7f14db865640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14d4101e90 0x7f14d419b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.174+0000 7f14db865640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d410f120 0x7f14d41a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.174+0000 7f14db865640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4114ca0 con 0x7f14d4101e90 2024-08-20T03:02:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.174+0000 7f14db865640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4114b20 con 0x7f14d410f120 2024-08-20T03:02:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.174+0000 7f14db865640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4114e20 con 0x7f14d4101570 2024-08-20T03:02:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.174+0000 7f14d8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14d4101e90 0x7f14d419b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.174+0000 7f14d8e00640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14d4101e90 0x7f14d419b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58648/0 (socket says 172.21.15.44:58648) 2024-08-20T03:02:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.174+0000 7f14d8e00640 1 -- 172.21.15.44:0/2508389976 learned_addr learned my addr 172.21.15.44:0/2508389976 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14d9e02640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d410f120 0x7f14d41a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14d9601640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d4101570 0x7f14d419b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:18.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14d8e00640 1 -- 172.21.15.44:0/2508389976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d4101570 msgr2=0x7f14d419b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14d8e00640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d4101570 0x7f14d419b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14d8e00640 1 -- 172.21.15.44:0/2508389976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d410f120 msgr2=0x7f14d41a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14d8e00640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d410f120 0x7f14d41a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14d8e00640 1 -- 172.21.15.44:0/2508389976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d41a1250 con 0x7f14d4101e90 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14d9e02640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d410f120 0x7f14d41a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14d8e00640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14d4101e90 0x7f14d419b6e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f14c4006010 tx=0x7f14c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14ca7fc640 1 -- 172.21.15.44:0/2508389976 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c403f030 con 0x7f14d4101e90 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14db865640 1 -- 172.21.15.44:0/2508389976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d41a14e0 con 0x7f14d4101e90 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14ca7fc640 1 -- 172.21.15.44:0/2508389976 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c4003800 con 0x7f14d4101e90 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14ca7fc640 1 -- 172.21.15.44:0/2508389976 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4035070 con 0x7f14d4101e90 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.175+0000 7f14db865640 1 -- 172.21.15.44:0/2508389976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d41a7c00 con 0x7f14d4101e90 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.176+0000 7f14db865640 1 -- 172.21.15.44:0/2508389976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d4101950 con 0x7f14d4101e90 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.178+0000 7f14ca7fc640 1 -- 172.21.15.44:0/2508389976 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f14c4003cb0 con 0x7f14d4101e90 2024-08-20T03:02:18.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.179+0000 7f14ca7fc640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f14c00b3cb0 0x7f14c00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.179+0000 7f14d9601640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f14c00b3cb0 0x7f14c00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.179+0000 7f14ca7fc640 1 -- 172.21.15.44:0/2508389976 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f14c4104ae0 con 0x7f14d4101e90 2024-08-20T03:02:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.180+0000 7f14d9601640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f14c00b3cb0 0x7f14c00b6170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f14d00058c0 tx=0x7f14d0005c60 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:18.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.182+0000 7f14ca7fc640 1 -- 172.21.15.44:0/2508389976 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f14c40cbf20 con 0x7f14d4101e90 2024-08-20T03:02:18.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:18 smithi165 ceph-mon[22357]: pgmap v314: 97 pgs: 97 active+clean; 653 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-08-20T03:02:18.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:18 smithi165 ceph-mon[22357]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:02:18.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:18 smithi165 ceph-mon[22357]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:02:18.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.409+0000 7f14db865640 1 -- 172.21.15.44:0/2508389976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14d410a2b0 con 0x7f14d4101e90 2024-08-20T03:02:18.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.409+0000 7f14ca7fc640 1 -- 172.21.15.44:0/2508389976 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7f14c40d41f0 con 0x7f14d4101e90 2024-08-20T03:02:18.413 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:02:18.413 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":28,"btime":"2024-08-20T03:02:12:953534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:02:12.906757+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24569,24583,24596,24589,24604]},"id":1}]} 2024-08-20T03:02:18.413 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 28 2024-08-20T03:02:18.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:18 smithi099 ceph-mon[21984]: pgmap v314: 97 pgs: 97 active+clean; 653 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-08-20T03:02:18.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:18 smithi099 ceph-mon[21984]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:02:18.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:18 smithi099 ceph-mon[21984]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:02:18.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.411+0000 7f14db865640 1 -- 172.21.15.44:0/2508389976 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f14c00b3cb0 msgr2=0x7f14c00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:18.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.411+0000 7f14db865640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f14c00b3cb0 0x7f14c00b6170 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f14d00058c0 tx=0x7f14d0005c60 comp rx=0 tx=0).stop 2024-08-20T03:02:18.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.411+0000 7f14db865640 1 -- 172.21.15.44:0/2508389976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14d4101e90 msgr2=0x7f14d419b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:18.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.411+0000 7f14db865640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14d4101e90 0x7f14d419b6e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f14c4006010 tx=0x7f14c40042f0 comp rx=0 tx=0).stop 2024-08-20T03:02:18.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.412+0000 7f14db865640 1 -- 172.21.15.44:0/2508389976 shutdown_connections 2024-08-20T03:02:18.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.412+0000 7f14db865640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f14c00b3cb0 0x7f14c00b6170 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:18.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.412+0000 7f14db865640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d410f120 0x7f14d41a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:18.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.412+0000 7f14db865640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14d4101e90 0x7f14d419b6e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:18.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.412+0000 7f14db865640 1 --2- 172.21.15.44:0/2508389976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d4101570 0x7f14d419b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:18.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.412+0000 7f14db865640 1 -- 172.21.15.44:0/2508389976 >> 172.21.15.44:0/2508389976 conn(0x7f14d40fd440 msgr2=0x7f14d4110470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:18.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.412+0000 7f14db865640 1 -- 172.21.15.44:0/2508389976 shutdown_connections 2024-08-20T03:02:18.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:18.412+0000 7f14db865640 1 -- 172.21.15.44:0/2508389976 wait complete. 2024-08-20T03:02:19.076 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:19 smithi044 ceph-mon[18790]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:02:19.076 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:19 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2508389976' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:02:19.107 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T03:02:19.362 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:19.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:19 smithi165 ceph-mon[22357]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:02:19.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:19 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2508389976' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:02:19.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:19 smithi099 ceph-mon[21984]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T03:02:19.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:19 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2508389976' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:02:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.900+0000 7f0982d03640 1 -- 172.21.15.44:0/2804299102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f097c1084c0 msgr2=0x7f097c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.900+0000 7f0982d03640 1 --2- 172.21.15.44:0/2804299102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f097c1084c0 0x7f097c108900 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0964009900 tx=0x7f096402a410 comp rx=0 tx=0).stop 2024-08-20T03:02:19.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 -- 172.21.15.44:0/2804299102 shutdown_connections 2024-08-20T03:02:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 --2- 172.21.15.44:0/2804299102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f097c108e40 0x7f097c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 --2- 172.21.15.44:0/2804299102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f097c1084c0 0x7f097c108900 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 --2- 172.21.15.44:0/2804299102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f097c102ab0 0x7f097c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 -- 172.21.15.44:0/2804299102 >> 172.21.15.44:0/2804299102 conn(0x7f097c0fe540 msgr2=0x7f097c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 -- 172.21.15.44:0/2804299102 shutdown_connections 2024-08-20T03:02:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 -- 172.21.15.44:0/2804299102 wait complete. 2024-08-20T03:02:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 Processor -- start 2024-08-20T03:02:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 -- start start 2024-08-20T03:02:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f097c102ab0 0x7f097c1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f097c1084c0 0x7f097c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f097c108e40 0x7f097c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:19.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f097c112ac0 con 0x7f097c108e40 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f097c112940 con 0x7f097c102ab0 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.901+0000 7f0982d03640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f097c112c40 con 0x7f097c1084c0 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f09812a0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f097c108e40 0x7f097c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f09812a0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f097c108e40 0x7f097c19e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58660/0 (socket says 172.21.15.44:58660) 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f09812a0640 1 -- 172.21.15.44:0/1974563622 learned_addr learned my addr 172.21.15.44:0/1974563622 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f097bfff640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f097c1084c0 0x7f097c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f0980a9f640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f097c102ab0 0x7f097c1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f09812a0640 1 -- 172.21.15.44:0/1974563622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f097c1084c0 msgr2=0x7f097c1995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f09812a0640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f097c1084c0 0x7f097c1995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f09812a0640 1 -- 172.21.15.44:0/1974563622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f097c102ab0 msgr2=0x7f097c1990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f09812a0640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f097c102ab0 0x7f097c1990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f09812a0640 1 -- 172.21.15.44:0/1974563622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c19f160 con 0x7f097c108e40 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f09812a0640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f097c108e40 0x7f097c19e990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f097400eb10 tx=0x7f097400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:19.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f0979ffb640 1 -- 172.21.15.44:0/1974563622 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974004b20 con 0x7f097c108e40 2024-08-20T03:02:19.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f0979ffb640 1 -- 172.21.15.44:0/1974563622 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0974004cc0 con 0x7f097c108e40 2024-08-20T03:02:19.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f0982d03640 1 -- 172.21.15.44:0/1974563622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097c19f450 con 0x7f097c108e40 2024-08-20T03:02:19.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f0979ffb640 1 -- 172.21.15.44:0/1974563622 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974010670 con 0x7f097c108e40 2024-08-20T03:02:19.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.902+0000 7f0982d03640 1 -- 172.21.15.44:0/1974563622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097c1a5b10 con 0x7f097c108e40 2024-08-20T03:02:19.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.903+0000 7f0982d03640 1 -- 172.21.15.44:0/1974563622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f097c102e90 con 0x7f097c108e40 2024-08-20T03:02:19.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.905+0000 7f0979ffb640 1 -- 172.21.15.44:0/1974563622 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f0974005000 con 0x7f097c108e40 2024-08-20T03:02:19.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.905+0000 7f0979ffb640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f09680b3d30 0x7f09680b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:19.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.905+0000 7f0980a9f640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f09680b3d30 0x7f09680b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:19.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.905+0000 7f0979ffb640 1 -- 172.21.15.44:0/1974563622 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f09740e4670 con 0x7f097c108e40 2024-08-20T03:02:19.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.906+0000 7f0980a9f640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f09680b3d30 0x7f09680b61f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f096c005fd0 tx=0x7f096c005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:19.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:19.908+0000 7f0979ffb640 1 -- 172.21.15.44:0/1974563622 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0974055e20 con 0x7f097c108e40 2024-08-20T03:02:20.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.093+0000 7f0982d03640 1 -- 172.21.15.44:0/1974563622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f097c10ac20 con 0x7f097c108e40 2024-08-20T03:02:20.097 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:20 smithi044 ceph-mon[18790]: pgmap v315: 97 pgs: 97 active+clean; 653 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-08-20T03:02:20.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.095+0000 7f0979ffb640 1 -- 172.21.15.44:0/1974563622 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22954 (secure 0 0 0) 0x7f09740abab0 con 0x7f097c108e40 2024-08-20T03:02:20.098 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:02:20.098 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":77,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","created":"2024-08-20T02:52:10.979264+0000","modified":"2024-08-20T03:00:59.484512+0000","last_up_change":"2024-08-20T02:59:26.404917+0000","last_in_change":"2024-08-20T02:59:03.707380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T02:55:46.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T03:00:14.354568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T03:00:16.869335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7708f8c-a799-4b8f-ab1e-78482d73ca1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6803","nonce":858898316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6805","nonce":858898316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6809","nonce":858898316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6807","nonce":858898316}]},"public_addr":"172.21.15.44:6803/858898316","cluster_addr":"172.21.15.44:6805/858898316","heartbeat_back_addr":"172.21.15.44:6809/858898316","heartbeat_front_addr":"172.21.15.44:6807/858898316","state":["exists","up"]},{"osd":1,"uuid":"12f07986-60bf-498f-a012-cd7ea607b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6801","nonce":4252911663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6803","nonce":4252911663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6807","nonce":4252911663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6805","nonce":4252911663}]},"public_addr":"172.21.15.99:6801/4252911663","cluster_addr":"172.21.15.99:6803/4252911663","heartbeat_back_addr":"172.21.15.99:6807/4252911663","heartbeat_front_addr":"172.21.15.99:6805/4252911663","state":["exists","up"]},{"osd":2,"uuid":"0f189000-ba7f-4412-b0d3-75e8265f6685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6801","nonce":1492740465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6803","nonce":1492740465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6807","nonce":1492740465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6805","nonce":1492740465}]},"public_addr":"172.21.15.165:6801/1492740465","cluster_addr":"172.21.15.165:6803/1492740465","heartbeat_back_addr":"172.21.15.165:6807/1492740465","heartbeat_front_addr":"172.21.15.165:6805/1492740465","state":["exists","up"]},{"osd":3,"uuid":"8f9b042c-d6b5-4eee-aa29-ffc41a48c20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6811","nonce":3423983702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6813","nonce":3423983702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6817","nonce":3423983702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6815","nonce":3423983702}]},"public_addr":"172.21.15.44:6811/3423983702","cluster_addr":"172.21.15.44:6813/3423983702","heartbeat_back_addr":"172.21.15.44:6817/3423983702","heartbeat_front_addr":"172.21.15.44:6815/3423983702","state":["exists","up"]},{"osd":4,"uuid":"4edebfdb-af10-49ee-b724-2a63b9219221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6809","nonce":3099617325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6811","nonce":3099617325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6815","nonce":3099617325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6813","nonce":3099617325}]},"public_addr":"172.21.15.99:6809/3099617325","cluster_addr":"172.21.15.99:6811/3099617325","heartbeat_back_addr":"172.21.15.99:6815/3099617325","heartbeat_front_addr":"172.21.15.99:6813/3099617325","state":["exists","up"]},{"osd":5,"uuid":"6d22c21f-cc43-4ef5-a75c-d4fc1a361c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6809","nonce":523245255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6811","nonce":523245255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6815","nonce":523245255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6813","nonce":523245255}]},"public_addr":"172.21.15.165:6809/523245255","cluster_addr":"172.21.15.165:6811/523245255","heartbeat_back_addr":"172.21.15.165:6815/523245255","heartbeat_front_addr":"172.21.15.165:6813/523245255","state":["exists","up"]},{"osd":6,"uuid":"03df0c2f-1b7c-4242-888c-e047322fd8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6819","nonce":1039098986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6821","nonce":1039098986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6825","nonce":1039098986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6823","nonce":1039098986}]},"public_addr":"172.21.15.44:6819/1039098986","cluster_addr":"172.21.15.44:6821/1039098986","heartbeat_back_addr":"172.21.15.44:6825/1039098986","heartbeat_front_addr":"172.21.15.44:6823/1039098986","state":["exists","up"]},{"osd":7,"uuid":"84bbc683-9b13-448e-897c-114458dae33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6817","nonce":1438613564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6819","nonce":1438613564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6823","nonce":1438613564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6821","nonce":1438613564}]},"public_addr":"172.21.15.99:6817/1438613564","cluster_addr":"172.21.15.99:6819/1438613564","heartbeat_back_addr":"172.21.15.99:6823/1438613564","heartbeat_front_addr":"172.21.15.99:6821/1438613564","state":["exists","up"]},{"osd":8,"uuid":"14d7ffd3-db05-4228-a12e-4c69ef65ef8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6817","nonce":632957688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6819","nonce":632957688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6823","nonce":632957688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6821","nonce":632957688}]},"public_addr":"172.21.15.165:6817/632957688","cluster_addr":"172.21.15.165:6819/632957688","heartbeat_back_addr":"172.21.15.165:6823/632957688","heartbeat_front_addr":"172.21.15.165:6821/632957688","state":["exists","up"]},{"osd":9,"uuid":"1ae468a5-3d72-4eb0-a1a6-cd87eae740b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6827","nonce":2426940093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6829","nonce":2426940093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6833","nonce":2426940093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6831","nonce":2426940093}]},"public_addr":"172.21.15.44:6827/2426940093","cluster_addr":"172.21.15.44:6829/2426940093","heartbeat_back_addr":"172.21.15.44:6833/2426940093","heartbeat_front_addr":"172.21.15.44:6831/2426940093","state":["exists","up"]},{"osd":10,"uuid":"fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6825","nonce":3275793038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6827","nonce":3275793038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6831","nonce":3275793038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6829","nonce":3275793038}]},"public_addr":"172.21.15.99:6825/3275793038","cluster_addr":"172.21.15.99:6827/3275793038","heartbeat_back_addr":"172.21.15.99:6831/3275793038","heartbeat_front_addr":"172.21.15.99:6829/3275793038","state":["exists","up"]},{"osd":11,"uuid":"50715410-1fc2-4818-b600-604a4b7cce9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6825","nonce":107688415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6827","nonce":107688415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6831","nonce":107688415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6829","nonce":107688415}]},"public_addr":"172.21.15.165:6825/107688415","cluster_addr":"172.21.15.165:6827/107688415","heartbeat_back_addr":"172.21.15.165:6831/107688415","heartbeat_front_addr":"172.21.15.165:6829/107688415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:54:56.697075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:20.486846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:43.353376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:08.333879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:32.186492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:55.891366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:19.681421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:45.475926+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:10.109795+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:34.444169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:59.147646+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:59:25.403071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2597827045":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/3582612619":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/1916809746":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2919033419":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:6800/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2331034359":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2572185146":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/4176802819":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/3654439062":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6801/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/2376733275":"2024-08-21T02:53:58.312659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T03:02:20.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.097+0000 7f0982d03640 1 -- 172.21.15.44:0/1974563622 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f09680b3d30 msgr2=0x7f09680b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:20.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.097+0000 7f0982d03640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f09680b3d30 0x7f09680b61f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f096c005fd0 tx=0x7f096c005850 comp rx=0 tx=0).stop 2024-08-20T03:02:20.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.097+0000 7f0982d03640 1 -- 172.21.15.44:0/1974563622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f097c108e40 msgr2=0x7f097c19e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:20.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.097+0000 7f0982d03640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f097c108e40 0x7f097c19e990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f097400eb10 tx=0x7f097400c2d0 comp rx=0 tx=0).stop 2024-08-20T03:02:20.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.097+0000 7f0982d03640 1 -- 172.21.15.44:0/1974563622 shutdown_connections 2024-08-20T03:02:20.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.097+0000 7f0982d03640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f09680b3d30 0x7f09680b61f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:20.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.097+0000 7f0982d03640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f097c108e40 0x7f097c19e990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:20.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.097+0000 7f0982d03640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f097c1084c0 0x7f097c1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:20.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.098+0000 7f0982d03640 1 --2- 172.21.15.44:0/1974563622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f097c102ab0 0x7f097c1990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:20.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.098+0000 7f0982d03640 1 -- 172.21.15.44:0/1974563622 >> 172.21.15.44:0/1974563622 conn(0x7f097c0fe540 msgr2=0x7f097c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:20.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.098+0000 7f0982d03640 1 -- 172.21.15.44:0/1974563622 shutdown_connections 2024-08-20T03:02:20.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.098+0000 7f0982d03640 1 -- 172.21.15.44:0/1974563622 wait complete. 2024-08-20T03:02:20.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:20 smithi165 ceph-mon[22357]: pgmap v315: 97 pgs: 97 active+clean; 653 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-08-20T03:02:20.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:20 smithi099 ceph-mon[21984]: pgmap v315: 97 pgs: 97 active+clean; 653 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-08-20T03:02:20.784 DEBUG:tasks.fwd_scrub:Yielding 2024-08-20T03:02:20.785 INFO:teuthology.run_tasks:Running task exec... 2024-08-20T03:02:20.794 INFO:teuthology.task.exec:Executing custom commands... 2024-08-20T03:02:20.794 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-08-20T03:02:20.795 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-08-20T03:02:20.795 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-08-20T03:02:20.795 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T03:02:20.795 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T03:02:20.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.903+0000 7f40623c2640 1 -- 172.21.15.44:0/2488639357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f405c1072d0 msgr2=0x7f405c107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:20.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.903+0000 7f40623c2640 1 --2- 172.21.15.44:0/2488639357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f405c1072d0 0x7f405c107710 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4044009900 tx=0x7f404402a410 comp rx=0 tx=0).stop 2024-08-20T03:02:20.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.903+0000 7f40623c2640 1 -- 172.21.15.44:0/2488639357 shutdown_connections 2024-08-20T03:02:20.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.903+0000 7f40623c2640 1 --2- 172.21.15.44:0/2488639357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f405c107c50 0x7f405c10e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:20.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.903+0000 7f40623c2640 1 --2- 172.21.15.44:0/2488639357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f405c1072d0 0x7f405c107710 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:20.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.903+0000 7f40623c2640 1 --2- 172.21.15.44:0/2488639357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f405c101920 0x7f405c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:20.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.903+0000 7f40623c2640 1 -- 172.21.15.44:0/2488639357 >> 172.21.15.44:0/2488639357 conn(0x7f405c078620 msgr2=0x7f405c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:20.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.903+0000 7f40623c2640 1 -- 172.21.15.44:0/2488639357 shutdown_connections 2024-08-20T03:02:20.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.904+0000 7f40623c2640 1 -- 172.21.15.44:0/2488639357 wait complete. 2024-08-20T03:02:20.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.904+0000 7f40623c2640 1 Processor -- start 2024-08-20T03:02:20.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.904+0000 7f40623c2640 1 -- start start 2024-08-20T03:02:20.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.904+0000 7f40623c2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f405c101920 0x7f405c10ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:20.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.904+0000 7f40623c2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f405c1072d0 0x7f405c10f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:20.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.904+0000 7f40623c2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f405c107c50 0x7f405c10f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:20.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f40623c2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f405c112a30 con 0x7f405c107c50 2024-08-20T03:02:20.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f40623c2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f405c1128b0 con 0x7f405c1072d0 2024-08-20T03:02:20.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f40623c2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f405c112bb0 con 0x7f405c101920 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f406095f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f405c107c50 0x7f405c10f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f406095f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f405c107c50 0x7f405c10f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58688/0 (socket says 172.21.15.44:58688) 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f406095f640 1 -- 172.21.15.44:0/1946499277 learned_addr learned my addr 172.21.15.44:0/1946499277 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f405bfff640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f405c101920 0x7f405c10ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f405b7fe640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f405c1072d0 0x7f405c10f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f406095f640 1 -- 172.21.15.44:0/1946499277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f405c101920 msgr2=0x7f405c10ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f406095f640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f405c101920 0x7f405c10ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f406095f640 1 -- 172.21.15.44:0/1946499277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f405c1072d0 msgr2=0x7f405c10f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f406095f640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f405c1072d0 0x7f405c10f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f406095f640 1 -- 172.21.15.44:0/1946499277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f405c1a73c0 con 0x7f405c107c50 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f405b7fe640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f405c1072d0 0x7f405c10f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f406095f640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f405c107c50 0x7f405c10f9e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f405400eaa0 tx=0x7f405400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:20.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f40597fa640 1 -- 172.21.15.44:0/1946499277 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405400cdb0 con 0x7f405c107c50 2024-08-20T03:02:20.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f40597fa640 1 -- 172.21.15.44:0/1946499277 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4054005020 con 0x7f405c107c50 2024-08-20T03:02:20.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f40623c2640 1 -- 172.21.15.44:0/1946499277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f405c1a76b0 con 0x7f405c107c50 2024-08-20T03:02:20.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f40597fa640 1 -- 172.21.15.44:0/1946499277 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40540116d0 con 0x7f405c107c50 2024-08-20T03:02:20.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.905+0000 7f40623c2640 1 -- 172.21.15.44:0/1946499277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f405c1a7c20 con 0x7f405c107c50 2024-08-20T03:02:20.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.907+0000 7f40623c2640 1 -- 172.21.15.44:0/1946499277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405c101d00 con 0x7f405c107c50 2024-08-20T03:02:20.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.909+0000 7f40597fa640 1 -- 172.21.15.44:0/1946499277 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f40540051d0 con 0x7f405c107c50 2024-08-20T03:02:20.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.910+0000 7f40597fa640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40480b3d30 0x7f40480b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:20.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.910+0000 7f405bfff640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40480b3d30 0x7f40480b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:20.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.910+0000 7f40597fa640 1 -- 172.21.15.44:0/1946499277 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f40540e4330 con 0x7f405c107c50 2024-08-20T03:02:20.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.910+0000 7f405bfff640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40480b3d30 0x7f40480b61f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f404c005d00 tx=0x7f404c00b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:20.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:20.912+0000 7f40597fa640 1 -- 172.21.15.44:0/1946499277 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4054016030 con 0x7f405c107c50 2024-08-20T03:02:21.062 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:21.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.128+0000 7f40623c2640 1 -- 172.21.15.44:0/1946499277 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f405c0632c0 con 0x7f40480b3d30 2024-08-20T03:02:21.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.278+0000 7f40597fa640 1 -- 172.21.15.44:0/1946499277 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f405c0632c0 con 0x7f40480b3d30 2024-08-20T03:02:21.281 INFO:teuthology.orchestra.run.smithi044.stdout:Schedule set for path / 2024-08-20T03:02:21.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.280+0000 7f40623c2640 1 -- 172.21.15.44:0/1946499277 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40480b3d30 msgr2=0x7f40480b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.280+0000 7f40623c2640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40480b3d30 0x7f40480b61f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f404c005d00 tx=0x7f404c00b410 comp rx=0 tx=0).stop 2024-08-20T03:02:21.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.280+0000 7f40623c2640 1 -- 172.21.15.44:0/1946499277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f405c107c50 msgr2=0x7f405c10f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.280+0000 7f40623c2640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f405c107c50 0x7f405c10f9e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f405400eaa0 tx=0x7f405400ef60 comp rx=0 tx=0).stop 2024-08-20T03:02:21.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.280+0000 7f40623c2640 1 -- 172.21.15.44:0/1946499277 shutdown_connections 2024-08-20T03:02:21.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.280+0000 7f40623c2640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f40480b3d30 0x7f40480b61f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.280+0000 7f40623c2640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f405c107c50 0x7f405c10f9e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.280+0000 7f40623c2640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f405c1072d0 0x7f405c10f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.280+0000 7f40623c2640 1 --2- 172.21.15.44:0/1946499277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f405c101920 0x7f405c10ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.280+0000 7f40623c2640 1 -- 172.21.15.44:0/1946499277 >> 172.21.15.44:0/1946499277 conn(0x7f405c078620 msgr2=0x7f405c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:21.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.280+0000 7f40623c2640 1 -- 172.21.15.44:0/1946499277 shutdown_connections 2024-08-20T03:02:21.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.280+0000 7f40623c2640 1 -- 172.21.15.44:0/1946499277 wait complete. 2024-08-20T03:02:21.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:21 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1974563622' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:02:21.294 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-08-20T03:02:21.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:21 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1974563622' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:02:21.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.397+0000 7fef13a77640 1 -- 172.21.15.44:0/3230624459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c107c50 msgr2=0x7fef0c10e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.397+0000 7fef13a77640 1 --2- 172.21.15.44:0/3230624459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c107c50 0x7fef0c10e450 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fef04009900 tx=0x7fef0402a410 comp rx=0 tx=0).stop 2024-08-20T03:02:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 -- 172.21.15.44:0/3230624459 shutdown_connections 2024-08-20T03:02:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 --2- 172.21.15.44:0/3230624459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c107c50 0x7fef0c10e450 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 --2- 172.21.15.44:0/3230624459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef0c1072d0 0x7fef0c107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 --2- 172.21.15.44:0/3230624459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c1018d0 0x7fef0c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 -- 172.21.15.44:0/3230624459 >> 172.21.15.44:0/3230624459 conn(0x7fef0c0fd350 msgr2=0x7fef0c0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 -- 172.21.15.44:0/3230624459 shutdown_connections 2024-08-20T03:02:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 -- 172.21.15.44:0/3230624459 wait complete. 2024-08-20T03:02:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 Processor -- start 2024-08-20T03:02:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 -- start start 2024-08-20T03:02:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c1018d0 0x7fef0c199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c1072d0 0x7fef0c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef0c107c50 0x7fef0c1a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c111920 con 0x7fef0c1018d0 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1117a0 con 0x7fef0c1072d0 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.398+0000 7fef13a77640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c111aa0 con 0x7fef0c107c50 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef11813640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c1018d0 0x7fef0c199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef11813640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c1018d0 0x7fef0c199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58710/0 (socket says 172.21.15.44:58710) 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef11813640 1 -- 172.21.15.44:0/4224173234 learned_addr learned my addr 172.21.15.44:0/4224173234 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef12014640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef0c107c50 0x7fef0c1a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef11012640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c1072d0 0x7fef0c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef11813640 1 -- 172.21.15.44:0/4224173234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef0c107c50 msgr2=0x7fef0c1a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef11813640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef0c107c50 0x7fef0c1a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef11813640 1 -- 172.21.15.44:0/4224173234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c1072d0 msgr2=0x7fef0c19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef11813640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c1072d0 0x7fef0c19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef11813640 1 -- 172.21.15.44:0/4224173234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c1a52d0 con 0x7fef0c1018d0 2024-08-20T03:02:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef11012640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c1072d0 0x7fef0c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef11813640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c1018d0 0x7fef0c199d10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7feefc00cce0 tx=0x7feefc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef0affd640 1 -- 172.21.15.44:0/4224173234 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc004b20 con 0x7fef0c1018d0 2024-08-20T03:02:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef0affd640 1 -- 172.21.15.44:0/4224173234 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feefc004cc0 con 0x7fef0c1018d0 2024-08-20T03:02:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef0affd640 1 -- 172.21.15.44:0/4224173234 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc00f6b0 con 0x7fef0c1018d0 2024-08-20T03:02:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef13a77640 1 -- 172.21.15.44:0/4224173234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0c1a54a0 con 0x7fef0c1018d0 2024-08-20T03:02:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.399+0000 7fef13a77640 1 -- 172.21.15.44:0/4224173234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0c1a59b0 con 0x7fef0c1018d0 2024-08-20T03:02:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.400+0000 7fef13a77640 1 -- 172.21.15.44:0/4224173234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef0c101cb0 con 0x7fef0c1018d0 2024-08-20T03:02:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.402+0000 7fef0affd640 1 -- 172.21.15.44:0/4224173234 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7feefc00f850 con 0x7fef0c1018d0 2024-08-20T03:02:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.402+0000 7fef0affd640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feef40b3d30 0x7feef40b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.403+0000 7fef11012640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feef40b3d30 0x7feef40b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.403+0000 7fef0affd640 1 -- 172.21.15.44:0/4224173234 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7feefc0e42d0 con 0x7fef0c1018d0 2024-08-20T03:02:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.403+0000 7fef11012640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feef40b3d30 0x7feef40b61f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef0c06b970 tx=0x7fef00005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:21.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.406+0000 7fef0affd640 1 -- 172.21.15.44:0/4224173234 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7feefc014030 con 0x7fef0c1018d0 2024-08-20T03:02:21.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:21 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1974563622' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:02:21.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.556+0000 7f25c58ef640 1 -- 172.21.15.44:0/2133827020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c010ce40 msgr2=0x7f25c010f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.556+0000 7f25c58ef640 1 --2- 172.21.15.44:0/2133827020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c010ce40 0x7f25c010f2a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009ca0 tx=0x7f25b802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.556+0000 7f25c58ef640 1 -- 172.21.15.44:0/2133827020 shutdown_connections 2024-08-20T03:02:21.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.556+0000 7f25c58ef640 1 --2- 172.21.15.44:0/2133827020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c010ce40 0x7f25c010f2a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.556+0000 7f25c58ef640 1 --2- 172.21.15.44:0/2133827020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c0104300 0x7f25c010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.556+0000 7f25c58ef640 1 --2- 172.21.15.44:0/2133827020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c01039e0 0x7f25c0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.556+0000 7f25c58ef640 1 -- 172.21.15.44:0/2133827020 >> 172.21.15.44:0/2133827020 conn(0x7f25c00fd440 msgr2=0x7f25c00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:21.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.556+0000 7f25c58ef640 1 -- 172.21.15.44:0/2133827020 shutdown_connections 2024-08-20T03:02:21.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.556+0000 7f25c58ef640 1 -- 172.21.15.44:0/2133827020 wait complete. 2024-08-20T03:02:21.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.557+0000 7f25c58ef640 1 Processor -- start 2024-08-20T03:02:21.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.557+0000 7f25c58ef640 1 -- start start 2024-08-20T03:02:21.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.557+0000 7f25c58ef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c01039e0 0x7f25c019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.557+0000 7f25c58ef640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c0104300 0x7f25c019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.557+0000 7f25c58ef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c010ce40 0x7f25c01a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.557+0000 7f25c58ef640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25c0112ab0 con 0x7f25c010ce40 2024-08-20T03:02:21.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.557+0000 7f25c58ef640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f25c0112930 con 0x7f25c01039e0 2024-08-20T03:02:21.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.557+0000 7f25c58ef640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25c0112c30 con 0x7f25c0104300 2024-08-20T03:02:21.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.557+0000 7f25bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c010ce40 0x7f25c01a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.557+0000 7f25bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c010ce40 0x7f25c01a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58736/0 (socket says 172.21.15.44:58736) 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25bf7fe640 1 -- 172.21.15.44:0/3005434269 learned_addr learned my addr 172.21.15.44:0/3005434269 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25b67fc640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c0104300 0x7f25c019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25beffd640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c01039e0 0x7f25c019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25bf7fe640 1 -- 172.21.15.44:0/3005434269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c0104300 msgr2=0x7f25c019b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25bf7fe640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c0104300 0x7f25c019b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25bf7fe640 1 -- 172.21.15.44:0/3005434269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c01039e0 msgr2=0x7f25c019b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25bf7fe640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c01039e0 0x7f25c019b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25bf7fe640 1 -- 172.21.15.44:0/3005434269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c01a1220 con 0x7f25c010ce40 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25b67fc640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c0104300 0x7f25c019b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25beffd640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c01039e0 0x7f25c019b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25bf7fe640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c010ce40 0x7f25c01a0a50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f25b8005cc0 tx=0x7f25b802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:21.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25c48ed640 1 -- 172.21.15.44:0/3005434269 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8003bf0 con 0x7f25c010ce40 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25c58ef640 1 -- 172.21.15.44:0/3005434269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c01a14b0 con 0x7f25c010ce40 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25c48ed640 1 -- 172.21.15.44:0/3005434269 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b8003d90 con 0x7f25c010ce40 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25c48ed640 1 -- 172.21.15.44:0/3005434269 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b802fbc0 con 0x7f25c010ce40 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.558+0000 7f25c58ef640 1 -- 172.21.15.44:0/3005434269 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c01a7bd0 con 0x7f25c010ce40 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.561+0000 7f25c7ce8740 1 Processor -- start 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25c48ed640 1 -- 172.21.15.44:0/3005434269 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f25b802fd60 con 0x7f25c010ce40 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25c7ce8740 1 -- start start 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25c7ce8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce33b0 0x558a78ce3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25c7ce8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce3cd0 0x558a78cef720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25c7ce8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78cf2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25c7ce8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558a78c59940 con 0x558a78ce3cd0 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25c7ce8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558a78c48e40 con 0x558a78ce33b0 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25c7ce8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558a78c4ce80 con 0x558a78cefc60 2024-08-20T03:02:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce3cd0 0x558a78cef720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce3cd0 0x558a78cef720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58750/0 (socket says 172.21.15.44:58750) 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25beffd640 1 -- 172.21.15.44:0/1346882954 learned_addr learned my addr 172.21.15.44:0/1346882954 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25b67fc640 1 --2- 172.21.15.44:0/1346882954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce33b0 0x558a78ce3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.562+0000 7f25b67fc640 1 --2- 172.21.15.44:0/1346882954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78cf2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25beffd640 1 -- 172.21.15.44:0/1346882954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 msgr2=0x558a78cf2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25beffd640 1 --2- 172.21.15.44:0/1346882954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78cf2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25beffd640 1 -- 172.21.15.44:0/1346882954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce33b0 msgr2=0x558a78ce3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25beffd640 1 --2- 172.21.15.44:0/1346882954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce33b0 0x558a78ce3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25beffd640 1 -- 172.21.15.44:0/1346882954 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a78be5520 con 0x558a78ce3cd0 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25b67fc640 1 --2- 172.21.15.44:0/1346882954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78cf2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25c48ed640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f25ac0b3cb0 0x7f25ac0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25beffd640 1 --2- 172.21.15.44:0/1346882954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce3cd0 0x558a78cef720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f25b000bd70 tx=0x7f25b000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e76491c95d00e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25b67fc640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f25ac0b3cb0 0x7f25ac0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25c48ed640 1 -- 172.21.15.44:0/3005434269 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f25b81054e0 con 0x7f25c010ce40 2024-08-20T03:02:21.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25b5ffb640 1 -- 172.21.15.44:0/1346882954 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0004020 con 0x558a78ce3cd0 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25b5ffb640 1 -- 172.21.15.44:0/1346882954 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b00041c0 con 0x558a78ce3cd0 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.563+0000 7f25b5ffb640 1 -- 172.21.15.44:0/1346882954 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0011710 con 0x558a78ce3cd0 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.564+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1346882954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce3cd0 msgr2=0x558a78cef720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.564+0000 7f25b67fc640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f25ac0b3cb0 0x7f25ac0b6170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x558a78ce3360 tx=0x7f25a8004730 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.564+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/1346882954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce3cd0 0x558a78cef720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f25b000bd70 tx=0x7f25b000f730 comp rx=0 tx=0).stop 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.564+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1346882954 shutdown_connections 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.564+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/1346882954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78cf2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.564+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/1346882954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce3cd0 0x558a78cef720 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.564+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/1346882954 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce33b0 0x558a78ce3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.564+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1346882954 >> 172.21.15.44:0/1346882954 conn(0x558a78ce11e0 msgr2=0x558a78ce2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.564+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1346882954 shutdown_connections 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.564+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1346882954 wait complete. 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.564+0000 7f25c7ce8740 1 -- start start 2024-08-20T03:02:21.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25c7ce8740 1 Processor -- start 2024-08-20T03:02:21.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25c7ce8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78cecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25c7ce8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce3cd0 0x558a78ced240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25c7ce8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78ced780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25c7ce8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558a78aab890 con 0x558a78ce33b0 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25c7ce8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558a78bcf700 con 0x558a78ce3cd0 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25c7ce8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558a78cf29b0 con 0x558a78cefc60 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25c7ce8740 10 client.0 _ll_register_callbacks cb 0x558a78c446f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78cecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78cecd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58762/0 (socket says 172.21.15.44:58762) 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25beffd640 1 -- 172.21.15.44:0/1895232029 learned_addr learned my addr 172.21.15.44:0/1895232029 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25bf7fe640 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78ced780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25b67fc640 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce3cd0 0x558a78ced240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25beffd640 1 -- 172.21.15.44:0/1895232029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 msgr2=0x558a78ced780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25beffd640 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78ced780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25beffd640 1 -- 172.21.15.44:0/1895232029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce3cd0 msgr2=0x558a78ced240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25beffd640 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce3cd0 0x558a78ced240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25beffd640 1 -- 172.21.15.44:0/1895232029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a789a8930 con 0x558a78ce33b0 2024-08-20T03:02:21.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.565+0000 7f25bf7fe640 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78ced780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f25beffd640 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78cecd00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f25b0000c00 tx=0x7f25b000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e676dcf5718a5f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f259effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f259effd640 1 -- 172.21.15.44:0/1895232029 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0003eb0 con 0x558a78ce33b0 2024-08-20T03:02:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f259effd640 1 -- 172.21.15.44:0/1895232029 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f25b00035f0 con 0x558a78ce33b0 2024-08-20T03:02:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1895232029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25b00041c0 con 0x558a78ce33b0 2024-08-20T03:02:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f259effd640 1 -- 172.21.15.44:0/1895232029 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f25b00037f0 con 0x558a78ce33b0 2024-08-20T03:02:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f25c7ce8740 10 client.15519 fetch_fsmap learned FSMap version 28 2024-08-20T03:02:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1895232029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a78cfe5b0 con 0x558a78ce33b0 2024-08-20T03:02:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f259effd640 1 -- 172.21.15.44:0/1895232029 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b00041c0 con 0x558a78ce33b0 2024-08-20T03:02:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f259effd640 1 -- 172.21.15.44:0/1895232029 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f25b0004790 con 0x558a78ce33b0 2024-08-20T03:02:21.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f25c7ce8740 10 client.15519 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:02:21.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f25c7ce8740 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:02:21.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f25c7ce8740 20 client.15519 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:21.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558a78cffa10 0x558a78d01e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f25c7ce8740 4 client.15519 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:02:21.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.566+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1895232029 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a78c5b890 con 0x558a78cffa10 2024-08-20T03:02:21.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.567+0000 7f25b67fc640 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558a78cffa10 0x558a78d01e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.567+0000 7f259effd640 10 client.15519 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:21.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.567+0000 7f25b67fc640 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558a78cffa10 0x558a78d01e00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eaacb054343e7cb server_cookie=ae3ee0ff8b528847 in_seq=0 out_seq=0 2024-08-20T03:02:21.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f259effd640 1 -- 172.21.15.44:0/1895232029 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a78c5b890 con 0x558a78cffa10 2024-08-20T03:02:21.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f259effd640 10 client.15519 handle_command_reply: tid=0 2024-08-20T03:02:21.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f25c7ce8740 1 client.15519 shutdown 2024-08-20T03:02:21.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1895232029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 msgr2=0x558a78cecd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78cecd00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f25b0000c00 tx=0x7f25b000bab0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1895232029 shutdown_connections 2024-08-20T03:02:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558a78cffa10 0x558a78d01e00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f259effd640 0 client.15519 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78ced780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce3cd0 0x558a78ced240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/1895232029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78cecd00 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1895232029 >> 172.21.15.44:0/1895232029 conn(0x558a78ce11e0 msgr2=0x558a78ce6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1895232029 shutdown_connections 2024-08-20T03:02:21.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f25c7ce8740 1 -- 172.21.15.44:0/1895232029 wait complete. 2024-08-20T03:02:21.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.568+0000 7f25c7ce8740 20 client.15519 trim_cache size 0 max 16384 2024-08-20T03:02:21.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.586+0000 7f25c7ce8740 1 Processor -- start 2024-08-20T03:02:21.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.586+0000 7f25c7ce8740 1 -- start start 2024-08-20T03:02:21.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.586+0000 7f25c7ce8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78d075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.586+0000 7f25c7ce8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce3cd0 0x558a78d07b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.586+0000 7f25c7ce8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78ce9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.586+0000 7f25c7ce8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25c01b04d0 con 0x558a78ce33b0 2024-08-20T03:02:21.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.586+0000 7f25c7ce8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f25c01b2010 con 0x558a78ce3cd0 2024-08-20T03:02:21.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.586+0000 7f25c7ce8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25c01b0ba0 con 0x558a78cefc60 2024-08-20T03:02:21.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.586+0000 7f25beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78d075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.586+0000 7f25beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78d075e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58778/0 (socket says 172.21.15.44:58778) 2024-08-20T03:02:21.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.586+0000 7f25beffd640 1 -- 172.21.15.44:0/801568952 learned_addr learned my addr 172.21.15.44:0/801568952 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:21.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.586+0000 7f25bf7fe640 1 --2- 172.21.15.44:0/801568952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78ce9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.587+0000 7f25beffd640 1 -- 172.21.15.44:0/801568952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 msgr2=0x558a78ce9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:21.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.587+0000 7f25b67fc640 1 --2- 172.21.15.44:0/801568952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce3cd0 0x558a78d07b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.587+0000 7f25beffd640 1 --2- 172.21.15.44:0/801568952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78ce9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.587+0000 7f25beffd640 1 -- 172.21.15.44:0/801568952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce3cd0 msgr2=0x558a78d07b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:21.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.587+0000 7f25beffd640 1 --2- 172.21.15.44:0/801568952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce3cd0 0x558a78d07b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.587+0000 7f25beffd640 1 -- 172.21.15.44:0/801568952 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a78ce1c60 con 0x558a78ce33b0 2024-08-20T03:02:21.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.587+0000 7f25bf7fe640 1 --2- 172.21.15.44:0/801568952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78ce9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:21.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.587+0000 7f25b67fc640 1 --2- 172.21.15.44:0/801568952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce3cd0 0x558a78d07b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:21.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.587+0000 7f25beffd640 1 --2- 172.21.15.44:0/801568952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78d075e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f25b0004bb0 tx=0x7f25b00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376b29a25b48ec9e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:21.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.587+0000 7f25b57fa640 1 -- 172.21.15.44:0/801568952 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0033070 con 0x558a78ce33b0 2024-08-20T03:02:21.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.587+0000 7f25b57fa640 1 -- 172.21.15.44:0/801568952 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b000fc30 con 0x558a78ce33b0 2024-08-20T03:02:21.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.587+0000 7f25b57fa640 1 -- 172.21.15.44:0/801568952 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b00049a0 con 0x558a78ce33b0 2024-08-20T03:02:21.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.588+0000 7f25c7ce8740 1 -- 172.21.15.44:0/801568952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 msgr2=0x558a78d075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.588+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/801568952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78d075e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f25b0004bb0 tx=0x7f25b00035f0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.588+0000 7f25c7ce8740 1 -- 172.21.15.44:0/801568952 shutdown_connections 2024-08-20T03:02:21.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.588+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/801568952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78cefc60 0x558a78ce9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.588+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/801568952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78ce3cd0 0x558a78d07b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.588+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/801568952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78d075e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.588+0000 7f25c7ce8740 1 -- 172.21.15.44:0/801568952 >> 172.21.15.44:0/801568952 conn(0x558a78ce11e0 msgr2=0x558a78ce6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:21.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.588+0000 7f25c7ce8740 1 -- 172.21.15.44:0/801568952 shutdown_connections 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.588+0000 7f25c7ce8740 1 -- 172.21.15.44:0/801568952 wait complete. 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.588+0000 7f25c7ce8740 1 -- start start 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.588+0000 7f25c7ce8740 1 Processor -- start 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25c7ce8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78cf8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25c7ce8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78ce3cd0 0x558a78cf3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25c7ce8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78cefc60 0x558a78cf3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25c7ce8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f25c01b1940 con 0x558a78ce33b0 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25c7ce8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f25c01b1270 con 0x558a78cefc60 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25c7ce8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f25c01ae4b0 con 0x558a78ce3cd0 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25c7ce8740 10 client.0 _ll_register_callbacks cb 0x7f25b0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78cf8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25b67fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78ce3cd0 0x558a78cf3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78cf8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58794/0 (socket says 172.21.15.44:58794) 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25b67fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78ce3cd0 0x558a78cf3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:34288/0 (socket says 172.21.15.44:34288) 2024-08-20T03:02:21.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25beffd640 1 -- 172.21.15.44:0/3562719643 learned_addr learned my addr 172.21.15.44:0/3562719643 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:21.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25bf7fe640 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78cefc60 0x558a78cf3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25beffd640 1 -- 172.21.15.44:0/3562719643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78ce3cd0 msgr2=0x558a78cf3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:21.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25beffd640 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78ce3cd0 0x558a78cf3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25beffd640 1 -- 172.21.15.44:0/3562719643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78cefc60 msgr2=0x558a78cf3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:21.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25beffd640 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78cefc60 0x558a78cf3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25beffd640 1 -- 172.21.15.44:0/3562719643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a78cf44a0 con 0x558a78ce33b0 2024-08-20T03:02:21.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25b67fc640 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78ce3cd0 0x558a78cf3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:21.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f25beffd640 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78cf8260 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f25b00045c0 tx=0x7f25b00049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35252b95620cd387 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:21.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.589+0000 7f259effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:21.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f259effd640 1 -- 172.21.15.44:0/3562719643 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b003b070 con 0x558a78ce33b0 2024-08-20T03:02:21.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f259effd640 1 -- 172.21.15.44:0/3562719643 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f25b0003ee0 con 0x558a78ce33b0 2024-08-20T03:02:21.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f25c7ce8740 1 -- 172.21.15.44:0/3562719643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25b000fc30 con 0x558a78ce33b0 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f259effd640 1 -- 172.21.15.44:0/3562719643 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f25b0047050 con 0x558a78ce33b0 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f25c7ce8740 10 client.15525 fetch_fsmap learned FSMap version 28 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f25c7ce8740 1 -- 172.21.15.44:0/3562719643 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a78d6a540 con 0x558a78ce33b0 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f259effd640 1 -- 172.21.15.44:0/3562719643 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b000fc30 con 0x558a78ce33b0 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f259effd640 1 -- 172.21.15.44:0/3562719643 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f25b0040740 con 0x558a78ce33b0 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f25c7ce8740 10 client.15525 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f25c7ce8740 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f25c7ce8740 20 client.15525 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558a78cffa10 0x558a78d6cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f25c7ce8740 4 client.15525 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.590+0000 7f25c7ce8740 1 -- 172.21.15.44:0/3562719643 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558a78c57330 con 0x558a78cffa10 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.591+0000 7f25b67fc640 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558a78cffa10 0x558a78d6cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.591+0000 7f259effd640 10 client.15525 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:21.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.591+0000 7f25b67fc640 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558a78cffa10 0x558a78d6cb60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11925a4cf883d4ef server_cookie=dfa4c7e5f7eff060 in_seq=0 out_seq=0 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f259effd640 1 -- 172.21.15.44:0/3562719643 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558a78c57330 con 0x558a78cffa10 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f259effd640 10 client.15525 handle_command_reply: tid=0 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f25c7ce8740 1 client.15525 shutdown 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f25c7ce8740 1 -- 172.21.15.44:0/3562719643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 msgr2=0x558a78cf8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78cf8260 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f25b00045c0 tx=0x7f25b00049c0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f25c7ce8740 1 -- 172.21.15.44:0/3562719643 shutdown_connections 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558a78cffa10 0x558a78d6cb60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558a78cefc60 0x558a78cf3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558a78ce3cd0 0x558a78cf3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f25c7ce8740 1 --2- 172.21.15.44:0/3562719643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558a78ce33b0 0x558a78cf8260 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f25c7ce8740 1 -- 172.21.15.44:0/3562719643 >> 172.21.15.44:0/3562719643 conn(0x558a78ce11e0 msgr2=0x558a78cdfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f259effd640 0 client.15525 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f25c7ce8740 1 -- 172.21.15.44:0/3562719643 shutdown_connections 2024-08-20T03:02:21.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.592+0000 7f25c7ce8740 1 -- 172.21.15.44:0/3562719643 wait complete. 2024-08-20T03:02:21.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c7ce8740 20 client.15525 trim_cache size 0 max 16384 2024-08-20T03:02:21.601 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:02:21.601 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-08-20T03:02:21.601 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "8185fb9d-acac-4186-bf07-28ceccb34d6d", 2024-08-20T03:02:21.601 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-08-20T03:02:21.601 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:02:21.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c58ef640 1 -- 172.21.15.44:0/3005434269 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f25ac0b3cb0 msgr2=0x7f25ac0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c58ef640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f25ac0b3cb0 0x7f25ac0b6170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x558a78ce3360 tx=0x7f25a8004730 comp rx=0 tx=0).stop 2024-08-20T03:02:21.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c58ef640 1 -- 172.21.15.44:0/3005434269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c010ce40 msgr2=0x7f25c01a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c58ef640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c010ce40 0x7f25c01a0a50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f25b8005cc0 tx=0x7f25b802f380 comp rx=0 tx=0).stop 2024-08-20T03:02:21.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c58ef640 1 -- 172.21.15.44:0/3005434269 shutdown_connections 2024-08-20T03:02:21.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c58ef640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f25ac0b3cb0 0x7f25ac0b6170 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c58ef640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25c010ce40 0x7f25c01a0a50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c58ef640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25c0104300 0x7f25c019b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c58ef640 1 --2- 172.21.15.44:0/3005434269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25c01039e0 0x7f25c019b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c58ef640 1 -- 172.21.15.44:0/3005434269 >> 172.21.15.44:0/3005434269 conn(0x7f25c00fd440 msgr2=0x7f25c010f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:21.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c58ef640 1 -- 172.21.15.44:0/3005434269 shutdown_connections 2024-08-20T03:02:21.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.593+0000 7f25c58ef640 1 -- 172.21.15.44:0/3005434269 wait complete. 2024-08-20T03:02:21.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.631+0000 7fef13a77640 1 -- 172.21.15.44:0/4224173234 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fef0c0632c0 con 0x7feef40b3d30 2024-08-20T03:02:21.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.698+0000 7fef0affd640 1 -- 172.21.15.44:0/4224173234 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fef0c0632c0 con 0x7feef40b3d30 2024-08-20T03:02:21.702 INFO:teuthology.orchestra.run.smithi044.stdout:Retention added to path / 2024-08-20T03:02:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.700+0000 7fef13a77640 1 -- 172.21.15.44:0/4224173234 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feef40b3d30 msgr2=0x7feef40b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.700+0000 7fef13a77640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feef40b3d30 0x7feef40b61f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fef0c06b970 tx=0x7fef00005f90 comp rx=0 tx=0).stop 2024-08-20T03:02:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.700+0000 7fef13a77640 1 -- 172.21.15.44:0/4224173234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c1018d0 msgr2=0x7fef0c199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.700+0000 7fef13a77640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c1018d0 0x7fef0c199d10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7feefc00cce0 tx=0x7feefc00a620 comp rx=0 tx=0).stop 2024-08-20T03:02:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.700+0000 7fef13a77640 1 -- 172.21.15.44:0/4224173234 shutdown_connections 2024-08-20T03:02:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.700+0000 7fef13a77640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7feef40b3d30 0x7feef40b61f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.700+0000 7fef13a77640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fef0c107c50 0x7fef0c1a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.700+0000 7fef13a77640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef0c1072d0 0x7fef0c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.700+0000 7fef13a77640 1 --2- 172.21.15.44:0/4224173234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef0c1018d0 0x7fef0c199d10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.700+0000 7fef13a77640 1 -- 172.21.15.44:0/4224173234 >> 172.21.15.44:0/4224173234 conn(0x7fef0c0fd350 msgr2=0x7fef0c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.700+0000 7fef13a77640 1 -- 172.21.15.44:0/4224173234 shutdown_connections 2024-08-20T03:02:21.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.700+0000 7fef13a77640 1 -- 172.21.15.44:0/4224173234 wait complete. 2024-08-20T03:02:21.714 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-20T03:02:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.814+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/1581988234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f081072d0 msgr2=0x7f2f08107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.814+0000 7f2f0e3a1640 1 --2- 172.21.15.44:0/1581988234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f081072d0 0x7f2f08107710 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009900 tx=0x7f2efc02a410 comp rx=0 tx=0).stop 2024-08-20T03:02:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.814+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/1581988234 shutdown_connections 2024-08-20T03:02:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.814+0000 7f2f0e3a1640 1 --2- 172.21.15.44:0/1581988234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f08107c50 0x7f2f0810e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.814+0000 7f2f0e3a1640 1 --2- 172.21.15.44:0/1581988234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f081072d0 0x7f2f08107710 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.814+0000 7f2f0e3a1640 1 --2- 172.21.15.44:0/1581988234 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f08101920 0x7f2f08101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.814+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/1581988234 >> 172.21.15.44:0/1581988234 conn(0x7f2f08078620 msgr2=0x7f2f080ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.814+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/1581988234 shutdown_connections 2024-08-20T03:02:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.814+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/1581988234 wait complete. 2024-08-20T03:02:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.815+0000 7f2f0e3a1640 1 Processor -- start 2024-08-20T03:02:21.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.815+0000 7f2f0e3a1640 1 -- start start 2024-08-20T03:02:21.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f0e3a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f08101920 0x7f2f08198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f0e3a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f081072d0 0x7f2f08199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f0e3a1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f08107c50 0x7f2f0819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f0e3a1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f08111910 con 0x7f2f08101920 2024-08-20T03:02:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f0e3a1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2f08111790 con 0x7f2f081072d0 2024-08-20T03:02:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f0e3a1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2f08111a90 con 0x7f2f08107c50 2024-08-20T03:02:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f07fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f08101920 0x7f2f08198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f07fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f08101920 0x7f2f08198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58808/0 (socket says 172.21.15.44:58808) 2024-08-20T03:02:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f07fff640 1 -- 172.21.15.44:0/3183204305 learned_addr learned my addr 172.21.15.44:0/3183204305 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f0c93e640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f08107c50 0x7f2f0819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f077fe640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f081072d0 0x7f2f08199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f07fff640 1 -- 172.21.15.44:0/3183204305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f08107c50 msgr2=0x7f2f0819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f07fff640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f08107c50 0x7f2f0819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f07fff640 1 -- 172.21.15.44:0/3183204305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f081072d0 msgr2=0x7f2f08199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f07fff640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f081072d0 0x7f2f08199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f07fff640 1 -- 172.21.15.44:0/3183204305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0819eff0 con 0x7f2f08101920 2024-08-20T03:02:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f0c93e640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f08107c50 0x7f2f0819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.816+0000 7f2f07fff640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f08101920 0x7f2f08198f50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800e9f0 tx=0x7f2ef800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.817+0000 7f2f057fa640 1 -- 172.21.15.44:0/3183204305 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef800cd80 con 0x7f2f08101920 2024-08-20T03:02:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.817+0000 7f2f057fa640 1 -- 172.21.15.44:0/3183204305 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef80054d0 con 0x7f2f08101920 2024-08-20T03:02:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.817+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/3183204305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0819f2e0 con 0x7f2f08101920 2024-08-20T03:02:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.817+0000 7f2f057fa640 1 -- 172.21.15.44:0/3183204305 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8010690 con 0x7f2f08101920 2024-08-20T03:02:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.817+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/3183204305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f081a59a0 con 0x7f2f08101920 2024-08-20T03:02:21.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.818+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/3183204305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f08101d00 con 0x7f2f08101920 2024-08-20T03:02:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.819+0000 7f2f057fa640 1 -- 172.21.15.44:0/3183204305 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f2ef8004e90 con 0x7f2f08101920 2024-08-20T03:02:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.820+0000 7f2f057fa640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2ef00b3cb0 0x7f2ef00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.820+0000 7f2f077fe640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2ef00b3cb0 0x7f2ef00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.820+0000 7f2f057fa640 1 -- 172.21.15.44:0/3183204305 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f2ef80e4270 con 0x7f2f08101920 2024-08-20T03:02:21.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.820+0000 7f2f077fe640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2ef00b3cb0 0x7f2ef00b6170 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2efc031440 tx=0x7f2efc005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:21.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:21.824+0000 7f2f057fa640 1 -- 172.21.15.44:0/3183204305 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2ef80ab6b0 con 0x7f2f08101920 2024-08-20T03:02:22.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.031+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/3183204305 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2f08103290 con 0x7f2ef00b3cb0 2024-08-20T03:02:22.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.038+0000 7f2f057fa640 1 -- 172.21.15.44:0/3183204305 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f2f08103290 con 0x7f2ef00b3cb0 2024-08-20T03:02:22.041 INFO:teuthology.orchestra.run.smithi044.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-20T00:00:00", "created": "2024-08-20T03:02:21", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-08-20T03:02:22.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.040+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/3183204305 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2ef00b3cb0 msgr2=0x7f2ef00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:22.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.040+0000 7f2f0e3a1640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2ef00b3cb0 0x7f2ef00b6170 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2efc031440 tx=0x7f2efc005b50 comp rx=0 tx=0).stop 2024-08-20T03:02:22.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.040+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/3183204305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f08101920 msgr2=0x7f2f08198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:22.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.040+0000 7f2f0e3a1640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f08101920 0x7f2f08198f50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800e9f0 tx=0x7f2ef800edf0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.040+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/3183204305 shutdown_connections 2024-08-20T03:02:22.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.040+0000 7f2f0e3a1640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2ef00b3cb0 0x7f2ef00b6170 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.040+0000 7f2f0e3a1640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f08107c50 0x7f2f0819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.040+0000 7f2f0e3a1640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2f081072d0 0x7f2f08199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.040+0000 7f2f0e3a1640 1 --2- 172.21.15.44:0/3183204305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f08101920 0x7f2f08198f50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.040+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/3183204305 >> 172.21.15.44:0/3183204305 conn(0x7f2f08078620 msgr2=0x7f2f08100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:22.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.040+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/3183204305 shutdown_connections 2024-08-20T03:02:22.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.040+0000 7f2f0e3a1640 1 -- 172.21.15.44:0/3183204305 wait complete. 2024-08-20T03:02:22.054 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-20T03:02:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.154+0000 7f26a8277640 1 -- 172.21.15.44:0/4101445911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a0107c50 msgr2=0x7f26a010e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.154+0000 7f26a8277640 1 --2- 172.21.15.44:0/4101445911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a0107c50 0x7f26a010e450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2698009ca0 tx=0x7f269802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.154+0000 7f26a8277640 1 -- 172.21.15.44:0/4101445911 shutdown_connections 2024-08-20T03:02:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.154+0000 7f26a8277640 1 --2- 172.21.15.44:0/4101445911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a0107c50 0x7f26a010e450 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.154+0000 7f26a8277640 1 --2- 172.21.15.44:0/4101445911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26a01072d0 0x7f26a0107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.154+0000 7f26a8277640 1 --2- 172.21.15.44:0/4101445911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26a0101920 0x7f26a0101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.154+0000 7f26a8277640 1 -- 172.21.15.44:0/4101445911 >> 172.21.15.44:0/4101445911 conn(0x7f26a0078620 msgr2=0x7f26a00ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:22.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.154+0000 7f26a8277640 1 -- 172.21.15.44:0/4101445911 shutdown_connections 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.154+0000 7f26a8277640 1 -- 172.21.15.44:0/4101445911 wait complete. 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a8277640 1 Processor -- start 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a8277640 1 -- start start 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a8277640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26a0101920 0x7f26a0197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a8277640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a01072d0 0x7f26a0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a8277640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26a0107c50 0x7f26a019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a8277640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0111960 con 0x7f26a01072d0 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a8277640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26a01117e0 con 0x7f26a0101920 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a8277640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f26a0111ae0 con 0x7f26a0107c50 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a5812640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a01072d0 0x7f26a0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a5812640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a01072d0 0x7f26a0198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58826/0 (socket says 172.21.15.44:58826) 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a5812640 1 -- 172.21.15.44:0/1149861384 learned_addr learned my addr 172.21.15.44:0/1149861384 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a6013640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26a0101920 0x7f26a0197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:22.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a6814640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26a0107c50 0x7f26a019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a5812640 1 -- 172.21.15.44:0/1149861384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26a0107c50 msgr2=0x7f26a019d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a5812640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26a0107c50 0x7f26a019d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a5812640 1 -- 172.21.15.44:0/1149861384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26a0101920 msgr2=0x7f26a0197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a5812640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26a0101920 0x7f26a0197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.155+0000 7f26a5812640 1 -- 172.21.15.44:0/1149861384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a019de80 con 0x7f26a01072d0 2024-08-20T03:02:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.156+0000 7f26a5812640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a01072d0 0x7f26a0198320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f269000d950 tx=0x7f269000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.156+0000 7f26977fe640 1 -- 172.21.15.44:0/1149861384 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2690017070 con 0x7f26a01072d0 2024-08-20T03:02:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.156+0000 7f26977fe640 1 -- 172.21.15.44:0/1149861384 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2690009c30 con 0x7f26a01072d0 2024-08-20T03:02:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.156+0000 7f26a8277640 1 -- 172.21.15.44:0/1149861384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a019e170 con 0x7f26a01072d0 2024-08-20T03:02:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.156+0000 7f26977fe640 1 -- 172.21.15.44:0/1149861384 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269000f620 con 0x7f26a01072d0 2024-08-20T03:02:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.156+0000 7f26a8277640 1 -- 172.21.15.44:0/1149861384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a01a4830 con 0x7f26a01072d0 2024-08-20T03:02:22.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.157+0000 7f26a8277640 1 -- 172.21.15.44:0/1149861384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a0101d00 con 0x7f26a01072d0 2024-08-20T03:02:22.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.158+0000 7f26977fe640 1 -- 172.21.15.44:0/1149861384 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 24) ==== 151127+0+0 (secure 0 0 0) 0x7f2690010460 con 0x7f26a01072d0 2024-08-20T03:02:22.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.159+0000 7f26977fe640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f268c0b3cb0 0x7f268c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:22.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.159+0000 7f26a6013640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f268c0b3cb0 0x7f268c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:22.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.159+0000 7f26977fe640 1 -- 172.21.15.44:0/1149861384 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f26900e3a10 con 0x7f26a01072d0 2024-08-20T03:02:22.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.160+0000 7f26a6013640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f268c0b3cb0 0x7f268c0b6170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f269c005fd0 tx=0x7f269c00b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:22.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.162+0000 7f26977fe640 1 -- 172.21.15.44:0/1149861384 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2690009de0 con 0x7f26a01072d0 2024-08-20T03:02:22.274 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:22 smithi044 ceph-mon[18790]: pgmap v316: 97 pgs: 97 active+clean; 653 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-08-20T03:02:22.274 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:22 smithi044 ceph-mon[18790]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:22.298 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:02:22.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.373+0000 7f26a8277640 1 -- 172.21.15.44:0/1149861384 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f26a00632c0 con 0x7f268c0b3cb0 2024-08-20T03:02:22.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.381+0000 7f26977fe640 1 -- 172.21.15.44:0/1149861384 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f26a00632c0 con 0x7f268c0b3cb0 2024-08-20T03:02:22.384 INFO:teuthology.orchestra.run.smithi044.stdout:/ 1m 6m3h 2024-08-20T03:02:22.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:22 smithi165 ceph-mon[22357]: pgmap v316: 97 pgs: 97 active+clean; 653 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-08-20T03:02:22.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:22 smithi165 ceph-mon[22357]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:22.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.384+0000 7f26a8277640 1 -- 172.21.15.44:0/1149861384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f268c0b3cb0 msgr2=0x7f268c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:22.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.384+0000 7f26a8277640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f268c0b3cb0 0x7f268c0b6170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f269c005fd0 tx=0x7f269c00b390 comp rx=0 tx=0).stop 2024-08-20T03:02:22.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.384+0000 7f26a8277640 1 -- 172.21.15.44:0/1149861384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a01072d0 msgr2=0x7f26a0198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:22.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.384+0000 7f26a8277640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a01072d0 0x7f26a0198320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f269000d950 tx=0x7f269000de10 comp rx=0 tx=0).stop 2024-08-20T03:02:22.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.384+0000 7f26a8277640 1 -- 172.21.15.44:0/1149861384 shutdown_connections 2024-08-20T03:02:22.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.384+0000 7f26a8277640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f268c0b3cb0 0x7f268c0b6170 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.384+0000 7f26a8277640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26a0107c50 0x7f26a019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.384+0000 7f26a8277640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26a01072d0 0x7f26a0198320 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.384+0000 7f26a8277640 1 --2- 172.21.15.44:0/1149861384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26a0101920 0x7f26a0197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:22.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.384+0000 7f26a8277640 1 -- 172.21.15.44:0/1149861384 >> 172.21.15.44:0/1149861384 conn(0x7f26a0078620 msgr2=0x7f26a0103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:22.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.384+0000 7f26a8277640 1 -- 172.21.15.44:0/1149861384 shutdown_connections 2024-08-20T03:02:22.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:22.384+0000 7f26a8277640 1 -- 172.21.15.44:0/1149861384 wait complete. 2024-08-20T03:02:22.401 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-08-20T03:02:22.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:22 smithi099 ceph-mon[21984]: pgmap v316: 97 pgs: 97 active+clean; 653 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-08-20T03:02:22.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:22 smithi099 ceph-mon[21984]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:22.428 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-08-20T03:02:22.438 INFO:teuthology.run_tasks:Running task background_exec... 2024-08-20T03:02:22.447 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-08-20T03:02:22.447 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-08-20T03:02:22.470 INFO:teuthology.run_tasks:Running task workunit... 2024-08-20T03:02:22.486 INFO:tasks.workunit:Pulling workunits from ref ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T03:02:22.486 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-20T03:02:22.486 INFO:tasks.workunit:timeout=3h 2024-08-20T03:02:22.486 INFO:tasks.workunit:cleanup=True 2024-08-20T03:02:22.486 DEBUG:teuthology.orchestra.run.smithi044:> stat -- /home/ubuntu/cephtest/mnt.0 2024-08-20T03:02:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-08-20T03:02:22.503 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-08-20T03:02:22.503 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2024-08-20T03:02:22.503 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T03:02:22.503 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-08-20T03:02:22.503 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-08-20 03:00:37.146338388 +0000 2024-08-20T03:02:22.503 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-08-20 03:00:37.146338388 +0000 2024-08-20T03:02:22.504 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-08-20 03:02:12.223917168 +0000 2024-08-20T03:02:22.504 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-08-20 03:00:37.146338388 +0000 2024-08-20T03:02:22.504 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-08-20T03:02:22.504 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-08-20T03:02:22.563 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:22.578 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T03:02:22.634 INFO:tasks.workunit.client.0.smithi044.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-08-20T03:02:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.060+0000 7fb5977eb640 1 -- 172.21.15.44:0/1502048683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb590108e40 msgr2=0x7fb59010f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.060+0000 7fb5977eb640 1 --2- 172.21.15.44:0/1502048683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb590108e40 0x7fb59010f640 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb588009ca0 tx=0x7fb58802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.060+0000 7fb5977eb640 1 -- 172.21.15.44:0/1502048683 shutdown_connections 2024-08-20T03:02:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.060+0000 7fb5977eb640 1 --2- 172.21.15.44:0/1502048683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb590108e40 0x7fb59010f640 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.060+0000 7fb5977eb640 1 --2- 172.21.15.44:0/1502048683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5901084c0 0x7fb590108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.060+0000 7fb5977eb640 1 --2- 172.21.15.44:0/1502048683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb590102ab0 0x7fb590102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.060+0000 7fb5977eb640 1 -- 172.21.15.44:0/1502048683 >> 172.21.15.44:0/1502048683 conn(0x7fb5900fe500 msgr2=0x7fb590100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.060+0000 7fb5977eb640 1 -- 172.21.15.44:0/1502048683 shutdown_connections 2024-08-20T03:02:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.060+0000 7fb5977eb640 1 -- 172.21.15.44:0/1502048683 wait complete. 2024-08-20T03:02:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb5977eb640 1 Processor -- start 2024-08-20T03:02:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb5977eb640 1 -- start start 2024-08-20T03:02:23.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb5977eb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb590102ab0 0x7fb590198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb5977eb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5901084c0 0x7fb5901994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb5977eb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb590108e40 0x7fb59019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb5977eb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb590112ac0 con 0x7fb590102ab0 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb5977eb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb590112940 con 0x7fb590108e40 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb5977eb640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb590112c40 con 0x7fb5901084c0 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb595587640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb590102ab0 0x7fb590198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb595587640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb590102ab0 0x7fb590198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58848/0 (socket says 172.21.15.44:58848) 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb595587640 1 -- 172.21.15.44:0/3301129893 learned_addr learned my addr 172.21.15.44:0/3301129893 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb594d86640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5901084c0 0x7fb5901994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.061+0000 7fb595d88640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb590108e40 0x7fb59019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.062+0000 7fb595587640 1 -- 172.21.15.44:0/3301129893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5901084c0 msgr2=0x7fb5901994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.062+0000 7fb595587640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5901084c0 0x7fb5901994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.062+0000 7fb595587640 1 -- 172.21.15.44:0/3301129893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb590108e40 msgr2=0x7fb59019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:23.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.062+0000 7fb595587640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb590108e40 0x7fb59019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.062+0000 7fb595587640 1 -- 172.21.15.44:0/3301129893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59019f060 con 0x7fb590102ab0 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.062+0000 7fb595d88640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb590108e40 0x7fb59019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.062+0000 7fb595587640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb590102ab0 0x7fb590198fb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00d950 tx=0x7fb58c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.062+0000 7fb5867fc640 1 -- 172.21.15.44:0/3301129893 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c017030 con 0x7fb590102ab0 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.062+0000 7fb5977eb640 1 -- 172.21.15.44:0/3301129893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59019f350 con 0x7fb590102ab0 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.062+0000 7fb5867fc640 1 -- 172.21.15.44:0/3301129893 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb58c004020 con 0x7fb590102ab0 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.062+0000 7fb5867fc640 1 -- 172.21.15.44:0/3301129893 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0047a0 con 0x7fb590102ab0 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.062+0000 7fb5977eb640 1 -- 172.21.15.44:0/3301129893 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5901a5a10 con 0x7fb590102ab0 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb599be4740 1 Processor -- start 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb5867fc640 1 -- 172.21.15.44:0/3301129893 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7fb58c009a30 con 0x7fb590102ab0 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb599be4740 1 -- start start 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb599be4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c089830 0x55e53c089c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb599be4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 0x55e53c095ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb599be4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c0960e0 0x55e53c0984d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb599be4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e53c0008e0 con 0x55e53c08a150 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb599be4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e53bfefde0 con 0x55e53c089830 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb599be4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e53bff3e20 con 0x55e53c0960e0 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb595587640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 0x55e53c095ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb595587640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 0x55e53c095ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58862/0 (socket says 172.21.15.44:58862) 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb595587640 1 -- 172.21.15.44:0/564517357 learned_addr learned my addr 172.21.15.44:0/564517357 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb595d88640 1 --2- 172.21.15.44:0/564517357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c089830 0x55e53c089c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.065+0000 7fb594d86640 1 --2- 172.21.15.44:0/564517357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c0960e0 0x55e53c0984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb595587640 1 -- 172.21.15.44:0/564517357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c0960e0 msgr2=0x55e53c0984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb595587640 1 --2- 172.21.15.44:0/564517357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c0960e0 0x55e53c0984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb595587640 1 -- 172.21.15.44:0/564517357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c089830 msgr2=0x55e53c089c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb595587640 1 --2- 172.21.15.44:0/564517357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c089830 0x55e53c089c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb595587640 1 -- 172.21.15.44:0/564517357 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e53bd4f650 con 0x55e53c08a150 2024-08-20T03:02:23.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb5867fc640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb57c0b3d40 0x7fb57c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb595d88640 1 --2- 172.21.15.44:0/564517357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c089830 0x55e53c089c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb595d88640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb57c0b3d40 0x7fb57c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb595587640 1 --2- 172.21.15.44:0/564517357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 0x55e53c095ba0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0e11a0 tx=0x7fb58c0a6d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be0614360a30cda server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb5867fc640 1 -- 172.21.15.44:0/3301129893 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb58c0e3aa0 con 0x7fb590102ab0 2024-08-20T03:02:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb57b7fe640 1 -- 172.21.15.44:0/564517357 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0e9020 con 0x55e53c08a150 2024-08-20T03:02:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb57b7fe640 1 -- 172.21.15.44:0/564517357 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb58c0b3110 con 0x55e53c08a150 2024-08-20T03:02:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.066+0000 7fb57b7fe640 1 -- 172.21.15.44:0/564517357 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0aa070 con 0x55e53c08a150 2024-08-20T03:02:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.067+0000 7fb599be4740 1 -- 172.21.15.44:0/564517357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 msgr2=0x55e53c095ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.067+0000 7fb599be4740 1 --2- 172.21.15.44:0/564517357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 0x55e53c095ba0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0e11a0 tx=0x7fb58c0a6d70 comp rx=0 tx=0).stop 2024-08-20T03:02:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.067+0000 7fb595d88640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb57c0b3d40 0x7fb57c0b6200 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb588005fa0 tx=0x7fb588009360 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:23.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.067+0000 7fb599be4740 1 -- 172.21.15.44:0/564517357 shutdown_connections 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.067+0000 7fb599be4740 1 --2- 172.21.15.44:0/564517357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c0960e0 0x55e53c0984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.067+0000 7fb599be4740 1 --2- 172.21.15.44:0/564517357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 0x55e53c095ba0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.067+0000 7fb599be4740 1 --2- 172.21.15.44:0/564517357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c089830 0x55e53c089c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.067+0000 7fb599be4740 1 -- 172.21.15.44:0/564517357 >> 172.21.15.44:0/564517357 conn(0x55e53c087660 msgr2=0x55e53c088800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.067+0000 7fb599be4740 1 -- 172.21.15.44:0/564517357 shutdown_connections 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.067+0000 7fb599be4740 1 -- 172.21.15.44:0/564517357 wait complete. 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.067+0000 7fb599be4740 1 -- start start 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.067+0000 7fb599be4740 1 Processor -- start 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb599be4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c089830 0x55e53c093040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb599be4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 0x55e53c093580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb599be4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c0960e0 0x55e53c093ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb599be4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e53be52700 con 0x55e53c08a150 2024-08-20T03:02:23.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb599be4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e53bf767c0 con 0x55e53c0960e0 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb599be4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e53c098d90 con 0x55e53c089830 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb599be4740 10 client.0 _ll_register_callbacks cb 0x55e53bfe96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb594d86640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 0x55e53c093580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb594d86640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 0x55e53c093580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58866/0 (socket says 172.21.15.44:58866) 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb594d86640 1 -- 172.21.15.44:0/1380073897 learned_addr learned my addr 172.21.15.44:0/1380073897 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb595d88640 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c0960e0 0x55e53c093ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb595587640 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c089830 0x55e53c093040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb594d86640 1 -- 172.21.15.44:0/1380073897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c089830 msgr2=0x55e53c093040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb594d86640 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c089830 0x55e53c093040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb594d86640 1 -- 172.21.15.44:0/1380073897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c0960e0 msgr2=0x55e53c093ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb594d86640 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c0960e0 0x55e53c093ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.068+0000 7fb594d86640 1 -- 172.21.15.44:0/1380073897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e53c0a2110 con 0x55e53c08a150 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb594d86640 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 0x55e53c093580 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb58000be30 tx=0x7fb5800117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b223c550cc524055 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb578ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:23.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb578ff9640 1 -- 172.21.15.44:0/1380073897 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580012650 con 0x55e53c08a150 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb578ff9640 1 -- 172.21.15.44:0/1380073897 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb5800127f0 con 0x55e53c08a150 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb599be4740 1 -- 172.21.15.44:0/1380073897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e53bf9f100 con 0x55e53c08a150 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb578ff9640 1 -- 172.21.15.44:0/1380073897 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb5800129f0 con 0x55e53c08a150 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb599be4740 10 client.15549 fetch_fsmap learned FSMap version 28 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb599be4740 1 -- 172.21.15.44:0/1380073897 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e53c0a4b20 con 0x55e53c08a150 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb578ff9640 1 -- 172.21.15.44:0/1380073897 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e53bf9f100 con 0x55e53c08a150 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb578ff9640 1 -- 172.21.15.44:0/1380073897 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb58001bc60 con 0x55e53c08a150 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb599be4740 10 client.15549 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb599be4740 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb599be4740 20 client.15549 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb599be4740 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e53c0a60d0 0x55e53c0a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb599be4740 4 client.15549 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:02:23.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.069+0000 7fb599be4740 1 -- 172.21.15.44:0/1380073897 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e53c002830 con 0x55e53c0a60d0 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.070+0000 7fb595587640 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e53c0a60d0 0x55e53c0a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.070+0000 7fb578ff9640 10 client.15549 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.070+0000 7fb595587640 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e53c0a60d0 0x55e53c0a84c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7c5ba0aad9817 server_cookie=3af8dbb964602fe9 in_seq=0 out_seq=0 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb578ff9640 1 -- 172.21.15.44:0/1380073897 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e53c002830 con 0x55e53c0a60d0 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb578ff9640 10 client.15549 handle_command_reply: tid=0 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb599be4740 1 client.15549 shutdown 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb599be4740 1 -- 172.21.15.44:0/1380073897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 msgr2=0x55e53c093580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb599be4740 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 0x55e53c093580 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb58000be30 tx=0x7fb5800117b0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb599be4740 1 -- 172.21.15.44:0/1380073897 shutdown_connections 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb599be4740 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e53c0a60d0 0x55e53c0a84c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb599be4740 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c0960e0 0x55e53c093ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb599be4740 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c08a150 0x55e53c093580 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb599be4740 1 --2- 172.21.15.44:0/1380073897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c089830 0x55e53c093040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb599be4740 1 -- 172.21.15.44:0/1380073897 >> 172.21.15.44:0/1380073897 conn(0x55e53c087660 msgr2=0x55e53c08d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:23.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb578ff9640 0 client.15549 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:23.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb599be4740 1 -- 172.21.15.44:0/1380073897 shutdown_connections 2024-08-20T03:02:23.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb599be4740 1 -- 172.21.15.44:0/1380073897 wait complete. 2024-08-20T03:02:23.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.071+0000 7fb599be4740 20 client.15549 trim_cache size 0 max 16384 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.088+0000 7fb599be4740 1 Processor -- start 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb599be4740 1 -- start start 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb599be4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c089830 0x55e53c0a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb599be4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c08a150 0x55e53c0a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb599be4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c0960e0 0x55e53c0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb599be4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901ae360 con 0x55e53c0960e0 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb599be4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901afea0 con 0x55e53c08a150 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb599be4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901aea30 con 0x55e53c089830 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb595d88640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c0960e0 0x55e53c0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb595d88640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c0960e0 0x55e53c0addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58874/0 (socket says 172.21.15.44:58874) 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb595d88640 1 -- 172.21.15.44:0/2280645183 learned_addr learned my addr 172.21.15.44:0/2280645183 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb594d86640 1 --2- 172.21.15.44:0/2280645183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c08a150 0x55e53c0a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb595587640 1 --2- 172.21.15.44:0/2280645183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c089830 0x55e53c0a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb595d88640 1 -- 172.21.15.44:0/2280645183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c089830 msgr2=0x55e53c0a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb595d88640 1 --2- 172.21.15.44:0/2280645183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c089830 0x55e53c0a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb595d88640 1 -- 172.21.15.44:0/2280645183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c08a150 msgr2=0x55e53c0a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb595d88640 1 --2- 172.21.15.44:0/2280645183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c08a150 0x55e53c0a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb595d88640 1 -- 172.21.15.44:0/2280645183 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e53c089f80 con 0x55e53c0960e0 2024-08-20T03:02:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.089+0000 7fb595d88640 1 --2- 172.21.15.44:0/2280645183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c0960e0 0x55e53c0addd0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb588031f30 tx=0x7fb58803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466ef23722090ee server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.090+0000 7fb57affd640 1 -- 172.21.15.44:0/2280645183 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588003b70 con 0x55e53c0960e0 2024-08-20T03:02:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.090+0000 7fb57affd640 1 -- 172.21.15.44:0/2280645183 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb588036040 con 0x55e53c0960e0 2024-08-20T03:02:23.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.090+0000 7fb57affd640 1 -- 172.21.15.44:0/2280645183 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58802f950 con 0x55e53c0960e0 2024-08-20T03:02:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.090+0000 7fb599be4740 1 -- 172.21.15.44:0/2280645183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c0960e0 msgr2=0x55e53c0addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.090+0000 7fb599be4740 1 --2- 172.21.15.44:0/2280645183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c0960e0 0x55e53c0addd0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb588031f30 tx=0x7fb58803b000 comp rx=0 tx=0).stop 2024-08-20T03:02:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.090+0000 7fb599be4740 1 -- 172.21.15.44:0/2280645183 shutdown_connections 2024-08-20T03:02:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.090+0000 7fb599be4740 1 --2- 172.21.15.44:0/2280645183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c0960e0 0x55e53c0addd0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.090+0000 7fb599be4740 1 --2- 172.21.15.44:0/2280645183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c08a150 0x55e53c0a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.090+0000 7fb599be4740 1 --2- 172.21.15.44:0/2280645183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c089830 0x55e53c0a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.090+0000 7fb599be4740 1 -- 172.21.15.44:0/2280645183 >> 172.21.15.44:0/2280645183 conn(0x55e53c087660 msgr2=0x55e53c0a0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:23.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.090+0000 7fb599be4740 1 -- 172.21.15.44:0/2280645183 shutdown_connections 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.090+0000 7fb599be4740 1 -- 172.21.15.44:0/2280645183 wait complete. 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb599be4740 1 -- start start 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb599be4740 1 Processor -- start 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb599be4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c089830 0x55e53c09b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb599be4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c08a150 0x55e53c09b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb599be4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c0960e0 0x55e53c10eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb599be4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901af7d0 con 0x55e53c089830 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb599be4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901af100 con 0x55e53c08a150 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb599be4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901ac340 con 0x55e53c0960e0 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb599be4740 10 client.0 _ll_register_callbacks cb 0x7fb58000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb595587640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c089830 0x55e53c09b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb595587640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c089830 0x55e53c09b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58878/0 (socket says 172.21.15.44:58878) 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb595587640 1 -- 172.21.15.44:0/528284526 learned_addr learned my addr 172.21.15.44:0/528284526 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:23.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb594d86640 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c08a150 0x55e53c09b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb595587640 1 -- 172.21.15.44:0/528284526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c0960e0 msgr2=0x55e53c10eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb595587640 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c0960e0 0x55e53c10eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb595587640 1 -- 172.21.15.44:0/528284526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c08a150 msgr2=0x55e53c09b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.091+0000 7fb595587640 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c08a150 0x55e53c09b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb595587640 1 -- 172.21.15.44:0/528284526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e53c10f2e0 con 0x55e53c089830 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb594d86640 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c08a150 0x55e53c09b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb595587640 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c089830 0x55e53c09b2b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0aa6b0 tx=0x7fb58c00df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f706e989f53e1f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb578ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb578ff9640 1 -- 172.21.15.44:0/528284526 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0e9040 con 0x55e53c089830 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb578ff9640 1 -- 172.21.15.44:0/528284526 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb58c0e14e0 con 0x55e53c089830 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb599be4740 1 -- 172.21.15.44:0/528284526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb588036040 con 0x55e53c089830 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb578ff9640 1 -- 172.21.15.44:0/528284526 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb58c0a9e90 con 0x55e53c089830 2024-08-20T03:02:23.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb599be4740 10 client.15555 fetch_fsmap learned FSMap version 28 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb599be4740 1 -- 172.21.15.44:0/528284526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e53c110580 con 0x55e53c089830 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb578ff9640 1 -- 172.21.15.44:0/528284526 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb588036040 con 0x55e53c089830 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb578ff9640 1 -- 172.21.15.44:0/528284526 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb58c0e73c0 con 0x55e53c089830 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb599be4740 10 client.15555 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb599be4740 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb599be4740 20 client.15555 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb599be4740 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e53c1108c0 0x55e53c112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb599be4740 4 client.15555 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.092+0000 7fb599be4740 1 -- 172.21.15.44:0/528284526 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e53bffe2d0 con 0x55e53c1108c0 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb594d86640 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e53c1108c0 0x55e53c112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb594d86640 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e53c1108c0 0x55e53c112cb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34fbe9b9b96c776a server_cookie=1f74534e1e41a21c in_seq=0 out_seq=0 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb578ff9640 10 client.15555 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb578ff9640 1 -- 172.21.15.44:0/528284526 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e53bffe2d0 con 0x55e53c1108c0 2024-08-20T03:02:23.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb578ff9640 10 client.15555 handle_command_reply: tid=0 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb599be4740 1 client.15555 shutdown 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb599be4740 1 -- 172.21.15.44:0/528284526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c089830 msgr2=0x55e53c09b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb599be4740 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c089830 0x55e53c09b2b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0aa6b0 tx=0x7fb58c00df60 comp rx=0 tx=0).stop 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb599be4740 1 -- 172.21.15.44:0/528284526 shutdown_connections 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb599be4740 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e53c1108c0 0x55e53c112cb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb599be4740 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e53c0960e0 0x55e53c10eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb599be4740 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e53c08a150 0x55e53c09b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb599be4740 1 --2- 172.21.15.44:0/528284526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53c089830 0x55e53c09b2b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb578ff9640 0 client.15555 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.093+0000 7fb599be4740 1 -- 172.21.15.44:0/528284526 >> 172.21.15.44:0/528284526 conn(0x55e53c087660 msgr2=0x55e53c0a16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb599be4740 1 -- 172.21.15.44:0/528284526 shutdown_connections 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb599be4740 1 -- 172.21.15.44:0/528284526 wait complete. 2024-08-20T03:02:23.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb599be4740 20 client.15555 trim_cache size 0 max 16384 2024-08-20T03:02:23.103 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:02:23.103 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-20T03:02:23.103 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-08-20T03:02:23.103 INFO:teuthology.orchestra.run.smithi044.stdout: "8185fb9d-acac-4186-bf07-28ceccb34d6d": { 2024-08-20T03:02:23.103 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-08-20T03:02:23.103 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "8185fb9d-acac-4186-bf07-28ceccb34d6d", 2024-08-20T03:02:23.103 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-08-20T03:02:23.103 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:02:23.103 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:02:23.103 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:02:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb5977eb640 1 -- 172.21.15.44:0/3301129893 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb57c0b3d40 msgr2=0x7fb57c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb5977eb640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb57c0b3d40 0x7fb57c0b6200 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb588005fa0 tx=0x7fb588009360 comp rx=0 tx=0).stop 2024-08-20T03:02:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb5977eb640 1 -- 172.21.15.44:0/3301129893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb590102ab0 msgr2=0x7fb590198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb5977eb640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb590102ab0 0x7fb590198fb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00d950 tx=0x7fb58c00de10 comp rx=0 tx=0).stop 2024-08-20T03:02:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb5977eb640 1 -- 172.21.15.44:0/3301129893 shutdown_connections 2024-08-20T03:02:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb5977eb640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb57c0b3d40 0x7fb57c0b6200 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb5977eb640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb590108e40 0x7fb59019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb5977eb640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5901084c0 0x7fb5901994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb5977eb640 1 --2- 172.21.15.44:0/3301129893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb590102ab0 0x7fb590198fb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:23.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb5977eb640 1 -- 172.21.15.44:0/3301129893 >> 172.21.15.44:0/3301129893 conn(0x7fb5900fe500 msgr2=0x7fb5900fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:23.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb5977eb640 1 -- 172.21.15.44:0/3301129893 shutdown_connections 2024-08-20T03:02:23.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:23.094+0000 7fb5977eb640 1 -- 172.21.15.44:0/3301129893 wait complete. 2024-08-20T03:02:23.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:23 smithi044 ceph-mon[18790]: scrub queued for path: / 2024-08-20T03:02:23.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:23 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:02:23.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:23 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:02:23.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:23 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:02:23.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:23 smithi044 ceph-mon[18790]: from='client.15507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:23.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:23 smithi044 ceph-mon[18790]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:23.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:23 smithi165 ceph-mon[22357]: scrub queued for path: / 2024-08-20T03:02:23.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:23 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:02:23.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:23 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:02:23.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:23 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:02:23.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:23 smithi165 ceph-mon[22357]: from='client.15507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:23.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:23 smithi165 ceph-mon[22357]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:23.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:23 smithi099 ceph-mon[21984]: scrub queued for path: / 2024-08-20T03:02:23.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:23 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:02:23.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:23 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:02:23.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:23 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:02:23.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:23 smithi099 ceph-mon[21984]: from='client.15507 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:23.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:23 smithi099 ceph-mon[21984]: from='client.15531 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:23.919 INFO:tasks.cephfs.filesystem:scrub status for tag:8185fb9d-acac-4186-bf07-28ceccb34d6d - {'path': '/', 'tag': '8185fb9d-acac-4186-bf07-28ceccb34d6d', 'options': 'recursive,force'} 2024-08-20T03:02:24.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:24 smithi044 ceph-mon[18790]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:24.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:24 smithi044 ceph-mon[18790]: pgmap v317: 97 pgs: 97 active+clean; 1.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 103 KiB/s wr, 49 op/s 2024-08-20T03:02:24.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:24 smithi044 ceph-mon[18790]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-08-20T03:02:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:24 smithi165 ceph-mon[22357]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:24 smithi165 ceph-mon[22357]: pgmap v317: 97 pgs: 97 active+clean; 1.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 103 KiB/s wr, 49 op/s 2024-08-20T03:02:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:24 smithi165 ceph-mon[22357]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-08-20T03:02:24.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:24 smithi099 ceph-mon[21984]: from='client.15537 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:02:24.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:24 smithi099 ceph-mon[21984]: pgmap v317: 97 pgs: 97 active+clean; 1.0 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 103 KiB/s wr, 49 op/s 2024-08-20T03:02:24.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:24 smithi099 ceph-mon[21984]: mgrmap e25: x(active, since 8m), standbys: z, y 2024-08-20T03:02:26.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:26 smithi044 ceph-mon[18790]: pgmap v318: 97 pgs: 97 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 176 KiB/s wr, 55 op/s 2024-08-20T03:02:26.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:26 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:02:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:26 smithi165 ceph-mon[22357]: pgmap v318: 97 pgs: 97 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 176 KiB/s wr, 55 op/s 2024-08-20T03:02:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:26 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:02:26.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:26 smithi099 ceph-mon[21984]: pgmap v318: 97 pgs: 97 active+clean; 1.4 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 176 KiB/s wr, 55 op/s 2024-08-20T03:02:26.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:26 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:02:28.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:27 smithi044 ceph-mon[18790]: pgmap v319: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 176 KiB/s wr, 56 op/s 2024-08-20T03:02:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:02:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:02:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:27 smithi165 ceph-mon[22357]: pgmap v319: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 176 KiB/s wr, 56 op/s 2024-08-20T03:02:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:02:28.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:02:28.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:27 smithi099 ceph-mon[21984]: pgmap v319: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 176 KiB/s wr, 56 op/s 2024-08-20T03:02:28.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:28.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:02:28.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:02:28.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:02:30.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:29 smithi044 ceph-mon[18790]: pgmap v320: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-08-20T03:02:30.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:29 smithi165 ceph-mon[22357]: pgmap v320: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-08-20T03:02:30.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:29 smithi099 ceph-mon[21984]: pgmap v320: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-08-20T03:02:32.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:31 smithi044 ceph-mon[18790]: pgmap v321: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-08-20T03:02:32.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:31 smithi165 ceph-mon[22357]: pgmap v321: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-08-20T03:02:32.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:31 smithi099 ceph-mon[21984]: pgmap v321: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-08-20T03:02:34.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:33 smithi044 ceph-mon[18790]: pgmap v322: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-08-20T03:02:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:33 smithi165 ceph-mon[22357]: pgmap v322: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-08-20T03:02:34.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:33 smithi099 ceph-mon[21984]: pgmap v322: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-08-20T03:02:35.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:34 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:02:35.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:34 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:02:35.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:34 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:02:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.607+0000 7fc2d20af640 1 -- 172.21.15.44:0/2030151672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2cc101980 msgr2=0x7fc2cc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.607+0000 7fc2d20af640 1 --2- 172.21.15.44:0/2030151672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2cc101980 0x7fc2cc101d60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009900 tx=0x7fc2bc02a410 comp rx=0 tx=0).stop 2024-08-20T03:02:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.607+0000 7fc2d20af640 1 -- 172.21.15.44:0/2030151672 shutdown_connections 2024-08-20T03:02:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.607+0000 7fc2d20af640 1 --2- 172.21.15.44:0/2030151672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2cc107d00 0x7fc2cc10e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.607+0000 7fc2d20af640 1 --2- 172.21.15.44:0/2030151672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2cc107380 0x7fc2cc1077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.608+0000 7fc2d20af640 1 --2- 172.21.15.44:0/2030151672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2cc101980 0x7fc2cc101d60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.608+0000 7fc2d20af640 1 -- 172.21.15.44:0/2030151672 >> 172.21.15.44:0/2030151672 conn(0x7fc2cc0fd3c0 msgr2=0x7fc2cc0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.608+0000 7fc2d20af640 1 -- 172.21.15.44:0/2030151672 shutdown_connections 2024-08-20T03:02:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.608+0000 7fc2d20af640 1 -- 172.21.15.44:0/2030151672 wait complete. 2024-08-20T03:02:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.608+0000 7fc2d20af640 1 Processor -- start 2024-08-20T03:02:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.608+0000 7fc2d20af640 1 -- start start 2024-08-20T03:02:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2d20af640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2cc101980 0x7fc2cc197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2d20af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2cc107380 0x7fc2cc198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2d20af640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2cc107d00 0x7fc2cc19d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2d20af640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1119c0 con 0x7fc2cc101980 2024-08-20T03:02:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2d20af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc111840 con 0x7fc2cc107380 2024-08-20T03:02:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2d20af640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc111b40 con 0x7fc2cc107d00 2024-08-20T03:02:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2cb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2cc101980 0x7fc2cc197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2cb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2cc101980 0x7fc2cc197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53030/0 (socket says 172.21.15.44:53030) 2024-08-20T03:02:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/1916125759 learned_addr learned my addr 172.21.15.44:0/1916125759 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2caffd640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2cc107380 0x7fc2cc198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2cbfff640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2cc107d00 0x7fc2cc19d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/1916125759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2cc107d00 msgr2=0x7fc2cc19d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2cc107d00 0x7fc2cc19d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/1916125759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2cc107380 msgr2=0x7fc2cc198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2cc107380 0x7fc2cc198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/1916125759 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc19dfc0 con 0x7fc2cc101980 2024-08-20T03:02:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2cbfff640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2cc107d00 0x7fc2cc19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2cc101980 0x7fc2cc197f20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc005940 tx=0x7fc2bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.609+0000 7fc2c8ff9640 1 -- 172.21.15.44:0/1916125759 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc046070 con 0x7fc2cc101980 2024-08-20T03:02:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.610+0000 7fc2c8ff9640 1 -- 172.21.15.44:0/1916125759 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc003aa0 con 0x7fc2cc101980 2024-08-20T03:02:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.610+0000 7fc2d20af640 1 -- 172.21.15.44:0/1916125759 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc19e250 con 0x7fc2cc101980 2024-08-20T03:02:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.610+0000 7fc2c8ff9640 1 -- 172.21.15.44:0/1916125759 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc035050 con 0x7fc2cc101980 2024-08-20T03:02:35.615 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.610+0000 7fc2d20af640 1 -- 172.21.15.44:0/1916125759 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc06b010 con 0x7fc2cc101980 2024-08-20T03:02:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.612+0000 7fc2c8ff9640 1 -- 172.21.15.44:0/1916125759 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7fc2bc049070 con 0x7fc2cc101980 2024-08-20T03:02:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.612+0000 7fc2d4e9e740 1 Processor -- start 2024-08-20T03:02:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.612+0000 7fc2d4e9e740 1 -- start start 2024-08-20T03:02:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2d4e9e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2d4e9e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791a5270 0x5628791b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2d4e9e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791b5610 0x5628791b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2d4e9e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56287911a360 con 0x5628791a4950 2024-08-20T03:02:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2d4e9e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562879109860 con 0x5628791b5610 2024-08-20T03:02:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2d4e9e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56287910d8a0 con 0x5628791a5270 2024-08-20T03:02:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2cbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2cbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791a4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53044/0 (socket says 172.21.15.44:53044) 2024-08-20T03:02:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2cbfff640 1 -- 172.21.15.44:0/2070559900 learned_addr learned my addr 172.21.15.44:0/2070559900 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2caffd640 1 --2- 172.21.15.44:0/2070559900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791b5610 0x5628791b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/2070559900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791a5270 0x5628791b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2c8ff9640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc2b40b3d40 0x7fc2b40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2caffd640 1 -- 172.21.15.44:0/2070559900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791a5270 msgr2=0x5628791b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2caffd640 1 --2- 172.21.15.44:0/2070559900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791a5270 0x5628791b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2caffd640 1 -- 172.21.15.44:0/2070559900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 msgr2=0x5628791a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2caffd640 1 --2- 172.21.15.44:0/2070559900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2caffd640 1 -- 172.21.15.44:0/2070559900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628790b0e40 con 0x5628791b5610 2024-08-20T03:02:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2cbfff640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc2b40b3d40 0x7fc2b40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2caffd640 1 --2- 172.21.15.44:0/2070559900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791b5610 0x5628791b7a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009bf0 tx=0x7fc2c0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d62de9eee7fe5845 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.613+0000 7fc2c8ff9640 1 -- 172.21.15.44:0/1916125759 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc2bc105ac0 con 0x7fc2cc101980 2024-08-20T03:02:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.614+0000 7fc2b1ffb640 1 -- 172.21.15.44:0/2070559900 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0004050 con 0x5628791b5610 2024-08-20T03:02:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.614+0000 7fc2b1ffb640 1 -- 172.21.15.44:0/2070559900 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c00054d0 con 0x5628791b5610 2024-08-20T03:02:35.623 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.614+0000 7fc2b1ffb640 1 -- 172.21.15.44:0/2070559900 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0004cd0 con 0x5628791b5610 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.614+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/2070559900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791b5610 msgr2=0x5628791b7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.614+0000 7fc2cbfff640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc2b40b3d40 0x7fc2b40b6200 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40092f0 tx=0x7fc2c4004a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.614+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/2070559900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791b5610 0x5628791b7a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009bf0 tx=0x7fc2c0011810 comp rx=0 tx=0).stop 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.614+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/2070559900 shutdown_connections 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.614+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/2070559900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791b5610 0x5628791b7a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.614+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/2070559900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791a5270 0x5628791b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.614+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/2070559900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.614+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/2070559900 >> 172.21.15.44:0/2070559900 conn(0x5628791a2730 msgr2=0x5628791a3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.615+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/2070559900 shutdown_connections 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.615+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/2070559900 wait complete. 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.615+0000 7fc2d4e9e740 1 -- start start 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.615+0000 7fc2d4e9e740 1 Processor -- start 2024-08-20T03:02:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.615+0000 7fc2d4e9e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.615+0000 7fc2d4e9e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.615+0000 7fc2d4e9e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.615+0000 7fc2d4e9e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562879042ab0 con 0x5628791a4950 2024-08-20T03:02:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.615+0000 7fc2d4e9e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562879090780 con 0x5628791a5270 2024-08-20T03:02:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.615+0000 7fc2d4e9e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5628791b82f0 con 0x5628791b5610 2024-08-20T03:02:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.615+0000 7fc2d4e9e740 10 client.0 _ll_register_callbacks cb 0x562878e18a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2cb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2cb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53054/0 (socket says 172.21.15.44:53054) 2024-08-20T03:02:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/207876021 learned_addr learned my addr 172.21.15.44:0/207876021 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2caffd640 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2cbfff640 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.625 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/207876021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 msgr2=0x5628791b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/207876021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 msgr2=0x5628791b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/207876021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562878e6c790 con 0x5628791a4950 2024-08-20T03:02:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2cbfff640 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2caffd640 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791b4f40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0cc6d0 tx=0x7fc2bc107040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165cbc4aeb292745 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2aaffd640 1 -- 172.21.15.44:0/207876021 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0c6e60 con 0x5628791a4950 2024-08-20T03:02:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2aaffd640 1 -- 172.21.15.44:0/207876021 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc2bc0ccba0 con 0x5628791a4950 2024-08-20T03:02:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/207876021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5628790b8e00 con 0x5628791a4950 2024-08-20T03:02:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2aaffd640 1 -- 172.21.15.44:0/207876021 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc2bc0cc510 con 0x5628791a4950 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2d4e9e740 10 client.15567 fetch_fsmap learned FSMap version 28 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.616+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/207876021 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562878ee5620 con 0x5628791a4950 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.617+0000 7fc2aaffd640 1 -- 172.21.15.44:0/207876021 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5628790b8e00 con 0x5628791a4950 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.617+0000 7fc2aaffd640 1 -- 172.21.15.44:0/207876021 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc2bc0d5360 con 0x5628791a4950 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.617+0000 7fc2d4e9e740 10 client.15567 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.617+0000 7fc2d4e9e740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.617+0000 7fc2d4e9e740 20 client.15567 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.617+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5628791c4f40 0x5628791c7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.617+0000 7fc2d4e9e740 4 client.15567 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.617+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/207876021 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56287911c2b0 con 0x5628791c4f40 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.617+0000 7fc2caffd640 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5628791c4f40 0x5628791c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.618+0000 7fc2caffd640 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5628791c4f40 0x5628791c7370 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f71cc24725c6404 server_cookie=56d113389546d77b in_seq=0 out_seq=0 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.618+0000 7fc2aaffd640 10 client.15567 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.618+0000 7fc2aaffd640 1 -- 172.21.15.44:0/207876021 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56287911c2b0 con 0x5628791c4f40 2024-08-20T03:02:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.618+0000 7fc2aaffd640 10 client.15567 handle_command_reply: tid=0 2024-08-20T03:02:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.618+0000 7fc2d4e9e740 1 client.15567 shutdown 2024-08-20T03:02:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.619+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/207876021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 msgr2=0x5628791b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.619+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791b4f40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0cc6d0 tx=0x7fc2bc107040 comp rx=0 tx=0).stop 2024-08-20T03:02:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.619+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/207876021 shutdown_connections 2024-08-20T03:02:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.619+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5628791c4f40 0x5628791c7370 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.619+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.619+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.619+0000 7fc2aaffd640 0 client.15567 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.619+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/207876021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791b4f40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.619+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/207876021 >> 172.21.15.44:0/207876021 conn(0x5628791a2730 msgr2=0x5628791a8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.619+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/207876021 shutdown_connections 2024-08-20T03:02:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.619+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/207876021 wait complete. 2024-08-20T03:02:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.619+0000 7fc2d4e9e740 20 client.15567 trim_cache size 0 max 16384 2024-08-20T03:02:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.633+0000 7fc2d4e9e740 1 Processor -- start 2024-08-20T03:02:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.633+0000 7fc2d4e9e740 1 -- start start 2024-08-20T03:02:35.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.633+0000 7fc2d4e9e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791c1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.633+0000 7fc2d4e9e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.633+0000 7fc2d4e9e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791a1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.633+0000 7fc2d4e9e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1abf90 con 0x5628791a4950 2024-08-20T03:02:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.633+0000 7fc2d4e9e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1adad0 con 0x5628791a5270 2024-08-20T03:02:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.633+0000 7fc2d4e9e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1ac660 con 0x5628791b5610 2024-08-20T03:02:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.633+0000 7fc2cb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.633+0000 7fc2cb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791c1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53064/0 (socket says 172.21.15.44:53064) 2024-08-20T03:02:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.633+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/2771745476 learned_addr learned my addr 172.21.15.44:0/2771745476 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2caffd640 1 --2- 172.21.15.44:0/2771745476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2cbfff640 1 --2- 172.21.15.44:0/2771745476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/2771745476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 msgr2=0x5628791a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/2771745476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/2771745476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 msgr2=0x5628791b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/2771745476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/2771745476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628791a50a0 con 0x5628791a4950 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2cbfff640 1 --2- 172.21.15.44:0/2771745476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2caffd640 1 --2- 172.21.15.44:0/2771745476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/2771745476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791c1410 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0cc440 tx=0x7fc2bc105e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2175e1bcfe979d41 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2b17fa640 1 -- 172.21.15.44:0/2771745476 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0cbeb0 con 0x5628791a4950 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2b17fa640 1 -- 172.21.15.44:0/2771745476 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc0d5130 con 0x5628791a4950 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.634+0000 7fc2b17fa640 1 -- 172.21.15.44:0/2771745476 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0cc090 con 0x5628791a4950 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.635+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/2771745476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 msgr2=0x5628791c1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.635+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/2771745476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791c1410 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0cc440 tx=0x7fc2bc105e50 comp rx=0 tx=0).stop 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.635+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/2771745476 shutdown_connections 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.635+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/2771745476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.635+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/2771745476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.635+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/2771745476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791c1410 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.635+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/2771745476 >> 172.21.15.44:0/2771745476 conn(0x5628791a2730 msgr2=0x5628791a3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.635+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/2771745476 shutdown_connections 2024-08-20T03:02:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.635+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/2771745476 wait complete. 2024-08-20T03:02:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.635+0000 7fc2d4e9e740 1 -- start start 2024-08-20T03:02:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.635+0000 7fc2d4e9e740 1 Processor -- start 2024-08-20T03:02:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2d4e9e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791bd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2d4e9e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2d4e9e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2d4e9e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1ad400 con 0x5628791a4950 2024-08-20T03:02:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2d4e9e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1acd30 con 0x5628791a5270 2024-08-20T03:02:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2d4e9e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1a9f70 con 0x5628791b5610 2024-08-20T03:02:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2d4e9e740 10 client.0 _ll_register_callbacks cb 0x7fc2bc109220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2cb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2cb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791bd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:53078/0 (socket says 172.21.15.44:53078) 2024-08-20T03:02:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/218025225 learned_addr learned my addr 172.21.15.44:0/218025225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:35.643 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2caffd640 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2cbfff640 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/218025225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 msgr2=0x5628791cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/218025225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 msgr2=0x5628791bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791bdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2cb7fe640 1 -- 172.21.15.44:0/218025225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628791cc720 con 0x5628791a4950 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2caffd640 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2cb7fe640 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791bd580 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc03a040 tx=0x7fc2bc0d0d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201bbc88797ebb8b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.636+0000 7fc2aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2aaffd640 1 -- 172.21.15.44:0/218025225 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0c6e60 con 0x5628791a4950 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2aaffd640 1 -- 172.21.15.44:0/218025225 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fc2bc0cb9a0 con 0x5628791a4950 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/218025225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562879117d50 con 0x5628791a4950 2024-08-20T03:02:35.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2aaffd640 1 -- 172.21.15.44:0/218025225 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc2bc0cbeb0 con 0x5628791a4950 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2d4e9e740 10 client.15573 fetch_fsmap learned FSMap version 28 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/218025225 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5628791cc720 con 0x5628791a4950 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2aaffd640 1 -- 172.21.15.44:0/218025225 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562879117d50 con 0x5628791a4950 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2aaffd640 1 -- 172.21.15.44:0/218025225 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc2bc031ac0 con 0x5628791a4950 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2d4e9e740 10 client.15573 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2d4e9e740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2d4e9e740 20 client.15573 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56287922f840 0x562879231c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2d4e9e740 4 client.15573 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.637+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/218025225 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562879115490 con 0x56287922f840 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.638+0000 7fc2caffd640 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56287922f840 0x562879231c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.638+0000 7fc2aaffd640 10 client.15573 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:35.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.638+0000 7fc2caffd640 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56287922f840 0x562879231c60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d6b92878d1ed6c server_cookie=808f7d241685e7d2 in_seq=0 out_seq=0 2024-08-20T03:02:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2aaffd640 1 -- 172.21.15.44:0/218025225 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562879115490 con 0x56287922f840 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2aaffd640 10 client.15573 handle_command_reply: tid=0 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2d4e9e740 1 client.15573 shutdown 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/218025225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 msgr2=0x5628791bd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791bd580 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc03a040 tx=0x7fc2bc0d0d40 comp rx=0 tx=0).stop 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/218025225 shutdown_connections 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56287922f840 0x562879231c60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5628791b5610 0x5628791cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5628791a5270 0x5628791bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2d4e9e740 1 --2- 172.21.15.44:0/218025225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5628791a4950 0x5628791bd580 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/218025225 >> 172.21.15.44:0/218025225 conn(0x5628791a2730 msgr2=0x5628791a1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2aaffd640 0 client.15573 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/218025225 shutdown_connections 2024-08-20T03:02:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2d4e9e740 1 -- 172.21.15.44:0/218025225 wait complete. 2024-08-20T03:02:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.651+0000 7fc2d4e9e740 20 client.15573 trim_cache size 0 max 16384 2024-08-20T03:02:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:02:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:02:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:02:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:02:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.652+0000 7fc2d20af640 1 -- 172.21.15.44:0/1916125759 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc2b40b3d40 msgr2=0x7fc2b40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.652+0000 7fc2d20af640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc2b40b3d40 0x7fc2b40b6200 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40092f0 tx=0x7fc2c4004a40 comp rx=0 tx=0).stop 2024-08-20T03:02:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.652+0000 7fc2d20af640 1 -- 172.21.15.44:0/1916125759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2cc101980 msgr2=0x7fc2cc197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.652+0000 7fc2d20af640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2cc101980 0x7fc2cc197f20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc005940 tx=0x7fc2bc004270 comp rx=0 tx=0).stop 2024-08-20T03:02:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.652+0000 7fc2d20af640 1 -- 172.21.15.44:0/1916125759 shutdown_connections 2024-08-20T03:02:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.652+0000 7fc2d20af640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc2b40b3d40 0x7fc2b40b6200 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.652+0000 7fc2d20af640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2cc107d00 0x7fc2cc19d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.652+0000 7fc2d20af640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2cc107380 0x7fc2cc198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.652+0000 7fc2d20af640 1 --2- 172.21.15.44:0/1916125759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2cc101980 0x7fc2cc197f20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.652+0000 7fc2d20af640 1 -- 172.21.15.44:0/1916125759 >> 172.21.15.44:0/1916125759 conn(0x7fc2cc0fd3c0 msgr2=0x7fc2cc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.652+0000 7fc2d20af640 1 -- 172.21.15.44:0/1916125759 shutdown_connections 2024-08-20T03:02:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:35.652+0000 7fc2d20af640 1 -- 172.21.15.44:0/1916125759 wait complete. 2024-08-20T03:02:36.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:35 smithi044 ceph-mon[18790]: pgmap v323: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-08-20T03:02:36.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:35 smithi165 ceph-mon[22357]: pgmap v323: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-08-20T03:02:36.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:35 smithi099 ceph-mon[21984]: pgmap v323: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-08-20T03:02:38.038 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:37 smithi044 ceph-mon[18790]: pgmap v324: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-08-20T03:02:38.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:37 smithi165 ceph-mon[22357]: pgmap v324: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-08-20T03:02:38.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:37 smithi099 ceph-mon[21984]: pgmap v324: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-08-20T03:02:40.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:39 smithi044 ceph-mon[18790]: pgmap v325: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 15 op/s 2024-08-20T03:02:40.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:39 smithi165 ceph-mon[22357]: pgmap v325: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 15 op/s 2024-08-20T03:02:40.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:39 smithi099 ceph-mon[21984]: pgmap v325: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.9 KiB/s wr, 15 op/s 2024-08-20T03:02:42.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:41 smithi044 ceph-mon[18790]: pgmap v326: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-08-20T03:02:42.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:41 smithi165 ceph-mon[22357]: pgmap v326: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-08-20T03:02:42.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:41 smithi099 ceph-mon[21984]: pgmap v326: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-08-20T03:02:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:43 smithi165 ceph-mon[22357]: pgmap v327: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2024-08-20T03:02:44.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:43 smithi099 ceph-mon[21984]: pgmap v327: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2024-08-20T03:02:44.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:43 smithi044 ceph-mon[18790]: pgmap v327: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2024-08-20T03:02:46.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:45 smithi165 ceph-mon[22357]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-08-20T03:02:46.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:45 smithi099 ceph-mon[21984]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-08-20T03:02:46.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:45 smithi044 ceph-mon[18790]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-08-20T03:02:48.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:47 smithi165 ceph-mon[22357]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-08-20T03:02:48.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:47 smithi099 ceph-mon[21984]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-08-20T03:02:48.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:47 smithi044 ceph-mon[18790]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.0 KiB/s wr, 14 op/s 2024-08-20T03:02:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.787+0000 7f3391fa0640 1 -- 172.21.15.44:0/4025805606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f338c104890 msgr2=0x7f338c10e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.787+0000 7f3391fa0640 1 --2- 172.21.15.44:0/4025805606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f338c104890 0x7f338c10e2b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3384009ca0 tx=0x7f338402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.787+0000 7f3391fa0640 1 -- 172.21.15.44:0/4025805606 shutdown_connections 2024-08-20T03:02:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.787+0000 7f3391fa0640 1 --2- 172.21.15.44:0/4025805606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f338c104890 0x7f338c10e2b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.787+0000 7f3391fa0640 1 --2- 172.21.15.44:0/4025805606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f338c103ef0 0x7f338c104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.787+0000 7f3391fa0640 1 --2- 172.21.15.44:0/4025805606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f338c101b30 0x7f338c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.787+0000 7f3391fa0640 1 -- 172.21.15.44:0/4025805606 >> 172.21.15.44:0/4025805606 conn(0x7f338c077270 msgr2=0x7f338c077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.788+0000 7f3391fa0640 1 -- 172.21.15.44:0/4025805606 shutdown_connections 2024-08-20T03:02:48.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.788+0000 7f3391fa0640 1 -- 172.21.15.44:0/4025805606 wait complete. 2024-08-20T03:02:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.788+0000 7f3391fa0640 1 Processor -- start 2024-08-20T03:02:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3391fa0640 1 -- start start 2024-08-20T03:02:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3391fa0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f338c101b30 0x7f338c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3391fa0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f338c103ef0 0x7f338c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3391fa0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f338c104890 0x7f338c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3391fa0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f338c111690 con 0x7f338c101b30 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3391fa0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f338c111510 con 0x7f338c103ef0 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3391fa0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f338c111810 con 0x7f338c104890 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3390f9e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f338c101b30 0x7f338c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3390f9e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f338c101b30 0x7f338c198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38696/0 (socket says 172.21.15.44:38696) 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3390f9e640 1 -- 172.21.15.44:0/4157474298 learned_addr learned my addr 172.21.15.44:0/4157474298 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f338bfff640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f338c103ef0 0x7f338c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f339179f640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f338c104890 0x7f338c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3390f9e640 1 -- 172.21.15.44:0/4157474298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f338c104890 msgr2=0x7f338c19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3390f9e640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f338c104890 0x7f338c19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3390f9e640 1 -- 172.21.15.44:0/4157474298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f338c103ef0 msgr2=0x7f338c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3390f9e640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f338c103ef0 0x7f338c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f3390f9e640 1 -- 172.21.15.44:0/4157474298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c19edd0 con 0x7f338c101b30 2024-08-20T03:02:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.789+0000 7f338bfff640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f338c103ef0 0x7f338c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.790+0000 7f3390f9e640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f338c101b30 0x7f338c198d30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f338000e9f0 tx=0x7f338000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.790+0000 7f3389ffb640 1 -- 172.21.15.44:0/4157474298 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338000cd80 con 0x7f338c101b30 2024-08-20T03:02:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.790+0000 7f3389ffb640 1 -- 172.21.15.44:0/4157474298 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33800108e0 con 0x7f338c101b30 2024-08-20T03:02:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.790+0000 7f3389ffb640 1 -- 172.21.15.44:0/4157474298 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380018980 con 0x7f338c101b30 2024-08-20T03:02:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.790+0000 7f3391fa0640 1 -- 172.21.15.44:0/4157474298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338c19f0c0 con 0x7f338c101b30 2024-08-20T03:02:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.791+0000 7f3391fa0640 1 -- 172.21.15.44:0/4157474298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338c1a5960 con 0x7f338c101b30 2024-08-20T03:02:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.794+0000 7f3389ffb640 1 -- 172.21.15.44:0/4157474298 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f3380010430 con 0x7f338c101b30 2024-08-20T03:02:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.794+0000 7f3395ff8740 1 Processor -- start 2024-08-20T03:02:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.794+0000 7f3395ff8740 1 -- start start 2024-08-20T03:02:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.795+0000 7f3395ff8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157517120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.795+0000 7f3395ff8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157517660 0x5611575230b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.795+0000 7f3395ff8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5611575235f0 0x5611575259e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.795+0000 7f3395ff8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56115748c360 con 0x561157516d40 2024-08-20T03:02:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.795+0000 7f3395ff8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56115747b860 con 0x561157517660 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.795+0000 7f3395ff8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56115747f8a0 con 0x5611575235f0 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.795+0000 7f3390f9e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157517660 0x5611575230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.795+0000 7f338bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5611575235f0 0x5611575259e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.795+0000 7f339179f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157517120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.795+0000 7f338bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5611575235f0 0x5611575259e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:51362/0 (socket says 172.21.15.44:51362) 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.795+0000 7f338bfff640 1 -- 172.21.15.44:0/2166265033 learned_addr learned my addr 172.21.15.44:0/2166265033 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.796+0000 7f338bfff640 1 -- 172.21.15.44:0/2166265033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157517660 msgr2=0x5611575230b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.796+0000 7f338bfff640 1 --2- 172.21.15.44:0/2166265033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157517660 0x5611575230b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.796+0000 7f338bfff640 1 -- 172.21.15.44:0/2166265033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 msgr2=0x561157517120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.796+0000 7f338bfff640 1 --2- 172.21.15.44:0/2166265033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157517120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.796+0000 7f338bfff640 1 -- 172.21.15.44:0/2166265033 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561157422e40 con 0x5611575235f0 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.796+0000 7f339179f640 1 --2- 172.21.15.44:0/2166265033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157517120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.796+0000 7f3390f9e640 1 --2- 172.21.15.44:0/2166265033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157517660 0x5611575230b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.796+0000 7f3389ffb640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f337c0b3e40 0x7f337c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.796+0000 7f338bfff640 1 --2- 172.21.15.44:0/2166265033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5611575235f0 0x5611575259e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f337800cbd0 tx=0x7f337800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc774e6fc7a36f02 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.796+0000 7f339179f640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f337c0b3e40 0x7f337c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.796+0000 7f3389ffb640 1 -- 172.21.15.44:0/4157474298 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f33800e46e0 con 0x7f338c101b30 2024-08-20T03:02:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.797+0000 7f339179f640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f337c0b3e40 0x7f337c0b6300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3384009a60 tx=0x7f3384005c30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.797+0000 7f3376ffd640 1 -- 172.21.15.44:0/2166265033 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378005220 con 0x5611575235f0 2024-08-20T03:02:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.797+0000 7f3376ffd640 1 -- 172.21.15.44:0/2166265033 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3378004c50 con 0x5611575235f0 2024-08-20T03:02:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.797+0000 7f3376ffd640 1 -- 172.21.15.44:0/2166265033 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378010770 con 0x5611575235f0 2024-08-20T03:02:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.798+0000 7f3395ff8740 1 -- 172.21.15.44:0/2166265033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5611575235f0 msgr2=0x5611575259e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.798+0000 7f3395ff8740 1 --2- 172.21.15.44:0/2166265033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5611575235f0 0x5611575259e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f337800cbd0 tx=0x7f337800e720 comp rx=0 tx=0).stop 2024-08-20T03:02:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.798+0000 7f3395ff8740 1 -- 172.21.15.44:0/2166265033 shutdown_connections 2024-08-20T03:02:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.798+0000 7f3395ff8740 1 --2- 172.21.15.44:0/2166265033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5611575235f0 0x5611575259e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.798+0000 7f3395ff8740 1 --2- 172.21.15.44:0/2166265033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157517660 0x5611575230b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.798+0000 7f3395ff8740 1 --2- 172.21.15.44:0/2166265033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157517120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.798+0000 7f3395ff8740 1 -- 172.21.15.44:0/2166265033 >> 172.21.15.44:0/2166265033 conn(0x561157514b70 msgr2=0x561157515d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.798+0000 7f3395ff8740 1 -- 172.21.15.44:0/2166265033 shutdown_connections 2024-08-20T03:02:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.798+0000 7f3395ff8740 1 -- 172.21.15.44:0/2166265033 wait complete. 2024-08-20T03:02:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.799+0000 7f3395ff8740 1 -- start start 2024-08-20T03:02:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.799+0000 7f3395ff8740 1 Processor -- start 2024-08-20T03:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.799+0000 7f3395ff8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157516d40 0x561157522e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f3395ff8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561157517660 0x561157520260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f3395ff8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611575235f0 0x5611575207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f338bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561157517660 0x561157520260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f3390f9e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157516d40 0x561157522e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f3390f9e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157516d40 0x561157522e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55240/0 (socket says 172.21.15.44:55240) 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f338bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561157517660 0x561157520260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:51376/0 (socket says 172.21.15.44:51376) 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f339179f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611575235f0 0x5611575207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f339179f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611575235f0 0x5611575207a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38718/0 (socket says 172.21.15.44:38718) 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f3390f9e640 1 -- 172.21.15.44:0/3621016820 learned_addr learned my addr 172.21.15.44:0/3621016820 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f3395ff8740 1 -- 172.21.15.44:0/3621016820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5611573b4ab0 con 0x5611575235f0 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f3395ff8740 1 -- 172.21.15.44:0/3621016820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561157402780 con 0x561157516d40 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f3395ff8740 1 -- 172.21.15.44:0/3621016820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56115752a6e0 con 0x561157517660 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f3390f9e640 1 -- 172.21.15.44:0/3621016820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561157517660 msgr2=0x561157520260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f3390f9e640 1 --2- 172.21.15.44:0/3621016820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561157517660 0x561157520260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.800+0000 7f3390f9e640 1 -- 172.21.15.44:0/3621016820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611575235f0 msgr2=0x5611575207a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:48.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.801+0000 7f3390f9e640 1 --2- 172.21.15.44:0/3621016820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611575235f0 0x5611575207a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.801+0000 7f3390f9e640 1 -- 172.21.15.44:0/3621016820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611571de790 con 0x561157516d40 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.801+0000 7f339179f640 1 --2- 172.21.15.44:0/3621016820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611575235f0 0x5611575207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.801+0000 7f3395ff8740 10 client.0 _ll_register_callbacks cb 0x56115718aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.801+0000 7f3395ff8740 1 -- 172.21.15.44:0/3621016820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56115742ae00 con 0x561157516d40 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.801+0000 7f3390f9e640 1 --2- 172.21.15.44:0/3621016820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157516d40 0x561157522e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f33800ab000 tx=0x7f33800ea300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68dcb1a43eb3f936 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.801+0000 7f336bfff640 10 client.25136 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.801+0000 7f336bfff640 1 -- 172.21.15.44:0/3621016820 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380010720 con 0x561157516d40 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.801+0000 7f336bfff640 1 -- 172.21.15.44:0/3621016820 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f33800b4130 con 0x561157516d40 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.801+0000 7f336bfff640 1 -- 172.21.15.44:0/3621016820 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33800aae10 con 0x561157516d40 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.801+0000 7f3395ff8740 10 client.25136 fetch_fsmap learned FSMap version 28 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.801+0000 7f3395ff8740 1 -- 172.21.15.44:0/3621016820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=28}) -- 0x561157257620 con 0x561157516d40 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.802+0000 7f336bfff640 1 -- 172.21.15.44:0/3621016820 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f33800aa560 con 0x561157516d40 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.802+0000 7f3395ff8740 10 client.25136 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:02:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.802+0000 7f3395ff8740 10 client.25136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.802+0000 7f3395ff8740 20 client.25136 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.802+0000 7f3395ff8740 1 --2- 172.21.15.44:0/3621016820 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561157537100 0x561157539590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.802+0000 7f3395ff8740 4 client.25136 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.802+0000 7f3395ff8740 1 -- 172.21.15.44:0/3621016820 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56115748e2b0 con 0x561157537100 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.802+0000 7f338bfff640 1 --2- 172.21.15.44:0/3621016820 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561157537100 0x561157539590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.803+0000 7f338bfff640 1 --2- 172.21.15.44:0/3621016820 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561157537100 0x561157539590 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2908d4fbe18b323c server_cookie=ca18391350389847 in_seq=0 out_seq=0 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.803+0000 7f336bfff640 10 client.25136 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.803+0000 7f336bfff640 1 -- 172.21.15.44:0/3621016820 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56115748e2b0 con 0x561157537100 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.803+0000 7f336bfff640 10 client.25136 handle_command_reply: tid=0 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.803+0000 7f3395ff8740 1 client.25136 shutdown 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.803+0000 7f3395ff8740 1 -- 172.21.15.44:0/3621016820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157516d40 msgr2=0x561157522e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.803+0000 7f3395ff8740 1 --2- 172.21.15.44:0/3621016820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157516d40 0x561157522e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f33800ab000 tx=0x7f33800ea300 comp rx=0 tx=0).stop 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.803+0000 7f3395ff8740 1 -- 172.21.15.44:0/3621016820 shutdown_connections 2024-08-20T03:02:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.803+0000 7f3395ff8740 1 --2- 172.21.15.44:0/3621016820 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561157537100 0x561157539590 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.804+0000 7f3395ff8740 1 --2- 172.21.15.44:0/3621016820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611575235f0 0x5611575207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.804+0000 7f336bfff640 0 client.25136 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.804+0000 7f3395ff8740 1 --2- 172.21.15.44:0/3621016820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561157517660 0x561157520260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.804+0000 7f3395ff8740 1 --2- 172.21.15.44:0/3621016820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157516d40 0x561157522e20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.804+0000 7f3395ff8740 1 -- 172.21.15.44:0/3621016820 >> 172.21.15.44:0/3621016820 conn(0x561157514b70 msgr2=0x56115751a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.804+0000 7f3395ff8740 1 -- 172.21.15.44:0/3621016820 shutdown_connections 2024-08-20T03:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.804+0000 7f3395ff8740 1 -- 172.21.15.44:0/3621016820 wait complete. 2024-08-20T03:02:48.810 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.804+0000 7f3395ff8740 20 client.25136 trim_cache size 0 max 16384 2024-08-20T03:02:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.833+0000 7f3395ff8740 1 Processor -- start 2024-08-20T03:02:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.833+0000 7f3395ff8740 1 -- start start 2024-08-20T03:02:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.833+0000 7f3395ff8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157533890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.833+0000 7f3395ff8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157517660 0x561157514420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.833+0000 7f3395ff8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5611575235f0 0x561157520230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.833+0000 7f3390f9e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157533890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.833+0000 7f3390f9e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157533890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38724/0 (socket says 172.21.15.44:38724) 2024-08-20T03:02:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.833+0000 7f3390f9e640 1 -- 172.21.15.44:0/3614040334 learned_addr learned my addr 172.21.15.44:0/3614040334 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.833+0000 7f3395ff8740 1 -- 172.21.15.44:0/3614040334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1abd90 con 0x561157516d40 2024-08-20T03:02:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.833+0000 7f3395ff8740 1 -- 172.21.15.44:0/3614040334 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1ad8d0 con 0x561157517660 2024-08-20T03:02:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.833+0000 7f3395ff8740 1 -- 172.21.15.44:0/3614040334 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1ac460 con 0x5611575235f0 2024-08-20T03:02:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.834+0000 7f338bfff640 1 --2- 172.21.15.44:0/3614040334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157517660 0x561157514420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.834+0000 7f339179f640 1 --2- 172.21.15.44:0/3614040334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5611575235f0 0x561157520230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.834+0000 7f3390f9e640 1 -- 172.21.15.44:0/3614040334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5611575235f0 msgr2=0x561157520230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.834+0000 7f3390f9e640 1 --2- 172.21.15.44:0/3614040334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5611575235f0 0x561157520230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.834+0000 7f3390f9e640 1 -- 172.21.15.44:0/3614040334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157517660 msgr2=0x561157514420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.834+0000 7f3390f9e640 1 --2- 172.21.15.44:0/3614040334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157517660 0x561157514420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.834+0000 7f3390f9e640 1 -- 172.21.15.44:0/3614040334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561157517490 con 0x561157516d40 2024-08-20T03:02:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.834+0000 7f338bfff640 1 --2- 172.21.15.44:0/3614040334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157517660 0x561157514420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.834+0000 7f3390f9e640 1 --2- 172.21.15.44:0/3614040334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157533890 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f33800097b0 tx=0x7f33800cd000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d57881365f001f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.834+0000 7f33767fc640 1 -- 172.21.15.44:0/3614040334 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33800ab300 con 0x561157516d40 2024-08-20T03:02:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.835+0000 7f33767fc640 1 -- 172.21.15.44:0/3614040334 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56115742ae00 con 0x561157516d40 2024-08-20T03:02:48.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.835+0000 7f33767fc640 1 -- 172.21.15.44:0/3614040334 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33800cc5e0 con 0x561157516d40 2024-08-20T03:02:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.835+0000 7f3395ff8740 1 -- 172.21.15.44:0/3614040334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 msgr2=0x561157533890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.835+0000 7f3395ff8740 1 --2- 172.21.15.44:0/3614040334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157533890 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f33800097b0 tx=0x7f33800cd000 comp rx=0 tx=0).stop 2024-08-20T03:02:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.836+0000 7f3395ff8740 1 -- 172.21.15.44:0/3614040334 shutdown_connections 2024-08-20T03:02:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.836+0000 7f3395ff8740 1 --2- 172.21.15.44:0/3614040334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5611575235f0 0x561157520230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.836+0000 7f3395ff8740 1 --2- 172.21.15.44:0/3614040334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561157517660 0x561157514420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.836+0000 7f3395ff8740 1 --2- 172.21.15.44:0/3614040334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157533890 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.836+0000 7f3395ff8740 1 -- 172.21.15.44:0/3614040334 >> 172.21.15.44:0/3614040334 conn(0x561157514b70 msgr2=0x56115751eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.836+0000 7f3395ff8740 1 -- 172.21.15.44:0/3614040334 shutdown_connections 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.836+0000 7f3395ff8740 1 -- 172.21.15.44:0/3614040334 wait complete. 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.836+0000 7f3395ff8740 1 -- start start 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.836+0000 7f3395ff8740 1 Processor -- start 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.837+0000 7f3395ff8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157530200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.837+0000 7f3395ff8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561157517660 0x56115752b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.837+0000 7f3395ff8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5611575235f0 0x56115752bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.837+0000 7f3395ff8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1ad200 con 0x561157516d40 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.837+0000 7f3395ff8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1acb30 con 0x5611575235f0 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.837+0000 7f3395ff8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f338c1a9d90 con 0x561157517660 2024-08-20T03:02:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.837+0000 7f3390f9e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157530200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.837+0000 7f3395ff8740 10 client.0 _ll_register_callbacks cb 0x7f33800ab0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.837+0000 7f3390f9e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157530200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38732/0 (socket says 172.21.15.44:38732) 2024-08-20T03:02:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.837+0000 7f3390f9e640 1 -- 172.21.15.44:0/1998559174 learned_addr learned my addr 172.21.15.44:0/1998559174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.837+0000 7f339179f640 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5611575235f0 0x56115752bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.837+0000 7f338bfff640 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561157517660 0x56115752b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.838+0000 7f338bfff640 1 -- 172.21.15.44:0/1998559174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5611575235f0 msgr2=0x56115752bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.838+0000 7f338bfff640 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5611575235f0 0x56115752bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.838+0000 7f338bfff640 1 -- 172.21.15.44:0/1998559174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 msgr2=0x561157530200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.838+0000 7f338bfff640 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157530200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.838+0000 7f338bfff640 1 -- 172.21.15.44:0/1998559174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611575a0e70 con 0x561157517660 2024-08-20T03:02:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.838+0000 7f3390f9e640 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157530200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.838+0000 7f339179f640 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5611575235f0 0x56115752bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.838+0000 7f338bfff640 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561157517660 0x56115752b6c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3378005700 tx=0x7f337800c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62f4e29beaf92be server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.838+0000 7f336bfff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.838+0000 7f336bfff640 1 -- 172.21.15.44:0/1998559174 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378005450 con 0x561157517660 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.838+0000 7f336bfff640 1 -- 172.21.15.44:0/1998559174 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3378004020 con 0x561157517660 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.838+0000 7f3395ff8740 1 -- 172.21.15.44:0/1998559174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56115742ae00 con 0x561157517660 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.839+0000 7f336bfff640 1 -- 172.21.15.44:0/1998559174 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3378037050 con 0x561157517660 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.839+0000 7f3395ff8740 10 client.25129 fetch_fsmap learned FSMap version 28 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.839+0000 7f3395ff8740 1 -- 172.21.15.44:0/1998559174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611575a2040 con 0x561157517660 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.839+0000 7f336bfff640 1 -- 172.21.15.44:0/1998559174 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3378004220 con 0x561157517660 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.839+0000 7f336bfff640 1 -- 172.21.15.44:0/1998559174 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f33780035f0 con 0x561157517660 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.839+0000 7f3395ff8740 10 client.25129 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:02:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.839+0000 7f3395ff8740 10 client.25129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:02:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.839+0000 7f3395ff8740 20 client.25129 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.839+0000 7f3395ff8740 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5611575a2530 0x5611575a4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.839+0000 7f3395ff8740 4 client.25129 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:02:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.839+0000 7f3395ff8740 1 -- 172.21.15.44:0/1998559174 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56115742ae00 con 0x5611575a2530 2024-08-20T03:02:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.839+0000 7f3390f9e640 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5611575a2530 0x5611575a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.840+0000 7f336bfff640 10 client.25129 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.840+0000 7f3390f9e640 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5611575a2530 0x5611575a4950 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeac585baf0c23e7 server_cookie=7dc0afb0739be3c1 in_seq=0 out_seq=0 2024-08-20T03:02:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f336bfff640 1 -- 172.21.15.44:0/1998559174 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56115742ae00 con 0x5611575a2530 2024-08-20T03:02:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f336bfff640 10 client.25129 handle_command_reply: tid=0 2024-08-20T03:02:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f3395ff8740 1 client.25129 shutdown 2024-08-20T03:02:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f3395ff8740 1 -- 172.21.15.44:0/1998559174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561157517660 msgr2=0x56115752b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f3395ff8740 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561157517660 0x56115752b6c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3378005700 tx=0x7f337800c560 comp rx=0 tx=0).stop 2024-08-20T03:02:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f3395ff8740 1 -- 172.21.15.44:0/1998559174 shutdown_connections 2024-08-20T03:02:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f3395ff8740 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5611575a2530 0x5611575a4950 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f3395ff8740 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5611575235f0 0x56115752bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f336bfff640 0 client.25129 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:48.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f3395ff8740 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561157517660 0x56115752b6c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f3395ff8740 1 --2- 172.21.15.44:0/1998559174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561157516d40 0x561157530200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f3395ff8740 1 -- 172.21.15.44:0/1998559174 >> 172.21.15.44:0/1998559174 conn(0x561157514b70 msgr2=0x561157521890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f3395ff8740 1 -- 172.21.15.44:0/1998559174 shutdown_connections 2024-08-20T03:02:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.848+0000 7f3395ff8740 1 -- 172.21.15.44:0/1998559174 wait complete. 2024-08-20T03:02:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.849+0000 7f3395ff8740 20 client.25129 trim_cache size 0 max 16384 2024-08-20T03:02:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:02:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:02:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:02:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:02:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.849+0000 7f3391fa0640 1 -- 172.21.15.44:0/4157474298 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f337c0b3e40 msgr2=0x7f337c0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.849+0000 7f3391fa0640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f337c0b3e40 0x7f337c0b6300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3384009a60 tx=0x7f3384005c30 comp rx=0 tx=0).stop 2024-08-20T03:02:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.849+0000 7f3391fa0640 1 -- 172.21.15.44:0/4157474298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f338c101b30 msgr2=0x7f338c198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.849+0000 7f3391fa0640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f338c101b30 0x7f338c198d30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f338000e9f0 tx=0x7f338000eeb0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.849+0000 7f3391fa0640 1 -- 172.21.15.44:0/4157474298 shutdown_connections 2024-08-20T03:02:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.849+0000 7f3391fa0640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f337c0b3e40 0x7f337c0b6300 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.849+0000 7f3391fa0640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f338c104890 0x7f338c19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.849+0000 7f3391fa0640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f338c103ef0 0x7f338c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.850+0000 7f3391fa0640 1 --2- 172.21.15.44:0/4157474298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f338c101b30 0x7f338c198d30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.850+0000 7f3391fa0640 1 -- 172.21.15.44:0/4157474298 >> 172.21.15.44:0/4157474298 conn(0x7f338c077270 msgr2=0x7f338c1072c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.850+0000 7f3391fa0640 1 -- 172.21.15.44:0/4157474298 shutdown_connections 2024-08-20T03:02:48.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:02:48.850+0000 7f3391fa0640 1 -- 172.21.15.44:0/4157474298 wait complete. 2024-08-20T03:02:50.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:49 smithi165 ceph-mon[22357]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-08-20T03:02:50.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:49 smithi099 ceph-mon[21984]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-08-20T03:02:50.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:49 smithi044 ceph-mon[18790]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-08-20T03:02:52.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:51 smithi165 ceph-mon[22357]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-08-20T03:02:52.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:51 smithi099 ceph-mon[21984]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-08-20T03:02:52.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:51 smithi044 ceph-mon[18790]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-08-20T03:02:53.921 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:02:54.088 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:53 smithi044 ceph-mon[18790]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 12 op/s 2024-08-20T03:02:54.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:53 smithi165 ceph-mon[22357]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 12 op/s 2024-08-20T03:02:54.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:53 smithi099 ceph-mon[21984]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 12 op/s 2024-08-20T03:02:54.199 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:54.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.728+0000 7ff8a0eea640 1 -- 172.21.15.44:0/434606753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff89c1042c0 msgr2=0x7ff89c10c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:54.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.728+0000 7ff8a0eea640 1 --2- 172.21.15.44:0/434606753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff89c1042c0 0x7ff89c10c7b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff890009900 tx=0x7ff89002a410 comp rx=0 tx=0).stop 2024-08-20T03:02:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.728+0000 7ff8a0eea640 1 -- 172.21.15.44:0/434606753 shutdown_connections 2024-08-20T03:02:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.728+0000 7ff8a0eea640 1 --2- 172.21.15.44:0/434606753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff89c10ce00 0x7ff89c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.728+0000 7ff8a0eea640 1 --2- 172.21.15.44:0/434606753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff89c1042c0 0x7ff89c10c7b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.728+0000 7ff8a0eea640 1 --2- 172.21.15.44:0/434606753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff89c1039a0 0x7ff89c103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.728+0000 7ff8a0eea640 1 -- 172.21.15.44:0/434606753 >> 172.21.15.44:0/434606753 conn(0x7ff89c0fd400 msgr2=0x7ff89c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.728+0000 7ff8a0eea640 1 -- 172.21.15.44:0/434606753 shutdown_connections 2024-08-20T03:02:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.728+0000 7ff8a0eea640 1 -- 172.21.15.44:0/434606753 wait complete. 2024-08-20T03:02:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.729+0000 7ff8a0eea640 1 Processor -- start 2024-08-20T03:02:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.729+0000 7ff8a0eea640 1 -- start start 2024-08-20T03:02:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.729+0000 7ff8a0eea640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff89c1039a0 0x7ff89c103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.729+0000 7ff8a0eea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff89c1042c0 0x7ff89c101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.729+0000 7ff8a0eea640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff89c10ce00 0x7ff89c101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.729+0000 7ff8a0eea640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c112ac0 con 0x7ff89c1042c0 2024-08-20T03:02:54.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.729+0000 7ff8a0eea640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c112940 con 0x7ff89c10ce00 2024-08-20T03:02:54.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.730+0000 7ff8a0eea640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c112c40 con 0x7ff89c1039a0 2024-08-20T03:02:54.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.730+0000 7ff899d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff89c1042c0 0x7ff89c101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.730+0000 7ff899d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff89c1042c0 0x7ff89c101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38746/0 (socket says 172.21.15.44:38746) 2024-08-20T03:02:54.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.730+0000 7ff899d9b640 1 -- 172.21.15.44:0/156012558 learned_addr learned my addr 172.21.15.44:0/156012558 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:54.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.730+0000 7ff89ad9d640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff89c10ce00 0x7ff89c101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.730+0000 7ff899d9b640 1 -- 172.21.15.44:0/156012558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff89c1039a0 msgr2=0x7ff89c103100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:54.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.730+0000 7ff899d9b640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff89c1039a0 0x7ff89c103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.730+0000 7ff899d9b640 1 -- 172.21.15.44:0/156012558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff89c10ce00 msgr2=0x7ff89c101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:54.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.730+0000 7ff899d9b640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff89c10ce00 0x7ff89c101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.730+0000 7ff899d9b640 1 -- 172.21.15.44:0/156012558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89c1ab790 con 0x7ff89c1042c0 2024-08-20T03:02:54.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.730+0000 7ff899d9b640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff89c1042c0 0x7ff89c101650 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff8900305d0 tx=0x7ff890030600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:54.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.731+0000 7ff88b7fe640 1 -- 172.21.15.44:0/156012558 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89003f030 con 0x7ff89c1042c0 2024-08-20T03:02:54.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.731+0000 7ff88b7fe640 1 -- 172.21.15.44:0/156012558 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff890030e60 con 0x7ff89c1042c0 2024-08-20T03:02:54.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.731+0000 7ff88b7fe640 1 -- 172.21.15.44:0/156012558 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890035070 con 0x7ff89c1042c0 2024-08-20T03:02:54.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.731+0000 7ff8a0eea640 1 -- 172.21.15.44:0/156012558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89c1ab960 con 0x7ff89c1042c0 2024-08-20T03:02:54.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.732+0000 7ff8a0eea640 1 -- 172.21.15.44:0/156012558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89c1abd60 con 0x7ff89c1042c0 2024-08-20T03:02:54.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.735+0000 7ff8a32e3740 1 Processor -- start 2024-08-20T03:02:54.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.735+0000 7ff88b7fe640 1 -- 172.21.15.44:0/156012558 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7ff890042030 con 0x7ff89c1042c0 2024-08-20T03:02:54.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.736+0000 7ff8a32e3740 1 -- start start 2024-08-20T03:02:54.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.736+0000 7ff8a32e3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e0fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.736+0000 7ff8a32e3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e10150 0x55d7c7e1bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.736+0000 7ff8a32e3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e1e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.736+0000 7ff8a32e3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d7c7d868e0 con 0x55d7c7e1c0e0 2024-08-20T03:02:54.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.736+0000 7ff8a32e3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d7c7d75de0 con 0x55d7c7e0f830 2024-08-20T03:02:54.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.736+0000 7ff8a32e3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d7c7d79e20 con 0x55d7c7e10150 2024-08-20T03:02:54.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.736+0000 7ff88b7fe640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff8840b3e70 0x7ff8840b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.736+0000 7ff89ad9d640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff8840b3e70 0x7ff8840b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff89ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e0fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff88b7fe640 1 -- 172.21.15.44:0/156012558 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff8901042f0 con 0x7ff89c1042c0 2024-08-20T03:02:54.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff89ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e0fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55296/0 (socket says 172.21.15.44:55296) 2024-08-20T03:02:54.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff89ad9d640 1 -- 172.21.15.44:0/875844541 learned_addr learned my addr 172.21.15.44:0/875844541 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:54.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff899d9b640 1 --2- 172.21.15.44:0/875844541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e1e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff89a59c640 1 --2- 172.21.15.44:0/875844541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e10150 0x55d7c7e1bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff89ad9d640 1 -- 172.21.15.44:0/875844541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e10150 msgr2=0x55d7c7e1bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff89ad9d640 1 --2- 172.21.15.44:0/875844541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e10150 0x55d7c7e1bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff89ad9d640 1 -- 172.21.15.44:0/875844541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 msgr2=0x55d7c7e1e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff89ad9d640 1 --2- 172.21.15.44:0/875844541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e1e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff89ad9d640 1 -- 172.21.15.44:0/875844541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c7ad5650 con 0x55d7c7e0f830 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff89ad9d640 1 --2- 172.21.15.44:0/875844541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e0fc10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff894012bf0 tx=0x7ff89401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=200765730b0dd065 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.737+0000 7ff89ad9d640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff8840b3e70 0x7ff8840b6330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff894014cf0 tx=0x7ff894009b40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.738+0000 7ff888ff9640 1 -- 172.21.15.44:0/875844541 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894026070 con 0x55d7c7e0f830 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.738+0000 7ff888ff9640 1 -- 172.21.15.44:0/875844541 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff894021410 con 0x55d7c7e0f830 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.738+0000 7ff888ff9640 1 -- 172.21.15.44:0/875844541 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894010660 con 0x55d7c7e0f830 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.739+0000 7ff8a32e3740 1 -- 172.21.15.44:0/875844541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 msgr2=0x55d7c7e0fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:54.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.739+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/875844541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e0fc10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff894012bf0 tx=0x7ff89401c000 comp rx=0 tx=0).stop 2024-08-20T03:02:54.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.739+0000 7ff8a32e3740 1 -- 172.21.15.44:0/875844541 shutdown_connections 2024-08-20T03:02:54.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.739+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/875844541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e1e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.739+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/875844541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e10150 0x55d7c7e1bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.739+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/875844541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e0fc10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.739+0000 7ff8a32e3740 1 -- 172.21.15.44:0/875844541 >> 172.21.15.44:0/875844541 conn(0x55d7c7e0d660 msgr2=0x55d7c7e0e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:54.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.740+0000 7ff8a32e3740 1 -- 172.21.15.44:0/875844541 shutdown_connections 2024-08-20T03:02:54.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.740+0000 7ff8a32e3740 1 -- 172.21.15.44:0/875844541 wait complete. 2024-08-20T03:02:54.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.740+0000 7ff8a32e3740 1 -- start start 2024-08-20T03:02:54.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.741+0000 7ff8a32e3740 1 Processor -- start 2024-08-20T03:02:54.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.741+0000 7ff8a32e3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e19050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff8a32e3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e10150 0x55d7c7e19590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff8a32e3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e19ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff899d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e10150 0x55d7c7e19590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff899d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e10150 0x55d7c7e19590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38766/0 (socket says 172.21.15.44:38766) 2024-08-20T03:02:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff899d9b640 1 -- 172.21.15.44:0/2231262934 learned_addr learned my addr 172.21.15.44:0/2231262934 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff89a59c640 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e19050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff8a32e3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d7c7bd8700 con 0x55d7c7e10150 2024-08-20T03:02:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2231262934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d7c7cfc7c0 con 0x55d7c7e0f830 2024-08-20T03:02:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2231262934 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d7c7e1ed30 con 0x55d7c7e1c0e0 2024-08-20T03:02:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff89ad9d640 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e19ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff8a32e3740 10 client.0 _ll_register_callbacks cb 0x55d7c7d6f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff89ad9d640 1 -- 172.21.15.44:0/2231262934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 msgr2=0x55d7c7e19050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.742+0000 7ff89ad9d640 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e19050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.743+0000 7ff89ad9d640 1 -- 172.21.15.44:0/2231262934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e10150 msgr2=0x55d7c7e19590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.743+0000 7ff89ad9d640 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e10150 0x55d7c7e19590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.743+0000 7ff89ad9d640 1 -- 172.21.15.44:0/2231262934 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c7e280e0 con 0x55d7c7e1c0e0 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.743+0000 7ff899d9b640 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e10150 0x55d7c7e19590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.743+0000 7ff89ad9d640 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e19ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff89400a010 tx=0x7ff8940122c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69e75fe582f5ddb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.743+0000 7ff881ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.743+0000 7ff881ffb640 1 -- 172.21.15.44:0/2231262934 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89402f030 con 0x55d7c7e1c0e0 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.743+0000 7ff881ffb640 1 -- 172.21.15.44:0/2231262934 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff894014490 con 0x55d7c7e1c0e0 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.743+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2231262934 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7c7d25100 con 0x55d7c7e1c0e0 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.744+0000 7ff881ffb640 1 -- 172.21.15.44:0/2231262934 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff894026040 con 0x55d7c7e1c0e0 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.744+0000 7ff8a32e3740 10 client.25138 fetch_fsmap learned FSMap version 28 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.744+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2231262934 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7c7e2aa50 con 0x55d7c7e1c0e0 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.744+0000 7ff881ffb640 1 -- 172.21.15.44:0/2231262934 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d7c7d25100 con 0x55d7c7e1c0e0 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.744+0000 7ff881ffb640 1 -- 172.21.15.44:0/2231262934 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff89400f040 con 0x55d7c7e1c0e0 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.744+0000 7ff8a32e3740 10 client.25138 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:02:54.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.744+0000 7ff8a32e3740 10 client.25138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:02:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.744+0000 7ff8a32e3740 20 client.25138 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.744+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7c7e2beb0 0x55d7c7e2e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.744+0000 7ff8a32e3740 4 client.25138 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:02:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.744+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2231262934 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7c7d88830 con 0x55d7c7e2beb0 2024-08-20T03:02:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.745+0000 7ff89a59c640 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7c7e2beb0 0x55d7c7e2e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.746+0000 7ff89a59c640 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7c7e2beb0 0x55d7c7e2e2a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e722ccaa1ecde65d server_cookie=e65b29c63cbe23a8 in_seq=0 out_seq=0 2024-08-20T03:02:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.746+0000 7ff881ffb640 10 client.25138 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.746+0000 7ff881ffb640 1 -- 172.21.15.44:0/2231262934 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7c7d88830 con 0x55d7c7e2beb0 2024-08-20T03:02:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.746+0000 7ff881ffb640 10 client.25138 handle_command_reply: tid=0 2024-08-20T03:02:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.746+0000 7ff8a32e3740 1 client.25138 shutdown 2024-08-20T03:02:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.748+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2231262934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e1c0e0 msgr2=0x55d7c7e19ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:54.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.748+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e19ad0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff89400a010 tx=0x7ff8940122c0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.748+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2231262934 shutdown_connections 2024-08-20T03:02:54.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.748+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7c7e2beb0 0x55d7c7e2e2a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.748+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e19ad0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.748+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e10150 0x55d7c7e19590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.748+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/2231262934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e19050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.748+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2231262934 >> 172.21.15.44:0/2231262934 conn(0x55d7c7e0d660 msgr2=0x55d7c7e13200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:54.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.748+0000 7ff881ffb640 0 client.25138 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:54.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.749+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2231262934 shutdown_connections 2024-08-20T03:02:54.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.749+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2231262934 wait complete. 2024-08-20T03:02:54.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.749+0000 7ff8a32e3740 20 client.25138 trim_cache size 0 max 16384 2024-08-20T03:02:54.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.783+0000 7ff8a32e3740 1 Processor -- start 2024-08-20T03:02:54.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.783+0000 7ff8a32e3740 1 -- start start 2024-08-20T03:02:54.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.783+0000 7ff8a32e3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e1ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.783+0000 7ff8a32e3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e10150 0x55d7c7e33900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.783+0000 7ff8a32e3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e33e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.783+0000 7ff8a32e3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1b4930 con 0x55d7c7e1c0e0 2024-08-20T03:02:54.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.783+0000 7ff8a32e3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1b6470 con 0x55d7c7e0f830 2024-08-20T03:02:54.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.783+0000 7ff8a32e3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1b5000 con 0x55d7c7e10150 2024-08-20T03:02:54.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.784+0000 7ff89ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e33e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.784+0000 7ff89ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e33e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38772/0 (socket says 172.21.15.44:38772) 2024-08-20T03:02:54.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.784+0000 7ff89ad9d640 1 -- 172.21.15.44:0/1841827807 learned_addr learned my addr 172.21.15.44:0/1841827807 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:54.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.784+0000 7ff899d9b640 1 --2- 172.21.15.44:0/1841827807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e10150 0x55d7c7e33900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.784+0000 7ff89a59c640 1 --2- 172.21.15.44:0/1841827807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e1ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.784+0000 7ff899d9b640 1 -- 172.21.15.44:0/1841827807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 msgr2=0x55d7c7e1ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.784+0000 7ff899d9b640 1 --2- 172.21.15.44:0/1841827807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e1ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.784+0000 7ff899d9b640 1 -- 172.21.15.44:0/1841827807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 msgr2=0x55d7c7e33e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.784+0000 7ff899d9b640 1 --2- 172.21.15.44:0/1841827807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e33e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.784+0000 7ff899d9b640 1 -- 172.21.15.44:0/1841827807 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c7e0d490 con 0x55d7c7e10150 2024-08-20T03:02:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.785+0000 7ff89ad9d640 1 --2- 172.21.15.44:0/1841827807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e33e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:54.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.785+0000 7ff899d9b640 1 --2- 172.21.15.44:0/1841827807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e10150 0x55d7c7e33900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff8900c80f0 tx=0x7ff890005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6612b34a9328a3fb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.785+0000 7ff89a59c640 1 --2- 172.21.15.44:0/1841827807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e1ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.785+0000 7ff883fff640 1 -- 172.21.15.44:0/1841827807 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890049040 con 0x55d7c7e10150 2024-08-20T03:02:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.785+0000 7ff883fff640 1 -- 172.21.15.44:0/1841827807 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8900caa20 con 0x55d7c7e10150 2024-08-20T03:02:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.785+0000 7ff883fff640 1 -- 172.21.15.44:0/1841827807 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8900d0a20 con 0x55d7c7e10150 2024-08-20T03:02:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.786+0000 7ff8a32e3740 1 -- 172.21.15.44:0/1841827807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e10150 msgr2=0x55d7c7e33900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.786+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/1841827807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e10150 0x55d7c7e33900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff8900c80f0 tx=0x7ff890005fd0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.786+0000 7ff8a32e3740 1 -- 172.21.15.44:0/1841827807 shutdown_connections 2024-08-20T03:02:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.786+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/1841827807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e33e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.786+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/1841827807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e10150 0x55d7c7e33900 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.786+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/1841827807 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e0f830 0x55d7c7e1ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.786+0000 7ff8a32e3740 1 -- 172.21.15.44:0/1841827807 >> 172.21.15.44:0/1841827807 conn(0x55d7c7e0d660 msgr2=0x55d7c7e1a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.786+0000 7ff8a32e3740 1 -- 172.21.15.44:0/1841827807 shutdown_connections 2024-08-20T03:02:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.786+0000 7ff8a32e3740 1 -- 172.21.15.44:0/1841827807 wait complete. 2024-08-20T03:02:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.786+0000 7ff8a32e3740 1 -- start start 2024-08-20T03:02:54.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.786+0000 7ff8a32e3740 1 Processor -- start 2024-08-20T03:02:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff8a32e3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e0f830 0x55d7c7e20e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff8a32e3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e10150 0x55d7c7e21350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff8a32e3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e21890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff899d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e10150 0x55d7c7e21350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff899d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e10150 0x55d7c7e21350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:55328/0 (socket says 172.21.15.44:55328) 2024-08-20T03:02:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff899d9b640 1 -- 172.21.15.44:0/2104433589 learned_addr learned my addr 172.21.15.44:0/2104433589 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2104433589 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1b5da0 con 0x55d7c7e1c0e0 2024-08-20T03:02:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2104433589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1b56d0 con 0x55d7c7e10150 2024-08-20T03:02:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2104433589 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c1b2910 con 0x55d7c7e0f830 2024-08-20T03:02:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff89ad9d640 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e21890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff8a32e3740 10 client.0 _ll_register_callbacks cb 0x7ff894024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:54.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff89a59c640 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e0f830 0x55d7c7e20e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff899d9b640 1 -- 172.21.15.44:0/2104433589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e0f830 msgr2=0x55d7c7e20e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:54.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff899d9b640 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e0f830 0x55d7c7e20e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff899d9b640 1 -- 172.21.15.44:0/2104433589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 msgr2=0x55d7c7e21890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:54.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff899d9b640 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e21890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff899d9b640 1 -- 172.21.15.44:0/2104433589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c7ea58e0 con 0x55d7c7e10150 2024-08-20T03:02:54.795 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:02:54.795 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-08-20T03:02:54.795 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-08-20T03:02:54.795 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:02:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.787+0000 7ff89a59c640 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e0f830 0x55d7c7e20e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:02:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.788+0000 7ff899d9b640 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e10150 0x55d7c7e21350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff8900ed000 tx=0x7ff890101f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e17334b19f47c933 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.788+0000 7ff881ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:02:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.788+0000 7ff881ffb640 1 -- 172.21.15.44:0/2104433589 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8900d06a0 con 0x55d7c7e10150 2024-08-20T03:02:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.788+0000 7ff881ffb640 1 -- 172.21.15.44:0/2104433589 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff8900cb170 con 0x55d7c7e10150 2024-08-20T03:02:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.788+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2104433589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8900caa20 con 0x55d7c7e10150 2024-08-20T03:02:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.788+0000 7ff881ffb640 1 -- 172.21.15.44:0/2104433589 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8900d0eb0 con 0x55d7c7e10150 2024-08-20T03:02:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.788+0000 7ff8a32e3740 10 client.25157 fetch_fsmap learned FSMap version 28 2024-08-20T03:02:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.788+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2104433589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7c7e8d910 con 0x55d7c7e10150 2024-08-20T03:02:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.788+0000 7ff881ffb640 1 -- 172.21.15.44:0/2104433589 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8900caa20 con 0x55d7c7e10150 2024-08-20T03:02:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.790+0000 7ff881ffb640 1 -- 172.21.15.44:0/2104433589 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff89004c060 con 0x55d7c7e10150 2024-08-20T03:02:54.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.790+0000 7ff8a32e3740 10 client.25157 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:02:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.790+0000 7ff8a32e3740 10 client.25157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:02:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.790+0000 7ff8a32e3740 20 client.25157 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.790+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7c7e8df40 0x55d7c7e90360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.790+0000 7ff8a32e3740 4 client.25157 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:02:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.790+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2104433589 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7c7d842d0 con 0x55d7c7e8df40 2024-08-20T03:02:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.790+0000 7ff89a59c640 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7c7e8df40 0x55d7c7e90360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.790+0000 7ff89a59c640 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7c7e8df40 0x55d7c7e90360 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de1fb020ec4d40df server_cookie=21e2943f3d5bbd44 in_seq=0 out_seq=0 2024-08-20T03:02:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.790+0000 7ff881ffb640 10 client.25157 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff881ffb640 1 -- 172.21.15.44:0/2104433589 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d7c7d842d0 con 0x55d7c7e8df40 2024-08-20T03:02:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff881ffb640 10 client.25157 handle_command_reply: tid=0 2024-08-20T03:02:54.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff8a32e3740 1 client.25157 shutdown 2024-08-20T03:02:54.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2104433589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e10150 msgr2=0x55d7c7e21350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:54.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e10150 0x55d7c7e21350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff8900ed000 tx=0x7ff890101f90 comp rx=0 tx=0).stop 2024-08-20T03:02:54.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2104433589 shutdown_connections 2024-08-20T03:02:54.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7c7e8df40 0x55d7c7e90360 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff881ffb640 0 client.25157 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:02:54.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7c7e1c0e0 0x55d7c7e21890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7c7e10150 0x55d7c7e21350 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff8a32e3740 1 --2- 172.21.15.44:0/2104433589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7c7e0f830 0x55d7c7e20e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2104433589 >> 172.21.15.44:0/2104433589 conn(0x55d7c7e0d660 msgr2=0x55d7c7e1aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:54.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2104433589 shutdown_connections 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff8a32e3740 1 -- 172.21.15.44:0/2104433589 wait complete. 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.791+0000 7ff8a32e3740 20 client.25157 trim_cache size 0 max 16384 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.792+0000 7ff8a0eea640 1 -- 172.21.15.44:0/156012558 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff8840b3e70 msgr2=0x7ff8840b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.792+0000 7ff8a0eea640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff8840b3e70 0x7ff8840b6330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff894014cf0 tx=0x7ff894009b40 comp rx=0 tx=0).stop 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.792+0000 7ff8a0eea640 1 -- 172.21.15.44:0/156012558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff89c1042c0 msgr2=0x7ff89c101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.792+0000 7ff8a0eea640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff89c1042c0 0x7ff89c101650 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff8900305d0 tx=0x7ff890030600 comp rx=0 tx=0).stop 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.792+0000 7ff8a0eea640 1 -- 172.21.15.44:0/156012558 shutdown_connections 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.792+0000 7ff8a0eea640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff8840b3e70 0x7ff8840b6330 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.792+0000 7ff8a0eea640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff89c10ce00 0x7ff89c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.792+0000 7ff8a0eea640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff89c1042c0 0x7ff89c101650 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.792+0000 7ff8a0eea640 1 --2- 172.21.15.44:0/156012558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff89c1039a0 0x7ff89c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.792+0000 7ff8a0eea640 1 -- 172.21.15.44:0/156012558 >> 172.21.15.44:0/156012558 conn(0x7ff89c0fd400 msgr2=0x7ff89c1046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.792+0000 7ff8a0eea640 1 -- 172.21.15.44:0/156012558 shutdown_connections 2024-08-20T03:02:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:54.792+0000 7ff8a0eea640 1 -- 172.21.15.44:0/156012558 wait complete. 2024-08-20T03:02:55.532 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T03:02:55.532 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:02:55.815 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:56.094 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:55 smithi044 ceph-mon[18790]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-20T03:02:56.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:55 smithi165 ceph-mon[22357]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-20T03:02:56.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:55 smithi099 ceph-mon[21984]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-20T03:02:56.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.370+0000 7f528b0c5640 1 -- 172.21.15.44:0/4031801816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5284101990 msgr2=0x7f5284101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:56.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.370+0000 7f528b0c5640 1 --2- 172.21.15.44:0/4031801816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5284101990 0x7f5284101d70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5278009900 tx=0x7f527802a3f0 comp rx=0 tx=0).stop 2024-08-20T03:02:56.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.370+0000 7f528b0c5640 1 -- 172.21.15.44:0/4031801816 shutdown_connections 2024-08-20T03:02:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.370+0000 7f528b0c5640 1 --2- 172.21.15.44:0/4031801816 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5284107d20 0x7f528410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.370+0000 7f528b0c5640 1 --2- 172.21.15.44:0/4031801816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52841073a0 0x7f52841077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.370+0000 7f528b0c5640 1 --2- 172.21.15.44:0/4031801816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5284101990 0x7f5284101d70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.370+0000 7f528b0c5640 1 -- 172.21.15.44:0/4031801816 >> 172.21.15.44:0/4031801816 conn(0x7f52840fd400 msgr2=0x7f52840ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.371+0000 7f528b0c5640 1 -- 172.21.15.44:0/4031801816 shutdown_connections 2024-08-20T03:02:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.371+0000 7f528b0c5640 1 -- 172.21.15.44:0/4031801816 wait complete. 2024-08-20T03:02:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.371+0000 7f528b0c5640 1 Processor -- start 2024-08-20T03:02:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.371+0000 7f528b0c5640 1 -- start start 2024-08-20T03:02:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.372+0000 7f528b0c5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5284101990 0x7f5284199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.372+0000 7f528b0c5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52841073a0 0x7f528419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.372+0000 7f528b0c5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284107d20 0x7f528419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:56.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.372+0000 7f528b0c5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52841119a0 con 0x7f5284101990 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.372+0000 7f528b0c5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5284111820 con 0x7f5284107d20 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.372+0000 7f528b0c5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5284111b20 con 0x7f52841073a0 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.372+0000 7f5288e61640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5284101990 0x7f5284199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.372+0000 7f5288e61640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5284101990 0x7f5284199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38796/0 (socket says 172.21.15.44:38796) 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.372+0000 7f5289662640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284107d20 0x7f528419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.372+0000 7f5288e61640 1 -- 172.21.15.44:0/1233334966 learned_addr learned my addr 172.21.15.44:0/1233334966 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.372+0000 7f5283fff640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52841073a0 0x7f528419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.373+0000 7f5288e61640 1 -- 172.21.15.44:0/1233334966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52841073a0 msgr2=0x7f528419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.373+0000 7f5288e61640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52841073a0 0x7f528419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.373+0000 7f5288e61640 1 -- 172.21.15.44:0/1233334966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284107d20 msgr2=0x7f528419a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.373+0000 7f5288e61640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284107d20 0x7f528419a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.373+0000 7f5288e61640 1 -- 172.21.15.44:0/1233334966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52841a51a0 con 0x7f5284101990 2024-08-20T03:02:56.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.373+0000 7f5283fff640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52841073a0 0x7f528419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:56.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.373+0000 7f5289662640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284107d20 0x7f528419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:56.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.373+0000 7f5288e61640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5284101990 0x7f5284199da0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5278005940 tx=0x7f52780043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:56.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.373+0000 7f5281ffb640 1 -- 172.21.15.44:0/1233334966 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527803f030 con 0x7f5284101990 2024-08-20T03:02:56.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.373+0000 7f5281ffb640 1 -- 172.21.15.44:0/1233334966 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5278003aa0 con 0x7f5284101990 2024-08-20T03:02:56.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.373+0000 7f528b0c5640 1 -- 172.21.15.44:0/1233334966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52841a5430 con 0x7f5284101990 2024-08-20T03:02:56.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.373+0000 7f5281ffb640 1 -- 172.21.15.44:0/1233334966 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278035070 con 0x7f5284101990 2024-08-20T03:02:56.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.374+0000 7f528b0c5640 1 -- 172.21.15.44:0/1233334966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52841a5960 con 0x7f5284101990 2024-08-20T03:02:56.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.376+0000 7f528b0c5640 1 -- 172.21.15.44:0/1233334966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5284101d70 con 0x7f5284101990 2024-08-20T03:02:56.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.379+0000 7f5281ffb640 1 -- 172.21.15.44:0/1233334966 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f5278042030 con 0x7f5284101990 2024-08-20T03:02:56.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.380+0000 7f5281ffb640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f52740b3e40 0x7f52740b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:56.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.380+0000 7f5281ffb640 1 -- 172.21.15.44:0/1233334966 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f52781045f0 con 0x7f5284101990 2024-08-20T03:02:56.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.380+0000 7f5283fff640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f52740b3e40 0x7f52740b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:56.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.381+0000 7f5283fff640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f52740b3e40 0x7f52740b6300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f528406b3d0 tx=0x7f5270005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:56.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.386+0000 7f5281ffb640 1 -- 172.21.15.44:0/1233334966 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f52780cb910 con 0x7f5284101990 2024-08-20T03:02:56.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.839+0000 7f528b0c5640 1 -- 172.21.15.44:0/1233334966 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5284109b00 con 0x7f5284101990 2024-08-20T03:02:56.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.840+0000 7f5281ffb640 1 -- 172.21.15.44:0/1233334966 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7f52780cb210 con 0x7f5284101990 2024-08-20T03:02:56.842 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:02:56.842 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":28,"btime":"2024-08-20T03:02:12:953534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:02:12.906757+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24569,24583,24596,24589,24604]},"id":1}]} 2024-08-20T03:02:56.843 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 28 2024-08-20T03:02:56.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.843+0000 7f528b0c5640 1 -- 172.21.15.44:0/1233334966 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f52740b3e40 msgr2=0x7f52740b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:56.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.843+0000 7f528b0c5640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f52740b3e40 0x7f52740b6300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f528406b3d0 tx=0x7f5270005f90 comp rx=0 tx=0).stop 2024-08-20T03:02:56.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.843+0000 7f528b0c5640 1 -- 172.21.15.44:0/1233334966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5284101990 msgr2=0x7f5284199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:56.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.843+0000 7f528b0c5640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5284101990 0x7f5284199da0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5278005940 tx=0x7f52780043b0 comp rx=0 tx=0).stop 2024-08-20T03:02:56.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.843+0000 7f528b0c5640 1 -- 172.21.15.44:0/1233334966 shutdown_connections 2024-08-20T03:02:56.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.843+0000 7f528b0c5640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f52740b3e40 0x7f52740b6300 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:56.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.843+0000 7f528b0c5640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5284107d20 0x7f528419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:56.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.843+0000 7f528b0c5640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f52841073a0 0x7f528419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:56.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.843+0000 7f528b0c5640 1 --2- 172.21.15.44:0/1233334966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5284101990 0x7f5284199da0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:56.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.843+0000 7f528b0c5640 1 -- 172.21.15.44:0/1233334966 >> 172.21.15.44:0/1233334966 conn(0x7f52840fd400 msgr2=0x7f52840ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:56.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.843+0000 7f528b0c5640 1 -- 172.21.15.44:0/1233334966 shutdown_connections 2024-08-20T03:02:56.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:56.843+0000 7f528b0c5640 1 -- 172.21.15.44:0/1233334966 wait complete. 2024-08-20T03:02:57.552 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-20T03:02:57.907 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:02:58.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:57 smithi165 ceph-mon[22357]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-20T03:02:58.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:57 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1233334966' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:02:58.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:57 smithi099 ceph-mon[21984]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-20T03:02:58.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:57 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1233334966' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:02:58.187 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:57 smithi044 ceph-mon[18790]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 682 B/s wr, 11 op/s 2024-08-20T03:02:58.187 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:57 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1233334966' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:02:58.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.435+0000 7f1ab53ab640 1 -- 172.21.15.44:0/4253681988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab01038f0 msgr2=0x7f1ab0103d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:58.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.435+0000 7f1ab53ab640 1 --2- 172.21.15.44:0/4253681988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab01038f0 0x7f1ab0103d50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009900 tx=0x7f1aa402a410 comp rx=0 tx=0).stop 2024-08-20T03:02:58.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.435+0000 7f1ab53ab640 1 -- 172.21.15.44:0/4253681988 shutdown_connections 2024-08-20T03:02:58.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.435+0000 7f1ab53ab640 1 --2- 172.21.15.44:0/4253681988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab0104290 0x7f1ab010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.435+0000 7f1ab53ab640 1 --2- 172.21.15.44:0/4253681988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab01038f0 0x7f1ab0103d50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.435+0000 7f1ab53ab640 1 --2- 172.21.15.44:0/4253681988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ab0101590 0x7f1ab0101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.435+0000 7f1ab53ab640 1 -- 172.21.15.44:0/4253681988 >> 172.21.15.44:0/4253681988 conn(0x7f1ab0076bd0 msgr2=0x7f1ab0076fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:58.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.436+0000 7f1ab53ab640 1 -- 172.21.15.44:0/4253681988 shutdown_connections 2024-08-20T03:02:58.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.436+0000 7f1ab53ab640 1 -- 172.21.15.44:0/4253681988 wait complete. 2024-08-20T03:02:58.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.437+0000 7f1ab53ab640 1 Processor -- start 2024-08-20T03:02:58.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.438+0000 7f1ab53ab640 1 -- start start 2024-08-20T03:02:58.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.438+0000 7f1ab53ab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab0101590 0x7f1ab0199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.438+0000 7f1ab53ab640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ab0104290 0x7f1ab0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.438+0000 7f1ab53ab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab019eb20 0x7f1ab019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.438+0000 7f1ab53ab640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab0112a90 con 0x7f1ab0101590 2024-08-20T03:02:58.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.438+0000 7f1ab53ab640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab0112910 con 0x7f1ab019eb20 2024-08-20T03:02:58.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.438+0000 7f1ab53ab640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab0112c10 con 0x7f1ab0104290 2024-08-20T03:02:58.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.438+0000 7f1aaeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab0101590 0x7f1ab0199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.438+0000 7f1aaeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab0101590 0x7f1ab0199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54778/0 (socket says 172.21.15.44:54778) 2024-08-20T03:02:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.438+0000 7f1aaeffd640 1 -- 172.21.15.44:0/3404095274 learned_addr learned my addr 172.21.15.44:0/3404095274 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.439+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab019eb20 0x7f1ab019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.439+0000 7f1aae7fc640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ab0104290 0x7f1ab0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.439+0000 7f1aaeffd640 1 -- 172.21.15.44:0/3404095274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ab0104290 msgr2=0x7f1ab0199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.439+0000 7f1aaeffd640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ab0104290 0x7f1ab0199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.439+0000 7f1aaeffd640 1 -- 172.21.15.44:0/3404095274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab019eb20 msgr2=0x7f1ab019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.439+0000 7f1aaeffd640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab019eb20 0x7f1ab019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.439+0000 7f1aaeffd640 1 -- 172.21.15.44:0/3404095274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab019f6b0 con 0x7f1ab0101590 2024-08-20T03:02:58.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.439+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab019eb20 0x7f1ab019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:58.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.439+0000 7f1aaeffd640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab0101590 0x7f1ab0199240 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000ed60 tx=0x7f1aa000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:58.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.440+0000 7f1a97fff640 1 -- 172.21.15.44:0/3404095274 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0009e30 con 0x7f1ab0101590 2024-08-20T03:02:58.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.440+0000 7f1a97fff640 1 -- 172.21.15.44:0/3404095274 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa00054d0 con 0x7f1ab0101590 2024-08-20T03:02:58.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.440+0000 7f1ab53ab640 1 -- 172.21.15.44:0/3404095274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab01a5ca0 con 0x7f1ab0101590 2024-08-20T03:02:58.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.440+0000 7f1a97fff640 1 -- 172.21.15.44:0/3404095274 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0010690 con 0x7f1ab0101590 2024-08-20T03:02:58.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.440+0000 7f1ab53ab640 1 -- 172.21.15.44:0/3404095274 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab01a6040 con 0x7f1ab0101590 2024-08-20T03:02:58.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.443+0000 7f1ab77a4740 1 Processor -- start 2024-08-20T03:02:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.443+0000 7f1ab77a4740 1 -- start start 2024-08-20T03:02:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.443+0000 7f1ab77a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 0x564dc2796c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.443+0000 7f1ab77a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc27a0be0 0x564dc27971a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.443+0000 7f1ab77a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc27adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.443+0000 7f1ab77a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564dc270d930 con 0x564dc27a0be0 2024-08-20T03:02:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.443+0000 7f1ab77a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564dc26fce30 con 0x564dc27ab7e0 2024-08-20T03:02:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.443+0000 7f1ab77a4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564dc2700e70 con 0x564dc2796880 2024-08-20T03:02:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.444+0000 7f1aaf7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 0x564dc2796c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.444+0000 7f1aaf7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 0x564dc2796c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47326/0 (socket says 172.21.15.44:47326) 2024-08-20T03:02:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.444+0000 7f1aaf7fe640 1 -- 172.21.15.44:0/2387187568 learned_addr learned my addr 172.21.15.44:0/2387187568 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:58.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.444+0000 7f1a97fff640 1 -- 172.21.15.44:0/3404095274 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f1aa00046c0 con 0x7f1ab0101590 2024-08-20T03:02:58.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.444+0000 7f1aaeffd640 1 --2- 172.21.15.44:0/2387187568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc27a0be0 0x564dc27971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.444+0000 7f1aae7fc640 1 --2- 172.21.15.44:0/2387187568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc27adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.444+0000 7f1aaf7fe640 1 -- 172.21.15.44:0/2387187568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 msgr2=0x564dc27adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:58.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.444+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/2387187568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc27adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.444+0000 7f1aaf7fe640 1 -- 172.21.15.44:0/2387187568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc27a0be0 msgr2=0x564dc27971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:58.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.444+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/2387187568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc27a0be0 0x564dc27971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.444+0000 7f1aaf7fe640 1 -- 172.21.15.44:0/2387187568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dc245c650 con 0x564dc2796880 2024-08-20T03:02:58.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.445+0000 7f1aaeffd640 1 --2- 172.21.15.44:0/2387187568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc27a0be0 0x564dc27971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:02:58.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.445+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/2387187568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 0x564dc2796c60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80117e0 tx=0x7f1aa8011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c73b85a10ca08e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:58.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.445+0000 7f1a957fa640 1 -- 172.21.15.44:0/2387187568 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8011310 con 0x564dc2796880 2024-08-20T03:02:58.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.445+0000 7f1a957fa640 1 -- 172.21.15.44:0/2387187568 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa8003e20 con 0x564dc2796880 2024-08-20T03:02:58.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.446+0000 7f1a957fa640 1 -- 172.21.15.44:0/2387187568 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80044f0 con 0x564dc2796880 2024-08-20T03:02:58.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.446+0000 7f1a97fff640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1a980b3e40 0x7f1a980b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.446+0000 7f1aaeffd640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1a980b3e40 0x7f1a980b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.446+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2387187568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 msgr2=0x564dc2796c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:58.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.446+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2387187568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 0x564dc2796c60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80117e0 tx=0x7f1aa8011ca0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.446+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2387187568 shutdown_connections 2024-08-20T03:02:58.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.447+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2387187568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc27adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.447+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2387187568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc27a0be0 0x564dc27971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.447+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2387187568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 0x564dc2796c60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.447+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2387187568 >> 172.21.15.44:0/2387187568 conn(0x564dc27946b0 msgr2=0x564dc2795850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:58.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.447+0000 7f1a97fff640 1 -- 172.21.15.44:0/3404095274 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1aa00e42d0 con 0x7f1ab0101590 2024-08-20T03:02:58.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.447+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2387187568 shutdown_connections 2024-08-20T03:02:58.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.447+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2387187568 wait complete. 2024-08-20T03:02:58.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.447+0000 7f1ab77a4740 1 -- start start 2024-08-20T03:02:58.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.448+0000 7f1ab77a4740 1 Processor -- start 2024-08-20T03:02:58.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.448+0000 7f1aaeffd640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1a980b3e40 0x7f1a980b6300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x564dc2796360 tx=0x7f1aa00e9040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:58.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.448+0000 7f1ab77a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 0x564dc279db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.448+0000 7f1ab77a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc27a0be0 0x564dc279e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.448+0000 7f1ab77a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc279e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.448+0000 7f1ab77a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564dc255f700 con 0x564dc27a0be0 2024-08-20T03:02:58.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1aaf7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 0x564dc279db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1aaf7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 0x564dc279db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47330/0 (socket says 172.21.15.44:47330) 2024-08-20T03:02:58.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1aaf7fe640 1 -- 172.21.15.44:0/2463076242 learned_addr learned my addr 172.21.15.44:0/2463076242 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:58.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1aae7fc640 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc27a0be0 0x564dc279e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2463076242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564dc2683810 con 0x564dc27ab7e0 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2463076242 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564dc27ae620 con 0x564dc2796880 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc279e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1ab77a4740 10 client.0 _ll_register_callbacks cb 0x564dc26f6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1aaf7fe640 1 -- 172.21.15.44:0/2463076242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 msgr2=0x564dc279db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 0x564dc279db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1aaf7fe640 1 -- 172.21.15.44:0/2463076242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc27a0be0 msgr2=0x564dc279e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc27a0be0 0x564dc279e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1aaf7fe640 1 -- 172.21.15.44:0/2463076242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dc27b79d0 con 0x564dc27ab7e0 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1aae7fc640 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc27a0be0 0x564dc279e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.449+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2463076242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564dc26ac150 con 0x564dc27ab7e0 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.450+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc279e5c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8005f40 tx=0x7f1aa8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb528c7be3c31ee server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.450+0000 7f1a92ff5640 10 client.25175 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:02:58.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.450+0000 7f1a92ff5640 1 -- 172.21.15.44:0/2463076242 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8025040 con 0x564dc27ab7e0 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.450+0000 7f1a92ff5640 1 -- 172.21.15.44:0/2463076242 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1aa800fdd0 con 0x564dc27ab7e0 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.450+0000 7f1a92ff5640 1 -- 172.21.15.44:0/2463076242 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1aa8004a40 con 0x564dc27ab7e0 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.450+0000 7f1ab77a4740 10 client.25175 fetch_fsmap learned FSMap version 28 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.450+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2463076242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=28}) -- 0x564dc27ba1e0 con 0x564dc27ab7e0 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.450+0000 7f1a92ff5640 1 -- 172.21.15.44:0/2463076242 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1aa8018020 con 0x564dc27ab7e0 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.451+0000 7f1ab77a4740 10 client.25175 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.451+0000 7f1ab77a4740 10 client.25175 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.451+0000 7f1ab77a4740 20 client.25175 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.451+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x564dc27bb530 0x564dc27bd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.451+0000 7f1ab77a4740 4 client.25175 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.451+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2463076242 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dc26ac150 con 0x564dc27bb530 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.451+0000 7f1aae7fc640 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x564dc27bb530 0x564dc27bd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.451+0000 7f1aae7fc640 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x564dc27bb530 0x564dc27bd920 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=39d528554b4227d6 server_cookie=dec8db2e2e3b9bf4 in_seq=0 out_seq=0 2024-08-20T03:02:58.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.451+0000 7f1a92ff5640 10 client.25175 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:02:58.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.452+0000 7f1a92ff5640 1 -- 172.21.15.44:0/2463076242 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564dc26ac150 con 0x564dc27bb530 2024-08-20T03:02:58.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.452+0000 7f1a92ff5640 10 client.25175 handle_command_reply: tid=0 2024-08-20T03:02:58.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.452+0000 7f1ab77a4740 1 client.25175 shutdown 2024-08-20T03:02:58.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.452+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2463076242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 msgr2=0x564dc279e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:58.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.452+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc279e5c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8005f40 tx=0x7f1aa8005fd0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.453+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2463076242 shutdown_connections 2024-08-20T03:02:58.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.453+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x564dc27bb530 0x564dc27bd920 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.453+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc279e5c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.453+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc27a0be0 0x564dc279e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.453+0000 7f1a92ff5640 0 client.25175 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:02:58.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.453+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2463076242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc2796880 0x564dc279db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.453+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2463076242 >> 172.21.15.44:0/2463076242 conn(0x564dc27946b0 msgr2=0x564dc27999c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:58.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.453+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2463076242 shutdown_connections 2024-08-20T03:02:58.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.453+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2463076242 wait complete. 2024-08-20T03:02:58.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.453+0000 7f1ab77a4740 20 client.25175 trim_cache size 0 max 16384 2024-08-20T03:02:58.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.479+0000 7f1ab77a4740 1 Processor -- start 2024-08-20T03:02:58.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.479+0000 7f1ab77a4740 1 -- start start 2024-08-20T03:02:58.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.480+0000 7f1ab77a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc2796880 0x564dc27ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.480+0000 7f1ab77a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 0x564dc27c3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.480+0000 7f1ab77a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc27c3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.480+0000 7f1ab77a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01ac670 con 0x564dc2796880 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.480+0000 7f1ab77a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01ae1b0 con 0x564dc27ab7e0 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.480+0000 7f1ab77a4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01acd40 con 0x564dc27a0be0 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.480+0000 7f1aae7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 0x564dc27c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.480+0000 7f1aaf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc2796880 0x564dc27ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.480+0000 7f1aaf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc2796880 0x564dc27ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54814/0 (socket says 172.21.15.44:54814) 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.480+0000 7f1aaf7fe640 1 -- 172.21.15.44:0/2135412673 learned_addr learned my addr 172.21.15.44:0/2135412673 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.480+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/2135412673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc27c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.480+0000 7f1aae7fc640 1 -- 172.21.15.44:0/2135412673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 msgr2=0x564dc27c3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.481+0000 7f1aae7fc640 1 --2- 172.21.15.44:0/2135412673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc27c3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.481+0000 7f1aae7fc640 1 -- 172.21.15.44:0/2135412673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc2796880 msgr2=0x564dc27ad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:58.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.481+0000 7f1aae7fc640 1 --2- 172.21.15.44:0/2135412673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc2796880 0x564dc27ad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.481+0000 7f1aae7fc640 1 -- 172.21.15.44:0/2135412673 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dc2796170 con 0x564dc27a0be0 2024-08-20T03:02:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.481+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/2135412673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc2796880 0x564dc27ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:02:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.481+0000 7f1aae7fc640 1 --2- 172.21.15.44:0/2135412673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 0x564dc27c3040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4008000 tx=0x7f1aa40058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6791e63b18fe7089 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.481+0000 7f1a94ff9640 1 -- 172.21.15.44:0/2135412673 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4000e60 con 0x564dc27a0be0 2024-08-20T03:02:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.481+0000 7f1a94ff9640 1 -- 172.21.15.44:0/2135412673 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa4031d90 con 0x564dc27a0be0 2024-08-20T03:02:58.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.481+0000 7f1a94ff9640 1 -- 172.21.15.44:0/2135412673 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4035040 con 0x564dc27a0be0 2024-08-20T03:02:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.482+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2135412673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 msgr2=0x564dc27c3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.482+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2135412673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 0x564dc27c3040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4008000 tx=0x7f1aa40058d0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.482+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2135412673 shutdown_connections 2024-08-20T03:02:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.482+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2135412673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc27c3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.482+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2135412673 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 0x564dc27c3040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.482+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/2135412673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc2796880 0x564dc27ad9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.482+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2135412673 >> 172.21.15.44:0/2135412673 conn(0x564dc27946b0 msgr2=0x564dc27ad2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.483+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2135412673 shutdown_connections 2024-08-20T03:02:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.483+0000 7f1ab77a4740 1 -- 172.21.15.44:0/2135412673 wait complete. 2024-08-20T03:02:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.483+0000 7f1ab77a4740 1 -- start start 2024-08-20T03:02:58.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.483+0000 7f1ab77a4740 1 Processor -- start 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1ab77a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc2796880 0x564dc27b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1ab77a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 0x564dc27b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1ab77a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc2824010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1ab77a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01adae0 con 0x564dc2796880 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1ab77a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01ad410 con 0x564dc27ab7e0 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1ab77a4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01aa650 con 0x564dc27a0be0 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1ab77a4740 10 client.0 _ll_register_callbacks cb 0x7f1aa8011370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1aae7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 0x564dc27b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1aae7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 0x564dc27b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47342/0 (socket says 172.21.15.44:47342) 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1aae7fc640 1 -- 172.21.15.44:0/3151308350 learned_addr learned my addr 172.21.15.44:0/3151308350 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1aae7fc640 1 -- 172.21.15.44:0/3151308350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 msgr2=0x564dc2824010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1aae7fc640 1 --2- 172.21.15.44:0/3151308350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc2824010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.484+0000 7f1aae7fc640 1 -- 172.21.15.44:0/3151308350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc2796880 msgr2=0x564dc27b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.485+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/3151308350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc2796880 0x564dc27b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.485+0000 7f1aae7fc640 1 --2- 172.21.15.44:0/3151308350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc2796880 0x564dc27b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.485+0000 7f1aae7fc640 1 -- 172.21.15.44:0/3151308350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dc2824750 con 0x564dc27a0be0 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.485+0000 7f1aae7fc640 1 --2- 172.21.15.44:0/3151308350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 0x564dc27b1640 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40316d0 tx=0x7f1aa4031700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e2ab545931b98d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.485+0000 7f1a92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.485+0000 7f1a92ff5640 1 -- 172.21.15.44:0/3151308350 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4035070 con 0x564dc27a0be0 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.485+0000 7f1a92ff5640 1 -- 172.21.15.44:0/3151308350 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1aa4004480 con 0x564dc27a0be0 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.485+0000 7f1ab77a4740 1 -- 172.21.15.44:0/3151308350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aa4031d90 con 0x564dc27a0be0 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.485+0000 7f1a92ff5640 1 -- 172.21.15.44:0/3151308350 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1aa405c050 con 0x564dc27a0be0 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.485+0000 7f1ab77a4740 10 client.25165 fetch_fsmap learned FSMap version 28 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.485+0000 7f1ab77a4740 1 -- 172.21.15.44:0/3151308350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564dc2825a50 con 0x564dc27a0be0 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.486+0000 7f1a92ff5640 1 -- 172.21.15.44:0/3151308350 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa4053070 con 0x564dc27a0be0 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.486+0000 7f1a92ff5640 1 -- 172.21.15.44:0/3151308350 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1aa404e5c0 con 0x564dc27a0be0 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.486+0000 7f1ab77a4740 10 client.25165 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:02:58.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.486+0000 7f1ab77a4740 10 client.25165 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:02:58.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.486+0000 7f1ab77a4740 20 client.25165 populate_metadata read hostname 'smithi044' 2024-08-20T03:02:58.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.486+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/3151308350 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x564dc2826180 0x564dc28285a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:02:58.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.486+0000 7f1ab77a4740 4 client.25165 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:02:58.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.486+0000 7f1ab77a4740 1 -- 172.21.15.44:0/3151308350 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1aa4031d90 con 0x564dc2826180 2024-08-20T03:02:58.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.486+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/3151308350 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x564dc2826180 0x564dc28285a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:02:58.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.487+0000 7f1aaf7fe640 1 --2- 172.21.15.44:0/3151308350 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x564dc2826180 0x564dc28285a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e262a47beab2788c server_cookie=75af3d0cb0e70db8 in_seq=0 out_seq=0 2024-08-20T03:02:58.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.487+0000 7f1a92ff5640 10 client.25165 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:02:58.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.487+0000 7f1a92ff5640 1 -- 172.21.15.44:0/3151308350 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1aa4031d90 con 0x564dc2826180 2024-08-20T03:02:58.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.487+0000 7f1a92ff5640 10 client.25165 handle_command_reply: tid=0 2024-08-20T03:02:58.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.487+0000 7f1ab77a4740 1 client.25165 shutdown 2024-08-20T03:02:58.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.487+0000 7f1ab77a4740 1 -- 172.21.15.44:0/3151308350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 msgr2=0x564dc27b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:58.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.487+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/3151308350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 0x564dc27b1640 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40316d0 tx=0x7f1aa4031700 comp rx=0 tx=0).stop 2024-08-20T03:02:58.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.488+0000 7f1ab77a4740 1 -- 172.21.15.44:0/3151308350 shutdown_connections 2024-08-20T03:02:58.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.488+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/3151308350 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x564dc2826180 0x564dc28285a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.488+0000 7f1a92ff5640 0 client.25165 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:02:58.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.488+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/3151308350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564dc27ab7e0 0x564dc2824010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.488+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/3151308350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564dc27a0be0 0x564dc27b1640 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.488+0000 7f1ab77a4740 1 --2- 172.21.15.44:0/3151308350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564dc2796880 0x564dc27b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.488+0000 7f1ab77a4740 1 -- 172.21.15.44:0/3151308350 >> 172.21.15.44:0/3151308350 conn(0x564dc27946b0 msgr2=0x564dc279f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:58.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.488+0000 7f1ab77a4740 1 -- 172.21.15.44:0/3151308350 shutdown_connections 2024-08-20T03:02:58.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.488+0000 7f1ab77a4740 1 -- 172.21.15.44:0/3151308350 wait complete. 2024-08-20T03:02:58.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.488+0000 7f1ab77a4740 20 client.25165 trim_cache size 0 max 16384 2024-08-20T03:02:58.495 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:02:58.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.489+0000 7f1ab53ab640 1 -- 172.21.15.44:0/3404095274 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1a980b3e40 msgr2=0x7f1a980b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:58.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.489+0000 7f1ab53ab640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1a980b3e40 0x7f1a980b6300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x564dc2796360 tx=0x7f1aa00e9040 comp rx=0 tx=0).stop 2024-08-20T03:02:58.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.490+0000 7f1ab53ab640 1 -- 172.21.15.44:0/3404095274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab0101590 msgr2=0x7f1ab0199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:02:58.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.490+0000 7f1ab53ab640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab0101590 0x7f1ab0199240 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000ed60 tx=0x7f1aa000c6a0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.490+0000 7f1ab53ab640 1 -- 172.21.15.44:0/3404095274 shutdown_connections 2024-08-20T03:02:58.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.490+0000 7f1ab53ab640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1a980b3e40 0x7f1a980b6300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.490+0000 7f1ab53ab640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ab019eb20 0x7f1ab019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.490+0000 7f1ab53ab640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ab0104290 0x7f1ab0199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.490+0000 7f1ab53ab640 1 --2- 172.21.15.44:0/3404095274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ab0101590 0x7f1ab0199240 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:02:58.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.490+0000 7f1ab53ab640 1 -- 172.21.15.44:0/3404095274 >> 172.21.15.44:0/3404095274 conn(0x7f1ab0076bd0 msgr2=0x7f1ab0104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:02:58.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.491+0000 7f1ab53ab640 1 -- 172.21.15.44:0/3404095274 shutdown_connections 2024-08-20T03:02:58.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:02:58.491+0000 7f1ab53ab640 1 -- 172.21.15.44:0/3404095274 wait complete. 2024-08-20T03:02:59.316 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-20T03:02:59.628 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:00.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:02:59 smithi044 ceph-mon[18790]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-20T03:03:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:02:59 smithi165 ceph-mon[22357]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-20T03:03:00.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:02:59 smithi099 ceph-mon[21984]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-20T03:03:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.199+0000 7f06df520640 1 -- 172.21.15.44:0/2042040349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06d8101990 msgr2=0x7f06d8101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.199+0000 7f06df520640 1 --2- 172.21.15.44:0/2042040349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06d8101990 0x7f06d8101d70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009900 tx=0x7f06d402a410 comp rx=0 tx=0).stop 2024-08-20T03:03:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.199+0000 7f06df520640 1 -- 172.21.15.44:0/2042040349 shutdown_connections 2024-08-20T03:03:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.199+0000 7f06df520640 1 --2- 172.21.15.44:0/2042040349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d8107d20 0x7f06d810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.199+0000 7f06df520640 1 --2- 172.21.15.44:0/2042040349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d81073a0 0x7f06d81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.199+0000 7f06df520640 1 --2- 172.21.15.44:0/2042040349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06d8101990 0x7f06d8101d70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.199+0000 7f06df520640 1 -- 172.21.15.44:0/2042040349 >> 172.21.15.44:0/2042040349 conn(0x7f06d80fd440 msgr2=0x7f06d80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.200+0000 7f06df520640 1 -- 172.21.15.44:0/2042040349 shutdown_connections 2024-08-20T03:03:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.200+0000 7f06df520640 1 -- 172.21.15.44:0/2042040349 wait complete. 2024-08-20T03:03:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06df520640 1 Processor -- start 2024-08-20T03:03:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06df520640 1 -- start start 2024-08-20T03:03:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06df520640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d8101990 0x7f06d810af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06df520640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06d81073a0 0x7f06d810b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06df520640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d8107d20 0x7f06d81ab150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06df520640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06d810fee0 con 0x7f06d81073a0 2024-08-20T03:03:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06df520640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06d810fd60 con 0x7f06d8101990 2024-08-20T03:03:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06df520640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8110060 con 0x7f06d8107d20 2024-08-20T03:03:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06dd2bc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d8101990 0x7f06d810af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06dd2bc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d8101990 0x7f06d810af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45624/0 (socket says 172.21.15.44:45624) 2024-08-20T03:03:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06dcabb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06d81073a0 0x7f06d810b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06dd2bc640 1 -- 172.21.15.44:0/1185767125 learned_addr learned my addr 172.21.15.44:0/1185767125 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.201+0000 7f06ddabd640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d8107d20 0x7f06d81ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.202+0000 7f06dcabb640 1 -- 172.21.15.44:0/1185767125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d8107d20 msgr2=0x7f06d81ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.202+0000 7f06dcabb640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d8107d20 0x7f06d81ab150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.202+0000 7f06dcabb640 1 -- 172.21.15.44:0/1185767125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d8101990 msgr2=0x7f06d810af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.202+0000 7f06dcabb640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d8101990 0x7f06d810af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.202+0000 7f06dcabb640 1 -- 172.21.15.44:0/1185767125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d81ab7f0 con 0x7f06d81073a0 2024-08-20T03:03:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.202+0000 7f06dcabb640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06d81073a0 0x7f06d810b490 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f06c8005fd0 tx=0x7f06c800de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.202+0000 7f06ce7fc640 1 -- 172.21.15.44:0/1185767125 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8004230 con 0x7f06d81073a0 2024-08-20T03:03:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.202+0000 7f06df520640 1 -- 172.21.15.44:0/1185767125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d81abae0 con 0x7f06d81073a0 2024-08-20T03:03:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.202+0000 7f06df520640 1 -- 172.21.15.44:0/1185767125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d81ac070 con 0x7f06d81073a0 2024-08-20T03:03:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.202+0000 7f06ce7fc640 1 -- 172.21.15.44:0/1185767125 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06c8009d90 con 0x7f06d81073a0 2024-08-20T03:03:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.202+0000 7f06ce7fc640 1 -- 172.21.15.44:0/1185767125 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8020060 con 0x7f06d81073a0 2024-08-20T03:03:00.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.206+0000 7f06e1919740 1 Processor -- start 2024-08-20T03:03:00.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.206+0000 7f06e1919740 1 -- start start 2024-08-20T03:03:00.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.206+0000 7f06e1919740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 0x5654ca86fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.206+0000 7f06e1919740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca870070 0x5654ca87bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.206+0000 7f06e1919740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca87c000 0x5654ca87e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.206+0000 7f06e1919740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5654ca7e6930 con 0x5654ca870070 2024-08-20T03:03:00.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.206+0000 7f06e1919740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5654ca7d5e30 con 0x5654ca86f750 2024-08-20T03:03:00.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.206+0000 7f06e1919740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5654ca7d9e70 con 0x5654ca87c000 2024-08-20T03:03:00.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06ce7fc640 1 -- 172.21.15.44:0/1185767125 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f06c80035f0 con 0x7f06d81073a0 2024-08-20T03:03:00.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06ddabd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 0x5654ca86fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06ddabd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 0x5654ca86fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45636/0 (socket says 172.21.15.44:45636) 2024-08-20T03:03:00.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06ddabd640 1 -- 172.21.15.44:0/747889154 learned_addr learned my addr 172.21.15.44:0/747889154 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:00.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06dcabb640 1 --2- 172.21.15.44:0/747889154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca87c000 0x5654ca87e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/747889154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca870070 0x5654ca87bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06ddabd640 1 -- 172.21.15.44:0/747889154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca87c000 msgr2=0x5654ca87e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:00.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06ddabd640 1 --2- 172.21.15.44:0/747889154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca87c000 0x5654ca87e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06ddabd640 1 -- 172.21.15.44:0/747889154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca870070 msgr2=0x5654ca87bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:00.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06ddabd640 1 --2- 172.21.15.44:0/747889154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca870070 0x5654ca87bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06ddabd640 1 -- 172.21.15.44:0/747889154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ca535650 con 0x5654ca86f750 2024-08-20T03:03:00.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06dcabb640 1 --2- 172.21.15.44:0/747889154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca87c000 0x5654ca87e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:00.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/747889154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca870070 0x5654ca87bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:00.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.207+0000 7f06ddabd640 1 --2- 172.21.15.44:0/747889154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 0x5654ca86fb30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009f70 tx=0x7f06d0010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=510e26ef2c04e7d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:00.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.208+0000 7f06c37fe640 1 -- 172.21.15.44:0/747889154 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0016070 con 0x5654ca86f750 2024-08-20T03:03:00.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.208+0000 7f06c37fe640 1 -- 172.21.15.44:0/747889154 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d0004d20 con 0x5654ca86f750 2024-08-20T03:03:00.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.208+0000 7f06c37fe640 1 -- 172.21.15.44:0/747889154 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d00127d0 con 0x5654ca86f750 2024-08-20T03:03:00.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.208+0000 7f06ce7fc640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f06c40b3dc0 0x7f06c40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.208+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f06c40b3dc0 0x7f06c40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.209+0000 7f06e1919740 1 -- 172.21.15.44:0/747889154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 msgr2=0x5654ca86fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:00.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.209+0000 7f06e1919740 1 --2- 172.21.15.44:0/747889154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 0x5654ca86fb30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009f70 tx=0x7f06d0010800 comp rx=0 tx=0).stop 2024-08-20T03:03:00.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.209+0000 7f06e1919740 1 -- 172.21.15.44:0/747889154 shutdown_connections 2024-08-20T03:03:00.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.209+0000 7f06e1919740 1 --2- 172.21.15.44:0/747889154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca87c000 0x5654ca87e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.209+0000 7f06e1919740 1 --2- 172.21.15.44:0/747889154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca870070 0x5654ca87bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.209+0000 7f06e1919740 1 --2- 172.21.15.44:0/747889154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 0x5654ca86fb30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.209+0000 7f06e1919740 1 -- 172.21.15.44:0/747889154 >> 172.21.15.44:0/747889154 conn(0x5654ca86d580 msgr2=0x5654ca86e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:00.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.209+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f06c40b3dc0 0x7f06c40b6280 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f06d4031440 tx=0x7f06d40058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:00.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.209+0000 7f06ce7fc640 1 -- 172.21.15.44:0/1185767125 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f06c80e3a60 con 0x7f06d81073a0 2024-08-20T03:03:00.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.209+0000 7f06e1919740 1 -- 172.21.15.44:0/747889154 shutdown_connections 2024-08-20T03:03:00.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.209+0000 7f06e1919740 1 -- 172.21.15.44:0/747889154 wait complete. 2024-08-20T03:03:00.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.210+0000 7f06e1919740 1 -- start start 2024-08-20T03:03:00.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.210+0000 7f06e1919740 1 Processor -- start 2024-08-20T03:03:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.210+0000 7f06e1919740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 0x5654ca878fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.210+0000 7f06e1919740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca870070 0x5654ca879500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.210+0000 7f06e1919740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca87c000 0x5654ca879a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06e1919740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5654ca638700 con 0x5654ca87c000 2024-08-20T03:03:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06e1919740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5654ca75c810 con 0x5654ca86f750 2024-08-20T03:03:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06e1919740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5654ca87ecb0 con 0x5654ca870070 2024-08-20T03:03:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06e1919740 10 client.0 _ll_register_callbacks cb 0x5654ca7cf700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06ddabd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca87c000 0x5654ca879a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06ddabd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca87c000 0x5654ca879a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54848/0 (socket says 172.21.15.44:54848) 2024-08-20T03:03:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06ddabd640 1 -- 172.21.15.44:0/754779603 learned_addr learned my addr 172.21.15.44:0/754779603 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 0x5654ca878fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06dcabb640 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca870070 0x5654ca879500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06ddabd640 1 -- 172.21.15.44:0/754779603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca870070 msgr2=0x5654ca879500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06ddabd640 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca870070 0x5654ca879500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06ddabd640 1 -- 172.21.15.44:0/754779603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 msgr2=0x5654ca878fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06ddabd640 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 0x5654ca878fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06ddabd640 1 -- 172.21.15.44:0/754779603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ca888090 con 0x5654ca87c000 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 0x5654ca878fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06ddabd640 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca87c000 0x5654ca879a40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f06d0011010 tx=0x7f06d0012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79d15f9f5ab1864 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.211+0000 7f06c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.212+0000 7f06c0ff9640 1 -- 172.21.15.44:0/754779603 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0016070 con 0x5654ca87c000 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.212+0000 7f06c0ff9640 1 -- 172.21.15.44:0/754779603 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f06d0012d10 con 0x5654ca87c000 2024-08-20T03:03:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.212+0000 7f06e1919740 1 -- 172.21.15.44:0/754779603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654ca785150 con 0x5654ca87c000 2024-08-20T03:03:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.212+0000 7f06c0ff9640 1 -- 172.21.15.44:0/754779603 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f06d0031050 con 0x5654ca87c000 2024-08-20T03:03:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.212+0000 7f06e1919740 10 client.15639 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.212+0000 7f06e1919740 1 -- 172.21.15.44:0/754779603 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654ca88aa40 con 0x5654ca87c000 2024-08-20T03:03:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.212+0000 7f06c0ff9640 1 -- 172.21.15.44:0/754779603 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5654ca785150 con 0x5654ca87c000 2024-08-20T03:03:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.212+0000 7f06c0ff9640 1 -- 172.21.15.44:0/754779603 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f06d000f040 con 0x5654ca87c000 2024-08-20T03:03:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.212+0000 7f06e1919740 10 client.15639 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.212+0000 7f06e1919740 10 client.15639 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:03:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.212+0000 7f06e1919740 20 client.15639 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.213+0000 7f06e1919740 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5654ca88bd90 0x5654ca88e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.213+0000 7f06e1919740 4 client.15639 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:00.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.213+0000 7f06e1919740 1 -- 172.21.15.44:0/754779603 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654ca7e8880 con 0x5654ca88bd90 2024-08-20T03:03:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.214+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5654ca88bd90 0x5654ca88e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.214+0000 7f06c0ff9640 10 client.15639 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:03:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.214+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5654ca88bd90 0x5654ca88e180 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e3497f6fcf78a001 server_cookie=9b519fd4aa649d6f in_seq=0 out_seq=0 2024-08-20T03:03:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.215+0000 7f06c0ff9640 1 -- 172.21.15.44:0/754779603 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654ca7e8880 con 0x5654ca88bd90 2024-08-20T03:03:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.215+0000 7f06c0ff9640 10 client.15639 handle_command_reply: tid=0 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.215+0000 7f06e1919740 1 client.15639 shutdown 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.215+0000 7f06e1919740 1 -- 172.21.15.44:0/754779603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca87c000 msgr2=0x5654ca879a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.215+0000 7f06e1919740 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca87c000 0x5654ca879a40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f06d0011010 tx=0x7f06d0012760 comp rx=0 tx=0).stop 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.216+0000 7f06e1919740 1 -- 172.21.15.44:0/754779603 shutdown_connections 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.216+0000 7f06e1919740 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5654ca88bd90 0x5654ca88e180 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.216+0000 7f06e1919740 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca87c000 0x5654ca879a40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.216+0000 7f06e1919740 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca870070 0x5654ca879500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.216+0000 7f06c0ff9640 0 client.15639 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.216+0000 7f06e1919740 1 --2- 172.21.15.44:0/754779603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca86f750 0x5654ca878fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.216+0000 7f06e1919740 1 -- 172.21.15.44:0/754779603 >> 172.21.15.44:0/754779603 conn(0x5654ca86d580 msgr2=0x5654ca8730f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.216+0000 7f06e1919740 1 -- 172.21.15.44:0/754779603 shutdown_connections 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.216+0000 7f06e1919740 1 -- 172.21.15.44:0/754779603 wait complete. 2024-08-20T03:03:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.216+0000 7f06e1919740 20 client.15639 trim_cache size 0 max 16384 2024-08-20T03:03:00.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.242+0000 7f06e1919740 1 Processor -- start 2024-08-20T03:03:00.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.243+0000 7f06e1919740 1 -- start start 2024-08-20T03:03:00.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.243+0000 7f06e1919740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 0x5654ca8737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.243+0000 7f06e1919740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca870070 0x5654ca878720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.243+0000 7f06e1919740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca87c000 0x5654ca8937e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.243+0000 7f06e1919740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b48e0 con 0x5654ca86f750 2024-08-20T03:03:00.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.243+0000 7f06e1919740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b6420 con 0x5654ca87c000 2024-08-20T03:03:00.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.243+0000 7f06e1919740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b4fb0 con 0x5654ca870070 2024-08-20T03:03:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.243+0000 7f06dd2bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 0x5654ca8737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.243+0000 7f06dd2bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 0x5654ca8737c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54862/0 (socket says 172.21.15.44:54862) 2024-08-20T03:03:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.243+0000 7f06dd2bc640 1 -- 172.21.15.44:0/1668539903 learned_addr learned my addr 172.21.15.44:0/1668539903 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.243+0000 7f06ddabd640 1 --2- 172.21.15.44:0/1668539903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca87c000 0x5654ca8937e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.244+0000 7f06dd2bc640 1 -- 172.21.15.44:0/1668539903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca870070 msgr2=0x5654ca878720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.244+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/1668539903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca870070 0x5654ca878720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.244+0000 7f06dd2bc640 1 -- 172.21.15.44:0/1668539903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca87c000 msgr2=0x5654ca8937e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.244+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/1668539903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca87c000 0x5654ca8937e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.244+0000 7f06dd2bc640 1 -- 172.21.15.44:0/1668539903 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654ca86e000 con 0x5654ca86f750 2024-08-20T03:03:00.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.244+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/1668539903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 0x5654ca8737c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f06d4052fd0 tx=0x7f06d403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27775311783ad99 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:00.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.245+0000 7f06c2ffd640 1 -- 172.21.15.44:0/1668539903 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4035040 con 0x5654ca86f750 2024-08-20T03:03:00.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.245+0000 7f06c2ffd640 1 -- 172.21.15.44:0/1668539903 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d4058070 con 0x5654ca86f750 2024-08-20T03:03:00.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.245+0000 7f06c2ffd640 1 -- 172.21.15.44:0/1668539903 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4052610 con 0x5654ca86f750 2024-08-20T03:03:00.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.246+0000 7f06e1919740 1 -- 172.21.15.44:0/1668539903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 msgr2=0x5654ca8737c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:00.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.246+0000 7f06e1919740 1 --2- 172.21.15.44:0/1668539903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 0x5654ca8737c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f06d4052fd0 tx=0x7f06d403e000 comp rx=0 tx=0).stop 2024-08-20T03:03:00.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.246+0000 7f06e1919740 1 -- 172.21.15.44:0/1668539903 shutdown_connections 2024-08-20T03:03:00.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.246+0000 7f06e1919740 1 --2- 172.21.15.44:0/1668539903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca87c000 0x5654ca8937e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.246+0000 7f06e1919740 1 --2- 172.21.15.44:0/1668539903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca870070 0x5654ca878720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.246+0000 7f06e1919740 1 --2- 172.21.15.44:0/1668539903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 0x5654ca8737c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.246+0000 7f06e1919740 1 -- 172.21.15.44:0/1668539903 >> 172.21.15.44:0/1668539903 conn(0x5654ca86d580 msgr2=0x5654ca87ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:00.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.246+0000 7f06e1919740 1 -- 172.21.15.44:0/1668539903 shutdown_connections 2024-08-20T03:03:00.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.246+0000 7f06e1919740 1 -- 172.21.15.44:0/1668539903 wait complete. 2024-08-20T03:03:00.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.246+0000 7f06e1919740 1 -- start start 2024-08-20T03:03:00.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.246+0000 7f06e1919740 1 Processor -- start 2024-08-20T03:03:00.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.247+0000 7f06e1919740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 0x5654ca87ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.247+0000 7f06e1919740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca870070 0x5654ca87f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.247+0000 7f06dd2bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 0x5654ca87ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.247+0000 7f06dd2bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 0x5654ca87ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54864/0 (socket says 172.21.15.44:54864) 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.247+0000 7f06dd2bc640 1 -- 172.21.15.44:0/956771114 learned_addr learned my addr 172.21.15.44:0/956771114 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.247+0000 7f06e1919740 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca87c000 0x5654ca87f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.247+0000 7f06e1919740 1 -- 172.21.15.44:0/956771114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b5d50 con 0x5654ca86f750 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.247+0000 7f06e1919740 1 -- 172.21.15.44:0/956771114 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b5680 con 0x5654ca870070 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.247+0000 7f06e1919740 1 -- 172.21.15.44:0/956771114 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b28c0 con 0x5654ca87c000 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.247+0000 7f06e1919740 10 client.0 _ll_register_callbacks cb 0x7f06d0009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06dcabb640 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca870070 0x5654ca87f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06dd2bc640 1 -- 172.21.15.44:0/956771114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca87c000 msgr2=0x5654ca87f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca87c000 0x5654ca87f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06dd2bc640 1 -- 172.21.15.44:0/956771114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca870070 msgr2=0x5654ca87f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:00.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca870070 0x5654ca87f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06dd2bc640 1 -- 172.21.15.44:0/956771114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654ca8f4950 con 0x5654ca86f750 2024-08-20T03:03:00.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06dcabb640 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca870070 0x5654ca87f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:00.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06dd2bc640 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 0x5654ca87ef00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f06d402a450 tx=0x7f06d4003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e013de6af092825 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:00.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:00.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06c0ff9640 1 -- 172.21.15.44:0/956771114 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4061070 con 0x5654ca86f750 2024-08-20T03:03:00.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06c0ff9640 1 -- 172.21.15.44:0/956771114 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f06d40037f0 con 0x5654ca86f750 2024-08-20T03:03:00.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06e1919740 1 -- 172.21.15.44:0/956771114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d4058070 con 0x5654ca86f750 2024-08-20T03:03:00.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06c0ff9640 1 -- 172.21.15.44:0/956771114 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f06d40039f0 con 0x5654ca86f750 2024-08-20T03:03:00.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06e1919740 10 client.15645 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:00.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.248+0000 7f06e1919740 1 -- 172.21.15.44:0/956771114 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5654ca8f5c10 con 0x5654ca86f750 2024-08-20T03:03:00.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.249+0000 7f06c0ff9640 1 -- 172.21.15.44:0/956771114 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d4003b90 con 0x5654ca86f750 2024-08-20T03:03:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.249+0000 7f06c0ff9640 1 -- 172.21.15.44:0/956771114 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f06d403c560 con 0x5654ca86f750 2024-08-20T03:03:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.249+0000 7f06e1919740 10 client.15645 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.249+0000 7f06e1919740 10 client.15645 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:03:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.249+0000 7f06e1919740 20 client.15645 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.249+0000 7f06e1919740 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5654ca8f62d0 0x5654ca8f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.249+0000 7f06e1919740 4 client.15645 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.249+0000 7f06e1919740 1 -- 172.21.15.44:0/956771114 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f06d4058070 con 0x5654ca8f62d0 2024-08-20T03:03:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.249+0000 7f06dcabb640 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5654ca8f62d0 0x5654ca8f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.250+0000 7f06dcabb640 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5654ca8f62d0 0x5654ca8f86c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b36f7ee7ec41a9a server_cookie=e8b58cd02b8ce4f6 in_seq=0 out_seq=0 2024-08-20T03:03:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.250+0000 7f06c0ff9640 10 client.15645 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:03:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.250+0000 7f06c0ff9640 1 -- 172.21.15.44:0/956771114 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f06d4058070 con 0x5654ca8f62d0 2024-08-20T03:03:00.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.250+0000 7f06c0ff9640 10 client.15645 handle_command_reply: tid=0 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.250+0000 7f06e1919740 1 client.15645 shutdown 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.250+0000 7f06e1919740 1 -- 172.21.15.44:0/956771114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 msgr2=0x5654ca87ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.250+0000 7f06e1919740 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 0x5654ca87ef00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f06d402a450 tx=0x7f06d4003ef0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.251+0000 7f06e1919740 1 -- 172.21.15.44:0/956771114 shutdown_connections 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.251+0000 7f06e1919740 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5654ca8f62d0 0x5654ca8f86c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.251+0000 7f06e1919740 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654ca87c000 0x5654ca87f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.251+0000 7f06e1919740 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654ca870070 0x5654ca87f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.251+0000 7f06e1919740 1 --2- 172.21.15.44:0/956771114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654ca86f750 0x5654ca87ef00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.251+0000 7f06e1919740 1 -- 172.21.15.44:0/956771114 >> 172.21.15.44:0/956771114 conn(0x5654ca86d580 msgr2=0x5654ca86bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.251+0000 7f06c0ff9640 0 client.15645 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.251+0000 7f06e1919740 1 -- 172.21.15.44:0/956771114 shutdown_connections 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.251+0000 7f06e1919740 1 -- 172.21.15.44:0/956771114 wait complete. 2024-08-20T03:03:00.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.251+0000 7f06e1919740 20 client.15645 trim_cache size 0 max 16384 2024-08-20T03:03:00.256 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:00.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.252+0000 7f06df520640 1 -- 172.21.15.44:0/1185767125 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f06c40b3dc0 msgr2=0x7f06c40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:00.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.252+0000 7f06df520640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f06c40b3dc0 0x7f06c40b6280 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f06d4031440 tx=0x7f06d40058d0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.252+0000 7f06df520640 1 -- 172.21.15.44:0/1185767125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06d81073a0 msgr2=0x7f06d810b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:00.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.252+0000 7f06df520640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06d81073a0 0x7f06d810b490 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f06c8005fd0 tx=0x7f06c800de70 comp rx=0 tx=0).stop 2024-08-20T03:03:00.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.252+0000 7f06df520640 1 -- 172.21.15.44:0/1185767125 shutdown_connections 2024-08-20T03:03:00.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.252+0000 7f06df520640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f06c40b3dc0 0x7f06c40b6280 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.252+0000 7f06df520640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d8107d20 0x7f06d81ab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.252+0000 7f06df520640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06d81073a0 0x7f06d810b490 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.252+0000 7f06df520640 1 --2- 172.21.15.44:0/1185767125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06d8101990 0x7f06d810af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:00.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.252+0000 7f06df520640 1 -- 172.21.15.44:0/1185767125 >> 172.21.15.44:0/1185767125 conn(0x7f06d80fd440 msgr2=0x7f06d8105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:00.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.252+0000 7f06df520640 1 -- 172.21.15.44:0/1185767125 shutdown_connections 2024-08-20T03:03:00.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:00.252+0000 7f06df520640 1 -- 172.21.15.44:0/1185767125 wait complete. 2024-08-20T03:03:00.995 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:03:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:03:00.711+0000 7efc54309640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:03:00.995 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:03:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:03:00.712+0000 7efc54309640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:03:00.995 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:03:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:03:00.712+0000 7efc54309640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:03:00.995 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:03:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:03:00.712+0000 7efc54309640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:03:00.996 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:03:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:03:00.712+0000 7efc54309640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T03:03:01.034 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T03:03:01.427 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.947+0000 7f5d9959a640 1 -- 172.21.15.44:0/1971926319 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9402c040 con 0x7f5d9c106a50 2024-08-20T03:03:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.947+0000 7f5da0d06640 1 -- 172.21.15.44:0/1971926319 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d9c106a50 msgr2=0x7f5d9c107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.947+0000 7f5da0d06640 1 --2- 172.21.15.44:0/1971926319 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d9c106a50 0x7f5d9c107360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009ca0 tx=0x7f5d9402a950 comp rx=0 tx=0).stop 2024-08-20T03:03:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.947+0000 7f5da0d06640 1 -- 172.21.15.44:0/1971926319 shutdown_connections 2024-08-20T03:03:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.947+0000 7f5da0d06640 1 --2- 172.21.15.44:0/1971926319 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d9c106a50 0x7f5d9c107360 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.947+0000 7f5da0d06640 1 --2- 172.21.15.44:0/1971926319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d9c101de0 0x7f5d9c1062a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.947+0000 7f5da0d06640 1 --2- 172.21.15.44:0/1971926319 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d9c1014c0 0x7f5d9c1018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.947+0000 7f5da0d06640 1 -- 172.21.15.44:0/1971926319 >> 172.21.15.44:0/1971926319 conn(0x7f5d9c076ba0 msgr2=0x7f5d9c076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.947+0000 7f5da0d06640 1 -- 172.21.15.44:0/1971926319 shutdown_connections 2024-08-20T03:03:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.948+0000 7f5da0d06640 1 -- 172.21.15.44:0/1971926319 wait complete. 2024-08-20T03:03:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.949+0000 7f5da0d06640 1 Processor -- start 2024-08-20T03:03:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.949+0000 7f5da0d06640 1 -- start start 2024-08-20T03:03:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.949+0000 7f5da0d06640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d9c1014c0 0x7f5d9c19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.949+0000 7f5da0d06640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d9c101de0 0x7f5d9c19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.949+0000 7f5da0d06640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d9c106a50 0x7f5d9c1a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.949+0000 7f5da0d06640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c119010 con 0x7f5d9c1014c0 2024-08-20T03:03:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.949+0000 7f5da0d06640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c118e90 con 0x7f5d9c106a50 2024-08-20T03:03:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.949+0000 7f5da0d06640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c119190 con 0x7f5d9c101de0 2024-08-20T03:03:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.950+0000 7f5d9a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d9c1014c0 0x7f5d9c19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.950+0000 7f5d9a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d9c1014c0 0x7f5d9c19f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54928/0 (socket says 172.21.15.44:54928) 2024-08-20T03:03:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.950+0000 7f5d9a59c640 1 -- 172.21.15.44:0/218292212 learned_addr learned my addr 172.21.15.44:0/218292212 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.950+0000 7f5d99d9b640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d9c101de0 0x7f5d9c19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.950+0000 7f5d9ad9d640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d9c106a50 0x7f5d9c1a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.950+0000 7f5d9a59c640 1 -- 172.21.15.44:0/218292212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d9c101de0 msgr2=0x7f5d9c19f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:01.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.950+0000 7f5d9a59c640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d9c101de0 0x7f5d9c19f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.950+0000 7f5d9a59c640 1 -- 172.21.15.44:0/218292212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d9c106a50 msgr2=0x7f5d9c1a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.950+0000 7f5d9a59c640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d9c106a50 0x7f5d9c1a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.950+0000 7f5d9a59c640 1 -- 172.21.15.44:0/218292212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c1a5530 con 0x7f5d9c1014c0 2024-08-20T03:03:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.951+0000 7f5d9ad9d640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d9c106a50 0x7f5d9c1a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.951+0000 7f5d9a59c640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d9c1014c0 0x7f5d9c19f490 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00e9f0 tx=0x7f5d8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.951+0000 7f5d8b7fe640 1 -- 172.21.15.44:0/218292212 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00cd80 con 0x7f5d9c1014c0 2024-08-20T03:03:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.951+0000 7f5da0d06640 1 -- 172.21.15.44:0/218292212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9c1a5820 con 0x7f5d9c1014c0 2024-08-20T03:03:01.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.951+0000 7f5da0d06640 1 -- 172.21.15.44:0/218292212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9c1ac0c0 con 0x7f5d9c1014c0 2024-08-20T03:03:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.951+0000 7f5d8b7fe640 1 -- 172.21.15.44:0/218292212 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c0108e0 con 0x7f5d9c1014c0 2024-08-20T03:03:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.951+0000 7f5d8b7fe640 1 -- 172.21.15.44:0/218292212 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c018860 con 0x7f5d9c1014c0 2024-08-20T03:03:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.955+0000 7f5da3bcf740 1 Processor -- start 2024-08-20T03:03:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.941+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3600433447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0e41026c0 msgr2=0x7fe0e4102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.941+0000 7fe0e9ad4640 1 --2- 172.21.15.44:0/3600433447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0e41026c0 0x7fe0e4102aa0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009900 tx=0x7fe0d402a410 comp rx=0 tx=0).stop 2024-08-20T03:03:01.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.941+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3600433447 shutdown_connections 2024-08-20T03:03:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.941+0000 7fe0e9ad4640 1 --2- 172.21.15.44:0/3600433447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0e4107e30 0x7fe0e410e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.941+0000 7fe0e9ad4640 1 --2- 172.21.15.44:0/3600433447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0e41074b0 0x7fe0e41078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.941+0000 7fe0e9ad4640 1 --2- 172.21.15.44:0/3600433447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0e41026c0 0x7fe0e4102aa0 secure :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009900 tx=0x7fe0d402a410 comp rx=0 tx=0).stop 2024-08-20T03:03:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.941+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3600433447 >> 172.21.15.44:0/3600433447 conn(0x7fe0e4076b50 msgr2=0x7fe0e4076f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.942+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3600433447 shutdown_connections 2024-08-20T03:03:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.942+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3600433447 wait complete. 2024-08-20T03:03:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.943+0000 7fe0e9ad4640 1 Processor -- start 2024-08-20T03:03:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.943+0000 7fe0e9ad4640 1 -- start start 2024-08-20T03:03:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.943+0000 7fe0e9ad4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0e41074b0 0x7fe0e4197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.943+0000 7fe0e9ad4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0e4107e30 0x7fe0e41975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.943+0000 7fe0e9ad4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0e419c940 0x7fe0e419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.943+0000 7fe0e9ad4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e4110920 con 0x7fe0e41074b0 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.943+0000 7fe0e9ad4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41107a0 con 0x7fe0e419c940 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.943+0000 7fe0e9ad4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e4110aa0 con 0x7fe0e4107e30 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.944+0000 7fe0e37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0e41074b0 0x7fe0e4197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.944+0000 7fe0e37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0e41074b0 0x7fe0e4197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54912/0 (socket says 172.21.15.44:54912) 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.944+0000 7fe0e37fe640 1 -- 172.21.15.44:0/3591717803 learned_addr learned my addr 172.21.15.44:0/3591717803 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.944+0000 7fe0e37fe640 1 -- 172.21.15.44:0/3591717803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0e4107e30 msgr2=0x7fe0e41975a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.944+0000 7fe0e37fe640 1 --2- 172.21.15.44:0/3591717803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0e4107e30 0x7fe0e41975a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.944+0000 7fe0e37fe640 1 -- 172.21.15.44:0/3591717803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0e419c940 msgr2=0x7fe0e419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.944+0000 7fe0e37fe640 1 --2- 172.21.15.44:0/3591717803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0e419c940 0x7fe0e419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.944+0000 7fe0e37fe640 1 -- 172.21.15.44:0/3591717803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e419d4d0 con 0x7fe0e41074b0 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.945+0000 7fe0e37fe640 1 --2- 172.21.15.44:0/3591717803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0e41074b0 0x7fe0e4197060 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4006010 tx=0x7fe0d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.945+0000 7fe0e0ff9640 1 -- 172.21.15.44:0/3591717803 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d403f030 con 0x7fe0e41074b0 2024-08-20T03:03:01.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.945+0000 7fe0e0ff9640 1 -- 172.21.15.44:0/3591717803 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0d4003aa0 con 0x7fe0e41074b0 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.945+0000 7fe0e0ff9640 1 -- 172.21.15.44:0/3591717803 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4035070 con 0x7fe0e41074b0 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.945+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3591717803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e419d760 con 0x7fe0e41074b0 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.946+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3591717803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e41a3d80 con 0x7fe0e41074b0 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.949+0000 7fe0ebecd740 1 Processor -- start 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.949+0000 7fe0ebecd740 1 -- start start 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.949+0000 7fe0ebecd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd4f880 0x55ddebd4fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.950+0000 7fe0ebecd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd501a0 0x55ddebd5bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.950+0000 7fe0ebecd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd5c130 0x55ddebd5e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.950+0000 7fe0ebecd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ddebcc6930 con 0x55ddebd5c130 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.950+0000 7fe0ebecd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ddebcb5e30 con 0x55ddebd4f880 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.950+0000 7fe0ebecd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ddebcb9e70 con 0x55ddebd501a0 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.950+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd4f880 0x55ddebd4fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.950+0000 7fe0e2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd5c130 0x55ddebd5e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.950+0000 7fe0e0ff9640 1 -- 172.21.15.44:0/3591717803 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7fe0d4042030 con 0x7fe0e41074b0 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.950+0000 7fe0e2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd5c130 0x55ddebd5e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54936/0 (socket says 172.21.15.44:54936) 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.950+0000 7fe0e2ffd640 1 -- 172.21.15.44:0/566158478 learned_addr learned my addr 172.21.15.44:0/566158478 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.951+0000 7fe0e37fe640 1 --2- 172.21.15.44:0/566158478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd501a0 0x55ddebd5bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.951+0000 7fe0e2ffd640 1 -- 172.21.15.44:0/566158478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd501a0 msgr2=0x55ddebd5bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.952+0000 7fe0e2ffd640 1 --2- 172.21.15.44:0/566158478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd501a0 0x55ddebd5bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.952+0000 7fe0e2ffd640 1 -- 172.21.15.44:0/566158478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd4f880 msgr2=0x55ddebd4fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.952+0000 7fe0e2ffd640 1 --2- 172.21.15.44:0/566158478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd4f880 0x55ddebd4fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.952+0000 7fe0e2ffd640 1 -- 172.21.15.44:0/566158478 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddeba15650 con 0x55ddebd5c130 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.952+0000 7fe0e3fff640 1 --2- 172.21.15.44:0/566158478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd4f880 0x55ddebd4fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.952+0000 7fe0e37fe640 1 --2- 172.21.15.44:0/566158478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd501a0 0x55ddebd5bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.952+0000 7fe0e2ffd640 1 --2- 172.21.15.44:0/566158478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd5c130 0x55ddebd5e520 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800bc00 tx=0x7fe0d80093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8ba143b1e3e2ac server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:01.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.953+0000 7fe0e0ff9640 1 --2- 172.21.15.44:0/3591717803 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0cc0b3dc0 0x7fe0cc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.953+0000 7fe0e0ff9640 1 -- 172.21.15.44:0/3591717803 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe0d41042f0 con 0x7fe0e41074b0 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.953+0000 7fe0e3fff640 1 --2- 172.21.15.44:0/3591717803 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0cc0b3dc0 0x7fe0cc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.953+0000 7fe0c9ffb640 1 -- 172.21.15.44:0/566158478 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8009e10 con 0x55ddebd5c130 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.953+0000 7fe0c9ffb640 1 -- 172.21.15.44:0/566158478 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0d80035f0 con 0x55ddebd5c130 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.953+0000 7fe0c9ffb640 1 -- 172.21.15.44:0/566158478 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8015650 con 0x55ddebd5c130 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.954+0000 7fe0ebecd740 1 -- 172.21.15.44:0/566158478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd5c130 msgr2=0x55ddebd5e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.954+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/566158478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd5c130 0x55ddebd5e520 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800bc00 tx=0x7fe0d80093d0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.955+0000 7fe0e3fff640 1 --2- 172.21.15.44:0/3591717803 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0cc0b3dc0 0x7fe0cc0b6280 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc005fd0 tx=0x7fe0dc0098e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.955+0000 7fe0ebecd740 1 -- 172.21.15.44:0/566158478 shutdown_connections 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.955+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/566158478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd5c130 0x55ddebd5e520 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.955+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/566158478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd501a0 0x55ddebd5bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.955+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/566158478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd4f880 0x55ddebd4fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.955+0000 7fe0ebecd740 1 -- 172.21.15.44:0/566158478 >> 172.21.15.44:0/566158478 conn(0x55ddebd4d6b0 msgr2=0x55ddebd4e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.955+0000 7f5da3bcf740 1 -- start start 2024-08-20T03:03:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.955+0000 7f5da3bcf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e15fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.955+0000 7f5da3bcf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 0x55b354e21f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.955+0000 7f5da3bcf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 0x55b354e24890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.955+0000 7f5da3bcf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b354d8b360 con 0x55b354e16510 2024-08-20T03:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.955+0000 7f5da3bcf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b354d7a860 con 0x55b354e15bf0 2024-08-20T03:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.955+0000 7f5da3bcf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b354d7e8a0 con 0x55b354e224a0 2024-08-20T03:03:01.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.955+0000 7fe0ebecd740 1 -- 172.21.15.44:0/566158478 shutdown_connections 2024-08-20T03:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.956+0000 7f5d99d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 0x55b354e24890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.956+0000 7f5d9ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e15fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.956+0000 7f5d99d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 0x55b354e24890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47448/0 (socket says 172.21.15.44:47448) 2024-08-20T03:03:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.956+0000 7f5d99d9b640 1 -- 172.21.15.44:0/105201931 learned_addr learned my addr 172.21.15.44:0/105201931 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.955+0000 7fe0ebecd740 1 -- 172.21.15.44:0/566158478 wait complete. 2024-08-20T03:03:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.956+0000 7fe0ebecd740 1 -- start start 2024-08-20T03:03:01.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.956+0000 7fe0ebecd740 1 Processor -- start 2024-08-20T03:03:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.956+0000 7f5d8b7fe640 1 -- 172.21.15.44:0/218292212 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 25) ==== 151184+0+0 (secure 0 0 0) 0x7f5d8c010430 con 0x7f5d9c1014c0 2024-08-20T03:03:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.956+0000 7fe0ebecd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd59120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.956+0000 7fe0ebecd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd501a0 0x55ddebd59660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.956+0000 7fe0ebecd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd5c130 0x55ddebd59ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.956+0000 7fe0ebecd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ddebb18700 con 0x55ddebd4f880 2024-08-20T03:03:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.956+0000 7fe0ebecd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ddebc3c810 con 0x55ddebd5c130 2024-08-20T03:03:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.956+0000 7fe0ebecd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ddebd5ede0 con 0x55ddebd501a0 2024-08-20T03:03:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.956+0000 7fe0ebecd740 10 client.0 _ll_register_callbacks cb 0x55ddebcaf700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.957+0000 7fe0e37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd59120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.957+0000 7fe0e2ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd501a0 0x55ddebd59660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.957+0000 7fe0e2ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd501a0 0x55ddebd59660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47460/0 (socket says 172.21.15.44:47460) 2024-08-20T03:03:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.957+0000 7fe0e2ffd640 1 -- 172.21.15.44:0/1441882772 learned_addr learned my addr 172.21.15.44:0/1441882772 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.957+0000 7fe0e37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd59120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54948/0 (socket says 172.21.15.44:54948) 2024-08-20T03:03:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.957+0000 7fe0e3fff640 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd5c130 0x55ddebd59ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.957+0000 7fe0e3fff640 1 -- 172.21.15.44:0/1441882772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd501a0 msgr2=0x55ddebd59660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.957+0000 7fe0e3fff640 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd501a0 0x55ddebd59660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.957+0000 7fe0e3fff640 1 -- 172.21.15.44:0/1441882772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 msgr2=0x55ddebd59120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.957+0000 7fe0e3fff640 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd59120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.957+0000 7fe0e3fff640 1 -- 172.21.15.44:0/1441882772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddebd5f180 con 0x55ddebd5c130 2024-08-20T03:03:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.958+0000 7f5d99d9b640 1 -- 172.21.15.44:0/105201931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 msgr2=0x55b354e15fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.958+0000 7f5d99d9b640 1 --2- 172.21.15.44:0/105201931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e15fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.958+0000 7f5d99d9b640 1 -- 172.21.15.44:0/105201931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 msgr2=0x55b354e21f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.958+0000 7f5d9a59c640 1 --2- 172.21.15.44:0/105201931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 0x55b354e21f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.958+0000 7fe0e37fe640 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd59120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:01.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.958+0000 7fe0e3fff640 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd5c130 0x55ddebd59ba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc011e90 tx=0x7fe0dc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39b46365fd02abf9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.958+0000 7fe0c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:03:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.958+0000 7f5d99d9b640 1 --2- 172.21.15.44:0/105201931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 0x55b354e21f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.958+0000 7f5d99d9b640 1 -- 172.21.15.44:0/105201931 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b354d21e40 con 0x55b354e224a0 2024-08-20T03:03:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.958+0000 7f5d9a59c640 1 --2- 172.21.15.44:0/105201931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 0x55b354e21f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.958+0000 7fe0c2ffd640 1 -- 172.21.15.44:0/1441882772 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc00f6e0 con 0x55ddebd5c130 2024-08-20T03:03:01.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.958+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1441882772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ddebc65150 con 0x55ddebd5c130 2024-08-20T03:03:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.959+0000 7f5d8b7fe640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d840b3dc0 0x7f5d840b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.959+0000 7f5d9ad9d640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d840b3dc0 0x7f5d840b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.958+0000 7fe0c2ffd640 1 -- 172.21.15.44:0/1441882772 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe0dc00f880 con 0x55ddebd5c130 2024-08-20T03:03:01.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.958+0000 7fe0c2ffd640 1 -- 172.21.15.44:0/1441882772 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe0dc009dc0 con 0x55ddebd5c130 2024-08-20T03:03:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.958+0000 7fe0ebecd740 10 client.25217 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.958+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1441882772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ddebd61b50 con 0x55ddebd5c130 2024-08-20T03:03:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.959+0000 7fe0c2ffd640 1 -- 172.21.15.44:0/1441882772 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ddebc65150 con 0x55ddebd5c130 2024-08-20T03:03:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.959+0000 7fe0c2ffd640 1 -- 172.21.15.44:0/1441882772 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe0dc026a90 con 0x55ddebd5c130 2024-08-20T03:03:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.959+0000 7fe0ebecd740 10 client.25217 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.959+0000 7fe0ebecd740 10 client.25217 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:03:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.959+0000 7fe0ebecd740 20 client.25217 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.959+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55ddebd63050 0x55ddebd654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.959+0000 7fe0ebecd740 4 client.25217 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:01.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.959+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1441882772 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddebcc8880 con 0x55ddebd63050 2024-08-20T03:03:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.960+0000 7f5d99d9b640 1 --2- 172.21.15.44:0/105201931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 0x55b354e24890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009c90 tx=0x7f5d90011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32f97665e3b32312 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:01.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.960+0000 7fe0e37fe640 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55ddebd63050 0x55ddebd654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.960+0000 7f5d8b7fe640 1 -- 172.21.15.44:0/218292212 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5d8c0e4030 con 0x7f5d9c1014c0 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.960+0000 7f5d88ff9640 1 -- 172.21.15.44:0/105201931 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9001e070 con 0x55b354e224a0 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.960+0000 7f5d88ff9640 1 -- 172.21.15.44:0/105201931 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d90019ae0 con 0x55b354e224a0 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.960+0000 7f5d88ff9640 1 -- 172.21.15.44:0/105201931 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9000ea40 con 0x55b354e224a0 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.961+0000 7f5da3bcf740 1 -- 172.21.15.44:0/105201931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 msgr2=0x55b354e24890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.961+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/105201931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 0x55b354e24890 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009c90 tx=0x7f5d90011810 comp rx=0 tx=0).stop 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.961+0000 7f5d9ad9d640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d840b3dc0 0x7f5d840b6280 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5d94006010 tx=0x7f5d94005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.961+0000 7f5da3bcf740 1 -- 172.21.15.44:0/105201931 shutdown_connections 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.961+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/105201931 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 0x55b354e24890 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.961+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/105201931 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 0x55b354e21f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.961+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/105201931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e15fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.961+0000 7f5da3bcf740 1 -- 172.21.15.44:0/105201931 >> 172.21.15.44:0/105201931 conn(0x55b354e13a20 msgr2=0x55b354e14bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:01.973 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.961+0000 7f5da3bcf740 1 -- 172.21.15.44:0/105201931 shutdown_connections 2024-08-20T03:03:01.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.961+0000 7fe0c2ffd640 10 client.25217 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:03:01.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.961+0000 7fe0e37fe640 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55ddebd63050 0x55ddebd654b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bf3b76f4b51730e server_cookie=eaf4fcbbc828c3d7 in_seq=0 out_seq=0 2024-08-20T03:03:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.962+0000 7f5da3bcf740 1 -- 172.21.15.44:0/105201931 wait complete. 2024-08-20T03:03:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.962+0000 7f5da3bcf740 1 -- start start 2024-08-20T03:03:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.962+0000 7f5da3bcf740 1 Processor -- start 2024-08-20T03:03:01.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.962+0000 7fe0c2ffd640 1 -- 172.21.15.44:0/1441882772 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddebcc8880 con 0x55ddebd63050 2024-08-20T03:03:01.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.962+0000 7fe0c2ffd640 10 client.25217 handle_command_reply: tid=0 2024-08-20T03:03:01.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.962+0000 7fe0ebecd740 1 client.25217 shutdown 2024-08-20T03:03:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.962+0000 7f5da3bcf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e21d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.962+0000 7f5da3bcf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e16510 0x55b354e1f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.962+0000 7f5da3bcf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e224a0 0x55b354e1f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.962+0000 7f5da3bcf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b354cb3ab0 con 0x55b354e224a0 2024-08-20T03:03:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.962+0000 7f5da3bcf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b354d01780 con 0x55b354e15bf0 2024-08-20T03:03:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.962+0000 7f5da3bcf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b354e25180 con 0x55b354e16510 2024-08-20T03:03:01.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.963+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1441882772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd5c130 msgr2=0x55ddebd59ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:01.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.963+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd5c130 0x55ddebd59ba0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc011e90 tx=0x7fe0dc004a40 comp rx=0 tx=0).stop 2024-08-20T03:03:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.962+0000 7f5da3bcf740 10 client.0 _ll_register_callbacks cb 0x55b354a89a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.963+0000 7f5d9a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e21d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.963+0000 7f5d9a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e21d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45762/0 (socket says 172.21.15.44:45762) 2024-08-20T03:03:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.963+0000 7f5d9a59c640 1 -- 172.21.15.44:0/1042516261 learned_addr learned my addr 172.21.15.44:0/1042516261 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.963+0000 7f5d99d9b640 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e16510 0x55b354e1f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.963+0000 7f5d9ad9d640 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e224a0 0x55b354e1f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.963+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1441882772 shutdown_connections 2024-08-20T03:03:01.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.963+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd5c130 0x55ddebd59ba0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.963+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55ddebd63050 0x55ddebd654b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.963+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd501a0 0x55ddebd59660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.963+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1441882772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd59120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.963+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1441882772 >> 172.21.15.44:0/1441882772 conn(0x55ddebd4d6b0 msgr2=0x55ddebd53250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:01.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.963+0000 7fe0c2ffd640 0 client.25217 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:03:01.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.963+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1441882772 shutdown_connections 2024-08-20T03:03:01.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.963+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1441882772 wait complete. 2024-08-20T03:03:01.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.963+0000 7fe0ebecd740 20 client.25217 trim_cache size 0 max 16384 2024-08-20T03:03:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.963+0000 7f5d9a59c640 1 -- 172.21.15.44:0/1042516261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e16510 msgr2=0x55b354e1f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.963+0000 7f5d9a59c640 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e16510 0x55b354e1f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.963+0000 7f5d9a59c640 1 -- 172.21.15.44:0/1042516261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e224a0 msgr2=0x55b354e1f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.963+0000 7f5d9a59c640 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e224a0 0x55b354e1f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.963+0000 7f5d9a59c640 1 -- 172.21.15.44:0/1042516261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b354add790 con 0x55b354e15bf0 2024-08-20T03:03:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.963+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1042516261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b354d29e00 con 0x55b354e15bf0 2024-08-20T03:03:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.963+0000 7f5d9ad9d640 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e224a0 0x55b354e1f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5d9a59c640 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e21d30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00e9c0 tx=0x7f5d8c0ab360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fc30359791833ca server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5d81ffb640 10 client.25220 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:03:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5d81ffb640 1 -- 172.21.15.44:0/1042516261 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c010720 con 0x55b354e15bf0 2024-08-20T03:03:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5d81ffb640 1 -- 172.21.15.44:0/1042516261 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5d8c0a7350 con 0x55b354e15bf0 2024-08-20T03:03:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5d81ffb640 1 -- 172.21.15.44:0/1042516261 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d8c0aa870 con 0x55b354e15bf0 2024-08-20T03:03:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5da3bcf740 10 client.25220 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1042516261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55b354b56620 con 0x55b354e15bf0 2024-08-20T03:03:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5d81ffb640 1 -- 172.21.15.44:0/1042516261 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5d8c0a66d0 con 0x55b354e15bf0 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5da3bcf740 10 client.25220 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5da3bcf740 10 client.25220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5da3bcf740 20 client.25220 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b354e31bf0 0x55b354e340b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5da3bcf740 4 client.25220 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.964+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1042516261 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b354d8d2b0 con 0x55b354e31bf0 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.965+0000 7f5d99d9b640 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b354e31bf0 0x55b354e340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.965+0000 7f5d81ffb640 10 client.25220 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.965+0000 7f5d99d9b640 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b354e31bf0 0x55b354e340b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb27cf91f8bd89f6 server_cookie=bc842b4d205ffb77 in_seq=0 out_seq=0 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5d81ffb640 1 -- 172.21.15.44:0/1042516261 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b354d8d2b0 con 0x55b354e31bf0 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5d81ffb640 10 client.25220 handle_command_reply: tid=0 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5da3bcf740 1 client.25220 shutdown 2024-08-20T03:03:01.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1042516261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 msgr2=0x55b354e21d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e21d30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00e9c0 tx=0x7f5d8c0ab360 comp rx=0 tx=0).stop 2024-08-20T03:03:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1042516261 shutdown_connections 2024-08-20T03:03:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b354e31bf0 0x55b354e340b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e224a0 0x55b354e1f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5d81ffb640 0 client.25220 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e16510 0x55b354e1f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/1042516261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e21d30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1042516261 >> 172.21.15.44:0/1042516261 conn(0x55b354e13a20 msgr2=0x55b354e195c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1042516261 shutdown_connections 2024-08-20T03:03:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.966+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1042516261 wait complete. 2024-08-20T03:03:01.983 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.967+0000 7f5da3bcf740 20 client.25220 trim_cache size 0 max 16384 2024-08-20T03:03:01.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.990+0000 7fe0ebecd740 1 Processor -- start 2024-08-20T03:03:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.990+0000 7fe0ebecd740 1 -- start start 2024-08-20T03:03:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.990+0000 7fe0ebecd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd70330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.990+0000 7fe0ebecd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 0x55ddebd538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.990+0000 7fe0ebecd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd5c130 0x55ddebd86960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.990+0000 7fe0ebecd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41ac6d0 con 0x55ddebd4f880 2024-08-20T03:03:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.990+0000 7fe0ebecd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41ae210 con 0x55ddebd501a0 2024-08-20T03:03:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.991+0000 7fe0e2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 0x55ddebd538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.991+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd5c130 0x55ddebd86960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.991+0000 7fe0e2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 0x55ddebd538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45768/0 (socket says 172.21.15.44:45768) 2024-08-20T03:03:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.991+0000 7fe0e2ffd640 1 -- 172.21.15.44:0/510481933 learned_addr learned my addr 172.21.15.44:0/510481933 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:01.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.990+0000 7fe0ebecd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41acda0 con 0x55ddebd5c130 2024-08-20T03:03:01.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.991+0000 7fe0e37fe640 1 --2- 172.21.15.44:0/510481933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd70330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.991+0000 7fe0e2ffd640 1 -- 172.21.15.44:0/510481933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd5c130 msgr2=0x55ddebd86960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:01.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.991+0000 7fe0e2ffd640 1 --2- 172.21.15.44:0/510481933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd5c130 0x55ddebd86960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.991+0000 7fe0e2ffd640 1 -- 172.21.15.44:0/510481933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 msgr2=0x55ddebd70330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:01.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.991+0000 7fe0e2ffd640 1 --2- 172.21.15.44:0/510481933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd70330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.991+0000 7fe0e2ffd640 1 -- 172.21.15.44:0/510481933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddebd4ffd0 con 0x55ddebd501a0 2024-08-20T03:03:01.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.991+0000 7fe0e2ffd640 1 --2- 172.21.15.44:0/510481933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 0x55ddebd538f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8003f70 tx=0x7fe0d8005890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f95ecbea17c38191 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:01.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.992+0000 7fe0c97fa640 1 -- 172.21.15.44:0/510481933 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8023070 con 0x55ddebd501a0 2024-08-20T03:03:01.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.992+0000 7fe0c97fa640 1 -- 172.21.15.44:0/510481933 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0d8009e10 con 0x55ddebd501a0 2024-08-20T03:03:01.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.992+0000 7fe0c97fa640 1 -- 172.21.15.44:0/510481933 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80053f0 con 0x55ddebd501a0 2024-08-20T03:03:01.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.993+0000 7fe0ebecd740 1 -- 172.21.15.44:0/510481933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 msgr2=0x55ddebd538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:01.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.993+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/510481933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 0x55ddebd538f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8003f70 tx=0x7fe0d8005890 comp rx=0 tx=0).stop 2024-08-20T03:03:01.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.993+0000 7fe0ebecd740 1 -- 172.21.15.44:0/510481933 shutdown_connections 2024-08-20T03:03:01.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.993+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/510481933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd5c130 0x55ddebd86960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.993+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/510481933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 0x55ddebd538f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.993+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/510481933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd70330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.993+0000 7fe0ebecd740 1 -- 172.21.15.44:0/510481933 >> 172.21.15.44:0/510481933 conn(0x55ddebd4d6b0 msgr2=0x55ddebd6fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:01.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.993+0000 7fe0ebecd740 1 -- 172.21.15.44:0/510481933 shutdown_connections 2024-08-20T03:03:01.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.993+0000 7fe0ebecd740 1 -- 172.21.15.44:0/510481933 wait complete. 2024-08-20T03:03:01.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.994+0000 7fe0ebecd740 1 -- start start 2024-08-20T03:03:01.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.994+0000 7fe0ebecd740 1 Processor -- start 2024-08-20T03:03:01.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.995+0000 7fe0ebecd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd67aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.995+0000 7fe0ebecd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 0x55ddebd68040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.995+0000 7fe0e37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd67aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.995+0000 7fe0e37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd67aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54970/0 (socket says 172.21.15.44:54970) 2024-08-20T03:03:01.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.995+0000 7fe0e37fe640 1 -- 172.21.15.44:0/1891361552 learned_addr learned my addr 172.21.15.44:0/1891361552 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:01.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.995+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd5c130 0x55ddebddf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:01.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.995+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1891361552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41adb40 con 0x55ddebd4f880 2024-08-20T03:03:01.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.995+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1891361552 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41ad470 con 0x55ddebd501a0 2024-08-20T03:03:01.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.995+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1891361552 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e41aa6b0 con 0x55ddebd5c130 2024-08-20T03:03:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.995+0000 7fe0e37fe640 1 -- 172.21.15.44:0/1891361552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd5c130 msgr2=0x55ddebddf8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:03:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.996+0000 7fe0e37fe640 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd5c130 0x55ddebddf8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.996+0000 7fe0e37fe640 1 -- 172.21.15.44:0/1891361552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 msgr2=0x55ddebd68040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.996+0000 7fe0e2ffd640 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 0x55ddebd68040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.996+0000 7fe0ebecd740 10 client.0 _ll_register_callbacks cb 0x7fe0dc009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.996+0000 7fe0e37fe640 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 0x55ddebd68040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.996+0000 7fe0e37fe640 1 -- 172.21.15.44:0/1891361552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddebde0020 con 0x55ddebd4f880 2024-08-20T03:03:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.996+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1891361552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0d8009e10 con 0x55ddebd4f880 2024-08-20T03:03:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.996+0000 7fe0e2ffd640 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 0x55ddebd68040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.996+0000 7fe0e37fe640 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd67aa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4005940 tx=0x7fe0d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735b7ed0aa591857 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.996+0000 7fe0c2ffd640 10 client.15678 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:01.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.996+0000 7fe0c2ffd640 1 -- 172.21.15.44:0/1891361552 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4049040 con 0x55ddebd4f880 2024-08-20T03:03:02.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.996+0000 7fe0c2ffd640 1 -- 172.21.15.44:0/1891361552 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe0d40c71a0 con 0x55ddebd4f880 2024-08-20T03:03:02.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.997+0000 7fe0c2ffd640 1 -- 172.21.15.44:0/1891361552 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe0d40d3fc0 con 0x55ddebd4f880 2024-08-20T03:03:02.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.997+0000 7fe0ebecd740 10 client.15678 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:02.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.997+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1891361552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55ddebde7300 con 0x55ddebd4f880 2024-08-20T03:03:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.997+0000 7f5da3bcf740 1 Processor -- start 2024-08-20T03:03:02.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.997+0000 7f5da3bcf740 1 -- start start 2024-08-20T03:03:02.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.997+0000 7fe0c2ffd640 1 -- 172.21.15.44:0/1891361552 <== mon.0 v2:172.21.15.44:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe0d4101cd0 con 0x55ddebd4f880 2024-08-20T03:03:02.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.997+0000 7fe0ebecd740 10 client.15678 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:02.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.997+0000 7fe0ebecd740 10 client.15678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:03:02.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.997+0000 7fe0ebecd740 20 client.15678 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:02.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.997+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55ddebde7a60 0x55ddebde9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:02.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.997+0000 7fe0ebecd740 4 client.15678 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:02.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.997+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1891361552 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddebcc4320 con 0x55ddebde7a60 2024-08-20T03:03:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.998+0000 7f5da3bcf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e2e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.998+0000 7f5da3bcf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 0x55b354e1e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:02.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.998+0000 7fe0e2ffd640 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55ddebde7a60 0x55ddebde9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.998+0000 7f5d9a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e2e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.998+0000 7f5d9a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e2e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45784/0 (socket says 172.21.15.44:45784) 2024-08-20T03:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.998+0000 7f5d99d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 0x55b354e1e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.998+0000 7f5d99d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 0x55b354e1e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54974/0 (socket says 172.21.15.44:54974) 2024-08-20T03:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.998+0000 7f5d9a59c640 1 -- 172.21.15.44:0/1520400905 learned_addr learned my addr 172.21.15.44:0/1520400905 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.998+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/1520400905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 0x55b354e39980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.998+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1520400905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c00e540 con 0x55b354e16510 2024-08-20T03:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.998+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1520400905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c010080 con 0x55b354e15bf0 2024-08-20T03:03:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.998+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1520400905 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c00ec10 con 0x55b354e224a0 2024-08-20T03:03:02.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.998+0000 7fe0e2ffd640 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55ddebde7a60 0x55ddebde9e80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6701373ae59ed9f5 server_cookie=c2fbd0adb8967325 in_seq=0 out_seq=0 2024-08-20T03:03:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.998+0000 7f5d9ad9d640 1 --2- 172.21.15.44:0/1520400905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 0x55b354e39980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:02.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.998+0000 7fe0c2ffd640 10 client.15678 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:03:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.999+0000 7f5d9ad9d640 1 -- 172.21.15.44:0/1520400905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 msgr2=0x55b354e2e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.999+0000 7f5d9ad9d640 1 --2- 172.21.15.44:0/1520400905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e2e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.999+0000 7f5d9ad9d640 1 -- 172.21.15.44:0/1520400905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 msgr2=0x55b354e1e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.999+0000 7f5d9ad9d640 1 --2- 172.21.15.44:0/1520400905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 0x55b354e1e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.999+0000 7f5d9ad9d640 1 -- 172.21.15.44:0/1520400905 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b354e16340 con 0x55b354e224a0 2024-08-20T03:03:02.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0c2ffd640 1 -- 172.21.15.44:0/1891361552 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddebcc4320 con 0x55ddebde7a60 2024-08-20T03:03:02.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0c2ffd640 10 client.15678 handle_command_reply: tid=0 2024-08-20T03:03:02.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0ebecd740 1 client.15678 shutdown 2024-08-20T03:03:02.006 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.999+0000 7f5d99d9b640 1 --2- 172.21.15.44:0/1520400905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 0x55b354e1e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:02.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1891361552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 msgr2=0x55ddebd67aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:02.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd67aa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4005940 tx=0x7fe0d4009260 comp rx=0 tx=0).stop 2024-08-20T03:03:02.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1891361552 shutdown_connections 2024-08-20T03:03:02.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55ddebde7a60 0x55ddebde9e80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.999+0000 7f5d9ad9d640 1 --2- 172.21.15.44:0/1520400905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 0x55b354e39980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5d9402af40 tx=0x7f5d94042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e09d0051c48f8497 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.999+0000 7f5d83fff640 1 -- 172.21.15.44:0/1520400905 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94056070 con 0x55b354e224a0 2024-08-20T03:03:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.999+0000 7f5d83fff640 1 -- 172.21.15.44:0/1520400905 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d94030cb0 con 0x55b354e224a0 2024-08-20T03:03:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:01.999+0000 7f5d83fff640 1 -- 172.21.15.44:0/1520400905 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94034040 con 0x55b354e224a0 2024-08-20T03:03:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ddebd5c130 0x55ddebddf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ddebd501a0 0x55ddebd68040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0ebecd740 1 --2- 172.21.15.44:0/1891361552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ddebd4f880 0x55ddebd67aa0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1891361552 >> 172.21.15.44:0/1891361552 conn(0x55ddebd4d6b0 msgr2=0x55ddebd5a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0c2ffd640 0 client.15678 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:03:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:01.999+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1891361552 shutdown_connections 2024-08-20T03:03:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.000+0000 7fe0ebecd740 1 -- 172.21.15.44:0/1891361552 wait complete. 2024-08-20T03:03:02.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.000+0000 7fe0ebecd740 20 client.15678 trim_cache size 0 max 16384 2024-08-20T03:03:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.000+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1520400905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 msgr2=0x55b354e39980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.000+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/1520400905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 0x55b354e39980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5d9402af40 tx=0x7f5d94042000 comp rx=0 tx=0).stop 2024-08-20T03:03:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.000+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1520400905 shutdown_connections 2024-08-20T03:03:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.000+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/1520400905 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e224a0 0x55b354e39980 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.000+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/1520400905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e16510 0x55b354e1e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.000+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/1520400905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e15bf0 0x55b354e2e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.000+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1520400905 >> 172.21.15.44:0/1520400905 conn(0x55b354e13a20 msgr2=0x55b354e20fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:02.009 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.000+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1520400905 shutdown_connections 2024-08-20T03:03:02.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.001+0000 7f5da3bcf740 1 -- 172.21.15.44:0/1520400905 wait complete. 2024-08-20T03:03:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.001+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3591717803 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0cc0b3dc0 msgr2=0x7fe0cc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:02.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.001+0000 7fe0e9ad4640 1 --2- 172.21.15.44:0/3591717803 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0cc0b3dc0 0x7fe0cc0b6280 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc005fd0 tx=0x7fe0dc0098e0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.001+0000 7f5da3bcf740 1 -- start start 2024-08-20T03:03:02.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.001+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3591717803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0e41074b0 msgr2=0x7fe0e4197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.001+0000 7f5da3bcf740 1 Processor -- start 2024-08-20T03:03:02.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.001+0000 7fe0e9ad4640 1 --2- 172.21.15.44:0/3591717803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0e41074b0 0x7fe0e4197060 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4006010 tx=0x7fe0d40042f0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.001+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3591717803 shutdown_connections 2024-08-20T03:03:02.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.001+0000 7fe0e9ad4640 1 --2- 172.21.15.44:0/3591717803 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0cc0b3dc0 0x7fe0cc0b6280 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.001+0000 7fe0e9ad4640 1 --2- 172.21.15.44:0/3591717803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0e419c940 0x7fe0e419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.002+0000 7fe0e9ad4640 1 --2- 172.21.15.44:0/3591717803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0e4107e30 0x7fe0e41975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.002+0000 7fe0e9ad4640 1 --2- 172.21.15.44:0/3591717803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0e41074b0 0x7fe0e4197060 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5da3bcf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e15bf0 0x55b354e26280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5da3bcf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e16510 0x55b354e267c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:02.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.002+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3591717803 >> 172.21.15.44:0/3591717803 conn(0x7fe0e4076b50 msgr2=0x7fe0e4100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5da3bcf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e224a0 0x55b354e26d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5d9ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e224a0 0x55b354e26d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5d99d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e16510 0x55b354e267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5d9a59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e15bf0 0x55b354e26280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5d9ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e224a0 0x55b354e26d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54988/0 (socket says 172.21.15.44:54988) 2024-08-20T03:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5d9ad9d640 1 -- 172.21.15.44:0/3948448524 learned_addr learned my addr 172.21.15.44:0/3948448524 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5d99d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e16510 0x55b354e267c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45792/0 (socket says 172.21.15.44:45792) 2024-08-20T03:03:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5da3bcf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c00f9b0 con 0x55b354e224a0 2024-08-20T03:03:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5da3bcf740 1 -- 172.21.15.44:0/3948448524 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c00f2e0 con 0x55b354e16510 2024-08-20T03:03:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5da3bcf740 1 -- 172.21.15.44:0/3948448524 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7c00c520 con 0x55b354e15bf0 2024-08-20T03:03:02.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.002+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3591717803 shutdown_connections 2024-08-20T03:03:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5da3bcf740 10 client.0 _ll_register_callbacks cb 0x7f5d8c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5d99d9b640 1 -- 172.21.15.44:0/3948448524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e15bf0 msgr2=0x55b354e26280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:02.018 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5d99d9b640 1 --2- 172.21.15.44:0/3948448524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e15bf0 0x55b354e26280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5d99d9b640 1 -- 172.21.15.44:0/3948448524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e224a0 msgr2=0x55b354e26d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5d99d9b640 1 --2- 172.21.15.44:0/3948448524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e224a0 0x55b354e26d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5d99d9b640 1 -- 172.21.15.44:0/3948448524 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b354e9b980 con 0x55b354e16510 2024-08-20T03:03:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.002+0000 7f5d9a59c640 1 --2- 172.21.15.44:0/3948448524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e15bf0 0x55b354e26280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:02.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:02.002+0000 7fe0e9ad4640 1 -- 172.21.15.44:0/3591717803 wait complete. 2024-08-20T03:03:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.003+0000 7f5da3bcf740 1 -- 172.21.15.44:0/3948448524 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d94030cb0 con 0x55b354e16510 2024-08-20T03:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.003+0000 7f5d99d9b640 1 --2- 172.21.15.44:0/3948448524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e16510 0x55b354e267c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009890 tx=0x7f5d90009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c1b4513143c5737 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.003+0000 7f5d81ffb640 10 client.25229 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.003+0000 7f5d81ffb640 1 -- 172.21.15.44:0/3948448524 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9001e040 con 0x55b354e16510 2024-08-20T03:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.003+0000 7f5d81ffb640 1 -- 172.21.15.44:0/3948448524 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5d900053c0 con 0x55b354e16510 2024-08-20T03:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.003+0000 7f5d81ffb640 1 -- 172.21.15.44:0/3948448524 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d90005640 con 0x55b354e16510 2024-08-20T03:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.003+0000 7f5da3bcf740 10 client.25229 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.003+0000 7f5da3bcf740 1 -- 172.21.15.44:0/3948448524 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55b354e9cab0 con 0x55b354e16510 2024-08-20T03:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.004+0000 7f5d81ffb640 1 -- 172.21.15.44:0/3948448524 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5d9000e6a0 con 0x55b354e16510 2024-08-20T03:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.004+0000 7f5da3bcf740 10 client.25229 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.004+0000 7f5da3bcf740 10 client.25229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.004+0000 7f5da3bcf740 20 client.25229 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:02.021 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.004+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/3948448524 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b354e9cf80 0x55b354e9f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.004+0000 7f5da3bcf740 4 client.25229 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:03:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.004+0000 7f5da3bcf740 1 -- 172.21.15.44:0/3948448524 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b354d88d50 con 0x55b354e9cf80 2024-08-20T03:03:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.004+0000 7f5d9a59c640 1 --2- 172.21.15.44:0/3948448524 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b354e9cf80 0x55b354e9f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.005+0000 7f5d9a59c640 1 --2- 172.21.15.44:0/3948448524 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b354e9cf80 0x55b354e9f370 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ec65b4c660ff16d server_cookie=6ed79e17603b733c in_seq=0 out_seq=0 2024-08-20T03:03:02.022 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.005+0000 7f5d81ffb640 10 client.25229 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:02.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:01 smithi165 ceph-mon[22357]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-20T03:03:02.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:01 smithi099 ceph-mon[21984]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-20T03:03:02.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:01 smithi044 ceph-mon[18790]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-08-20T03:03:02.789 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T03:03:02.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:02.837+0000 7f5d8b7fe640 1 -- 172.21.15.44:0/218292212 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f5d8c0a9a30 con 0x7f5d9c1014c0 2024-08-20T03:03:03.062 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:03.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.595+0000 7f7057a62640 1 -- 172.21.15.44:0/1700829992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f705010d4b0 msgr2=0x7f705010d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:03.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.595+0000 7f7057a62640 1 --2- 172.21.15.44:0/1700829992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f705010d4b0 0x7f705010d890 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f704c009900 tx=0x7f704c02a410 comp rx=0 tx=0).stop 2024-08-20T03:03:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.595+0000 7f7057a62640 1 -- 172.21.15.44:0/1700829992 shutdown_connections 2024-08-20T03:03:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.595+0000 7f7057a62640 1 --2- 172.21.15.44:0/1700829992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7050113840 0x7f705011a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.595+0000 7f7057a62640 1 --2- 172.21.15.44:0/1700829992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7050112ec0 0x7f7050113300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.595+0000 7f7057a62640 1 --2- 172.21.15.44:0/1700829992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f705010d4b0 0x7f705010d890 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.595+0000 7f7057a62640 1 -- 172.21.15.44:0/1700829992 >> 172.21.15.44:0/1700829992 conn(0x7f705006d9b0 msgr2=0x7f705006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.595+0000 7f7057a62640 1 -- 172.21.15.44:0/1700829992 shutdown_connections 2024-08-20T03:03:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.595+0000 7f7057a62640 1 -- 172.21.15.44:0/1700829992 wait complete. 2024-08-20T03:03:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.595+0000 7f7057a62640 1 Processor -- start 2024-08-20T03:03:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.595+0000 7f7057a62640 1 -- start start 2024-08-20T03:03:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7057a62640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f705010d4b0 0x7f70501a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7057a62640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7050112ec0 0x7f70501a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7057a62640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7050113840 0x7f70501a9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7057a62640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f705011d450 con 0x7f705010d4b0 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7057a62640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f705011d2d0 con 0x7f7050112ec0 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7057a62640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f705011d5d0 con 0x7f7050113840 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7054ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7050112ec0 0x7f70501a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7054ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7050112ec0 0x7f70501a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45812/0 (socket says 172.21.15.44:45812) 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7054ffd640 1 -- 172.21.15.44:0/1749981283 learned_addr learned my addr 172.21.15.44:0/1749981283 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7054ffd640 1 -- 172.21.15.44:0/1749981283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7050113840 msgr2=0x7f70501a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7054ffd640 1 --2- 172.21.15.44:0/1749981283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7050113840 0x7f70501a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7054ffd640 1 -- 172.21.15.44:0/1749981283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f705010d4b0 msgr2=0x7f70501a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7054ffd640 1 --2- 172.21.15.44:0/1749981283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f705010d4b0 0x7f70501a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.596+0000 7f7054ffd640 1 -- 172.21.15.44:0/1749981283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70501a9a30 con 0x7f7050112ec0 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.597+0000 7f7054ffd640 1 --2- 172.21.15.44:0/1749981283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7050112ec0 0x7f70501a3ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f704000d950 tx=0x7f704000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:03.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.597+0000 7f70467fc640 1 -- 172.21.15.44:0/1749981283 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040017070 con 0x7f7050112ec0 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.597+0000 7f70467fc640 1 -- 172.21.15.44:0/1749981283 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7040009c30 con 0x7f7050112ec0 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.597+0000 7f7057a62640 1 -- 172.21.15.44:0/1749981283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70501a9d20 con 0x7f7050112ec0 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.597+0000 7f70467fc640 1 -- 172.21.15.44:0/1749981283 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704000f620 con 0x7f7050112ec0 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.598+0000 7f7057a62640 1 -- 172.21.15.44:0/1749981283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705006e740 con 0x7f7050112ec0 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.600+0000 7f70467fc640 1 -- 172.21.15.44:0/1749981283 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f7040010460 con 0x7f7050112ec0 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.601+0000 7f70467fc640 1 --2- 172.21.15.44:0/1749981283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f703c0bc6b0 0x7f703c0beb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.601+0000 7f70557fe640 1 --2- 172.21.15.44:0/1749981283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f703c0bc6b0 0x7f703c0beb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.601+0000 7f70467fc640 1 -- 172.21.15.44:0/1749981283 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f70400e3e10 con 0x7f7050112ec0 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.601+0000 7f7059e5b740 1 Processor -- start 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.601+0000 7f7059e5b740 1 -- start start 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f70557fe640 1 --2- 172.21.15.44:0/1749981283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f703c0bc6b0 0x7f703c0beb70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f704c0096c0 tx=0x7f704c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7059e5b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba520dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7059e5b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba5219bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7059e5b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba521a130 0x561ba521c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7059e5b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561ba5184930 con 0x561ba520d880 2024-08-20T03:03:03.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7059e5b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561ba5173e30 con 0x561ba520e1a0 2024-08-20T03:03:03.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7059e5b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561ba5177e70 con 0x561ba521a130 2024-08-20T03:03:03.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f70557fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba520dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f70557fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba520dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55030/0 (socket says 172.21.15.44:55030) 2024-08-20T03:03:03.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f70557fe640 1 -- 172.21.15.44:0/225590941 learned_addr learned my addr 172.21.15.44:0/225590941 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:03.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7054ffd640 1 --2- 172.21.15.44:0/225590941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba5219bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7055fff640 1 --2- 172.21.15.44:0/225590941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba521a130 0x561ba521c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7054ffd640 1 -- 172.21.15.44:0/225590941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba521a130 msgr2=0x561ba521c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:03.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7054ffd640 1 --2- 172.21.15.44:0/225590941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba521a130 0x561ba521c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7054ffd640 1 -- 172.21.15.44:0/225590941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 msgr2=0x561ba520dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7054ffd640 1 --2- 172.21.15.44:0/225590941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba520dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.602+0000 7f7054ffd640 1 -- 172.21.15.44:0/225590941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ba4ed3650 con 0x561ba520e1a0 2024-08-20T03:03:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.603+0000 7f70557fe640 1 --2- 172.21.15.44:0/225590941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba520dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.603+0000 7f7054ffd640 1 --2- 172.21.15.44:0/225590941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba5219bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70400e14b0 tx=0x7f70400aa3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afae2bf0e8d9520 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.603+0000 7f703b7fe640 1 -- 172.21.15.44:0/225590941 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040020070 con 0x561ba520e1a0 2024-08-20T03:03:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.603+0000 7f703b7fe640 1 -- 172.21.15.44:0/225590941 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7040009de0 con 0x561ba520e1a0 2024-08-20T03:03:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.603+0000 7f703b7fe640 1 -- 172.21.15.44:0/225590941 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70400e1980 con 0x561ba520e1a0 2024-08-20T03:03:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.604+0000 7f7059e5b740 1 -- 172.21.15.44:0/225590941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 msgr2=0x561ba5219bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.604+0000 7f7059e5b740 1 --2- 172.21.15.44:0/225590941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba5219bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70400e14b0 tx=0x7f70400aa3d0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.604+0000 7f7059e5b740 1 -- 172.21.15.44:0/225590941 shutdown_connections 2024-08-20T03:03:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.604+0000 7f7059e5b740 1 --2- 172.21.15.44:0/225590941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba521a130 0x561ba521c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.604+0000 7f7059e5b740 1 --2- 172.21.15.44:0/225590941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba5219bf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.604+0000 7f7059e5b740 1 --2- 172.21.15.44:0/225590941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba520dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.604+0000 7f7059e5b740 1 -- 172.21.15.44:0/225590941 >> 172.21.15.44:0/225590941 conn(0x561ba520b6b0 msgr2=0x561ba520c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.604+0000 7f7059e5b740 1 -- 172.21.15.44:0/225590941 shutdown_connections 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.604+0000 7f7059e5b740 1 -- 172.21.15.44:0/225590941 wait complete. 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.605+0000 7f7059e5b740 1 -- start start 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.605+0000 7f7059e5b740 1 Processor -- start 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.605+0000 7f7059e5b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba5217150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.605+0000 7f70557fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba5217150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f70557fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba5217150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55034/0 (socket says 172.21.15.44:55034) 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f70557fe640 1 -- 172.21.15.44:0/2319521197 learned_addr learned my addr 172.21.15.44:0/2319521197 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f7059e5b740 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba520e1a0 0x561ba5217690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f7059e5b740 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba521a130 0x561ba5217bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f7054ffd640 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba520e1a0 0x561ba5217690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f7059e5b740 1 -- 172.21.15.44:0/2319521197 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561ba4fd6700 con 0x561ba520d880 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f7059e5b740 1 -- 172.21.15.44:0/2319521197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561ba50fa810 con 0x561ba521a130 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f7059e5b740 1 -- 172.21.15.44:0/2319521197 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561ba521cde0 con 0x561ba520e1a0 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f7055fff640 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba521a130 0x561ba5217bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f7055fff640 1 -- 172.21.15.44:0/2319521197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba520e1a0 msgr2=0x561ba5217690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f7055fff640 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba520e1a0 0x561ba5217690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f7055fff640 1 -- 172.21.15.44:0/2319521197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 msgr2=0x561ba5217150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.606+0000 7f7055fff640 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba5217150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.607+0000 7f7055fff640 1 -- 172.21.15.44:0/2319521197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ba5226210 con 0x561ba521a130 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.607+0000 7f70557fe640 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba5217150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.607+0000 7f7055fff640 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba521a130 0x561ba5217bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7048009270 tx=0x7f70480118e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb4d6ac722087605 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.607+0000 7f7038ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:03:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.607+0000 7f7054ffd640 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba520e1a0 0x561ba5217690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.607+0000 7f7038ff9640 1 -- 172.21.15.44:0/2319521197 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048004d20 con 0x561ba521a130 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.607+0000 7f7038ff9640 1 -- 172.21.15.44:0/2319521197 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7048012780 con 0x561ba521a130 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.607+0000 7f7038ff9640 10 client.0 inactive, discarding osd_map(77..77 src has 1..77) 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.607+0000 7f7059e5b740 10 client.0 _ll_register_callbacks cb 0x561ba516d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.607+0000 7f7059e5b740 1 -- 172.21.15.44:0/2319521197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561ba5123150 con 0x561ba521a130 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.607+0000 7f7038ff9640 1 -- 172.21.15.44:0/2319521197 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7048012980 con 0x561ba521a130 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.608+0000 7f7059e5b740 10 client.25241 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.608+0000 7f7059e5b740 1 -- 172.21.15.44:0/2319521197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=28}) -- 0x561ba52289d0 con 0x561ba521a130 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.608+0000 7f7038ff9640 1 -- 172.21.15.44:0/2319521197 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f704800f040 con 0x561ba521a130 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.608+0000 7f7059e5b740 10 client.25241 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.608+0000 7f7059e5b740 10 client.25241 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.608+0000 7f7059e5b740 20 client.25241 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:03.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.608+0000 7f7059e5b740 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561ba522a2a0 0x561ba522c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.608+0000 7f7059e5b740 4 client.25241 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:03.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.608+0000 7f7059e5b740 1 -- 172.21.15.44:0/2319521197 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ba5123150 con 0x561ba522a2a0 2024-08-20T03:03:03.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.608+0000 7f70557fe640 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561ba522a2a0 0x561ba522c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.609+0000 7f7038ff9640 10 client.25241 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:03:03.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.609+0000 7f70557fe640 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561ba522a2a0 0x561ba522c760 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=254fefe0b0dbe1f7 server_cookie=7f38499993f0df42 in_seq=0 out_seq=0 2024-08-20T03:03:03.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7038ff9640 1 -- 172.21.15.44:0/2319521197 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ba5123150 con 0x561ba522a2a0 2024-08-20T03:03:03.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7038ff9640 10 client.25241 handle_command_reply: tid=0 2024-08-20T03:03:03.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7059e5b740 1 client.25241 shutdown 2024-08-20T03:03:03.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7059e5b740 1 -- 172.21.15.44:0/2319521197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba521a130 msgr2=0x561ba5217bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:03.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7059e5b740 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba521a130 0x561ba5217bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7048009270 tx=0x7f70480118e0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7059e5b740 1 -- 172.21.15.44:0/2319521197 shutdown_connections 2024-08-20T03:03:03.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7059e5b740 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561ba522a2a0 0x561ba522c760 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7059e5b740 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba521a130 0x561ba5217bd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7059e5b740 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba520e1a0 0x561ba5217690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7038ff9640 0 client.25241 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:03:03.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7059e5b740 1 --2- 172.21.15.44:0/2319521197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba5217150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7059e5b740 1 -- 172.21.15.44:0/2319521197 >> 172.21.15.44:0/2319521197 conn(0x561ba520b6b0 msgr2=0x561ba5211250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:03.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7059e5b740 1 -- 172.21.15.44:0/2319521197 shutdown_connections 2024-08-20T03:03:03.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7059e5b740 1 -- 172.21.15.44:0/2319521197 wait complete. 2024-08-20T03:03:03.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.610+0000 7f7059e5b740 20 client.25241 trim_cache size 0 max 16384 2024-08-20T03:03:03.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.636+0000 7f7059e5b740 1 Processor -- start 2024-08-20T03:03:03.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.636+0000 7f7059e5b740 1 -- start start 2024-08-20T03:03:03.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.636+0000 7f7059e5b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba520d880 0x561ba5211a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.636+0000 7f7059e5b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba5217650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.636+0000 7f7059e5b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba521a130 0x561ba5217b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.636+0000 7f7059e5b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f70501b4850 con 0x561ba521a130 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.636+0000 7f7059e5b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f70501b6390 con 0x561ba520e1a0 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.636+0000 7f7059e5b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f70501b4f20 con 0x561ba520d880 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.636+0000 7f7055fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba521a130 0x561ba5217b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.636+0000 7f7055fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba521a130 0x561ba5217b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55050/0 (socket says 172.21.15.44:55050) 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.636+0000 7f7055fff640 1 -- 172.21.15.44:0/3906100159 learned_addr learned my addr 172.21.15.44:0/3906100159 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.636+0000 7f7054ffd640 1 --2- 172.21.15.44:0/3906100159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba5217650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.637+0000 7f70557fe640 1 --2- 172.21.15.44:0/3906100159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba520d880 0x561ba5211a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.637+0000 7f7054ffd640 1 -- 172.21.15.44:0/3906100159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba520d880 msgr2=0x561ba5211a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.637+0000 7f7054ffd640 1 --2- 172.21.15.44:0/3906100159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba520d880 0x561ba5211a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.637+0000 7f7054ffd640 1 -- 172.21.15.44:0/3906100159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba521a130 msgr2=0x561ba5217b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.637+0000 7f7054ffd640 1 --2- 172.21.15.44:0/3906100159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba521a130 0x561ba5217b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.637+0000 7f7054ffd640 1 -- 172.21.15.44:0/3906100159 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ba520dfd0 con 0x561ba520e1a0 2024-08-20T03:03:03.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.637+0000 7f7055fff640 1 --2- 172.21.15.44:0/3906100159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba521a130 0x561ba5217b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:03.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.637+0000 7f7054ffd640 1 --2- 172.21.15.44:0/3906100159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba5217650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f70400a61c0 tx=0x7f7040005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68a76b8b6c6f2c6b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:03.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.637+0000 7f703affd640 1 -- 172.21.15.44:0/3906100159 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040020030 con 0x561ba520e1a0 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.637+0000 7f703affd640 1 -- 172.21.15.44:0/3906100159 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7040009de0 con 0x561ba520e1a0 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.637+0000 7f703affd640 1 -- 172.21.15.44:0/3906100159 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70400a6490 con 0x561ba520e1a0 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.638+0000 7f7059e5b740 1 -- 172.21.15.44:0/3906100159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 msgr2=0x561ba5217650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.638+0000 7f7059e5b740 1 --2- 172.21.15.44:0/3906100159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba5217650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f70400a61c0 tx=0x7f7040005880 comp rx=0 tx=0).stop 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.638+0000 7f7059e5b740 1 -- 172.21.15.44:0/3906100159 shutdown_connections 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.638+0000 7f7059e5b740 1 --2- 172.21.15.44:0/3906100159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba521a130 0x561ba5217b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.638+0000 7f7059e5b740 1 --2- 172.21.15.44:0/3906100159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba5217650 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.638+0000 7f7059e5b740 1 --2- 172.21.15.44:0/3906100159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba520d880 0x561ba5211a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.638+0000 7f7059e5b740 1 -- 172.21.15.44:0/3906100159 >> 172.21.15.44:0/3906100159 conn(0x561ba520b6b0 msgr2=0x561ba5218fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.638+0000 7f7059e5b740 1 -- 172.21.15.44:0/3906100159 shutdown_connections 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.639+0000 7f7059e5b740 1 -- 172.21.15.44:0/3906100159 wait complete. 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.639+0000 7f7059e5b740 1 -- start start 2024-08-20T03:03:03.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.639+0000 7f7059e5b740 1 Processor -- start 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.640+0000 7f7059e5b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba522f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.640+0000 7f7059e5b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba522fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.640+0000 7f7059e5b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba521a130 0x561ba528cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.640+0000 7f70557fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba522f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.640+0000 7f70557fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba522f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55060/0 (socket says 172.21.15.44:55060) 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.640+0000 7f7054ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba522fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.640+0000 7f7054ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba522fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45858/0 (socket says 172.21.15.44:45858) 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.640+0000 7f70557fe640 1 -- 172.21.15.44:0/4081184340 learned_addr learned my addr 172.21.15.44:0/4081184340 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.640+0000 7f7059e5b740 1 -- 172.21.15.44:0/4081184340 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f70501b5cc0 con 0x561ba520d880 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.640+0000 7f7059e5b740 1 -- 172.21.15.44:0/4081184340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f70501b55f0 con 0x561ba520e1a0 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.640+0000 7f7059e5b740 1 -- 172.21.15.44:0/4081184340 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f70501b2830 con 0x561ba521a130 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.640+0000 7f7059e5b740 10 client.0 _ll_register_callbacks cb 0x7f7048009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:03.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.641+0000 7f7055fff640 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba521a130 0x561ba528cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.641+0000 7f7054ffd640 1 -- 172.21.15.44:0/4081184340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba521a130 msgr2=0x561ba528cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.641+0000 7f7054ffd640 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba521a130 0x561ba528cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.641+0000 7f7054ffd640 1 -- 172.21.15.44:0/4081184340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 msgr2=0x561ba522f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.641+0000 7f7054ffd640 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba522f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.641+0000 7f7054ffd640 1 -- 172.21.15.44:0/4081184340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ba528d4c0 con 0x561ba520e1a0 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.641+0000 7f70557fe640 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba522f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.641+0000 7f7054ffd640 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba522fc70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f70400a61c0 tx=0x7f70400a6e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78a12d2d58b71ba2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.641+0000 7f7038ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.641+0000 7f7038ff9640 1 -- 172.21.15.44:0/4081184340 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7040020070 con 0x561ba520e1a0 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.641+0000 7f7038ff9640 1 -- 172.21.15.44:0/4081184340 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f70400e1970 con 0x561ba520e1a0 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.641+0000 7f7059e5b740 1 -- 172.21.15.44:0/4081184340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7040009de0 con 0x561ba520e1a0 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.642+0000 7f7038ff9640 1 -- 172.21.15.44:0/4081184340 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70400aa8c0 con 0x561ba520e1a0 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.642+0000 7f7059e5b740 10 client.25247 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.642+0000 7f7059e5b740 1 -- 172.21.15.44:0/4081184340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561ba5294910 con 0x561ba520e1a0 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.642+0000 7f7038ff9640 1 -- 172.21.15.44:0/4081184340 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70400b3970 con 0x561ba520e1a0 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.642+0000 7f7038ff9640 1 -- 172.21.15.44:0/4081184340 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7040010750 con 0x561ba520e1a0 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.642+0000 7f7059e5b740 10 client.25247 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:03.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.642+0000 7f7059e5b740 10 client.25247 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:03:03.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.642+0000 7f7059e5b740 20 client.25247 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:03.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.642+0000 7f7059e5b740 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561ba522a2a0 0x561ba5296b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:03.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.642+0000 7f7059e5b740 4 client.25247 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:03.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.642+0000 7f7059e5b740 1 -- 172.21.15.44:0/4081184340 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7040009de0 con 0x561ba522a2a0 2024-08-20T03:03:03.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.642+0000 7f70557fe640 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561ba522a2a0 0x561ba5296b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:03.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f70557fe640 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561ba522a2a0 0x561ba5296b50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b31481eb426bd575 server_cookie=972bd11e9eaf74ec in_seq=0 out_seq=0 2024-08-20T03:03:03.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7038ff9640 10 client.25247 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:03:03.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7038ff9640 1 -- 172.21.15.44:0/4081184340 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7040009de0 con 0x561ba522a2a0 2024-08-20T03:03:03.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7038ff9640 10 client.25247 handle_command_reply: tid=0 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7059e5b740 1 client.25247 shutdown 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7059e5b740 1 -- 172.21.15.44:0/4081184340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 msgr2=0x561ba522fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7059e5b740 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba522fc70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f70400a61c0 tx=0x7f70400a6e20 comp rx=0 tx=0).stop 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7059e5b740 1 -- 172.21.15.44:0/4081184340 shutdown_connections 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7059e5b740 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561ba522a2a0 0x561ba5296b50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7059e5b740 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ba521a130 0x561ba528cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7059e5b740 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ba520e1a0 0x561ba522fc70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7059e5b740 1 --2- 172.21.15.44:0/4081184340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ba520d880 0x561ba522f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7038ff9640 0 client.25247 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7059e5b740 1 -- 172.21.15.44:0/4081184340 >> 172.21.15.44:0/4081184340 conn(0x561ba520b6b0 msgr2=0x561ba5225070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7059e5b740 1 -- 172.21.15.44:0/4081184340 shutdown_connections 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7059e5b740 1 -- 172.21.15.44:0/4081184340 wait complete. 2024-08-20T03:03:03.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.643+0000 7f7059e5b740 20 client.25247 trim_cache size 0 max 16384 2024-08-20T03:03:03.650 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:03.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.644+0000 7f7057a62640 1 -- 172.21.15.44:0/1749981283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f703c0bc6b0 msgr2=0x7f703c0beb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:03.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.644+0000 7f7057a62640 1 --2- 172.21.15.44:0/1749981283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f703c0bc6b0 0x7f703c0beb70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f704c0096c0 tx=0x7f704c031400 comp rx=0 tx=0).stop 2024-08-20T03:03:03.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.644+0000 7f7057a62640 1 -- 172.21.15.44:0/1749981283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7050112ec0 msgr2=0x7f70501a3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:03.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.644+0000 7f7057a62640 1 --2- 172.21.15.44:0/1749981283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7050112ec0 0x7f70501a3ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f704000d950 tx=0x7f704000de10 comp rx=0 tx=0).stop 2024-08-20T03:03:03.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.644+0000 7f7057a62640 1 -- 172.21.15.44:0/1749981283 shutdown_connections 2024-08-20T03:03:03.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.644+0000 7f7057a62640 1 --2- 172.21.15.44:0/1749981283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f703c0bc6b0 0x7f703c0beb70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.644+0000 7f7057a62640 1 --2- 172.21.15.44:0/1749981283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7050113840 0x7f70501a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.644+0000 7f7057a62640 1 --2- 172.21.15.44:0/1749981283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7050112ec0 0x7f70501a3ec0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.644+0000 7f7057a62640 1 --2- 172.21.15.44:0/1749981283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f705010d4b0 0x7f70501a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:03.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.644+0000 7f7057a62640 1 -- 172.21.15.44:0/1749981283 >> 172.21.15.44:0/1749981283 conn(0x7f705006d9b0 msgr2=0x7f705010ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:03.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.644+0000 7f7057a62640 1 -- 172.21.15.44:0/1749981283 shutdown_connections 2024-08-20T03:03:03.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:03.644+0000 7f7057a62640 1 -- 172.21.15.44:0/1749981283 wait complete. 2024-08-20T03:03:04.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:03 smithi165 ceph-mon[22357]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-08-20T03:03:04.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:03 smithi165 ceph-mon[22357]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-08-20T03:03:04.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:03 smithi099 ceph-mon[21984]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-08-20T03:03:04.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:03 smithi099 ceph-mon[21984]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-08-20T03:03:04.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:03 smithi044 ceph-mon[18790]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-08-20T03:03:04.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:03 smithi044 ceph-mon[18790]: mgrmap e26: x(active, since 9m), standbys: z, y 2024-08-20T03:03:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5d81ffb640 1 -- 172.21.15.44:0/3948448524 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b354d88d50 con 0x55b354e9cf80 2024-08-20T03:03:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5d81ffb640 10 client.25229 handle_command_reply: tid=0 2024-08-20T03:03:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5da3bcf740 1 client.25229 shutdown 2024-08-20T03:03:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5da3bcf740 1 -- 172.21.15.44:0/3948448524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e16510 msgr2=0x55b354e267c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/3948448524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e16510 0x55b354e267c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009890 tx=0x7f5d90009640 comp rx=0 tx=0).stop 2024-08-20T03:03:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5da3bcf740 1 -- 172.21.15.44:0/3948448524 shutdown_connections 2024-08-20T03:03:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/3948448524 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b354e9cf80 0x55b354e9f370 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/3948448524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b354e224a0 0x55b354e26d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/3948448524 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b354e16510 0x55b354e267c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5da3bcf740 1 --2- 172.21.15.44:0/3948448524 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b354e15bf0 0x55b354e26280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5da3bcf740 1 -- 172.21.15.44:0/3948448524 >> 172.21.15.44:0/3948448524 conn(0x55b354e13a20 msgr2=0x55b354e122c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5d81ffb640 0 client.25229 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5da3bcf740 1 -- 172.21.15.44:0/3948448524 shutdown_connections 2024-08-20T03:03:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5da3bcf740 1 -- 172.21.15.44:0/3948448524 wait complete. 2024-08-20T03:03:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.335+0000 7f5da3bcf740 20 client.25229 trim_cache size 0 max 16384 2024-08-20T03:03:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:03:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:03:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:03:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:03:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.336+0000 7f5da0d06640 1 -- 172.21.15.44:0/218292212 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d840b3dc0 msgr2=0x7f5d840b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.336+0000 7f5da0d06640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d840b3dc0 0x7f5d840b6280 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5d94006010 tx=0x7f5d94005d80 comp rx=0 tx=0).stop 2024-08-20T03:03:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.336+0000 7f5da0d06640 1 -- 172.21.15.44:0/218292212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d9c1014c0 msgr2=0x7f5d9c19f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.336+0000 7f5da0d06640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d9c1014c0 0x7f5d9c19f490 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00e9f0 tx=0x7f5d8c00eeb0 comp rx=0 tx=0).stop 2024-08-20T03:03:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.336+0000 7f5da0d06640 1 -- 172.21.15.44:0/218292212 shutdown_connections 2024-08-20T03:03:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.336+0000 7f5da0d06640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5d840b3dc0 0x7f5d840b6280 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.336+0000 7f5da0d06640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d9c106a50 0x7f5d9c1a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.336+0000 7f5da0d06640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d9c101de0 0x7f5d9c19f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.336+0000 7f5da0d06640 1 --2- 172.21.15.44:0/218292212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d9c1014c0 0x7f5d9c19f490 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.336+0000 7f5da0d06640 1 -- 172.21.15.44:0/218292212 >> 172.21.15.44:0/218292212 conn(0x7f5d9c076ba0 msgr2=0x7f5d9c104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.336+0000 7f5da0d06640 1 -- 172.21.15.44:0/218292212 shutdown_connections 2024-08-20T03:03:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:04.336+0000 7f5da0d06640 1 -- 172.21.15.44:0/218292212 wait complete. 2024-08-20T03:03:04.420 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T03:03:04.689 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.195+0000 7fbe46021640 1 -- 172.21.15.44:0/1146128610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401042d0 msgr2=0x7fbe4010c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.195+0000 7fbe46021640 1 --2- 172.21.15.44:0/1146128610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401042d0 0x7fbe4010c7c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009900 tx=0x7fbe3402a3f0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.195+0000 7fbe46021640 1 -- 172.21.15.44:0/1146128610 shutdown_connections 2024-08-20T03:03:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.195+0000 7fbe46021640 1 --2- 172.21.15.44:0/1146128610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe4010ce10 0x7fbe4010f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.195+0000 7fbe46021640 1 --2- 172.21.15.44:0/1146128610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401042d0 0x7fbe4010c7c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.195+0000 7fbe46021640 1 --2- 172.21.15.44:0/1146128610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe401039b0 0x7fbe40103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.195+0000 7fbe46021640 1 -- 172.21.15.44:0/1146128610 >> 172.21.15.44:0/1146128610 conn(0x7fbe400fd430 msgr2=0x7fbe400ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.195+0000 7fbe46021640 1 -- 172.21.15.44:0/1146128610 shutdown_connections 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.196+0000 7fbe46021640 1 -- 172.21.15.44:0/1146128610 wait complete. 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.196+0000 7fbe46021640 1 Processor -- start 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.196+0000 7fbe46021640 1 -- start start 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe46021640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401039b0 0x7fbe40198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe46021640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe401042d0 0x7fbe401994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe46021640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe4010ce10 0x7fbe4019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe3f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401039b0 0x7fbe40198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe3f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401039b0 0x7fbe40198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55064/0 (socket says 172.21.15.44:55064) 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe3f7fe640 1 -- 172.21.15.44:0/2271009217 learned_addr learned my addr 172.21.15.44:0/2271009217 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe46021640 1 -- 172.21.15.44:0/2271009217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40112a80 con 0x7fbe401039b0 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe46021640 1 -- 172.21.15.44:0/2271009217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40112900 con 0x7fbe401042d0 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe46021640 1 -- 172.21.15.44:0/2271009217 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40112c00 con 0x7fbe4010ce10 2024-08-20T03:03:05.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe3effd640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe401042d0 0x7fbe401994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe3ffff640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe4010ce10 0x7fbe4019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe3f7fe640 1 -- 172.21.15.44:0/2271009217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe4010ce10 msgr2=0x7fbe4019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe4010ce10 0x7fbe4019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe3f7fe640 1 -- 172.21.15.44:0/2271009217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe401042d0 msgr2=0x7fbe401994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe401042d0 0x7fbe401994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe3f7fe640 1 -- 172.21.15.44:0/2271009217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4019f040 con 0x7fbe401039b0 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.197+0000 7fbe3ffff640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe4010ce10 0x7fbe4019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.198+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401039b0 0x7fbe40198f90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000e9f0 tx=0x7fbe3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.198+0000 7fbe3cff9640 1 -- 172.21.15.44:0/2271009217 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000cd80 con 0x7fbe401039b0 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.198+0000 7fbe3cff9640 1 -- 172.21.15.44:0/2271009217 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe300108e0 con 0x7fbe401039b0 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.198+0000 7fbe46021640 1 -- 172.21.15.44:0/2271009217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4019f330 con 0x7fbe401039b0 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.198+0000 7fbe3cff9640 1 -- 172.21.15.44:0/2271009217 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30018980 con 0x7fbe401039b0 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.198+0000 7fbe46021640 1 -- 172.21.15.44:0/2271009217 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe401a59f0 con 0x7fbe401039b0 2024-08-20T03:03:05.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe4841a740 1 Processor -- start 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe4841a740 1 -- start start 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe4841a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d27e880 0x56532d27ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe4841a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27f1a0 0x56532d28abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe4841a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d28b130 0x56532d28d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe4841a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56532d1f5930 con 0x56532d28b130 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe4841a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56532d1e4e30 con 0x56532d27e880 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe4841a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56532d1e8e70 con 0x56532d27f1a0 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe3cff9640 1 -- 172.21.15.44:0/2271009217 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7fbe30010430 con 0x7fbe401039b0 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe3effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d28b130 0x56532d28d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe3effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d28b130 0x56532d28d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55070/0 (socket says 172.21.15.44:55070) 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe3effd640 1 -- 172.21.15.44:0/2496306795 learned_addr learned my addr 172.21.15.44:0/2496306795 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.201+0000 7fbe3ffff640 1 --2- 172.21.15.44:0/2496306795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d27e880 0x56532d27ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/2496306795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27f1a0 0x56532d28abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe3effd640 1 -- 172.21.15.44:0/2496306795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27f1a0 msgr2=0x56532d28abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe3effd640 1 --2- 172.21.15.44:0/2496306795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27f1a0 0x56532d28abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe3effd640 1 -- 172.21.15.44:0/2496306795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d27e880 msgr2=0x56532d27ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe3effd640 1 --2- 172.21.15.44:0/2496306795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d27e880 0x56532d27ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe3effd640 1 -- 172.21.15.44:0/2496306795 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532cf44650 con 0x56532d28b130 2024-08-20T03:03:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe3ffff640 1 --2- 172.21.15.44:0/2496306795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d27e880 0x56532d27ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/2496306795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27f1a0 0x56532d28abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe3effd640 1 --2- 172.21.15.44:0/2496306795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d28b130 0x56532d28d520 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbe34033000 tx=0x7fbe34032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0657024c5b8d05c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe3cff9640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbe280b3e50 0x7fbe280b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe25ffb640 1 -- 172.21.15.44:0/2496306795 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34035040 con 0x56532d28b130 2024-08-20T03:03:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe25ffb640 1 -- 172.21.15.44:0/2496306795 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe34004030 con 0x56532d28b130 2024-08-20T03:03:05.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.202+0000 7fbe25ffb640 1 -- 172.21.15.44:0/2496306795 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34005580 con 0x56532d28b130 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.203+0000 7fbe3ffff640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbe280b3e50 0x7fbe280b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.203+0000 7fbe3cff9640 1 -- 172.21.15.44:0/2271009217 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbe300e4540 con 0x7fbe401039b0 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.203+0000 7fbe4841a740 1 -- 172.21.15.44:0/2496306795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d28b130 msgr2=0x56532d28d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.203+0000 7fbe3ffff640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbe280b3e50 0x7fbe280b6310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbe38007ef0 tx=0x7fbe38007480 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.203+0000 7fbe4841a740 1 --2- 172.21.15.44:0/2496306795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d28b130 0x56532d28d520 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbe34033000 tx=0x7fbe34032bb0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.203+0000 7fbe4841a740 1 -- 172.21.15.44:0/2496306795 shutdown_connections 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.203+0000 7fbe4841a740 1 --2- 172.21.15.44:0/2496306795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d28b130 0x56532d28d520 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.203+0000 7fbe4841a740 1 --2- 172.21.15.44:0/2496306795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27f1a0 0x56532d28abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.204+0000 7fbe4841a740 1 --2- 172.21.15.44:0/2496306795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d27e880 0x56532d27ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.204+0000 7fbe4841a740 1 -- 172.21.15.44:0/2496306795 >> 172.21.15.44:0/2496306795 conn(0x56532d27c6b0 msgr2=0x56532d27d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.204+0000 7fbe4841a740 1 -- 172.21.15.44:0/2496306795 shutdown_connections 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.204+0000 7fbe4841a740 1 -- 172.21.15.44:0/2496306795 wait complete. 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.204+0000 7fbe4841a740 1 -- start start 2024-08-20T03:03:05.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.204+0000 7fbe4841a740 1 Processor -- start 2024-08-20T03:03:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.204+0000 7fbe4841a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27e880 0x56532d28ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.204+0000 7fbe4841a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d287fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe4841a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d28b130 0x56532d288750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe4841a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56532d047700 con 0x56532d27f1a0 2024-08-20T03:03:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe4841a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56532d16b810 con 0x56532d28b130 2024-08-20T03:03:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe4841a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56532d28dd80 con 0x56532d27e880 2024-08-20T03:03:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe3effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d287fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe4841a740 10 client.0 _ll_register_callbacks cb 0x56532d1de700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe3effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d287fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55072/0 (socket says 172.21.15.44:55072) 2024-08-20T03:03:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe3effd640 1 -- 172.21.15.44:0/3912644812 learned_addr learned my addr 172.21.15.44:0/3912644812 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27e880 0x56532d28ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe3ffff640 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d28b130 0x56532d288750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe3f7fe640 1 -- 172.21.15.44:0/3912644812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d28b130 msgr2=0x56532d288750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d28b130 0x56532d288750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe3f7fe640 1 -- 172.21.15.44:0/3912644812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 msgr2=0x56532d287fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d287fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.205+0000 7fbe3f7fe640 1 -- 172.21.15.44:0/3912644812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56532d29f8c0 con 0x56532d27e880 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.206+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27e880 0x56532d28ab70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbe300e1be0 tx=0x7fbe300aa890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd76811ce856ff82 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.206+0000 7fbe1effd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.206+0000 7fbe1effd640 1 -- 172.21.15.44:0/3912644812 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300e20a0 con 0x56532d27e880 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.206+0000 7fbe1effd640 1 -- 172.21.15.44:0/3912644812 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbe300e40d0 con 0x56532d27e880 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.206+0000 7fbe4841a740 1 -- 172.21.15.44:0/3912644812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56532d194150 con 0x56532d27e880 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.206+0000 7fbe1effd640 1 -- 172.21.15.44:0/3912644812 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe300ab6f0 con 0x56532d27e880 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.206+0000 7fbe4841a740 10 client.25228 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:05.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.206+0000 7fbe4841a740 1 -- 172.21.15.44:0/3912644812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56532d2a2160 con 0x56532d27e880 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.206+0000 7fbe1effd640 1 -- 172.21.15.44:0/3912644812 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe300aaff0 con 0x56532d27e880 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.207+0000 7fbe1effd640 1 -- 172.21.15.44:0/3912644812 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbe30018b20 con 0x56532d27e880 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.207+0000 7fbe4841a740 10 client.25228 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.207+0000 7fbe4841a740 10 client.25228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.207+0000 7fbe4841a740 20 client.25228 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.207+0000 7fbe4841a740 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56532d2a3620 0x56532d2a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.207+0000 7fbe4841a740 4 client.25228 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.207+0000 7fbe4841a740 1 -- 172.21.15.44:0/3912644812 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56532d194150 con 0x56532d2a3620 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.207+0000 7fbe3effd640 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56532d2a3620 0x56532d2a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.207+0000 7fbe1effd640 10 client.25228 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.208+0000 7fbe3effd640 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56532d2a3620 0x56532d2a5a50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4807747ca3157a server_cookie=2b55408ea6ae4470 in_seq=0 out_seq=0 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe1effd640 1 -- 172.21.15.44:0/3912644812 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56532d194150 con 0x56532d2a3620 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe1effd640 10 client.25228 handle_command_reply: tid=0 2024-08-20T03:03:05.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe4841a740 1 client.25228 shutdown 2024-08-20T03:03:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe4841a740 1 -- 172.21.15.44:0/3912644812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27e880 msgr2=0x56532d28ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe4841a740 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27e880 0x56532d28ab70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbe300e1be0 tx=0x7fbe300aa890 comp rx=0 tx=0).stop 2024-08-20T03:03:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe4841a740 1 -- 172.21.15.44:0/3912644812 shutdown_connections 2024-08-20T03:03:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe4841a740 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56532d2a3620 0x56532d2a5a50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe4841a740 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d28b130 0x56532d288750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe4841a740 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d287fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe1effd640 0 client.25228 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe4841a740 1 --2- 172.21.15.44:0/3912644812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27e880 0x56532d28ab70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe4841a740 1 -- 172.21.15.44:0/3912644812 >> 172.21.15.44:0/3912644812 conn(0x56532d27c6b0 msgr2=0x56532d282220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe4841a740 1 -- 172.21.15.44:0/3912644812 shutdown_connections 2024-08-20T03:03:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe4841a740 1 -- 172.21.15.44:0/3912644812 wait complete. 2024-08-20T03:03:05.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.209+0000 7fbe4841a740 20 client.25228 trim_cache size 0 max 16384 2024-08-20T03:03:05.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.230+0000 7fbe4841a740 1 Processor -- start 2024-08-20T03:03:05.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.230+0000 7fbe4841a740 1 -- start start 2024-08-20T03:03:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.230+0000 7fbe4841a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d27e880 0x56532d287720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.230+0000 7fbe4841a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d2ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.230+0000 7fbe4841a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d28b130 0x56532d2ab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe4841a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401ac070 con 0x56532d27f1a0 2024-08-20T03:03:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe4841a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401adbb0 con 0x56532d27e880 2024-08-20T03:03:05.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe4841a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401ac740 con 0x56532d28b130 2024-08-20T03:03:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe3effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d2ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe3effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d2ab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55078/0 (socket says 172.21.15.44:55078) 2024-08-20T03:03:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe3effd640 1 -- 172.21.15.44:0/2684208515 learned_addr learned my addr 172.21.15.44:0/2684208515 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/2684208515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d27e880 0x56532d287720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe3ffff640 1 --2- 172.21.15.44:0/2684208515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d28b130 0x56532d2ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe3effd640 1 -- 172.21.15.44:0/2684208515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d28b130 msgr2=0x56532d2ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe3effd640 1 --2- 172.21.15.44:0/2684208515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d28b130 0x56532d2ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe3effd640 1 -- 172.21.15.44:0/2684208515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d27e880 msgr2=0x56532d287720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:05.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe3effd640 1 --2- 172.21.15.44:0/2684208515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d27e880 0x56532d287720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe3effd640 1 -- 172.21.15.44:0/2684208515 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532d28a7a0 con 0x56532d27f1a0 2024-08-20T03:03:05.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe3ffff640 1 --2- 172.21.15.44:0/2684208515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d28b130 0x56532d2ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:05.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.231+0000 7fbe3effd640 1 --2- 172.21.15.44:0/2684208515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d2ab2f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbe34032c20 tx=0x7fbe3402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7dfad15fc4745b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:05.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.232+0000 7fbe257fa640 1 -- 172.21.15.44:0/2684208515 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34035040 con 0x56532d27f1a0 2024-08-20T03:03:05.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.232+0000 7fbe257fa640 1 -- 172.21.15.44:0/2684208515 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe34005620 con 0x56532d27f1a0 2024-08-20T03:03:05.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.232+0000 7fbe257fa640 1 -- 172.21.15.44:0/2684208515 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34004920 con 0x56532d27f1a0 2024-08-20T03:03:05.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.232+0000 7fbe4841a740 1 -- 172.21.15.44:0/2684208515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 msgr2=0x56532d2ab2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:05.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.232+0000 7fbe4841a740 1 --2- 172.21.15.44:0/2684208515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d2ab2f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbe34032c20 tx=0x7fbe3402a3f0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.232+0000 7fbe4841a740 1 -- 172.21.15.44:0/2684208515 shutdown_connections 2024-08-20T03:03:05.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.232+0000 7fbe4841a740 1 --2- 172.21.15.44:0/2684208515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d28b130 0x56532d2ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.232+0000 7fbe4841a740 1 --2- 172.21.15.44:0/2684208515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d2ab2f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.232+0000 7fbe4841a740 1 --2- 172.21.15.44:0/2684208515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d27e880 0x56532d287720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.233+0000 7fbe4841a740 1 -- 172.21.15.44:0/2684208515 >> 172.21.15.44:0/2684208515 conn(0x56532d27c6b0 msgr2=0x56532d289e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.233+0000 7fbe4841a740 1 -- 172.21.15.44:0/2684208515 shutdown_connections 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.233+0000 7fbe4841a740 1 -- 172.21.15.44:0/2684208515 wait complete. 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.233+0000 7fbe4841a740 1 -- start start 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.233+0000 7fbe4841a740 1 Processor -- start 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.233+0000 7fbe4841a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27e880 0x56532d29bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.233+0000 7fbe4841a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d284b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.233+0000 7fbe4841a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d28b130 0x56532d2850c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe4841a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401ad4e0 con 0x56532d27f1a0 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe4841a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401ace10 con 0x56532d28b130 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe4841a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401aa070 con 0x56532d27e880 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe4841a740 10 client.0 _ll_register_callbacks cb 0x7fbe300e1a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:05.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe3effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d284b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe3effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d284b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55086/0 (socket says 172.21.15.44:55086) 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe3effd640 1 -- 172.21.15.44:0/508421419 learned_addr learned my addr 172.21.15.44:0/508421419 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe3ffff640 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d28b130 0x56532d2850c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27e880 0x56532d29bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe3effd640 1 -- 172.21.15.44:0/508421419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27e880 msgr2=0x56532d29bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe3effd640 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27e880 0x56532d29bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe3effd640 1 -- 172.21.15.44:0/508421419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d28b130 msgr2=0x56532d2850c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe3effd640 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d28b130 0x56532d2850c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe3effd640 1 -- 172.21.15.44:0/508421419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56532d285730 con 0x56532d27f1a0 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27e880 0x56532d29bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe3effd640 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d284b80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbe3402a910 tx=0x7fbe3402ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d211d0f9d8d876 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.234+0000 7fbe1effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:05.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.235+0000 7fbe1effd640 1 -- 172.21.15.44:0/508421419 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34035040 con 0x56532d27f1a0 2024-08-20T03:03:05.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.235+0000 7fbe1effd640 1 -- 172.21.15.44:0/508421419 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fbe340316b0 con 0x56532d27f1a0 2024-08-20T03:03:05.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.235+0000 7fbe4841a740 1 -- 172.21.15.44:0/508421419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe34005620 con 0x56532d27f1a0 2024-08-20T03:03:05.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.235+0000 7fbe1effd640 1 -- 172.21.15.44:0/508421419 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe340318b0 con 0x56532d27f1a0 2024-08-20T03:03:05.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.235+0000 7fbe4841a740 10 client.15717 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:05.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.235+0000 7fbe4841a740 1 -- 172.21.15.44:0/508421419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56532d29a690 con 0x56532d27f1a0 2024-08-20T03:03:05.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.235+0000 7fbe1effd640 1 -- 172.21.15.44:0/508421419 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe34005620 con 0x56532d27f1a0 2024-08-20T03:03:05.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.235+0000 7fbe1effd640 1 -- 172.21.15.44:0/508421419 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbe34004880 con 0x56532d27f1a0 2024-08-20T03:03:05.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.235+0000 7fbe4841a740 10 client.15717 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:05.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.235+0000 7fbe4841a740 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:05.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.235+0000 7fbe4841a740 20 client.15717 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:05.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.235+0000 7fbe4841a740 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56532d30e2c0 0x56532d3106b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:05.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.236+0000 7fbe4841a740 4 client.15717 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:05.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.236+0000 7fbe4841a740 1 -- 172.21.15.44:0/508421419 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56532d1f7880 con 0x56532d30e2c0 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.236+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56532d30e2c0 0x56532d3106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.236+0000 7fbe3f7fe640 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56532d30e2c0 0x56532d3106b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72dac2f8a5034363 server_cookie=59f6c272c6d6711f in_seq=0 out_seq=0 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.236+0000 7fbe1effd640 10 client.15717 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.236+0000 7fbe1effd640 1 -- 172.21.15.44:0/508421419 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56532d1f7880 con 0x56532d30e2c0 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.236+0000 7fbe1effd640 10 client.15717 handle_command_reply: tid=0 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.236+0000 7fbe4841a740 1 client.15717 shutdown 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.237+0000 7fbe4841a740 1 -- 172.21.15.44:0/508421419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 msgr2=0x56532d284b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.237+0000 7fbe4841a740 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d284b80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbe3402a910 tx=0x7fbe3402ab50 comp rx=0 tx=0).stop 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.237+0000 7fbe4841a740 1 -- 172.21.15.44:0/508421419 shutdown_connections 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.237+0000 7fbe4841a740 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56532d30e2c0 0x56532d3106b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.237+0000 7fbe4841a740 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56532d28b130 0x56532d2850c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.237+0000 7fbe1effd640 0 client.15717 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:05.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.237+0000 7fbe4841a740 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56532d27f1a0 0x56532d284b80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.237+0000 7fbe4841a740 1 --2- 172.21.15.44:0/508421419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56532d27e880 0x56532d29bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.237+0000 7fbe4841a740 1 -- 172.21.15.44:0/508421419 >> 172.21.15.44:0/508421419 conn(0x56532d27c6b0 msgr2=0x56532d27af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:05.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.237+0000 7fbe4841a740 1 -- 172.21.15.44:0/508421419 shutdown_connections 2024-08-20T03:03:05.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.237+0000 7fbe4841a740 1 -- 172.21.15.44:0/508421419 wait complete. 2024-08-20T03:03:05.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.237+0000 7fbe4841a740 20 client.15717 trim_cache size 0 max 16384 2024-08-20T03:03:05.244 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:05.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.238+0000 7fbe46021640 1 -- 172.21.15.44:0/2271009217 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbe280b3e50 msgr2=0x7fbe280b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:05.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.238+0000 7fbe46021640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbe280b3e50 0x7fbe280b6310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbe38007ef0 tx=0x7fbe38007480 comp rx=0 tx=0).stop 2024-08-20T03:03:05.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.238+0000 7fbe46021640 1 -- 172.21.15.44:0/2271009217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401039b0 msgr2=0x7fbe40198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:05.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.238+0000 7fbe46021640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401039b0 0x7fbe40198f90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000e9f0 tx=0x7fbe3000eeb0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.238+0000 7fbe46021640 1 -- 172.21.15.44:0/2271009217 shutdown_connections 2024-08-20T03:03:05.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.238+0000 7fbe46021640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbe280b3e50 0x7fbe280b6310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.238+0000 7fbe46021640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe4010ce10 0x7fbe4019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.238+0000 7fbe46021640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe401042d0 0x7fbe401994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.238+0000 7fbe46021640 1 --2- 172.21.15.44:0/2271009217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401039b0 0x7fbe40198f90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:05.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.238+0000 7fbe46021640 1 -- 172.21.15.44:0/2271009217 >> 172.21.15.44:0/2271009217 conn(0x7fbe400fd430 msgr2=0x7fbe4010f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:05.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.238+0000 7fbe46021640 1 -- 172.21.15.44:0/2271009217 shutdown_connections 2024-08-20T03:03:05.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:05.238+0000 7fbe46021640 1 -- 172.21.15.44:0/2271009217 wait complete. 2024-08-20T03:03:06.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:05 smithi165 ceph-mon[22357]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-08-20T03:03:06.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:05 smithi099 ceph-mon[21984]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-08-20T03:03:06.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:05 smithi044 ceph-mon[18790]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-08-20T03:03:07.055 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T03:03:07.055 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T03:03:07.418 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:07.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.931+0000 7f626453a640 1 -- 172.21.15.44:0/3332546925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625c107d20 msgr2=0x7f625c10e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:07.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.931+0000 7f626453a640 1 --2- 172.21.15.44:0/3332546925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625c107d20 0x7f625c10e520 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6254009ca0 tx=0x7f625402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.932+0000 7f626453a640 1 -- 172.21.15.44:0/3332546925 shutdown_connections 2024-08-20T03:03:07.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.932+0000 7f626453a640 1 --2- 172.21.15.44:0/3332546925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625c107d20 0x7f625c10e520 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.932+0000 7f626453a640 1 --2- 172.21.15.44:0/3332546925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625c1073a0 0x7f625c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.932+0000 7f626453a640 1 --2- 172.21.15.44:0/3332546925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f625c101990 0x7f625c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.932+0000 7f626453a640 1 -- 172.21.15.44:0/3332546925 >> 172.21.15.44:0/3332546925 conn(0x7f625c0fd400 msgr2=0x7f625c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:07.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.932+0000 7f626453a640 1 -- 172.21.15.44:0/3332546925 shutdown_connections 2024-08-20T03:03:07.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.932+0000 7f626453a640 1 -- 172.21.15.44:0/3332546925 wait complete. 2024-08-20T03:03:07.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.932+0000 7f626453a640 1 Processor -- start 2024-08-20T03:03:07.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f626453a640 1 -- start start 2024-08-20T03:03:07.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f626453a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625c101990 0x7f625c074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f626453a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625c1073a0 0x7f625c074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f626453a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f625c107d20 0x7f625c06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f626453a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1108e0 con 0x7f625c101990 2024-08-20T03:03:07.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f626453a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f625c110760 con 0x7f625c1073a0 2024-08-20T03:03:07.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f626453a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f625c110a60 con 0x7f625c107d20 2024-08-20T03:03:07.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f62622d6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625c101990 0x7f625c074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f62622d6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625c101990 0x7f625c074500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54304/0 (socket says 172.21.15.44:54304) 2024-08-20T03:03:07.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f62622d6640 1 -- 172.21.15.44:0/3256726809 learned_addr learned my addr 172.21.15.44:0/3256726809 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:07.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f6261ad5640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625c1073a0 0x7f625c074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f6262ad7640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f625c107d20 0x7f625c06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.933+0000 7f62622d6640 1 -- 172.21.15.44:0/3256726809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f625c107d20 msgr2=0x7f625c06d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:07.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.934+0000 7f62622d6640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f625c107d20 0x7f625c06d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.934+0000 7f62622d6640 1 -- 172.21.15.44:0/3256726809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625c1073a0 msgr2=0x7f625c074a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:07.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.934+0000 7f62622d6640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625c1073a0 0x7f625c074a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.934+0000 7f62622d6640 1 -- 172.21.15.44:0/3256726809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625c06e170 con 0x7f625c101990 2024-08-20T03:03:07.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.934+0000 7f6262ad7640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f625c107d20 0x7f625c06d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:07.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.934+0000 7f62622d6640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625c101990 0x7f625c074500 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f625800c920 tx=0x7f625800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:07.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.934+0000 7f62537fe640 1 -- 172.21.15.44:0/3256726809 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258007c70 con 0x7f625c101990 2024-08-20T03:03:07.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.934+0000 7f62537fe640 1 -- 172.21.15.44:0/3256726809 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6258007e10 con 0x7f625c101990 2024-08-20T03:03:07.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.934+0000 7f626453a640 1 -- 172.21.15.44:0/3256726809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625c06e460 con 0x7f625c101990 2024-08-20T03:03:07.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.934+0000 7f62537fe640 1 -- 172.21.15.44:0/3256726809 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258018820 con 0x7f625c101990 2024-08-20T03:03:07.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.934+0000 7f626453a640 1 -- 172.21.15.44:0/3256726809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f625c0733f0 con 0x7f625c101990 2024-08-20T03:03:07.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.937+0000 7f6266933740 1 Processor -- start 2024-08-20T03:03:07.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.937+0000 7f6266933740 1 -- start start 2024-08-20T03:03:07.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.937+0000 7f6266933740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb133b0 0x56471bb13790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.937+0000 7f6266933740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb1d710 0x56471bb13cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.937+0000 7f6266933740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb28310 0x56471bb2a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.937+0000 7f6266933740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56471ba89940 con 0x56471bb133b0 2024-08-20T03:03:07.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.937+0000 7f6266933740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56471ba78e40 con 0x56471bb28310 2024-08-20T03:03:07.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.937+0000 7f6266933740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56471ba7ce80 con 0x56471bb1d710 2024-08-20T03:03:07.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.938+0000 7f6262ad7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb133b0 0x56471bb13790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.938+0000 7f6262ad7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb133b0 0x56471bb13790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54306/0 (socket says 172.21.15.44:54306) 2024-08-20T03:03:07.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.938+0000 7f6262ad7640 1 -- 172.21.15.44:0/1839441518 learned_addr learned my addr 172.21.15.44:0/1839441518 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:07.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.938+0000 7f6261ad5640 1 --2- 172.21.15.44:0/1839441518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb28310 0x56471bb2a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.938+0000 7f6262ad7640 1 -- 172.21.15.44:0/1839441518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb1d710 msgr2=0x56471bb13cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.938+0000 7f6262ad7640 1 --2- 172.21.15.44:0/1839441518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb1d710 0x56471bb13cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.938+0000 7f6262ad7640 1 -- 172.21.15.44:0/1839441518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb28310 msgr2=0x56471bb2a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.938+0000 7f6262ad7640 1 --2- 172.21.15.44:0/1839441518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb28310 0x56471bb2a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.938+0000 7f6262ad7640 1 -- 172.21.15.44:0/1839441518 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56471ba15520 con 0x56471bb133b0 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.938+0000 7f62537fe640 1 -- 172.21.15.44:0/3256726809 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f625800f450 con 0x7f625c101990 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.938+0000 7f6262ad7640 1 --2- 172.21.15.44:0/1839441518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb133b0 0x56471bb13790 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f62540098e0 tx=0x7f6254009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe83e7d291f16b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.939+0000 7f6250ff9640 1 -- 172.21.15.44:0/1839441518 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254031df0 con 0x56471bb133b0 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.939+0000 7f6250ff9640 1 -- 172.21.15.44:0/1839441518 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6254036070 con 0x56471bb133b0 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.939+0000 7f6250ff9640 1 -- 172.21.15.44:0/1839441518 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254004330 con 0x56471bb133b0 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.939+0000 7f62537fe640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f62480b3dd0 0x7f62480b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.939+0000 7f62622d6640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f62480b3dd0 0x7f62480b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.939+0000 7f6266933740 1 -- 172.21.15.44:0/1839441518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb133b0 msgr2=0x56471bb13790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:07.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.939+0000 7f6266933740 1 --2- 172.21.15.44:0/1839441518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb133b0 0x56471bb13790 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f62540098e0 tx=0x7f6254009910 comp rx=0 tx=0).stop 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.939+0000 7f62537fe640 1 -- 172.21.15.44:0/3256726809 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f62580e42f0 con 0x7f625c101990 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.940+0000 7f6266933740 1 -- 172.21.15.44:0/1839441518 shutdown_connections 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.940+0000 7f6266933740 1 --2- 172.21.15.44:0/1839441518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb28310 0x56471bb2a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.940+0000 7f6266933740 1 --2- 172.21.15.44:0/1839441518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb1d710 0x56471bb13cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.940+0000 7f6266933740 1 --2- 172.21.15.44:0/1839441518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb133b0 0x56471bb13790 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.940+0000 7f6266933740 1 -- 172.21.15.44:0/1839441518 >> 172.21.15.44:0/1839441518 conn(0x56471bb111e0 msgr2=0x56471bb12380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.940+0000 7f62622d6640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f62480b3dd0 0x7f62480b6290 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f62580e19f0 tx=0x7f6258092040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.940+0000 7f6266933740 1 -- 172.21.15.44:0/1839441518 shutdown_connections 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.940+0000 7f6266933740 1 -- 172.21.15.44:0/1839441518 wait complete. 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.940+0000 7f6266933740 1 -- start start 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.940+0000 7f6266933740 1 Processor -- start 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6266933740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6266933740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 0x56471bb1aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6266933740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb28310 0x56471bb1b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6266933740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56471b8db890 con 0x56471bb1d710 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6266933740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56471b9ff700 con 0x56471bb133b0 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6266933740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56471bb2b140 con 0x56471bb28310 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6266933740 10 client.0 _ll_register_callbacks cb 0x56471ba746f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6261ad5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 0x56471bb1aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6261ad5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 0x56471bb1aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54314/0 (socket says 172.21.15.44:54314) 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6261ad5640 1 -- 172.21.15.44:0/1241403943 learned_addr learned my addr 172.21.15.44:0/1241403943 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6262ad7640 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb1a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6262ad7640 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb28310 0x56471bb1b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6261ad5640 1 -- 172.21.15.44:0/1241403943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb28310 msgr2=0x56471bb1b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6261ad5640 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb28310 0x56471bb1b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6261ad5640 1 -- 172.21.15.44:0/1241403943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 msgr2=0x56471bb1a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:07.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6261ad5640 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb1a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6261ad5640 1 -- 172.21.15.44:0/1241403943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56471b7d8930 con 0x56471bb1d710 2024-08-20T03:03:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6262ad7640 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb28310 0x56471bb1b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6261ad5640 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 0x56471bb1aca0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f624c00b940 tx=0x7f624c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b2ecf6f6fd466b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.941+0000 7f6245ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6245ffb640 1 -- 172.21.15.44:0/1241403943 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c01b640 con 0x56471bb1d710 2024-08-20T03:03:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6266933740 1 -- 172.21.15.44:0/1241403943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6254036070 con 0x56471bb1d710 2024-08-20T03:03:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6245ffb640 1 -- 172.21.15.44:0/1241403943 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f624c011440 con 0x56471bb1d710 2024-08-20T03:03:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6245ffb640 1 -- 172.21.15.44:0/1241403943 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f624c003e40 con 0x56471bb1d710 2024-08-20T03:03:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6266933740 10 client.15729 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6266933740 1 -- 172.21.15.44:0/1241403943 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56471bb36de0 con 0x56471bb1d710 2024-08-20T03:03:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6245ffb640 1 -- 172.21.15.44:0/1241403943 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6254036070 con 0x56471bb1d710 2024-08-20T03:03:07.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6245ffb640 1 -- 172.21.15.44:0/1241403943 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f624c023050 con 0x56471bb1d710 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6266933740 10 client.15729 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6266933740 10 client.15729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6266933740 20 client.15729 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6266933740 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56471bb381f0 0x56471bb3a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6266933740 4 client.15729 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.942+0000 7f6266933740 1 -- 172.21.15.44:0/1241403943 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56471ba8b890 con 0x56471bb381f0 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.943+0000 7f6262ad7640 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56471bb381f0 0x56471bb3a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.943+0000 7f6262ad7640 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56471bb381f0 0x56471bb3a5e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71842dc1f3a13cc3 server_cookie=326e9b4a2d70786b in_seq=0 out_seq=0 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.943+0000 7f6245ffb640 10 client.15729 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6245ffb640 1 -- 172.21.15.44:0/1241403943 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56471ba8b890 con 0x56471bb381f0 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6245ffb640 10 client.15729 handle_command_reply: tid=0 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6266933740 1 client.15729 shutdown 2024-08-20T03:03:07.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6266933740 1 -- 172.21.15.44:0/1241403943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 msgr2=0x56471bb1aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6266933740 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 0x56471bb1aca0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f624c00b940 tx=0x7f624c00be00 comp rx=0 tx=0).stop 2024-08-20T03:03:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6266933740 1 -- 172.21.15.44:0/1241403943 shutdown_connections 2024-08-20T03:03:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6266933740 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56471bb381f0 0x56471bb3a5e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6245ffb640 0 client.15729 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6266933740 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb28310 0x56471bb1b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6266933740 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 0x56471bb1aca0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6266933740 1 --2- 172.21.15.44:0/1241403943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6266933740 1 -- 172.21.15.44:0/1241403943 >> 172.21.15.44:0/1241403943 conn(0x56471bb111e0 msgr2=0x56471bb0fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6266933740 1 -- 172.21.15.44:0/1241403943 shutdown_connections 2024-08-20T03:03:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6266933740 1 -- 172.21.15.44:0/1241403943 wait complete. 2024-08-20T03:03:07.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.944+0000 7f6266933740 20 client.15729 trim_cache size 0 max 16384 2024-08-20T03:03:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.962+0000 7f6266933740 1 Processor -- start 2024-08-20T03:03:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.962+0000 7f6266933740 1 -- start start 2024-08-20T03:03:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.962+0000 7f6266933740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb2a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.962+0000 7f6266933740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb1d710 0x56471bb3fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.962+0000 7f6266933740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb28310 0x56471bb40430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.962+0000 7f6266933740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1b48b0 con 0x56471bb28310 2024-08-20T03:03:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.962+0000 7f6266933740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1b63f0 con 0x56471bb133b0 2024-08-20T03:03:07.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.962+0000 7f6266933740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1b4f80 con 0x56471bb1d710 2024-08-20T03:03:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.963+0000 7f6261ad5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb1d710 0x56471bb3fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.963+0000 7f6261ad5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb1d710 0x56471bb3fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41170/0 (socket says 172.21.15.44:41170) 2024-08-20T03:03:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.963+0000 7f6261ad5640 1 -- 172.21.15.44:0/3321455102 learned_addr learned my addr 172.21.15.44:0/3321455102 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.963+0000 7f6262ad7640 1 --2- 172.21.15.44:0/3321455102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb2a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.963+0000 7f6262ad7640 1 --2- 172.21.15.44:0/3321455102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb28310 0x56471bb40430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.963+0000 7f6261ad5640 1 -- 172.21.15.44:0/3321455102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 msgr2=0x56471bb2a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.963+0000 7f6261ad5640 1 --2- 172.21.15.44:0/3321455102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb2a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.963+0000 7f6261ad5640 1 -- 172.21.15.44:0/3321455102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb28310 msgr2=0x56471bb40430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.963+0000 7f6261ad5640 1 --2- 172.21.15.44:0/3321455102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb28310 0x56471bb40430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.963+0000 7f6261ad5640 1 -- 172.21.15.44:0/3321455102 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56471bb13e70 con 0x56471bb1d710 2024-08-20T03:03:07.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.963+0000 7f6262ad7640 1 --2- 172.21.15.44:0/3321455102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb28310 0x56471bb40430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:07.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.963+0000 7f6261ad5640 1 --2- 172.21.15.44:0/3321455102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb1d710 0x56471bb3fef0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f624c004570 tx=0x7f624c0045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e8725d0fb6c48f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:07.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.964+0000 7f6247fff640 1 -- 172.21.15.44:0/3321455102 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c011640 con 0x56471bb1d710 2024-08-20T03:03:07.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.964+0000 7f6247fff640 1 -- 172.21.15.44:0/3321455102 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f624c0054f0 con 0x56471bb1d710 2024-08-20T03:03:07.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.964+0000 7f6247fff640 1 -- 172.21.15.44:0/3321455102 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c004d20 con 0x56471bb1d710 2024-08-20T03:03:07.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.965+0000 7f6266933740 1 -- 172.21.15.44:0/3321455102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb1d710 msgr2=0x56471bb3fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:07.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.965+0000 7f6266933740 1 --2- 172.21.15.44:0/3321455102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb1d710 0x56471bb3fef0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f624c004570 tx=0x7f624c0045a0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.965+0000 7f6266933740 1 -- 172.21.15.44:0/3321455102 shutdown_connections 2024-08-20T03:03:07.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.965+0000 7f6266933740 1 --2- 172.21.15.44:0/3321455102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb28310 0x56471bb40430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.965+0000 7f6266933740 1 --2- 172.21.15.44:0/3321455102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb1d710 0x56471bb3fef0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.965+0000 7f6266933740 1 --2- 172.21.15.44:0/3321455102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb2a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.965+0000 7f6266933740 1 -- 172.21.15.44:0/3321455102 >> 172.21.15.44:0/3321455102 conn(0x56471bb111e0 msgr2=0x56471bb29e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.965+0000 7f6266933740 1 -- 172.21.15.44:0/3321455102 shutdown_connections 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.965+0000 7f6266933740 1 -- 172.21.15.44:0/3321455102 wait complete. 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.966+0000 7f6266933740 1 -- start start 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.966+0000 7f6266933740 1 Processor -- start 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.966+0000 7f6266933740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb9c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.966+0000 7f6266933740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 0x56471bb30420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6262ad7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb9c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6262ad7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb9c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58084/0 (socket says 172.21.15.44:58084) 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6261ad5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 0x56471bb30420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6261ad5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 0x56471bb30420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54330/0 (socket says 172.21.15.44:54330) 2024-08-20T03:03:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6262ad7640 1 -- 172.21.15.44:0/1807252722 learned_addr learned my addr 172.21.15.44:0/1807252722 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6266933740 1 --2- 172.21.15.44:0/1807252722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb28310 0x56471bb30bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6266933740 1 -- 172.21.15.44:0/1807252722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1b5d20 con 0x56471bb1d710 2024-08-20T03:03:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6266933740 1 -- 172.21.15.44:0/1807252722 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1b5650 con 0x56471bb133b0 2024-08-20T03:03:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6266933740 1 -- 172.21.15.44:0/1807252722 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f625c1b2890 con 0x56471bb28310 2024-08-20T03:03:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6261ad5640 1 -- 172.21.15.44:0/1807252722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb28310 msgr2=0x56471bb30bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6261ad5640 1 --2- 172.21.15.44:0/1807252722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb28310 0x56471bb30bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6261ad5640 1 -- 172.21.15.44:0/1807252722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 msgr2=0x56471bb9c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6261ad5640 1 --2- 172.21.15.44:0/1807252722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb9c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6261ad5640 1 -- 172.21.15.44:0/1807252722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56471bb2b540 con 0x56471bb1d710 2024-08-20T03:03:07.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.967+0000 7f6266933740 10 client.0 _ll_register_callbacks cb 0x7f624c004100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:07.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.968+0000 7f6266933740 1 -- 172.21.15.44:0/1807252722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f624c0054f0 con 0x56471bb1d710 2024-08-20T03:03:07.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.968+0000 7f6261ad5640 1 --2- 172.21.15.44:0/1807252722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 0x56471bb30420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f624c0059f0 tx=0x7f624c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c256e4e8a4447c8a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:07.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.968+0000 7f6245ffb640 10 client.15732 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:07.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.968+0000 7f6245ffb640 1 -- 172.21.15.44:0/1807252722 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c004ca0 con 0x56471bb1d710 2024-08-20T03:03:07.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.968+0000 7f6245ffb640 1 -- 172.21.15.44:0/1807252722 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f624c004e40 con 0x56471bb1d710 2024-08-20T03:03:07.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.968+0000 7f6245ffb640 1 -- 172.21.15.44:0/1807252722 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f624c005080 con 0x56471bb1d710 2024-08-20T03:03:07.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.968+0000 7f6266933740 10 client.15732 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:07.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.968+0000 7f6266933740 1 -- 172.21.15.44:0/1807252722 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56471bb23dd0 con 0x56471bb1d710 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.969+0000 7f6245ffb640 1 -- 172.21.15.44:0/1807252722 <== mon.0 v2:172.21.15.44:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f624c034070 con 0x56471bb1d710 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.969+0000 7f6266933740 10 client.15732 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.969+0000 7f6266933740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.969+0000 7f6266933740 20 client.15732 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.969+0000 7f6266933740 1 --2- 172.21.15.44:0/1807252722 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56471bb381f0 0x56471bb243a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.969+0000 7f6266933740 4 client.15732 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.969+0000 7f6266933740 1 -- 172.21.15.44:0/1807252722 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56471ba87330 con 0x56471bb381f0 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.969+0000 7f6262ad7640 1 --2- 172.21.15.44:0/1807252722 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56471bb381f0 0x56471bb243a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.970+0000 7f6262ad7640 1 --2- 172.21.15.44:0/1807252722 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56471bb381f0 0x56471bb243a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67b314bc9d62269a server_cookie=58d05944c9e97ea5 in_seq=0 out_seq=0 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.970+0000 7f6245ffb640 10 client.15732 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.971+0000 7f6245ffb640 1 -- 172.21.15.44:0/1807252722 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56471ba87330 con 0x56471bb381f0 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.971+0000 7f6245ffb640 10 client.15732 handle_command_reply: tid=0 2024-08-20T03:03:07.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.971+0000 7f6266933740 1 client.15732 shutdown 2024-08-20T03:03:07.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.971+0000 7f6266933740 1 -- 172.21.15.44:0/1807252722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 msgr2=0x56471bb30420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.971+0000 7f6266933740 1 --2- 172.21.15.44:0/1807252722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 0x56471bb30420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f624c0059f0 tx=0x7f624c00b8a0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.972+0000 7f6266933740 1 -- 172.21.15.44:0/1807252722 shutdown_connections 2024-08-20T03:03:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.972+0000 7f6266933740 1 --2- 172.21.15.44:0/1807252722 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56471bb381f0 0x56471bb243a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.972+0000 7f6266933740 1 --2- 172.21.15.44:0/1807252722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56471bb28310 0x56471bb30bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.972+0000 7f6266933740 1 --2- 172.21.15.44:0/1807252722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56471bb1d710 0x56471bb30420 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.972+0000 7f6266933740 1 --2- 172.21.15.44:0/1807252722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56471bb133b0 0x56471bb9c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.972+0000 7f6266933740 1 -- 172.21.15.44:0/1807252722 >> 172.21.15.44:0/1807252722 conn(0x56471bb111e0 msgr2=0x56471bb1bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.972+0000 7f6245ffb640 0 client.15732 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.972+0000 7f6266933740 1 -- 172.21.15.44:0/1807252722 shutdown_connections 2024-08-20T03:03:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.972+0000 7f6266933740 1 -- 172.21.15.44:0/1807252722 wait complete. 2024-08-20T03:03:07.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.972+0000 7f6266933740 20 client.15732 trim_cache size 0 max 16384 2024-08-20T03:03:07.977 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:03:07.977 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-08-20T03:03:07.977 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "a8aacdc1-a4e2-4122-8feb-7dcb7d05261f", 2024-08-20T03:03:07.977 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-08-20T03:03:07.977 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:03:07.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.973+0000 7f626453a640 1 -- 172.21.15.44:0/3256726809 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f62480b3dd0 msgr2=0x7f62480b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:07.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.973+0000 7f626453a640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f62480b3dd0 0x7f62480b6290 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f62580e19f0 tx=0x7f6258092040 comp rx=0 tx=0).stop 2024-08-20T03:03:07.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.973+0000 7f626453a640 1 -- 172.21.15.44:0/3256726809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625c101990 msgr2=0x7f625c074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:07.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.973+0000 7f626453a640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625c101990 0x7f625c074500 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f625800c920 tx=0x7f625800cde0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.973+0000 7f626453a640 1 -- 172.21.15.44:0/3256726809 shutdown_connections 2024-08-20T03:03:07.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.973+0000 7f626453a640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f62480b3dd0 0x7f62480b6290 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.973+0000 7f626453a640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f625c107d20 0x7f625c06d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.973+0000 7f626453a640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f625c1073a0 0x7f625c074a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.973+0000 7f626453a640 1 --2- 172.21.15.44:0/3256726809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f625c101990 0x7f625c074500 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:07.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.973+0000 7f626453a640 1 -- 172.21.15.44:0/3256726809 >> 172.21.15.44:0/3256726809 conn(0x7f625c0fd400 msgr2=0x7f625c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:07.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.973+0000 7f626453a640 1 -- 172.21.15.44:0/3256726809 shutdown_connections 2024-08-20T03:03:07.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:07.973+0000 7f626453a640 1 -- 172.21.15.44:0/3256726809 wait complete. 2024-08-20T03:03:08.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:07 smithi165 ceph-mon[22357]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-08-20T03:03:08.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:07 smithi099 ceph-mon[21984]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-08-20T03:03:08.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:07 smithi044 ceph-mon[18790]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-08-20T03:03:08.821 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:03:09.096 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:09.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:08 smithi165 ceph-mon[22357]: scrub queued for path: / 2024-08-20T03:03:09.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:08 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:09.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:08 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:03:09.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:08 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:09.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:08 smithi099 ceph-mon[21984]: scrub queued for path: / 2024-08-20T03:03:09.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:08 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:09.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:08 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:03:09.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:08 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:09.217 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:08 smithi044 ceph-mon[18790]: scrub queued for path: / 2024-08-20T03:03:09.217 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:08 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:09.217 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:08 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:03:09.217 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:08 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:09.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.593+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4275854058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea81042c0 msgr2=0x7f1ea810c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:09.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.593+0000 7f1eadaf4640 1 --2- 172.21.15.44:0/4275854058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea81042c0 0x7f1ea810c7b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009900 tx=0x7f1e9c02a410 comp rx=0 tx=0).stop 2024-08-20T03:03:09.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.593+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4275854058 shutdown_connections 2024-08-20T03:03:09.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.593+0000 7f1eadaf4640 1 --2- 172.21.15.44:0/4275854058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea810ce00 0x7f1ea810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.593+0000 7f1eadaf4640 1 --2- 172.21.15.44:0/4275854058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea81042c0 0x7f1ea810c7b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.593+0000 7f1eadaf4640 1 --2- 172.21.15.44:0/4275854058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ea81039a0 0x7f1ea8103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.593+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4275854058 >> 172.21.15.44:0/4275854058 conn(0x7f1ea80fd440 msgr2=0x7f1ea80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.593+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4275854058 shutdown_connections 2024-08-20T03:03:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.593+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4275854058 wait complete. 2024-08-20T03:03:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.593+0000 7f1eadaf4640 1 Processor -- start 2024-08-20T03:03:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.594+0000 7f1eadaf4640 1 -- start start 2024-08-20T03:03:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.594+0000 7f1eadaf4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ea81039a0 0x7f1ea8101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.594+0000 7f1eadaf4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea81042c0 0x7f1ea8101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.594+0000 7f1eadaf4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea810ce00 0x7f1ea81020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.594+0000 7f1eadaf4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8112a70 con 0x7f1ea810ce00 2024-08-20T03:03:09.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.594+0000 7f1eadaf4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81128f0 con 0x7f1ea81042c0 2024-08-20T03:03:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.594+0000 7f1eadaf4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8112bf0 con 0x7f1ea81039a0 2024-08-20T03:03:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.594+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea810ce00 0x7f1ea81020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.594+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea810ce00 0x7f1ea81020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54336/0 (socket says 172.21.15.44:54336) 2024-08-20T03:03:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.594+0000 7f1ea7fff640 1 -- 172.21.15.44:0/4052761914 learned_addr learned my addr 172.21.15.44:0/4052761914 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.594+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea81042c0 0x7f1ea8101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.594+0000 7f1ea77fe640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ea81039a0 0x7f1ea8101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.595+0000 7f1ea7fff640 1 -- 172.21.15.44:0/4052761914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ea81039a0 msgr2=0x7f1ea8101650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.595+0000 7f1ea7fff640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ea81039a0 0x7f1ea8101650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.595+0000 7f1ea7fff640 1 -- 172.21.15.44:0/4052761914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea81042c0 msgr2=0x7f1ea8101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.595+0000 7f1ea7fff640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea81042c0 0x7f1ea8101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.595+0000 7f1ea7fff640 1 -- 172.21.15.44:0/4052761914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea81a7400 con 0x7f1ea810ce00 2024-08-20T03:03:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.595+0000 7f1ea7fff640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea810ce00 0x7f1ea81020d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000eb10 tx=0x7f1ea000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.595+0000 7f1ea4ff9640 1 -- 172.21.15.44:0/4052761914 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0004b20 con 0x7f1ea810ce00 2024-08-20T03:03:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.595+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4052761914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea81a76f0 con 0x7f1ea810ce00 2024-08-20T03:03:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.595+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4052761914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea81a7c00 con 0x7f1ea810ce00 2024-08-20T03:03:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.595+0000 7f1ea4ff9640 1 -- 172.21.15.44:0/4052761914 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea0004cc0 con 0x7f1ea810ce00 2024-08-20T03:03:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.595+0000 7f1ea4ff9640 1 -- 172.21.15.44:0/4052761914 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0010690 con 0x7f1ea810ce00 2024-08-20T03:03:09.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.599+0000 7f1eafeed740 1 Processor -- start 2024-08-20T03:03:09.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.599+0000 7f1ea4ff9640 1 -- 172.21.15.44:0/4052761914 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f1ea0005000 con 0x7f1ea810ce00 2024-08-20T03:03:09.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.599+0000 7f1eafeed740 1 -- start start 2024-08-20T03:03:09.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1eafeed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 0x5640ce8edac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1eafeed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 0x5640ce8f9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1eafeed740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8f9f90 0x5640ce8fc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1eafeed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5640ce8648e0 con 0x5640ce8ed6e0 2024-08-20T03:03:09.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1eafeed740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5640ce853de0 con 0x5640ce8f9f90 2024-08-20T03:03:09.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1eafeed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5640ce857e20 con 0x5640ce8ee000 2024-08-20T03:03:09.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1ea6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 0x5640ce8edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1ea6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 0x5640ce8edac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54350/0 (socket says 172.21.15.44:54350) 2024-08-20T03:03:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1ea6ffd640 1 -- 172.21.15.44:0/1307978122 learned_addr learned my addr 172.21.15.44:0/1307978122 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1ea77fe640 1 --2- 172.21.15.44:0/1307978122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 0x5640ce8f9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/1307978122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8f9f90 0x5640ce8fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1ea6ffd640 1 -- 172.21.15.44:0/1307978122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 msgr2=0x5640ce8f9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/1307978122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 0x5640ce8f9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1ea6ffd640 1 -- 172.21.15.44:0/1307978122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8f9f90 msgr2=0x5640ce8fc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/1307978122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8f9f90 0x5640ce8fc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1ea6ffd640 1 -- 172.21.15.44:0/1307978122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640ce5b3650 con 0x5640ce8ed6e0 2024-08-20T03:03:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.600+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/1307978122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 0x5640ce8edac0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009900 tx=0x7f1e9c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da546da8509a0e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.601+0000 7f1e8dffb640 1 -- 172.21.15.44:0/1307978122 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c044020 con 0x5640ce8ed6e0 2024-08-20T03:03:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.601+0000 7f1e8dffb640 1 -- 172.21.15.44:0/1307978122 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e9c0316a0 con 0x5640ce8ed6e0 2024-08-20T03:03:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.601+0000 7f1e8dffb640 1 -- 172.21.15.44:0/1307978122 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c035040 con 0x5640ce8ed6e0 2024-08-20T03:03:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.601+0000 7f1ea4ff9640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1e900b3dd0 0x7f1e900b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.601+0000 7f1ea77fe640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1e900b3dd0 0x7f1e900b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.601+0000 7f1ea4ff9640 1 -- 172.21.15.44:0/4052761914 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1ea00e47a0 con 0x7f1ea810ce00 2024-08-20T03:03:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.602+0000 7f1eafeed740 1 -- 172.21.15.44:0/1307978122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 msgr2=0x5640ce8edac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.602+0000 7f1eafeed740 1 --2- 172.21.15.44:0/1307978122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 0x5640ce8edac0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009900 tx=0x7f1e9c031400 comp rx=0 tx=0).stop 2024-08-20T03:03:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.602+0000 7f1ea77fe640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1e900b3dd0 0x7f1e900b6290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5640ce8f0860 tx=0x7f1e9800b4d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.602+0000 7f1eafeed740 1 -- 172.21.15.44:0/1307978122 shutdown_connections 2024-08-20T03:03:09.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.602+0000 7f1eafeed740 1 --2- 172.21.15.44:0/1307978122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8f9f90 0x5640ce8fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.602+0000 7f1eafeed740 1 --2- 172.21.15.44:0/1307978122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 0x5640ce8f9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.602+0000 7f1eafeed740 1 --2- 172.21.15.44:0/1307978122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 0x5640ce8edac0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.602+0000 7f1eafeed740 1 -- 172.21.15.44:0/1307978122 >> 172.21.15.44:0/1307978122 conn(0x5640ce8eb510 msgr2=0x5640ce8ec6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.602+0000 7f1eafeed740 1 -- 172.21.15.44:0/1307978122 shutdown_connections 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.602+0000 7f1eafeed740 1 -- 172.21.15.44:0/1307978122 wait complete. 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.602+0000 7f1eafeed740 1 -- start start 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1eafeed740 1 Processor -- start 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1eafeed740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 0x5640ce8f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1eafeed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 0x5640ce8f7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1eafeed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8f9f90 0x5640ce8f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1eafeed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5640ce6b6700 con 0x5640ce8f9f90 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1eafeed740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5640ce7da7c0 con 0x5640ce8ed6e0 2024-08-20T03:03:09.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1eafeed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5640ce8fcbe0 con 0x5640ce8ee000 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1eafeed740 10 client.0 _ll_register_callbacks cb 0x5640ce84d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1ea77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 0x5640ce8f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1ea77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 0x5640ce8f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58112/0 (socket says 172.21.15.44:58112) 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1ea77fe640 1 -- 172.21.15.44:0/118531489 learned_addr learned my addr 172.21.15.44:0/118531489 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8f9f90 0x5640ce8f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.603+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/118531489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 0x5640ce8f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.604+0000 7f1ea77fe640 1 -- 172.21.15.44:0/118531489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 msgr2=0x5640ce8f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.604+0000 7f1ea77fe640 1 --2- 172.21.15.44:0/118531489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 0x5640ce8f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.604+0000 7f1ea77fe640 1 -- 172.21.15.44:0/118531489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8f9f90 msgr2=0x5640ce8f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.604+0000 7f1ea77fe640 1 --2- 172.21.15.44:0/118531489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8f9f90 0x5640ce8f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.604+0000 7f1ea77fe640 1 -- 172.21.15.44:0/118531489 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640ce906070 con 0x5640ce8ed6e0 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.604+0000 7f1ea77fe640 1 --2- 172.21.15.44:0/118531489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 0x5640ce8f6fe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e98005ba0 tx=0x7f1e980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28bfb84fe2bce439 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:09.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.604+0000 7f1e86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:03:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.604+0000 7f1e86ffd640 1 -- 172.21.15.44:0/118531489 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98029680 con 0x5640ce8ed6e0 2024-08-20T03:03:09.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.604+0000 7f1e86ffd640 1 -- 172.21.15.44:0/118531489 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1e98029820 con 0x5640ce8ed6e0 2024-08-20T03:03:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.604+0000 7f1eafeed740 1 -- 172.21.15.44:0/118531489 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5640ce803100 con 0x5640ce8ed6e0 2024-08-20T03:03:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.605+0000 7f1e86ffd640 1 -- 172.21.15.44:0/118531489 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e98029a20 con 0x5640ce8ed6e0 2024-08-20T03:03:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.605+0000 7f1eafeed740 10 client.25289 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.605+0000 7f1eafeed740 1 -- 172.21.15.44:0/118531489 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640ce908a80 con 0x5640ce8ed6e0 2024-08-20T03:03:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.605+0000 7f1e86ffd640 1 -- 172.21.15.44:0/118531489 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5640ce803100 con 0x5640ce8ed6e0 2024-08-20T03:03:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.605+0000 7f1e86ffd640 1 -- 172.21.15.44:0/118531489 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1e98033c60 con 0x5640ce8ed6e0 2024-08-20T03:03:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.605+0000 7f1eafeed740 10 client.25289 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.605+0000 7f1eafeed740 10 client.25289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.605+0000 7f1eafeed740 20 client.25289 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.605+0000 7f1eafeed740 1 --2- 172.21.15.44:0/118531489 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5640ce909e90 0x5640ce90c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.605+0000 7f1eafeed740 4 client.25289 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:09.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.605+0000 7f1eafeed740 1 -- 172.21.15.44:0/118531489 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640ce866830 con 0x5640ce909e90 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.606+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/118531489 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5640ce909e90 0x5640ce90c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.606+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/118531489 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5640ce909e90 0x5640ce90c2c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa2713484e5aa15 server_cookie=f97415eb82c9aa96 in_seq=0 out_seq=0 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.606+0000 7f1e86ffd640 10 client.25289 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1e86ffd640 1 -- 172.21.15.44:0/118531489 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640ce866830 con 0x5640ce909e90 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1e86ffd640 10 client.25289 handle_command_reply: tid=0 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1eafeed740 1 client.25289 shutdown 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1eafeed740 1 -- 172.21.15.44:0/118531489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 msgr2=0x5640ce8f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1eafeed740 1 --2- 172.21.15.44:0/118531489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 0x5640ce8f6fe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e98005ba0 tx=0x7f1e980046c0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1eafeed740 1 -- 172.21.15.44:0/118531489 shutdown_connections 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1eafeed740 1 --2- 172.21.15.44:0/118531489 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5640ce909e90 0x5640ce90c2c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1e86ffd640 0 client.25289 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1eafeed740 1 --2- 172.21.15.44:0/118531489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8f9f90 0x5640ce8f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1eafeed740 1 --2- 172.21.15.44:0/118531489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 0x5640ce8f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1eafeed740 1 --2- 172.21.15.44:0/118531489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 0x5640ce8f6fe0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1eafeed740 1 -- 172.21.15.44:0/118531489 >> 172.21.15.44:0/118531489 conn(0x5640ce8eb510 msgr2=0x5640ce8f10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1eafeed740 1 -- 172.21.15.44:0/118531489 shutdown_connections 2024-08-20T03:03:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1eafeed740 1 -- 172.21.15.44:0/118531489 wait complete. 2024-08-20T03:03:09.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.607+0000 7f1eafeed740 20 client.25289 trim_cache size 0 max 16384 2024-08-20T03:03:09.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.638+0000 7f1eafeed740 1 Processor -- start 2024-08-20T03:03:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.638+0000 7f1eafeed740 1 -- start start 2024-08-20T03:03:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.638+0000 7f1eafeed740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 0x5640ce8ea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.638+0000 7f1eafeed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 0x5640ce8f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.638+0000 7f1eafeed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8f9f90 0x5640ce8f7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.638+0000 7f1eafeed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81b03a0 con 0x5640ce8f9f90 2024-08-20T03:03:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.638+0000 7f1eafeed740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81b1ee0 con 0x5640ce8ed6e0 2024-08-20T03:03:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.638+0000 7f1eafeed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81b0a70 con 0x5640ce8ee000 2024-08-20T03:03:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.638+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8f9f90 0x5640ce8f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.638+0000 7f1ea7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8f9f90 0x5640ce8f7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54372/0 (socket says 172.21.15.44:54372) 2024-08-20T03:03:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.638+0000 7f1ea7fff640 1 -- 172.21.15.44:0/3059930975 learned_addr learned my addr 172.21.15.44:0/3059930975 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.639+0000 7f1ea77fe640 1 --2- 172.21.15.44:0/3059930975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 0x5640ce8ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.639+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/3059930975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 0x5640ce8f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.639+0000 7f1ea7fff640 1 -- 172.21.15.44:0/3059930975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 msgr2=0x5640ce8f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:09.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.639+0000 7f1ea7fff640 1 --2- 172.21.15.44:0/3059930975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 0x5640ce8f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.639+0000 7f1ea7fff640 1 -- 172.21.15.44:0/3059930975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 msgr2=0x5640ce8ea750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:09.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.639+0000 7f1ea7fff640 1 --2- 172.21.15.44:0/3059930975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 0x5640ce8ea750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.639+0000 7f1ea7fff640 1 -- 172.21.15.44:0/3059930975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640ce8ebf90 con 0x5640ce8f9f90 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.639+0000 7f1ea77fe640 1 --2- 172.21.15.44:0/3059930975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 0x5640ce8ea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.639+0000 7f1ea7fff640 1 --2- 172.21.15.44:0/3059930975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8f9f90 0x5640ce8f7660 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00049d0 tx=0x7f1ea00aacc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7c53f1e6e8dc9d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.639+0000 7f1e8d7fa640 1 -- 172.21.15.44:0/3059930975 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00ab2b0 con 0x5640ce8f9f90 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.639+0000 7f1e8d7fa640 1 -- 172.21.15.44:0/3059930975 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea00af8b0 con 0x5640ce8f9f90 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.639+0000 7f1e8d7fa640 1 -- 172.21.15.44:0/3059930975 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00ab770 con 0x5640ce8f9f90 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.640+0000 7f1eafeed740 1 -- 172.21.15.44:0/3059930975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8f9f90 msgr2=0x5640ce8f7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.640+0000 7f1eafeed740 1 --2- 172.21.15.44:0/3059930975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8f9f90 0x5640ce8f7660 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00049d0 tx=0x7f1ea00aacc0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.640+0000 7f1eafeed740 1 -- 172.21.15.44:0/3059930975 shutdown_connections 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.640+0000 7f1eafeed740 1 --2- 172.21.15.44:0/3059930975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8f9f90 0x5640ce8f7660 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.640+0000 7f1eafeed740 1 --2- 172.21.15.44:0/3059930975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8ee000 0x5640ce8f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.640+0000 7f1eafeed740 1 --2- 172.21.15.44:0/3059930975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ed6e0 0x5640ce8ea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.640+0000 7f1eafeed740 1 -- 172.21.15.44:0/3059930975 >> 172.21.15.44:0/3059930975 conn(0x5640ce8eb510 msgr2=0x5640ce8f6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.640+0000 7f1eafeed740 1 -- 172.21.15.44:0/3059930975 shutdown_connections 2024-08-20T03:03:09.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.641+0000 7f1eafeed740 1 -- 172.21.15.44:0/3059930975 wait complete. 2024-08-20T03:03:09.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.641+0000 7f1eafeed740 1 -- start start 2024-08-20T03:03:09.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.641+0000 7f1eafeed740 1 Processor -- start 2024-08-20T03:03:09.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.641+0000 7f1eafeed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 0x5640ce8fd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.641+0000 7f1eafeed740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ee000 0x5640ce8fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1eafeed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8f9f90 0x5640ce96cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1eafeed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81b1810 con 0x5640ce8ed6e0 2024-08-20T03:03:09.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1eafeed740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81b1140 con 0x5640ce8ee000 2024-08-20T03:03:09.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1eafeed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81ae380 con 0x5640ce8f9f90 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1eafeed740 10 client.0 _ll_register_callbacks cb 0x7f1e98005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1ea77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 0x5640ce8fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1ea77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 0x5640ce8fd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54386/0 (socket says 172.21.15.44:54386) 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1ea77fe640 1 -- 172.21.15.44:0/2035339095 learned_addr learned my addr 172.21.15.44:0/2035339095 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ee000 0x5640ce8fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1ea7fff640 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8f9f90 0x5640ce96cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1ea6ffd640 1 -- 172.21.15.44:0/2035339095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8f9f90 msgr2=0x5640ce96cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8f9f90 0x5640ce96cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1ea6ffd640 1 -- 172.21.15.44:0/2035339095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 msgr2=0x5640ce8fd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 0x5640ce8fd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1ea6ffd640 1 -- 172.21.15.44:0/2035339095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640ce96d260 con 0x5640ce8ee000 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1ea77fe640 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 0x5640ce8fd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:09.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1ea6ffd640 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ee000 0x5640ce8fdbf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0053f0 tx=0x7f1e9c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b9552226231c36b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.642+0000 7f1e86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:03:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.643+0000 7f1e86ffd640 1 -- 172.21.15.44:0/2035339095 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c044070 con 0x5640ce8ee000 2024-08-20T03:03:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.643+0000 7f1eafeed740 1 -- 172.21.15.44:0/2035339095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea00af8b0 con 0x5640ce8ee000 2024-08-20T03:03:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.643+0000 7f1e86ffd640 1 -- 172.21.15.44:0/2035339095 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1e9c004240 con 0x5640ce8ee000 2024-08-20T03:03:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.643+0000 7f1e86ffd640 1 -- 172.21.15.44:0/2035339095 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1e9c05a050 con 0x5640ce8ee000 2024-08-20T03:03:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.643+0000 7f1eafeed740 10 client.25295 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.643+0000 7f1eafeed740 1 -- 172.21.15.44:0/2035339095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640ce9746a0 con 0x5640ce8ee000 2024-08-20T03:03:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.643+0000 7f1e86ffd640 1 -- 172.21.15.44:0/2035339095 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea00af8b0 con 0x5640ce8ee000 2024-08-20T03:03:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.644+0000 7f1e86ffd640 1 -- 172.21.15.44:0/2035339095 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1e9c0320d0 con 0x5640ce8ee000 2024-08-20T03:03:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.644+0000 7f1eafeed740 10 client.25295 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.644+0000 7f1eafeed740 10 client.25295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:09.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.644+0000 7f1eafeed740 20 client.25295 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.644+0000 7f1eafeed740 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5640ce909e90 0x5640ce976a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.644+0000 7f1eafeed740 4 client.25295 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.644+0000 7f1eafeed740 1 -- 172.21.15.44:0/2035339095 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5640ce8622d0 con 0x5640ce909e90 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.644+0000 7f1ea77fe640 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5640ce909e90 0x5640ce976a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.644+0000 7f1ea77fe640 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5640ce909e90 0x5640ce976a40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36e5a70362d9cd1f server_cookie=fde88a89e4208079 in_seq=0 out_seq=0 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.644+0000 7f1e86ffd640 10 client.25295 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1e86ffd640 1 -- 172.21.15.44:0/2035339095 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5640ce8622d0 con 0x5640ce909e90 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1e86ffd640 10 client.25295 handle_command_reply: tid=0 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1eafeed740 1 client.25295 shutdown 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1eafeed740 1 -- 172.21.15.44:0/2035339095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ee000 msgr2=0x5640ce8fdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1eafeed740 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ee000 0x5640ce8fdbf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0053f0 tx=0x7f1e9c009680 comp rx=0 tx=0).stop 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1eafeed740 1 -- 172.21.15.44:0/2035339095 shutdown_connections 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1eafeed740 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5640ce909e90 0x5640ce976a40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1eafeed740 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640ce8f9f90 0x5640ce96cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1eafeed740 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5640ce8ee000 0x5640ce8fdbf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1eafeed740 1 --2- 172.21.15.44:0/2035339095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640ce8ed6e0 0x5640ce8fd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1eafeed740 1 -- 172.21.15.44:0/2035339095 >> 172.21.15.44:0/2035339095 conn(0x5640ce8eb510 msgr2=0x5640ce905540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:09.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1e86ffd640 0 client.25295 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:09.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.645+0000 7f1eafeed740 1 -- 172.21.15.44:0/2035339095 shutdown_connections 2024-08-20T03:03:09.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.646+0000 7f1eafeed740 1 -- 172.21.15.44:0/2035339095 wait complete. 2024-08-20T03:03:09.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.646+0000 7f1eafeed740 20 client.25295 trim_cache size 0 max 16384 2024-08-20T03:03:09.652 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:03:09.652 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-08-20T03:03:09.652 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-08-20T03:03:09.652 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:03:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.646+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4052761914 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1e900b3dd0 msgr2=0x7f1e900b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.646+0000 7f1eadaf4640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1e900b3dd0 0x7f1e900b6290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5640ce8f0860 tx=0x7f1e9800b4d0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.646+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4052761914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea810ce00 msgr2=0x7f1ea81020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.646+0000 7f1eadaf4640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea810ce00 0x7f1ea81020d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000eb10 tx=0x7f1ea000c2d0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.646+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4052761914 shutdown_connections 2024-08-20T03:03:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.646+0000 7f1eadaf4640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1e900b3dd0 0x7f1e900b6290 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.646+0000 7f1eadaf4640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ea810ce00 0x7f1ea81020d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.646+0000 7f1eadaf4640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ea81042c0 0x7f1ea8101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.646+0000 7f1eadaf4640 1 --2- 172.21.15.44:0/4052761914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ea81039a0 0x7f1ea8101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:09.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.646+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4052761914 >> 172.21.15.44:0/4052761914 conn(0x7f1ea80fd440 msgr2=0x7f1ea80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:09.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.647+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4052761914 shutdown_connections 2024-08-20T03:03:09.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:09.647+0000 7f1eadaf4640 1 -- 172.21.15.44:0/4052761914 wait complete. 2024-08-20T03:03:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:09 smithi165 ceph-mon[22357]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-08-20T03:03:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:09 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:03:10.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:09 smithi099 ceph-mon[21984]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-08-20T03:03:10.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:09 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:03:10.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:09 smithi044 ceph-mon[18790]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-08-20T03:03:10.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:09 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:03:10.495 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T03:03:10.496 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:03:10.766 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:11.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.240+0000 7f04276f1640 1 -- 172.21.15.44:0/918248802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0420101990 msgr2=0x7f0420101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:11.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.240+0000 7f04276f1640 1 --2- 172.21.15.44:0/918248802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0420101990 0x7f0420101d70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f041c009900 tx=0x7f041c02a410 comp rx=0 tx=0).stop 2024-08-20T03:03:11.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.241+0000 7f04276f1640 1 -- 172.21.15.44:0/918248802 shutdown_connections 2024-08-20T03:03:11.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.241+0000 7f04276f1640 1 --2- 172.21.15.44:0/918248802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0420107d20 0x7f042010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:11.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.241+0000 7f04276f1640 1 --2- 172.21.15.44:0/918248802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04201073a0 0x7f04201077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:11.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.241+0000 7f04276f1640 1 --2- 172.21.15.44:0/918248802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0420101990 0x7f0420101d70 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:11.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.241+0000 7f04276f1640 1 -- 172.21.15.44:0/918248802 >> 172.21.15.44:0/918248802 conn(0x7f04200fd400 msgr2=0x7f04200ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:11.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.241+0000 7f04276f1640 1 -- 172.21.15.44:0/918248802 shutdown_connections 2024-08-20T03:03:11.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.241+0000 7f04276f1640 1 -- 172.21.15.44:0/918248802 wait complete. 2024-08-20T03:03:11.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.241+0000 7f04276f1640 1 Processor -- start 2024-08-20T03:03:11.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.242+0000 7f04276f1640 1 -- start start 2024-08-20T03:03:11.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.242+0000 7f04276f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0420101990 0x7f0420198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:11.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.242+0000 7f04276f1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04201073a0 0x7f0420199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.242+0000 7f04276f1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0420107d20 0x7f042019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.242+0000 7f04276f1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f04201119e0 con 0x7f04201073a0 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.242+0000 7f04276f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0420111860 con 0x7f0420101990 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.242+0000 7f04276f1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0420111b60 con 0x7f0420107d20 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.242+0000 7f0424c8c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04201073a0 0x7f0420199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.242+0000 7f042548d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0420101990 0x7f0420198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.242+0000 7f042548d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0420101990 0x7f0420198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:58154/0 (socket says 172.21.15.44:58154) 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.242+0000 7f042548d640 1 -- 172.21.15.44:0/3438499314 learned_addr learned my addr 172.21.15.44:0/3438499314 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.242+0000 7f0425c8e640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0420107d20 0x7f042019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.243+0000 7f042548d640 1 -- 172.21.15.44:0/3438499314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0420107d20 msgr2=0x7f042019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.243+0000 7f042548d640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0420107d20 0x7f042019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.243+0000 7f042548d640 1 -- 172.21.15.44:0/3438499314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04201073a0 msgr2=0x7f0420199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:11.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.243+0000 7f042548d640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04201073a0 0x7f0420199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:11.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.243+0000 7f042548d640 1 -- 172.21.15.44:0/3438499314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042019f070 con 0x7f0420101990 2024-08-20T03:03:11.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.243+0000 7f042548d640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0420101990 0x7f0420198fc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f041c005940 tx=0x7f041c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:11.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.243+0000 7f04167fc640 1 -- 172.21.15.44:0/3438499314 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c03f030 con 0x7f0420101990 2024-08-20T03:03:11.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.243+0000 7f04167fc640 1 -- 172.21.15.44:0/3438499314 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f041c003aa0 con 0x7f0420101990 2024-08-20T03:03:11.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.243+0000 7f04276f1640 1 -- 172.21.15.44:0/3438499314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042019f300 con 0x7f0420101990 2024-08-20T03:03:11.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.244+0000 7f04167fc640 1 -- 172.21.15.44:0/3438499314 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c035070 con 0x7f0420101990 2024-08-20T03:03:11.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.244+0000 7f04276f1640 1 -- 172.21.15.44:0/3438499314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04201a5a20 con 0x7f0420101990 2024-08-20T03:03:11.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.245+0000 7f04276f1640 1 -- 172.21.15.44:0/3438499314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0420102c40 con 0x7f0420101990 2024-08-20T03:03:11.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.247+0000 7f04167fc640 1 -- 172.21.15.44:0/3438499314 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f041c042030 con 0x7f0420101990 2024-08-20T03:03:11.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.248+0000 7f04167fc640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f040c0b3cb0 0x7f040c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:11.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.248+0000 7f04167fc640 1 -- 172.21.15.44:0/3438499314 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f041c104410 con 0x7f0420101990 2024-08-20T03:03:11.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.248+0000 7f0424c8c640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f040c0b3cb0 0x7f040c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:11.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.249+0000 7f0424c8c640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f040c0b3cb0 0x7f040c0b6170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0410005fd0 tx=0x7f0410005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:11.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.252+0000 7f04167fc640 1 -- 172.21.15.44:0/3438499314 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f041c0cb5c0 con 0x7f0420101990 2024-08-20T03:03:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.662+0000 7f04276f1640 1 -- 172.21.15.44:0/3438499314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f042019a0f0 con 0x7f0420101990 2024-08-20T03:03:11.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.663+0000 7f04167fc640 1 -- 172.21.15.44:0/3438499314 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7f041c0caec0 con 0x7f0420101990 2024-08-20T03:03:11.666 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:03:11.666 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":28,"btime":"2024-08-20T03:02:12:953534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:02:12.906757+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24569,24583,24596,24589,24604]},"id":1}]} 2024-08-20T03:03:11.666 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 28 2024-08-20T03:03:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.666+0000 7f04276f1640 1 -- 172.21.15.44:0/3438499314 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f040c0b3cb0 msgr2=0x7f040c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.666+0000 7f04276f1640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f040c0b3cb0 0x7f040c0b6170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0410005fd0 tx=0x7f0410005850 comp rx=0 tx=0).stop 2024-08-20T03:03:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.666+0000 7f04276f1640 1 -- 172.21.15.44:0/3438499314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0420101990 msgr2=0x7f0420198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.666+0000 7f04276f1640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0420101990 0x7f0420198fc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f041c005940 tx=0x7f041c004270 comp rx=0 tx=0).stop 2024-08-20T03:03:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.666+0000 7f04276f1640 1 -- 172.21.15.44:0/3438499314 shutdown_connections 2024-08-20T03:03:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.666+0000 7f04276f1640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f040c0b3cb0 0x7f040c0b6170 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.666+0000 7f04276f1640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0420107d20 0x7f042019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.666+0000 7f04276f1640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04201073a0 0x7f0420199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.666+0000 7f04276f1640 1 --2- 172.21.15.44:0/3438499314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0420101990 0x7f0420198fc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.666+0000 7f04276f1640 1 -- 172.21.15.44:0/3438499314 >> 172.21.15.44:0/3438499314 conn(0x7f04200fd400 msgr2=0x7f042006aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.666+0000 7f04276f1640 1 -- 172.21.15.44:0/3438499314 shutdown_connections 2024-08-20T03:03:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:11.666+0000 7f04276f1640 1 -- 172.21.15.44:0/3438499314 wait complete. 2024-08-20T03:03:12.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:11 smithi044 ceph-mon[18790]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-08-20T03:03:12.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:11 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3438499314' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:03:12.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:11 smithi165 ceph-mon[22357]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-08-20T03:03:12.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:11 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3438499314' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:03:12.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:11 smithi099 ceph-mon[21984]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-08-20T03:03:12.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:11 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3438499314' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:03:12.487 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-20T03:03:12.754 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:13.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.504+0000 7f1fa0878640 1 -- 172.21.15.44:0/2178944596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f9c103850 msgr2=0x7f1f9c103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:13.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.504+0000 7f1fa0878640 1 --2- 172.21.15.44:0/2178944596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f9c103850 0x7f1f9c103c30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009900 tx=0x7f1f8402a410 comp rx=0 tx=0).stop 2024-08-20T03:03:13.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.504+0000 7f1fa0878640 1 -- 172.21.15.44:0/2178944596 shutdown_connections 2024-08-20T03:03:13.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.504+0000 7f1fa0878640 1 --2- 172.21.15.44:0/2178944596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f9c10ce00 0x7f1f9c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.504+0000 7f1fa0878640 1 --2- 172.21.15.44:0/2178944596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f9c104250 0x7f1f9c10c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.504+0000 7f1fa0878640 1 --2- 172.21.15.44:0/2178944596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f9c103850 0x7f1f9c103c30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.504+0000 7f1fa0878640 1 -- 172.21.15.44:0/2178944596 >> 172.21.15.44:0/2178944596 conn(0x7f1f9c0fd400 msgr2=0x7f1f9c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.504+0000 7f1fa0878640 1 -- 172.21.15.44:0/2178944596 shutdown_connections 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.504+0000 7f1fa0878640 1 -- 172.21.15.44:0/2178944596 wait complete. 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.505+0000 7f1fa0878640 1 Processor -- start 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.505+0000 7f1fa0878640 1 -- start start 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.505+0000 7f1fa0878640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f9c103850 0x7f1f9c198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.505+0000 7f1fa0878640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f9c104250 0x7f1f9c199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.505+0000 7f1fa0878640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f9c10ce00 0x7f1f9c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.505+0000 7f1fa0878640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c112a60 con 0x7f1f9c104250 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.505+0000 7f1fa0878640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1128e0 con 0x7f1f9c10ce00 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.505+0000 7f1fa0878640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c112be0 con 0x7f1f9c103850 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f99d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f9c104250 0x7f1f9c199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f99d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f9c104250 0x7f1f9c199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54420/0 (socket says 172.21.15.44:54420) 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f99d9b640 1 -- 172.21.15.44:0/3230095713 learned_addr learned my addr 172.21.15.44:0/3230095713 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f9c103850 0x7f1f9c198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f9ad9d640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f9c10ce00 0x7f1f9c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f99d9b640 1 -- 172.21.15.44:0/3230095713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f9c103850 msgr2=0x7f1f9c198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f9c103850 0x7f1f9c198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f99d9b640 1 -- 172.21.15.44:0/3230095713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f9c10ce00 msgr2=0x7f1f9c19e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f9c10ce00 0x7f1f9c19e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f99d9b640 1 -- 172.21.15.44:0/3230095713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c19f000 con 0x7f1f9c104250 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f9c104250 0x7f1f9c199490 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00e9f0 tx=0x7f1f8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f937fe640 1 -- 172.21.15.44:0/3230095713 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00cd80 con 0x7f1f9c104250 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1fa0878640 1 -- 172.21.15.44:0/3230095713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c19f2f0 con 0x7f1f9c104250 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f937fe640 1 -- 172.21.15.44:0/3230095713 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f8c0108e0 con 0x7f1f9c104250 2024-08-20T03:03:13.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1f937fe640 1 -- 172.21.15.44:0/3230095713 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c018980 con 0x7f1f9c104250 2024-08-20T03:03:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.506+0000 7f1fa0878640 1 -- 172.21.15.44:0/3230095713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c1015b0 con 0x7f1f9c104250 2024-08-20T03:03:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1fa2c71740 1 Processor -- start 2024-08-20T03:03:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1fa2c71740 1 -- start start 2024-08-20T03:03:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1f937fe640 1 -- 172.21.15.44:0/3230095713 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f1f8c010430 con 0x7f1f9c104250 2024-08-20T03:03:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1fa2c71740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed34880 0x56480ed34c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1fa2c71740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed351a0 0x56480ed40bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1fa2c71740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed41130 0x56480ed43520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1fa2c71740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56480ecab930 con 0x56480ed351a0 2024-08-20T03:03:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1fa2c71740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56480ec9ae30 con 0x56480ed34880 2024-08-20T03:03:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1fa2c71740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56480ec9ee70 con 0x56480ed41130 2024-08-20T03:03:13.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1f9a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed351a0 0x56480ed40bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1f9a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed351a0 0x56480ed40bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54436/0 (socket says 172.21.15.44:54436) 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1f9a59c640 1 -- 172.21.15.44:0/3401808986 learned_addr learned my addr 172.21.15.44:0/3401808986 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.510+0000 7f1f9ad9d640 1 --2- 172.21.15.44:0/3401808986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed34880 0x56480ed34c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/3401808986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed41130 0x56480ed43520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f9a59c640 1 -- 172.21.15.44:0/3401808986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed41130 msgr2=0x56480ed43520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/3401808986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed41130 0x56480ed43520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f9a59c640 1 -- 172.21.15.44:0/3401808986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed34880 msgr2=0x56480ed34c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/3401808986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed34880 0x56480ed34c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f9a59c640 1 -- 172.21.15.44:0/3401808986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480e9fa650 con 0x56480ed351a0 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/3401808986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed351a0 0x56480ed40bf0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1f84033000 tx=0x7f1f84032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730a473e836f51d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f937fe640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f880b3dd0 0x7f1f880b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f90ff9640 1 -- 172.21.15.44:0/3401808986 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84035040 con 0x56480ed351a0 2024-08-20T03:03:13.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f90ff9640 1 -- 172.21.15.44:0/3401808986 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f84004030 con 0x56480ed351a0 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f90ff9640 1 -- 172.21.15.44:0/3401808986 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f840054f0 con 0x56480ed351a0 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f9ad9d640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f880b3dd0 0x7f1f880b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.511+0000 7f1f937fe640 1 -- 172.21.15.44:0/3230095713 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1f8c0e4440 con 0x7f1f9c104250 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.512+0000 7f1fa2c71740 1 -- 172.21.15.44:0/3401808986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed351a0 msgr2=0x56480ed40bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.512+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/3401808986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed351a0 0x56480ed40bf0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f1f84033000 tx=0x7f1f84032b20 comp rx=0 tx=0).stop 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.512+0000 7f1f9ad9d640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f880b3dd0 0x7f1f880b6290 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1f94005fd0 tx=0x7f1f94009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.512+0000 7f1fa2c71740 1 -- 172.21.15.44:0/3401808986 shutdown_connections 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.512+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/3401808986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed41130 0x56480ed43520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.512+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/3401808986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed351a0 0x56480ed40bf0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.512+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/3401808986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed34880 0x56480ed34c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.512+0000 7f1fa2c71740 1 -- 172.21.15.44:0/3401808986 >> 172.21.15.44:0/3401808986 conn(0x56480ed326b0 msgr2=0x56480ed33850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.512+0000 7f1fa2c71740 1 -- 172.21.15.44:0/3401808986 shutdown_connections 2024-08-20T03:03:13.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.512+0000 7f1fa2c71740 1 -- 172.21.15.44:0/3401808986 wait complete. 2024-08-20T03:03:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1fa2c71740 1 -- start start 2024-08-20T03:03:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1fa2c71740 1 Processor -- start 2024-08-20T03:03:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1fa2c71740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed3e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1fa2c71740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed351a0 0x56480ed3e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1fa2c71740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed41130 0x56480ed3eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1fa2c71740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56480eafd700 con 0x56480ed34880 2024-08-20T03:03:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1fa2c71740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56480ec21810 con 0x56480ed41130 2024-08-20T03:03:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1fa2c71740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56480ed43d80 con 0x56480ed351a0 2024-08-20T03:03:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1fa2c71740 10 client.0 _ll_register_callbacks cb 0x56480ec94700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1f9a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed3e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1f9a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed3e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54440/0 (socket says 172.21.15.44:54440) 2024-08-20T03:03:13.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1f9a59c640 1 -- 172.21.15.44:0/717291258 learned_addr learned my addr 172.21.15.44:0/717291258 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed351a0 0x56480ed3e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.513+0000 7f1f9ad9d640 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed41130 0x56480ed3eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f99d9b640 1 -- 172.21.15.44:0/717291258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed41130 msgr2=0x56480ed3eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed41130 0x56480ed3eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f99d9b640 1 -- 172.21.15.44:0/717291258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 msgr2=0x56480ed3e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed3e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f99d9b640 1 -- 172.21.15.44:0/717291258 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56480ed4d110 con 0x56480ed351a0 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed3e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f9ad9d640 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed41130 0x56480ed3eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed351a0 0x56480ed3e5b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0e1b40 tx=0x7f1f8c0aa950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf9f9340ac7df79a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f81ffb640 1 -- 172.21.15.44:0/717291258 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0aab10 con 0x56480ed351a0 2024-08-20T03:03:13.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1fa2c71740 1 -- 172.21.15.44:0/717291258 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56480ec4a150 con 0x56480ed351a0 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f81ffb640 1 -- 172.21.15.44:0/717291258 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1f8c0e1df0 con 0x56480ed351a0 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f81ffb640 1 -- 172.21.15.44:0/717291258 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f8c0e1ff0 con 0x56480ed351a0 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1fa2c71740 10 client.25270 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1fa2c71740 1 -- 172.21.15.44:0/717291258 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56480ed4fac0 con 0x56480ed351a0 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.514+0000 7f1f81ffb640 1 -- 172.21.15.44:0/717291258 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56480ec4a150 con 0x56480ed351a0 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.515+0000 7f1f81ffb640 1 -- 172.21.15.44:0/717291258 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1f8c018bb0 con 0x56480ed351a0 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.515+0000 7f1fa2c71740 10 client.25270 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.515+0000 7f1fa2c71740 10 client.25270 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.515+0000 7f1fa2c71740 20 client.25270 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.515+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x56480ed50f10 0x56480ed53340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.515+0000 7f1fa2c71740 4 client.25270 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.515+0000 7f1fa2c71740 1 -- 172.21.15.44:0/717291258 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56480ecad880 con 0x56480ed50f10 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.515+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x56480ed50f10 0x56480ed53340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.515+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x56480ed50f10 0x56480ed53340 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c16b6d74e2139810 server_cookie=529c10d63dbbfcd7 in_seq=0 out_seq=0 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.515+0000 7f1f81ffb640 10 client.25270 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1f81ffb640 1 -- 172.21.15.44:0/717291258 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56480ecad880 con 0x56480ed50f10 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1f81ffb640 10 client.25270 handle_command_reply: tid=0 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1fa2c71740 1 client.25270 shutdown 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1fa2c71740 1 -- 172.21.15.44:0/717291258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed351a0 msgr2=0x56480ed3e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed351a0 0x56480ed3e5b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0e1b40 tx=0x7f1f8c0aa950 comp rx=0 tx=0).stop 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1fa2c71740 1 -- 172.21.15.44:0/717291258 shutdown_connections 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x56480ed50f10 0x56480ed53340 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed41130 0x56480ed3eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed351a0 0x56480ed3e5b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1f81ffb640 0 client.25270 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/717291258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed3e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1fa2c71740 1 -- 172.21.15.44:0/717291258 >> 172.21.15.44:0/717291258 conn(0x56480ed326b0 msgr2=0x56480ed38250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:13.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1fa2c71740 1 -- 172.21.15.44:0/717291258 shutdown_connections 2024-08-20T03:03:13.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.516+0000 7f1fa2c71740 1 -- 172.21.15.44:0/717291258 wait complete. 2024-08-20T03:03:13.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.517+0000 7f1fa2c71740 20 client.25270 trim_cache size 0 max 16384 2024-08-20T03:03:13.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1fa2c71740 1 Processor -- start 2024-08-20T03:03:13.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1fa2c71740 1 -- start start 2024-08-20T03:03:13.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1fa2c71740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed40170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1fa2c71740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed351a0 0x56480ed38960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1fa2c71740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed41130 0x56480ed3d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1fa2c71740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1b04a0 con 0x56480ed34880 2024-08-20T03:03:13.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1fa2c71740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1b1fe0 con 0x56480ed351a0 2024-08-20T03:03:13.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1fa2c71740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1b0b70 con 0x56480ed41130 2024-08-20T03:03:13.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1f9a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed40170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1f9a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed40170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54450/0 (socket says 172.21.15.44:54450) 2024-08-20T03:03:13.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1f9a59c640 1 -- 172.21.15.44:0/3334435418 learned_addr learned my addr 172.21.15.44:0/3334435418 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:13.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/3334435418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed351a0 0x56480ed38960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1f9a59c640 1 -- 172.21.15.44:0/3334435418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed41130 msgr2=0x56480ed3d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:13.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/3334435418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed41130 0x56480ed3d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1f9a59c640 1 -- 172.21.15.44:0/3334435418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed351a0 msgr2=0x56480ed38960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:13.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/3334435418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed351a0 0x56480ed38960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.530+0000 7f1f9a59c640 1 -- 172.21.15.44:0/3334435418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480ed2d830 con 0x56480ed34880 2024-08-20T03:03:13.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.531+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/3334435418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed40170 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1f840096c0 tx=0x7f1f840050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c59d97c2c45ff69 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:13.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.531+0000 7f1f83fff640 1 -- 172.21.15.44:0/3334435418 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84035040 con 0x56480ed34880 2024-08-20T03:03:13.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.531+0000 7f1f83fff640 1 -- 172.21.15.44:0/3334435418 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f84031950 con 0x56480ed34880 2024-08-20T03:03:13.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.531+0000 7f1f83fff640 1 -- 172.21.15.44:0/3334435418 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f840049b0 con 0x56480ed34880 2024-08-20T03:03:13.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.531+0000 7f1fa2c71740 1 -- 172.21.15.44:0/3334435418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 msgr2=0x56480ed40170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:13.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.531+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/3334435418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed40170 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1f840096c0 tx=0x7f1f840050f0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 -- 172.21.15.44:0/3334435418 shutdown_connections 2024-08-20T03:03:13.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/3334435418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed41130 0x56480ed3d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/3334435418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed351a0 0x56480ed38960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/3334435418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed40170 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 -- 172.21.15.44:0/3334435418 >> 172.21.15.44:0/3334435418 conn(0x56480ed326b0 msgr2=0x56480ed31770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:13.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 -- 172.21.15.44:0/3334435418 shutdown_connections 2024-08-20T03:03:13.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 -- 172.21.15.44:0/3334435418 wait complete. 2024-08-20T03:03:13.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 -- start start 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 Processor -- start 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed44000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed351a0 0x56480ed44540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed41130 0x56480ed44a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1b1910 con 0x56480ed34880 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1b1240 con 0x56480ed41130 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c1ae4a0 con 0x56480ed351a0 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1fa2c71740 10 client.0 _ll_register_callbacks cb 0x7f1f8c0e19d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1f9a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed44000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1f9a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed44000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54454/0 (socket says 172.21.15.44:54454) 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.532+0000 7f1f9a59c640 1 -- 172.21.15.44:0/47975785 learned_addr learned my addr 172.21.15.44:0/47975785 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f9ad9d640 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed41130 0x56480ed44a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed351a0 0x56480ed44540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f9a59c640 1 -- 172.21.15.44:0/47975785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed351a0 msgr2=0x56480ed44540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed351a0 0x56480ed44540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f9a59c640 1 -- 172.21.15.44:0/47975785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed41130 msgr2=0x56480ed44a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed41130 0x56480ed44a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f9a59c640 1 -- 172.21.15.44:0/47975785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56480edb9d00 con 0x56480ed34880 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed351a0 0x56480ed44540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f9a59c640 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed44000 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1f8402abe0 tx=0x7f1f84031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a4f304800b3c48 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f81ffb640 1 -- 172.21.15.44:0/47975785 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84035070 con 0x56480ed34880 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f81ffb640 1 -- 172.21.15.44:0/47975785 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1f84004040 con 0x56480ed34880 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1fa2c71740 1 -- 172.21.15.44:0/47975785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56480eca9320 con 0x56480ed34880 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1f81ffb640 1 -- 172.21.15.44:0/47975785 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f84004280 con 0x56480ed34880 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1fa2c71740 10 client.15771 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.533+0000 7f1fa2c71740 1 -- 172.21.15.44:0/47975785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56480edbb000 con 0x56480ed34880 2024-08-20T03:03:13.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.534+0000 7f1f81ffb640 1 -- 172.21.15.44:0/47975785 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56480eca9320 con 0x56480ed34880 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.534+0000 7f1f81ffb640 1 -- 172.21.15.44:0/47975785 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1f84003680 con 0x56480ed34880 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.534+0000 7f1fa2c71740 10 client.15771 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.534+0000 7f1fa2c71740 10 client.15771 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.534+0000 7f1fa2c71740 20 client.15771 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.534+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x56480ed50f10 0x56480edbd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.534+0000 7f1fa2c71740 4 client.15771 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.534+0000 7f1fa2c71740 1 -- 172.21.15.44:0/47975785 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56480eca6a60 con 0x56480ed50f10 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.534+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x56480ed50f10 0x56480edbd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1f81ffb640 10 client.15771 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1f99d9b640 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x56480ed50f10 0x56480edbd340 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=532b43dc2e3c8d80 server_cookie=1acad4d4898aa00f in_seq=0 out_seq=0 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1f81ffb640 1 -- 172.21.15.44:0/47975785 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56480eca6a60 con 0x56480ed50f10 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1f81ffb640 10 client.15771 handle_command_reply: tid=0 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1fa2c71740 1 client.15771 shutdown 2024-08-20T03:03:13.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1fa2c71740 1 -- 172.21.15.44:0/47975785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 msgr2=0x56480ed44000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed44000 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1f8402abe0 tx=0x7f1f84031400 comp rx=0 tx=0).stop 2024-08-20T03:03:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1fa2c71740 1 -- 172.21.15.44:0/47975785 shutdown_connections 2024-08-20T03:03:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x56480ed50f10 0x56480edbd340 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480ed41130 0x56480ed44a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480ed351a0 0x56480ed44540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1fa2c71740 1 --2- 172.21.15.44:0/47975785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480ed34880 0x56480ed44000 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1fa2c71740 1 -- 172.21.15.44:0/47975785 >> 172.21.15.44:0/47975785 conn(0x56480ed326b0 msgr2=0x56480ed4bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1f81ffb640 0 client.15771 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:03:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1fa2c71740 1 -- 172.21.15.44:0/47975785 shutdown_connections 2024-08-20T03:03:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1fa2c71740 1 -- 172.21.15.44:0/47975785 wait complete. 2024-08-20T03:03:13.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.535+0000 7f1fa2c71740 20 client.15771 trim_cache size 0 max 16384 2024-08-20T03:03:13.543 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.536+0000 7f1fa0878640 1 -- 172.21.15.44:0/3230095713 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f880b3dd0 msgr2=0x7f1f880b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.536+0000 7f1fa0878640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f880b3dd0 0x7f1f880b6290 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1f94005fd0 tx=0x7f1f94009500 comp rx=0 tx=0).stop 2024-08-20T03:03:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.536+0000 7f1fa0878640 1 -- 172.21.15.44:0/3230095713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f9c104250 msgr2=0x7f1f9c199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.536+0000 7f1fa0878640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f9c104250 0x7f1f9c199490 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00e9f0 tx=0x7f1f8c00eeb0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.536+0000 7f1fa0878640 1 -- 172.21.15.44:0/3230095713 shutdown_connections 2024-08-20T03:03:13.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.536+0000 7f1fa0878640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f880b3dd0 0x7f1f880b6290 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.536+0000 7f1fa0878640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f9c10ce00 0x7f1f9c19e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.536+0000 7f1fa0878640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f9c104250 0x7f1f9c199490 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.536+0000 7f1fa0878640 1 --2- 172.21.15.44:0/3230095713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f9c103850 0x7f1f9c198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.536+0000 7f1fa0878640 1 -- 172.21.15.44:0/3230095713 >> 172.21.15.44:0/3230095713 conn(0x7f1f9c0fd400 msgr2=0x7f1f9c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.536+0000 7f1fa0878640 1 -- 172.21.15.44:0/3230095713 shutdown_connections 2024-08-20T03:03:13.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:13.536+0000 7f1fa0878640 1 -- 172.21.15.44:0/3230095713 wait complete. 2024-08-20T03:03:14.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:13 smithi165 ceph-mon[22357]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-08-20T03:03:14.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:13 smithi099 ceph-mon[21984]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-08-20T03:03:14.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:13 smithi044 ceph-mon[18790]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-08-20T03:03:14.349 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-20T03:03:14.606 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:15.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.105+0000 7f01b743c640 1 -- 172.21.15.44:0/1427585339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b01073a0 msgr2=0x7f01b01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:15.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.105+0000 7f01b743c640 1 --2- 172.21.15.44:0/1427585339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b01073a0 0x7f01b01077e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009900 tx=0x7f01a002a410 comp rx=0 tx=0).stop 2024-08-20T03:03:15.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.105+0000 7f01b743c640 1 -- 172.21.15.44:0/1427585339 shutdown_connections 2024-08-20T03:03:15.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.105+0000 7f01b743c640 1 --2- 172.21.15.44:0/1427585339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01b0107d20 0x7f01b010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.105+0000 7f01b743c640 1 --2- 172.21.15.44:0/1427585339 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b01073a0 0x7f01b01077e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.105+0000 7f01b743c640 1 --2- 172.21.15.44:0/1427585339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b0101990 0x7f01b0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.105+0000 7f01b743c640 1 -- 172.21.15.44:0/1427585339 >> 172.21.15.44:0/1427585339 conn(0x7f01b00fd400 msgr2=0x7f01b00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.105+0000 7f01b743c640 1 -- 172.21.15.44:0/1427585339 shutdown_connections 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.105+0000 7f01b743c640 1 -- 172.21.15.44:0/1427585339 wait complete. 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b743c640 1 Processor -- start 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b743c640 1 -- start start 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b743c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b0101990 0x7f01b0199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b743c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01b01073a0 0x7f01b0199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b743c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b0107d20 0x7f01b019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b743c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01b01119e0 con 0x7f01b0101990 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b743c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01b0111860 con 0x7f01b0107d20 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b743c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01b0111b60 con 0x7f01b01073a0 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b51d8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b0101990 0x7f01b0199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b51d8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b0101990 0x7f01b0199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54468/0 (socket says 172.21.15.44:54468) 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b51d8640 1 -- 172.21.15.44:0/3654847305 learned_addr learned my addr 172.21.15.44:0/3654847305 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:15.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b59d9640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b0107d20 0x7f01b019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b49d7640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01b01073a0 0x7f01b0199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b51d8640 1 -- 172.21.15.44:0/3654847305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01b01073a0 msgr2=0x7f01b0199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b51d8640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01b01073a0 0x7f01b0199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b51d8640 1 -- 172.21.15.44:0/3654847305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b0107d20 msgr2=0x7f01b019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b51d8640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b0107d20 0x7f01b019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b51d8640 1 -- 172.21.15.44:0/3654847305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b019f0c0 con 0x7f01b0101990 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b59d9640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b0107d20 0x7f01b019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.106+0000 7f01b49d7640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01b01073a0 0x7f01b0199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.107+0000 7f01b51d8640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b0101990 0x7f01b0199010 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00d950 tx=0x7f01ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.107+0000 7f01a67fc640 1 -- 172.21.15.44:0/3654847305 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac017070 con 0x7f01b0101990 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.107+0000 7f01a67fc640 1 -- 172.21.15.44:0/3654847305 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ac009c30 con 0x7f01b0101990 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.107+0000 7f01b743c640 1 -- 172.21.15.44:0/3654847305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b019f3b0 con 0x7f01b0101990 2024-08-20T03:03:15.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.107+0000 7f01a67fc640 1 -- 172.21.15.44:0/3654847305 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00f620 con 0x7f01b0101990 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.107+0000 7f01b743c640 1 -- 172.21.15.44:0/3654847305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b01a5af0 con 0x7f01b0101990 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.110+0000 7f01a67fc640 1 -- 172.21.15.44:0/3654847305 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f01ac010460 con 0x7f01b0101990 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.110+0000 7f01b9835740 1 Processor -- start 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.110+0000 7f01b9835740 1 -- start start 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.110+0000 7f01b9835740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78787880 0x560f78787c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.110+0000 7f01b9835740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 0x560f78793bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.110+0000 7f01b9835740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78794130 0x560f78796520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.110+0000 7f01b9835740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560f786fe930 con 0x560f787881a0 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.110+0000 7f01b9835740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f786ede30 con 0x560f78794130 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.110+0000 7f01b9835740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560f786f1e70 con 0x560f78787880 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.110+0000 7f01b51d8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 0x560f78793bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.110+0000 7f01b51d8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 0x560f78793bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54478/0 (socket says 172.21.15.44:54478) 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.110+0000 7f01b51d8640 1 -- 172.21.15.44:0/4162971004 learned_addr learned my addr 172.21.15.44:0/4162971004 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f01b49d7640 1 --2- 172.21.15.44:0/4162971004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78794130 0x560f78796520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f01b59d9640 1 --2- 172.21.15.44:0/4162971004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78787880 0x560f78787c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f01a67fc640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f019c0b3dd0 0x7f019c0b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f01b49d7640 1 -- 172.21.15.44:0/4162971004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78787880 msgr2=0x560f78787c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f01b49d7640 1 --2- 172.21.15.44:0/4162971004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78787880 0x560f78787c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f01b49d7640 1 -- 172.21.15.44:0/4162971004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 msgr2=0x560f78793bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f01b49d7640 1 --2- 172.21.15.44:0/4162971004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 0x560f78793bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f01b49d7640 1 -- 172.21.15.44:0/4162971004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f7844d650 con 0x560f78794130 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f01b59d9640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f019c0b3dd0 0x7f019c0b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f01a67fc640 1 -- 172.21.15.44:0/3654847305 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f01ac0e3e20 con 0x7f01b0101990 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f01b49d7640 1 --2- 172.21.15.44:0/4162971004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78794130 0x560f78796520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f01a0000c00 tx=0x7f01a0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee88e8cd02db9e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f01b59d9640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f019c0b3dd0 0x7f019c0b6290 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f01a800f4f0 tx=0x7f01a8004b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f019b7fe640 1 -- 172.21.15.44:0/4162971004 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a003f070 con 0x560f78794130 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.111+0000 7f019b7fe640 1 -- 172.21.15.44:0/4162971004 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a00316a0 con 0x560f78794130 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.112+0000 7f019b7fe640 1 -- 172.21.15.44:0/4162971004 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0035070 con 0x560f78794130 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.112+0000 7f01b9835740 1 -- 172.21.15.44:0/4162971004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78794130 msgr2=0x560f78796520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:15.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.112+0000 7f01b9835740 1 --2- 172.21.15.44:0/4162971004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78794130 0x560f78796520 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f01a0000c00 tx=0x7f01a0032ce0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.112+0000 7f01b9835740 1 -- 172.21.15.44:0/4162971004 shutdown_connections 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.112+0000 7f01b9835740 1 --2- 172.21.15.44:0/4162971004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78794130 0x560f78796520 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.112+0000 7f01b9835740 1 --2- 172.21.15.44:0/4162971004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 0x560f78793bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.112+0000 7f01b9835740 1 --2- 172.21.15.44:0/4162971004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78787880 0x560f78787c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.112+0000 7f01b9835740 1 -- 172.21.15.44:0/4162971004 >> 172.21.15.44:0/4162971004 conn(0x560f787856b0 msgr2=0x560f78786850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.112+0000 7f01b9835740 1 -- 172.21.15.44:0/4162971004 shutdown_connections 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.112+0000 7f01b9835740 1 -- 172.21.15.44:0/4162971004 wait complete. 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b9835740 1 -- start start 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b9835740 1 Processor -- start 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b9835740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78787880 0x560f78791090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b9835740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 0x560f787915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b9835740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78794130 0x560f78791b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b9835740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560f78550700 con 0x560f787881a0 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b9835740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560f78674810 con 0x560f78787880 2024-08-20T03:03:15.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b9835740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560f78796d80 con 0x560f78794130 2024-08-20T03:03:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b9835740 10 client.0 _ll_register_callbacks cb 0x560f786e7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b49d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 0x560f787915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b49d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 0x560f787915d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54494/0 (socket says 172.21.15.44:54494) 2024-08-20T03:03:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b49d7640 1 -- 172.21.15.44:0/3171325913 learned_addr learned my addr 172.21.15.44:0/3171325913 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b59d9640 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78794130 0x560f78791b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.113+0000 7f01b51d8640 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78787880 0x560f78791090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f01b49d7640 1 -- 172.21.15.44:0/3171325913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78794130 msgr2=0x560f78791b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f01b49d7640 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78794130 0x560f78791b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f01b49d7640 1 -- 172.21.15.44:0/3171325913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78787880 msgr2=0x560f78791090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f01b49d7640 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78787880 0x560f78791090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f01b49d7640 1 -- 172.21.15.44:0/3171325913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f787a0100 con 0x560f787881a0 2024-08-20T03:03:15.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f01b49d7640 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 0x560f787915d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005600 tx=0x7f01a0005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c3f43c3ca04763 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f0198ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f0198ff9640 1 -- 172.21.15.44:0/3171325913 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a003f070 con 0x560f787881a0 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f0198ff9640 1 -- 172.21.15.44:0/3171325913 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f01a0004050 con 0x560f787881a0 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f01b9835740 1 -- 172.21.15.44:0/3171325913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f7869d150 con 0x560f787881a0 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f0198ff9640 1 -- 172.21.15.44:0/3171325913 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f01a0055050 con 0x560f787881a0 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f01b9835740 10 client.15783 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f01b9835740 1 -- 172.21.15.44:0/3171325913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f787a2ab0 con 0x560f787881a0 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.114+0000 7f0198ff9640 1 -- 172.21.15.44:0/3171325913 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560f7869d150 con 0x560f787881a0 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.115+0000 7f0198ff9640 1 -- 172.21.15.44:0/3171325913 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f01a003b460 con 0x560f787881a0 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.115+0000 7f01b9835740 10 client.15783 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.115+0000 7f01b9835740 10 client.15783 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.115+0000 7f01b9835740 20 client.15783 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.115+0000 7f01b9835740 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x560f787a3e00 0x560f787a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.115+0000 7f01b9835740 4 client.15783 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.115+0000 7f01b9835740 1 -- 172.21.15.44:0/3171325913 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f78700880 con 0x560f787a3e00 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.115+0000 7f01b51d8640 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x560f787a3e00 0x560f787a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.115+0000 7f01b51d8640 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x560f787a3e00 0x560f787a61f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=777772b9283dd8e9 server_cookie=485e0e808c9d4379 in_seq=0 out_seq=0 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.115+0000 7f0198ff9640 10 client.15783 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.116+0000 7f0198ff9640 1 -- 172.21.15.44:0/3171325913 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f78700880 con 0x560f787a3e00 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.116+0000 7f0198ff9640 10 client.15783 handle_command_reply: tid=0 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.116+0000 7f01b9835740 1 client.15783 shutdown 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.116+0000 7f01b9835740 1 -- 172.21.15.44:0/3171325913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 msgr2=0x560f787915d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.116+0000 7f01b9835740 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 0x560f787915d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005600 tx=0x7f01a0005630 comp rx=0 tx=0).stop 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.116+0000 7f01b9835740 1 -- 172.21.15.44:0/3171325913 shutdown_connections 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.116+0000 7f01b9835740 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x560f787a3e00 0x560f787a61f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.116+0000 7f01b9835740 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78794130 0x560f78791b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.116+0000 7f0198ff9640 0 client.15783 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:03:15.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.116+0000 7f01b9835740 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f787881a0 0x560f787915d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.116+0000 7f01b9835740 1 --2- 172.21.15.44:0/3171325913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78787880 0x560f78791090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.116+0000 7f01b9835740 1 -- 172.21.15.44:0/3171325913 >> 172.21.15.44:0/3171325913 conn(0x560f787856b0 msgr2=0x560f7878b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.117+0000 7f01b9835740 1 -- 172.21.15.44:0/3171325913 shutdown_connections 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.117+0000 7f01b9835740 1 -- 172.21.15.44:0/3171325913 wait complete. 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.117+0000 7f01b9835740 20 client.15783 trim_cache size 0 max 16384 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b9835740 1 Processor -- start 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b9835740 1 -- start start 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b9835740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78787880 0x560f78786e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b9835740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f787881a0 0x560f787ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b9835740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78794130 0x560f787abd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b9835740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01b01ae3c0 con 0x560f78794130 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b9835740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01b01aff00 con 0x560f78787880 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b9835740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01b01aea90 con 0x560f787881a0 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b59d9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78794130 0x560f787abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b59d9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78794130 0x560f787abd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54500/0 (socket says 172.21.15.44:54500) 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b59d9640 1 -- 172.21.15.44:0/2599655773 learned_addr learned my addr 172.21.15.44:0/2599655773 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b49d7640 1 --2- 172.21.15.44:0/2599655773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f787881a0 0x560f787ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.130+0000 7f01b51d8640 1 --2- 172.21.15.44:0/2599655773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78787880 0x560f78786e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.131+0000 7f01b59d9640 1 -- 172.21.15.44:0/2599655773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f787881a0 msgr2=0x560f787ab7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.131+0000 7f01b59d9640 1 --2- 172.21.15.44:0/2599655773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f787881a0 0x560f787ab7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.131+0000 7f01b59d9640 1 -- 172.21.15.44:0/2599655773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78787880 msgr2=0x560f78786e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.131+0000 7f01b59d9640 1 --2- 172.21.15.44:0/2599655773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78787880 0x560f78786e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.131+0000 7f01b59d9640 1 -- 172.21.15.44:0/2599655773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f78786130 con 0x560f78794130 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.131+0000 7f01b51d8640 1 --2- 172.21.15.44:0/2599655773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78787880 0x560f78786e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.131+0000 7f01b59d9640 1 --2- 172.21.15.44:0/2599655773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78794130 0x560f787abd00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f01a8011890 tx=0x7f01a8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bddde0c96da07c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.131+0000 7f019affd640 1 -- 172.21.15.44:0/2599655773 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8010db0 con 0x560f78794130 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.131+0000 7f019affd640 1 -- 172.21.15.44:0/2599655773 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a8022030 con 0x560f78794130 2024-08-20T03:03:15.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.131+0000 7f019affd640 1 -- 172.21.15.44:0/2599655773 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a801d6a0 con 0x560f78794130 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.132+0000 7f01b9835740 1 -- 172.21.15.44:0/2599655773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78794130 msgr2=0x560f787abd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.132+0000 7f01b9835740 1 --2- 172.21.15.44:0/2599655773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78794130 0x560f787abd00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f01a8011890 tx=0x7f01a8012000 comp rx=0 tx=0).stop 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.132+0000 7f01b9835740 1 -- 172.21.15.44:0/2599655773 shutdown_connections 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.132+0000 7f01b9835740 1 --2- 172.21.15.44:0/2599655773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78794130 0x560f787abd00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.132+0000 7f01b9835740 1 --2- 172.21.15.44:0/2599655773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f787881a0 0x560f787ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.132+0000 7f01b9835740 1 --2- 172.21.15.44:0/2599655773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f78787880 0x560f78786e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.132+0000 7f01b9835740 1 -- 172.21.15.44:0/2599655773 >> 172.21.15.44:0/2599655773 conn(0x560f787856b0 msgr2=0x560f78786700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.132+0000 7f01b9835740 1 -- 172.21.15.44:0/2599655773 shutdown_connections 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.132+0000 7f01b9835740 1 -- 172.21.15.44:0/2599655773 wait complete. 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.132+0000 7f01b9835740 1 -- start start 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.132+0000 7f01b9835740 1 Processor -- start 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b9835740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78787880 0x560f787977c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b9835740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f787881a0 0x560f78797d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b9835740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78794130 0x560f78810b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b9835740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f01b01af830 con 0x560f78787880 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b9835740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f01b01af160 con 0x560f787881a0 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b9835740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f01b01ac3a0 con 0x560f78794130 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b9835740 10 client.0 _ll_register_callbacks cb 0x7f01ac0e13b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b51d8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78787880 0x560f787977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b51d8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78787880 0x560f787977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54514/0 (socket says 172.21.15.44:54514) 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b51d8640 1 -- 172.21.15.44:0/1855765189 learned_addr learned my addr 172.21.15.44:0/1855765189 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b59d9640 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78794130 0x560f78810b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b49d7640 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f787881a0 0x560f78797d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b51d8640 1 -- 172.21.15.44:0/1855765189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78794130 msgr2=0x560f78810b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b51d8640 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78794130 0x560f78810b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b51d8640 1 -- 172.21.15.44:0/1855765189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f787881a0 msgr2=0x560f78797d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b51d8640 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f787881a0 0x560f78797d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b51d8640 1 -- 172.21.15.44:0/1855765189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f78811200 con 0x560f78787880 2024-08-20T03:03:15.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b59d9640 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78794130 0x560f78810b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.133+0000 7f01b51d8640 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78787880 0x560f787977c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0aa8c0 tx=0x7f01ac00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d294433ebecfed server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f0198ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f0198ff9640 1 -- 172.21.15.44:0/1855765189 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac020040 con 0x560f78787880 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f0198ff9640 1 -- 172.21.15.44:0/1855765189 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f01ac0a9b90 con 0x560f78787880 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f01b9835740 1 -- 172.21.15.44:0/1855765189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01a8022030 con 0x560f78787880 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f0198ff9640 1 -- 172.21.15.44:0/1855765189 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f01ac0aa700 con 0x560f78787880 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f01b9835740 10 client.15789 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f01b9835740 1 -- 172.21.15.44:0/1855765189 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f78812540 con 0x560f78787880 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f0198ff9640 1 -- 172.21.15.44:0/1855765189 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a8022030 con 0x560f78787880 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f0198ff9640 1 -- 172.21.15.44:0/1855765189 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f01ac0a62a0 con 0x560f78787880 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f01b9835740 10 client.15789 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f01b9835740 10 client.15789 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f01b9835740 20 client.15789 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:15.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f01b9835740 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x560f787a3e00 0x560f78814d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f01b9835740 4 client.15789 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.134+0000 7f01b9835740 1 -- 172.21.15.44:0/1855765189 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f78780880 con 0x560f787a3e00 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.135+0000 7f01b49d7640 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x560f787a3e00 0x560f78814d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.135+0000 7f0198ff9640 10 client.15789 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.135+0000 7f01b49d7640 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x560f787a3e00 0x560f78814d00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4dd9d220f77ef6fe server_cookie=f71ddb23b5c7cf2 in_seq=0 out_seq=0 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.135+0000 7f0198ff9640 1 -- 172.21.15.44:0/1855765189 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f78780880 con 0x560f787a3e00 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.135+0000 7f0198ff9640 10 client.15789 handle_command_reply: tid=0 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.135+0000 7f01b9835740 1 client.15789 shutdown 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.135+0000 7f01b9835740 1 -- 172.21.15.44:0/1855765189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78787880 msgr2=0x560f787977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.135+0000 7f01b9835740 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78787880 0x560f787977c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0aa8c0 tx=0x7f01ac00bf60 comp rx=0 tx=0).stop 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.136+0000 7f01b9835740 1 -- 172.21.15.44:0/1855765189 shutdown_connections 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.136+0000 7f01b9835740 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x560f787a3e00 0x560f78814d00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.154 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.136+0000 7f01b9835740 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560f78794130 0x560f78810b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.136+0000 7f0198ff9640 0 client.15789 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:03:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.136+0000 7f01b9835740 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560f787881a0 0x560f78797d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.136+0000 7f01b9835740 1 --2- 172.21.15.44:0/1855765189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f78787880 0x560f787977c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.136+0000 7f01b9835740 1 -- 172.21.15.44:0/1855765189 >> 172.21.15.44:0/1855765189 conn(0x560f787856b0 msgr2=0x560f787926c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.136+0000 7f01b9835740 1 -- 172.21.15.44:0/1855765189 shutdown_connections 2024-08-20T03:03:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.136+0000 7f01b9835740 1 -- 172.21.15.44:0/1855765189 wait complete. 2024-08-20T03:03:15.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.136+0000 7f01b9835740 20 client.15789 trim_cache size 0 max 16384 2024-08-20T03:03:15.155 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.136+0000 7f01b743c640 1 -- 172.21.15.44:0/3654847305 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f019c0b3dd0 msgr2=0x7f019c0b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.136+0000 7f01b743c640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f019c0b3dd0 0x7f019c0b6290 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f01a800f4f0 tx=0x7f01a8004b50 comp rx=0 tx=0).stop 2024-08-20T03:03:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.137+0000 7f01b743c640 1 -- 172.21.15.44:0/3654847305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b0101990 msgr2=0x7f01b0199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.137+0000 7f01b743c640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b0101990 0x7f01b0199010 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00d950 tx=0x7f01ac00de10 comp rx=0 tx=0).stop 2024-08-20T03:03:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.137+0000 7f01b743c640 1 -- 172.21.15.44:0/3654847305 shutdown_connections 2024-08-20T03:03:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.137+0000 7f01b743c640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f019c0b3dd0 0x7f019c0b6290 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.137+0000 7f01b743c640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01b0107d20 0x7f01b019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.137+0000 7f01b743c640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01b01073a0 0x7f01b0199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.137+0000 7f01b743c640 1 --2- 172.21.15.44:0/3654847305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f01b0101990 0x7f01b0199010 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.137+0000 7f01b743c640 1 -- 172.21.15.44:0/3654847305 >> 172.21.15.44:0/3654847305 conn(0x7f01b00fd400 msgr2=0x7f01b0104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.137+0000 7f01b743c640 1 -- 172.21.15.44:0/3654847305 shutdown_connections 2024-08-20T03:03:15.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:15.137+0000 7f01b743c640 1 -- 172.21.15.44:0/3654847305 wait complete. 2024-08-20T03:03:15.259 INFO:tasks.workunit.client.0.smithi044.stderr:Updating files: 90% (10981/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-08-20T03:03:15.978 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T03:03:16.026 INFO:tasks.workunit.client.0.smithi044.stderr:Note: switching to 'ef4275410d8483698016fe484b2a3c664080b4c1'. 2024-08-20T03:03:16.026 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-08-20T03:03:16.026 INFO:tasks.workunit.client.0.smithi044.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-20T03:03:16.026 INFO:tasks.workunit.client.0.smithi044.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-20T03:03:16.026 INFO:tasks.workunit.client.0.smithi044.stderr:state without impacting any branches by switching back to a branch. 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr: git switch -c 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr:Or undo this operation with: 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr: git switch - 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-08-20T03:03:16.027 INFO:tasks.workunit.client.0.smithi044.stderr:HEAD is now at ef4275410d8 Merge branch 'mds-conf-sorted' of https://github.com/batrick/ceph into wip-hemanth-testing-2024-08-19-1357 2024-08-20T03:03:16.032 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-08-20T03:03:16.058 INFO:tasks.workunit.client.0.smithi044.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-20T03:03:16.060 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-20T03:03:16.060 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-20T03:03:16.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:15 smithi165 ceph-mon[22357]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-08-20T03:03:16.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:15 smithi099 ceph-mon[21984]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-08-20T03:03:16.232 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:16.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:15 smithi044 ceph-mon[18790]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 12 KiB/s wr, 15 op/s 2024-08-20T03:03:16.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.801+0000 7f991fc41640 1 -- 172.21.15.44:0/2824459617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9918101570 msgr2=0x7f9918101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:16.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.801+0000 7f991fc41640 1 --2- 172.21.15.44:0/2824459617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9918101570 0x7f9918101950 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9914009ca0 tx=0x7f991402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.801+0000 7f991fc41640 1 -- 172.21.15.44:0/2824459617 shutdown_connections 2024-08-20T03:03:16.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.801+0000 7f991fc41640 1 --2- 172.21.15.44:0/2824459617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f991810f120 0x7f9918111510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.801+0000 7f991fc41640 1 --2- 172.21.15.44:0/2824459617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9918101e90 0x7f991810ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.801+0000 7f991fc41640 1 --2- 172.21.15.44:0/2824459617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9918101570 0x7f9918101950 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.801+0000 7f991fc41640 1 -- 172.21.15.44:0/2824459617 >> 172.21.15.44:0/2824459617 conn(0x7f99180fd440 msgr2=0x7f99180ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:16.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.802+0000 7f991fc41640 1 -- 172.21.15.44:0/2824459617 shutdown_connections 2024-08-20T03:03:16.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.802+0000 7f991fc41640 1 -- 172.21.15.44:0/2824459617 wait complete. 2024-08-20T03:03:16.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.802+0000 7f991fc41640 1 Processor -- start 2024-08-20T03:03:16.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.802+0000 7f991fc41640 1 -- start start 2024-08-20T03:03:16.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.802+0000 7f991fc41640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9918101570 0x7f991819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.802+0000 7f991fc41640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9918101e90 0x7f991819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.802+0000 7f991fc41640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991810f120 0x7f99181a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.802+0000 7f991fc41640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9918114ce0 con 0x7f9918101e90 2024-08-20T03:03:16.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.802+0000 7f991fc41640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9918114b60 con 0x7f9918101570 2024-08-20T03:03:16.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.802+0000 7f991fc41640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9918114e60 con 0x7f991810f120 2024-08-20T03:03:16.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.802+0000 7f991d1dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9918101e90 0x7f991819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991d1dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9918101e90 0x7f991819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54544/0 (socket says 172.21.15.44:54544) 2024-08-20T03:03:16.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991d1dc640 1 -- 172.21.15.44:0/430043488 learned_addr learned my addr 172.21.15.44:0/430043488 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:16.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991d9dd640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9918101570 0x7f991819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991e1de640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991810f120 0x7f99181a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991d1dc640 1 -- 172.21.15.44:0/430043488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991810f120 msgr2=0x7f99181a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991d1dc640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991810f120 0x7f99181a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991d1dc640 1 -- 172.21.15.44:0/430043488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9918101570 msgr2=0x7f991819b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991d1dc640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9918101570 0x7f991819b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991d1dc640 1 -- 172.21.15.44:0/430043488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99181a12e0 con 0x7f9918101e90 2024-08-20T03:03:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991e1de640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991810f120 0x7f99181a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:16.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991d1dc640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9918101e90 0x7f991819b770 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f990800b4a0 tx=0x7f990800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:16.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f990effd640 1 -- 172.21.15.44:0/430043488 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908014070 con 0x7f9918101e90 2024-08-20T03:03:16.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f990effd640 1 -- 172.21.15.44:0/430043488 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9908010ac0 con 0x7f9918101e90 2024-08-20T03:03:16.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991fc41640 1 -- 172.21.15.44:0/430043488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99181a15d0 con 0x7f9918101e90 2024-08-20T03:03:16.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f990effd640 1 -- 172.21.15.44:0/430043488 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908017c30 con 0x7f9918101e90 2024-08-20T03:03:16.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.803+0000 7f991fc41640 1 -- 172.21.15.44:0/430043488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99181a7d10 con 0x7f9918101e90 2024-08-20T03:03:16.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.806+0000 7f992203a740 1 Processor -- start 2024-08-20T03:03:16.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.806+0000 7f990effd640 1 -- 172.21.15.44:0/430043488 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f9908017070 con 0x7f9918101e90 2024-08-20T03:03:16.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.806+0000 7f992203a740 1 -- start start 2024-08-20T03:03:16.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.806+0000 7f992203a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8b730 0x559b1ef8bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.806+0000 7f992203a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef8c050 0x559b1ef97aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.806+0000 7f992203a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef97fe0 0x559b1ef9a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.806+0000 7f992203a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559b1ef02930 con 0x559b1ef97fe0 2024-08-20T03:03:16.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.806+0000 7f992203a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559b1eef1e30 con 0x559b1ef8c050 2024-08-20T03:03:16.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.806+0000 7f992203a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559b1eef5e70 con 0x559b1ef8b730 2024-08-20T03:03:16.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.806+0000 7f991d1dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef97fe0 0x559b1ef9a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.806+0000 7f991d1dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef97fe0 0x559b1ef9a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54558/0 (socket says 172.21.15.44:54558) 2024-08-20T03:03:16.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.806+0000 7f991d1dc640 1 -- 172.21.15.44:0/1546015594 learned_addr learned my addr 172.21.15.44:0/1546015594 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:16.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f991d9dd640 1 --2- 172.21.15.44:0/1546015594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef8c050 0x559b1ef97aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f991e1de640 1 --2- 172.21.15.44:0/1546015594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8b730 0x559b1ef8bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f991d1dc640 1 -- 172.21.15.44:0/1546015594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8b730 msgr2=0x559b1ef8bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:16.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f990effd640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f99040b3dd0 0x7f99040b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f991d1dc640 1 --2- 172.21.15.44:0/1546015594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8b730 0x559b1ef8bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f991d1dc640 1 -- 172.21.15.44:0/1546015594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef8c050 msgr2=0x559b1ef97aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:16.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f991d1dc640 1 --2- 172.21.15.44:0/1546015594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef8c050 0x559b1ef97aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f991d1dc640 1 -- 172.21.15.44:0/1546015594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b1ec51650 con 0x559b1ef97fe0 2024-08-20T03:03:16.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f991d9dd640 1 --2- 172.21.15.44:0/1546015594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef8c050 0x559b1ef97aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:16.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f991e1de640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f99040b3dd0 0x7f99040b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f990effd640 1 -- 172.21.15.44:0/430043488 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f99080e3750 con 0x7f9918101e90 2024-08-20T03:03:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f991d1dc640 1 --2- 172.21.15.44:0/1546015594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef97fe0 0x559b1ef9a3d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f99080e0ed0 tx=0x7f99080a6230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fb43f076670781 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f9903fff640 1 -- 172.21.15.44:0/1546015594 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990801e070 con 0x559b1ef97fe0 2024-08-20T03:03:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f9903fff640 1 -- 172.21.15.44:0/1546015594 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9908017360 con 0x559b1ef97fe0 2024-08-20T03:03:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.807+0000 7f9903fff640 1 -- 172.21.15.44:0/1546015594 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99080a9cc0 con 0x559b1ef97fe0 2024-08-20T03:03:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.808+0000 7f992203a740 1 -- 172.21.15.44:0/1546015594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef97fe0 msgr2=0x559b1ef9a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:16.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.808+0000 7f991e1de640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f99040b3dd0 0x7f99040b6290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9910005fd0 tx=0x7f99100046c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:16.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.808+0000 7f992203a740 1 --2- 172.21.15.44:0/1546015594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef97fe0 0x559b1ef9a3d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f99080e0ed0 tx=0x7f99080a6230 comp rx=0 tx=0).stop 2024-08-20T03:03:16.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.808+0000 7f992203a740 1 -- 172.21.15.44:0/1546015594 shutdown_connections 2024-08-20T03:03:16.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.808+0000 7f992203a740 1 --2- 172.21.15.44:0/1546015594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef97fe0 0x559b1ef9a3d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.808+0000 7f992203a740 1 --2- 172.21.15.44:0/1546015594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef8c050 0x559b1ef97aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.808+0000 7f992203a740 1 --2- 172.21.15.44:0/1546015594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8b730 0x559b1ef8bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.808+0000 7f992203a740 1 -- 172.21.15.44:0/1546015594 >> 172.21.15.44:0/1546015594 conn(0x559b1ef89560 msgr2=0x559b1ef8a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:16.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.808+0000 7f992203a740 1 -- 172.21.15.44:0/1546015594 shutdown_connections 2024-08-20T03:03:16.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.808+0000 7f992203a740 1 -- 172.21.15.44:0/1546015594 wait complete. 2024-08-20T03:03:16.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f992203a740 1 -- start start 2024-08-20T03:03:16.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f992203a740 1 Processor -- start 2024-08-20T03:03:16.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f992203a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 0x559b1ef94fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f992203a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8c050 0x559b1ef954f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f992203a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef97fe0 0x559b1ef95a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f992203a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559b1ed54700 con 0x559b1ef8b730 2024-08-20T03:03:16.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f992203a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559b1ee78810 con 0x559b1ef97fe0 2024-08-20T03:03:16.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f992203a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559b1ef9ac90 con 0x559b1ef8c050 2024-08-20T03:03:16.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f992203a740 10 client.0 _ll_register_callbacks cb 0x559b1eeeb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:16.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f991d9dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 0x559b1ef94fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f991d9dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 0x559b1ef94fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54568/0 (socket says 172.21.15.44:54568) 2024-08-20T03:03:16.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f991d9dd640 1 -- 172.21.15.44:0/1750006756 learned_addr learned my addr 172.21.15.44:0/1750006756 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:16.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f991e1de640 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef97fe0 0x559b1ef95a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f991d1dc640 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8c050 0x559b1ef954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f991d9dd640 1 -- 172.21.15.44:0/1750006756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8c050 msgr2=0x559b1ef954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:16.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.809+0000 7f991d9dd640 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8c050 0x559b1ef954f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f991d9dd640 1 -- 172.21.15.44:0/1750006756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef97fe0 msgr2=0x559b1ef95a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:16.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f991d9dd640 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef97fe0 0x559b1ef95a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f991d9dd640 1 -- 172.21.15.44:0/1750006756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b1efa40a0 con 0x559b1ef8b730 2024-08-20T03:03:16.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f991e1de640 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef97fe0 0x559b1ef95a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:16.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f991d9dd640 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 0x559b1ef94fb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9914009ca0 tx=0x7f9914009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a3a94a76f2d584 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:16.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f99017fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:16.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f99017fa640 1 -- 172.21.15.44:0/1750006756 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99140317a0 con 0x559b1ef8b730 2024-08-20T03:03:16.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f99017fa640 1 -- 172.21.15.44:0/1750006756 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f9914003970 con 0x559b1ef8b730 2024-08-20T03:03:16.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f992203a740 1 -- 172.21.15.44:0/1750006756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559b1eea1150 con 0x559b1ef8b730 2024-08-20T03:03:16.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f99017fa640 1 -- 172.21.15.44:0/1750006756 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9914003b70 con 0x559b1ef8b730 2024-08-20T03:03:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f992203a740 10 client.15801 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f992203a740 1 -- 172.21.15.44:0/1750006756 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b1efa6aa0 con 0x559b1ef8b730 2024-08-20T03:03:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.810+0000 7f99017fa640 1 -- 172.21.15.44:0/1750006756 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559b1eea1150 con 0x559b1ef8b730 2024-08-20T03:03:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.811+0000 7f99017fa640 1 -- 172.21.15.44:0/1750006756 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f99140045b0 con 0x559b1ef8b730 2024-08-20T03:03:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.811+0000 7f992203a740 10 client.15801 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.811+0000 7f992203a740 10 client.15801 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:03:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.811+0000 7f992203a740 20 client.15801 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.811+0000 7f992203a740 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x559b1efa7fb0 0x559b1efaa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.811+0000 7f992203a740 4 client.15801 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.811+0000 7f992203a740 1 -- 172.21.15.44:0/1750006756 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b1ef04880 con 0x559b1efa7fb0 2024-08-20T03:03:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.811+0000 7f991d1dc640 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x559b1efa7fb0 0x559b1efaa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.811+0000 7f99017fa640 10 client.15801 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:03:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.811+0000 7f991d1dc640 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x559b1efa7fb0 0x559b1efaa3a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41e3a2a488087b3c server_cookie=6a99fd239d0e0c5a in_seq=0 out_seq=0 2024-08-20T03:03:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f99017fa640 1 -- 172.21.15.44:0/1750006756 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b1ef04880 con 0x559b1efa7fb0 2024-08-20T03:03:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f99017fa640 10 client.15801 handle_command_reply: tid=0 2024-08-20T03:03:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f992203a740 1 client.15801 shutdown 2024-08-20T03:03:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f992203a740 1 -- 172.21.15.44:0/1750006756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 msgr2=0x559b1ef94fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:16.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f992203a740 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 0x559b1ef94fb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9914009ca0 tx=0x7f9914009c20 comp rx=0 tx=0).stop 2024-08-20T03:03:16.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f992203a740 1 -- 172.21.15.44:0/1750006756 shutdown_connections 2024-08-20T03:03:16.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f992203a740 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef97fe0 0x559b1ef95a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f992203a740 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x559b1efa7fb0 0x559b1efaa3a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f992203a740 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8c050 0x559b1ef954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f992203a740 1 --2- 172.21.15.44:0/1750006756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 0x559b1ef94fb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f99017fa640 0 client.15801 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:03:16.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f992203a740 1 -- 172.21.15.44:0/1750006756 >> 172.21.15.44:0/1750006756 conn(0x559b1ef89560 msgr2=0x559b1ef8f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:16.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.812+0000 7f992203a740 1 -- 172.21.15.44:0/1750006756 shutdown_connections 2024-08-20T03:03:16.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.813+0000 7f992203a740 1 -- 172.21.15.44:0/1750006756 wait complete. 2024-08-20T03:03:16.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.813+0000 7f992203a740 20 client.15801 trim_cache size 0 max 16384 2024-08-20T03:03:16.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f992203a740 1 Processor -- start 2024-08-20T03:03:16.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f992203a740 1 -- start start 2024-08-20T03:03:16.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f992203a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 0x559b1ef8f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f992203a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef8c050 0x559b1ef954b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f992203a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef97fe0 0x559b1ef959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f992203a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f99181b05e0 con 0x559b1ef8b730 2024-08-20T03:03:16.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f992203a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f99181b2120 con 0x559b1ef8c050 2024-08-20T03:03:16.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f992203a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99181b0cb0 con 0x559b1ef97fe0 2024-08-20T03:03:16.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f991d9dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 0x559b1ef8f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f991d9dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 0x559b1ef8f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54574/0 (socket says 172.21.15.44:54574) 2024-08-20T03:03:16.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f991d9dd640 1 -- 172.21.15.44:0/905438472 learned_addr learned my addr 172.21.15.44:0/905438472 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:16.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f991d1dc640 1 --2- 172.21.15.44:0/905438472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef8c050 0x559b1ef954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f991e1de640 1 --2- 172.21.15.44:0/905438472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef97fe0 0x559b1ef959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f991d9dd640 1 -- 172.21.15.44:0/905438472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef97fe0 msgr2=0x559b1ef959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:16.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f991d9dd640 1 --2- 172.21.15.44:0/905438472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef97fe0 0x559b1ef959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f991d9dd640 1 -- 172.21.15.44:0/905438472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef8c050 msgr2=0x559b1ef954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:16.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f991d9dd640 1 --2- 172.21.15.44:0/905438472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef8c050 0x559b1ef954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f991d9dd640 1 -- 172.21.15.44:0/905438472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b1ef8be80 con 0x559b1ef8b730 2024-08-20T03:03:16.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.826+0000 7f991d9dd640 1 --2- 172.21.15.44:0/905438472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 0x559b1ef8f830 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f991402ade0 tx=0x7f9914003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36d97c4f25a732f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:16.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.827+0000 7f99037fe640 1 -- 172.21.15.44:0/905438472 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991402f310 con 0x559b1ef8b730 2024-08-20T03:03:16.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.827+0000 7f99037fe640 1 -- 172.21.15.44:0/905438472 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9914036050 con 0x559b1ef8b730 2024-08-20T03:03:16.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.827+0000 7f99037fe640 1 -- 172.21.15.44:0/905438472 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914030820 con 0x559b1ef8b730 2024-08-20T03:03:16.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.827+0000 7f992203a740 1 -- 172.21.15.44:0/905438472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 msgr2=0x559b1ef8f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:16.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.827+0000 7f992203a740 1 --2- 172.21.15.44:0/905438472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 0x559b1ef8f830 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f991402ade0 tx=0x7f9914003b00 comp rx=0 tx=0).stop 2024-08-20T03:03:16.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.827+0000 7f992203a740 1 -- 172.21.15.44:0/905438472 shutdown_connections 2024-08-20T03:03:16.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.827+0000 7f992203a740 1 --2- 172.21.15.44:0/905438472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef97fe0 0x559b1ef959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.827+0000 7f992203a740 1 --2- 172.21.15.44:0/905438472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef8c050 0x559b1ef954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.827+0000 7f992203a740 1 --2- 172.21.15.44:0/905438472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8b730 0x559b1ef8f830 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.827+0000 7f992203a740 1 -- 172.21.15.44:0/905438472 >> 172.21.15.44:0/905438472 conn(0x559b1ef89560 msgr2=0x559b1ef884f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:16.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.827+0000 7f992203a740 1 -- 172.21.15.44:0/905438472 shutdown_connections 2024-08-20T03:03:16.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.827+0000 7f992203a740 1 -- 172.21.15.44:0/905438472 wait complete. 2024-08-20T03:03:16.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f992203a740 1 -- start start 2024-08-20T03:03:16.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f992203a740 1 Processor -- start 2024-08-20T03:03:16.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f992203a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8b730 0x559b1efa0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f992203a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8c050 0x559b1efa0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f992203a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef97fe0 0x559b1ef9ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f992203a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f99181b1a50 con 0x559b1ef8c050 2024-08-20T03:03:16.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f992203a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f99181b1380 con 0x559b1ef97fe0 2024-08-20T03:03:16.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f992203a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f99181ae610 con 0x559b1ef8b730 2024-08-20T03:03:16.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f992203a740 10 client.0 _ll_register_callbacks cb 0x7f9914005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:16.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f991d1dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8c050 0x559b1efa0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f991d1dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8c050 0x559b1efa0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54582/0 (socket says 172.21.15.44:54582) 2024-08-20T03:03:16.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f991d1dc640 1 -- 172.21.15.44:0/1781781444 learned_addr learned my addr 172.21.15.44:0/1781781444 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:16.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f991e1de640 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef97fe0 0x559b1ef9ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.828+0000 7f991d9dd640 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8b730 0x559b1efa0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f991d1dc640 1 -- 172.21.15.44:0/1781781444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8b730 msgr2=0x559b1efa0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:16.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f991d1dc640 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8b730 0x559b1efa0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f991d1dc640 1 -- 172.21.15.44:0/1781781444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef97fe0 msgr2=0x559b1ef9ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:16.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f991d1dc640 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef97fe0 0x559b1ef9ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f991d1dc640 1 -- 172.21.15.44:0/1781781444 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b1ef9c2a0 con 0x559b1ef8c050 2024-08-20T03:03:16.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f991d1dc640 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8c050 0x559b1efa0880 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f990800be60 tx=0x7f99080047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482993ae22980aea server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:16.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f99017fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:16.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f99017fa640 1 -- 172.21.15.44:0/1781781444 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990801e040 con 0x559b1ef8c050 2024-08-20T03:03:16.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f99017fa640 1 -- 172.21.15.44:0/1781781444 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f99080a6600 con 0x559b1ef8c050 2024-08-20T03:03:16.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f992203a740 1 -- 172.21.15.44:0/1781781444 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9914036050 con 0x559b1ef8c050 2024-08-20T03:03:16.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f99017fa640 1 -- 172.21.15.44:0/1781781444 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f99080a9ae0 con 0x559b1ef8c050 2024-08-20T03:03:16.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f992203a740 10 client.15807 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:16.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f992203a740 1 -- 172.21.15.44:0/1781781444 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b1f0129e0 con 0x559b1ef8c050 2024-08-20T03:03:16.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f99017fa640 1 -- 172.21.15.44:0/1781781444 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9914036050 con 0x559b1ef8c050 2024-08-20T03:03:16.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f99017fa640 1 -- 172.21.15.44:0/1781781444 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f99080e1100 con 0x559b1ef8c050 2024-08-20T03:03:16.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f992203a740 10 client.15807 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:16.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f992203a740 10 client.15807 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:03:16.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f992203a740 20 client.15807 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:16.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.829+0000 7f992203a740 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x559b1f012d40 0x559b1f015160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:16.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.830+0000 7f992203a740 4 client.15807 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:16.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.830+0000 7f992203a740 1 -- 172.21.15.44:0/1781781444 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b1ef00320 con 0x559b1f012d40 2024-08-20T03:03:16.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.830+0000 7f991d9dd640 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x559b1f012d40 0x559b1f015160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:16.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.830+0000 7f991d9dd640 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x559b1f012d40 0x559b1f015160 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32df540cfc91aa3d server_cookie=1953932e167970ed in_seq=0 out_seq=0 2024-08-20T03:03:16.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.830+0000 7f99017fa640 10 client.15807 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:03:16.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.830+0000 7f99017fa640 1 -- 172.21.15.44:0/1781781444 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b1ef00320 con 0x559b1f012d40 2024-08-20T03:03:16.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.830+0000 7f99017fa640 10 client.15807 handle_command_reply: tid=0 2024-08-20T03:03:16.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.830+0000 7f992203a740 1 client.15807 shutdown 2024-08-20T03:03:16.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.830+0000 7f992203a740 1 -- 172.21.15.44:0/1781781444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8c050 msgr2=0x559b1efa0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:16.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.830+0000 7f992203a740 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8c050 0x559b1efa0880 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f990800be60 tx=0x7f99080047d0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f992203a740 1 -- 172.21.15.44:0/1781781444 shutdown_connections 2024-08-20T03:03:16.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f992203a740 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b1ef97fe0 0x559b1ef9ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f992203a740 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b1ef8c050 0x559b1efa0880 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f992203a740 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x559b1f012d40 0x559b1f015160 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f992203a740 1 --2- 172.21.15.44:0/1781781444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b1ef8b730 0x559b1efa0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f992203a740 1 -- 172.21.15.44:0/1781781444 >> 172.21.15.44:0/1781781444 conn(0x559b1ef89560 msgr2=0x559b1ef965e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:16.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f99017fa640 0 client.15807 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:03:16.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f992203a740 1 -- 172.21.15.44:0/1781781444 shutdown_connections 2024-08-20T03:03:16.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f992203a740 1 -- 172.21.15.44:0/1781781444 wait complete. 2024-08-20T03:03:16.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f992203a740 20 client.15807 trim_cache size 0 max 16384 2024-08-20T03:03:16.899 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:16.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f991fc41640 1 -- 172.21.15.44:0/430043488 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f99040b3dd0 msgr2=0x7f99040b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:16.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f991fc41640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f99040b3dd0 0x7f99040b6290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9910005fd0 tx=0x7f99100046c0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f991fc41640 1 -- 172.21.15.44:0/430043488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9918101e90 msgr2=0x7f991819b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:16.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f991fc41640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9918101e90 0x7f991819b770 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f990800b4a0 tx=0x7f990800b960 comp rx=0 tx=0).stop 2024-08-20T03:03:16.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f991fc41640 1 -- 172.21.15.44:0/430043488 shutdown_connections 2024-08-20T03:03:16.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f991fc41640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f99040b3dd0 0x7f99040b6290 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f991fc41640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991810f120 0x7f99181a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f991fc41640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9918101e90 0x7f991819b770 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f991fc41640 1 --2- 172.21.15.44:0/430043488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9918101570 0x7f991819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:16.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f991fc41640 1 -- 172.21.15.44:0/430043488 >> 172.21.15.44:0/430043488 conn(0x7f99180fd440 msgr2=0x7f99180ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:16.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f991fc41640 1 -- 172.21.15.44:0/430043488 shutdown_connections 2024-08-20T03:03:16.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:16.831+0000 7f991fc41640 1 -- 172.21.15.44:0/430043488 wait complete. 2024-08-20T03:03:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.428+0000 7fe5919aa640 1 -- 172.21.15.44:0/698314179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe58c107c70 msgr2=0x7fe58c10e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.428+0000 7fe5919aa640 1 --2- 172.21.15.44:0/698314179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe58c107c70 0x7fe58c10e470 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe584009ca0 tx=0x7fe58402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.428+0000 7fe5919aa640 1 -- 172.21.15.44:0/698314179 shutdown_connections 2024-08-20T03:03:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.428+0000 7fe5919aa640 1 --2- 172.21.15.44:0/698314179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe58c107c70 0x7fe58c10e470 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.428+0000 7fe5919aa640 1 --2- 172.21.15.44:0/698314179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58c1072f0 0x7fe58c107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.428+0000 7fe5919aa640 1 --2- 172.21.15.44:0/698314179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe58c1018f0 0x7fe58c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.428+0000 7fe5919aa640 1 -- 172.21.15.44:0/698314179 >> 172.21.15.44:0/698314179 conn(0x7fe58c0fd350 msgr2=0x7fe58c0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.428+0000 7fe5919aa640 1 -- 172.21.15.44:0/698314179 shutdown_connections 2024-08-20T03:03:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.428+0000 7fe5919aa640 1 -- 172.21.15.44:0/698314179 wait complete. 2024-08-20T03:03:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.429+0000 7fe5919aa640 1 Processor -- start 2024-08-20T03:03:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.429+0000 7fe5919aa640 1 -- start start 2024-08-20T03:03:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.429+0000 7fe5919aa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58c1018f0 0x7fe58c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.429+0000 7fe5919aa640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe58c1072f0 0x7fe58c199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.429+0000 7fe5919aa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe58c107c70 0x7fe58c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.429+0000 7fe5919aa640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c111930 con 0x7fe58c107c70 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.429+0000 7fe5919aa640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1117b0 con 0x7fe58c1018f0 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.429+0000 7fe5919aa640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c111ab0 con 0x7fe58c1072f0 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.429+0000 7fe58b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe58c107c70 0x7fe58c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.429+0000 7fe58b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe58c107c70 0x7fe58c19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54612/0 (socket says 172.21.15.44:54612) 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.429+0000 7fe58b7fe640 1 -- 172.21.15.44:0/3517966055 learned_addr learned my addr 172.21.15.44:0/3517966055 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.429+0000 7fe58affd640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58c1018f0 0x7fe58c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe58a7fc640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe58c1072f0 0x7fe58c199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe58b7fe640 1 -- 172.21.15.44:0/3517966055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe58c1072f0 msgr2=0x7fe58c199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe58b7fe640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe58c1072f0 0x7fe58c199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe58b7fe640 1 -- 172.21.15.44:0/3517966055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58c1018f0 msgr2=0x7fe58c198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:17.435 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe58b7fe640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58c1018f0 0x7fe58c198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe58b7fe640 1 -- 172.21.15.44:0/3517966055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58c19ef80 con 0x7fe58c107c70 2024-08-20T03:03:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe58affd640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58c1018f0 0x7fe58c198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe58b7fe640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe58c107c70 0x7fe58c19e7b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe584005cc0 tx=0x7fe58402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe5909a8640 1 -- 172.21.15.44:0/3517966055 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584003bf0 con 0x7fe58c107c70 2024-08-20T03:03:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe5919aa640 1 -- 172.21.15.44:0/3517966055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58c19f210 con 0x7fe58c107c70 2024-08-20T03:03:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe5909a8640 1 -- 172.21.15.44:0/3517966055 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe584003d90 con 0x7fe58c107c70 2024-08-20T03:03:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe5909a8640 1 -- 172.21.15.44:0/3517966055 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58402fbc0 con 0x7fe58c107c70 2024-08-20T03:03:17.436 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.430+0000 7fe5919aa640 1 -- 172.21.15.44:0/3517966055 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe58c1a5930 con 0x7fe58c107c70 2024-08-20T03:03:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.432+0000 7fe59487a740 1 Processor -- start 2024-08-20T03:03:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe5909a8640 1 -- 172.21.15.44:0/3517966055 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7fe58402fd60 con 0x7fe58c107c70 2024-08-20T03:03:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe59487a740 1 -- start start 2024-08-20T03:03:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe59487a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 0x55b688882fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe59487a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688883510 0x55b68888ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe59487a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 0x55b688891890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe59487a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b6887f8360 con 0x55b688882bf0 2024-08-20T03:03:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe59487a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b6887e7860 con 0x55b688883510 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe59487a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6887eb8a0 con 0x55b68888f4a0 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe58a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 0x55b688882fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe58a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 0x55b688882fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54616/0 (socket says 172.21.15.44:54616) 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe58a7fc640 1 -- 172.21.15.44:0/1884835473 learned_addr learned my addr 172.21.15.44:0/1884835473 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe58affd640 1 --2- 172.21.15.44:0/1884835473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688883510 0x55b68888ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe58a7fc640 1 --2- 172.21.15.44:0/1884835473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 0x55b688891890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe58a7fc640 1 -- 172.21.15.44:0/1884835473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 msgr2=0x55b688891890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe58a7fc640 1 --2- 172.21.15.44:0/1884835473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 0x55b688891890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe58a7fc640 1 -- 172.21.15.44:0/1884835473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688883510 msgr2=0x55b68888ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe58a7fc640 1 --2- 172.21.15.44:0/1884835473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688883510 0x55b68888ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe58a7fc640 1 -- 172.21.15.44:0/1884835473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b68878ee40 con 0x55b688882bf0 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe5909a8640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe5780b3dd0 0x7fe5780b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.433+0000 7fe58a7fc640 1 --2- 172.21.15.44:0/1884835473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 0x55b688882fd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe574005fd0 tx=0x7fe574009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524a63ad5a83f3a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe58affd640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe5780b3dd0 0x7fe5780b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe581ffb640 1 -- 172.21.15.44:0/1884835473 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5740150f0 con 0x55b688882bf0 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe5909a8640 1 -- 172.21.15.44:0/3517966055 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe584105780 con 0x7fe58c107c70 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe581ffb640 1 -- 172.21.15.44:0/1884835473 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5740054d0 con 0x55b688882bf0 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe581ffb640 1 -- 172.21.15.44:0/1884835473 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574004d90 con 0x55b688882bf0 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe59487a740 1 -- 172.21.15.44:0/1884835473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 msgr2=0x55b688882fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe58affd640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe5780b3dd0 0x7fe5780b6290 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55b688885d70 tx=0x7fe57c00b5a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe59487a740 1 --2- 172.21.15.44:0/1884835473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 0x55b688882fd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe574005fd0 tx=0x7fe574009590 comp rx=0 tx=0).stop 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe59487a740 1 -- 172.21.15.44:0/1884835473 shutdown_connections 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe59487a740 1 --2- 172.21.15.44:0/1884835473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 0x55b688891890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe59487a740 1 --2- 172.21.15.44:0/1884835473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688883510 0x55b68888ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe59487a740 1 --2- 172.21.15.44:0/1884835473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 0x55b688882fd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.434+0000 7fe59487a740 1 -- 172.21.15.44:0/1884835473 >> 172.21.15.44:0/1884835473 conn(0x55b688880a20 msgr2=0x55b688881bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe59487a740 1 -- 172.21.15.44:0/1884835473 shutdown_connections 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe59487a740 1 -- 172.21.15.44:0/1884835473 wait complete. 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe59487a740 1 -- start start 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe59487a740 1 Processor -- start 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe59487a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688882bf0 0x55b68888ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe59487a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688883510 0x55b68888c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe59487a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 0x55b68888c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe59487a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b688720ab0 con 0x55b688883510 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe59487a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b68876e780 con 0x55b688882bf0 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe59487a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b688892120 con 0x55b68888f4a0 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe59487a740 10 client.0 _ll_register_callbacks cb 0x55b6884f6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe58a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688883510 0x55b68888c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe58a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688883510 0x55b68888c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54622/0 (socket says 172.21.15.44:54622) 2024-08-20T03:03:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.435+0000 7fe58a7fc640 1 -- 172.21.15.44:0/1131107607 learned_addr learned my addr 172.21.15.44:0/1131107607 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe58affd640 1 --2- 172.21.15.44:0/1131107607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688882bf0 0x55b68888ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe58a7fc640 1 -- 172.21.15.44:0/1131107607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 msgr2=0x55b68888c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe58a7fc640 1 --2- 172.21.15.44:0/1131107607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 0x55b68888c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe58a7fc640 1 -- 172.21.15.44:0/1131107607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688882bf0 msgr2=0x55b68888ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe58a7fc640 1 --2- 172.21.15.44:0/1131107607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688882bf0 0x55b68888ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe58a7fc640 1 -- 172.21.15.44:0/1131107607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b68854a790 con 0x55b688883510 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe58a7fc640 1 --2- 172.21.15.44:0/1131107607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688883510 0x55b68888c100 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe5740092e0 tx=0x7fe574009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d23020a962c00c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe56affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe56affd640 1 -- 172.21.15.44:0/1131107607 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574025020 con 0x55b688883510 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe59487a740 1 -- 172.21.15.44:0/1131107607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b688796e00 con 0x55b688883510 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe56affd640 1 -- 172.21.15.44:0/1131107607 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe5740134c0 con 0x55b688883510 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe56affd640 1 -- 172.21.15.44:0/1131107607 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe574030050 con 0x55b688883510 2024-08-20T03:03:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe59487a740 10 client.15819 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe59487a740 1 -- 172.21.15.44:0/1131107607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6885c3620 con 0x55b688883510 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.436+0000 7fe56affd640 1 -- 172.21.15.44:0/1131107607 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b688796e00 con 0x55b688883510 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.437+0000 7fe56affd640 1 -- 172.21.15.44:0/1131107607 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe57400c070 con 0x55b688883510 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.437+0000 7fe59487a740 10 client.15819 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.437+0000 7fe59487a740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.437+0000 7fe59487a740 20 client.15819 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.437+0000 7fe59487a740 1 --2- 172.21.15.44:0/1131107607 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b68889ec90 0x55b6888a1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.437+0000 7fe59487a740 4 client.15819 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.437+0000 7fe59487a740 1 -- 172.21.15.44:0/1131107607 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6887fa2b0 con 0x55b68889ec90 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.437+0000 7fe58affd640 1 --2- 172.21.15.44:0/1131107607 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b68889ec90 0x55b6888a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.437+0000 7fe58affd640 1 --2- 172.21.15.44:0/1131107607 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b68889ec90 0x55b6888a1080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5122f4f2d7748111 server_cookie=93a51ee12c8a7307 in_seq=0 out_seq=0 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.437+0000 7fe56affd640 10 client.15819 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.438+0000 7fe56affd640 1 -- 172.21.15.44:0/1131107607 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6887fa2b0 con 0x55b68889ec90 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.438+0000 7fe56affd640 10 client.15819 handle_command_reply: tid=0 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.438+0000 7fe59487a740 1 client.15819 shutdown 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.438+0000 7fe59487a740 1 -- 172.21.15.44:0/1131107607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688883510 msgr2=0x55b68888c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.438+0000 7fe59487a740 1 --2- 172.21.15.44:0/1131107607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688883510 0x55b68888c100 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe5740092e0 tx=0x7fe574009310 comp rx=0 tx=0).stop 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.438+0000 7fe59487a740 1 -- 172.21.15.44:0/1131107607 shutdown_connections 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.438+0000 7fe59487a740 1 --2- 172.21.15.44:0/1131107607 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b68889ec90 0x55b6888a1080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.438+0000 7fe59487a740 1 --2- 172.21.15.44:0/1131107607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 0x55b68888c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.438+0000 7fe59487a740 1 --2- 172.21.15.44:0/1131107607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688883510 0x55b68888c100 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.438+0000 7fe56affd640 0 client.15819 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.438+0000 7fe59487a740 1 --2- 172.21.15.44:0/1131107607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688882bf0 0x55b68888ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.438+0000 7fe59487a740 1 -- 172.21.15.44:0/1131107607 >> 172.21.15.44:0/1131107607 conn(0x55b688880a20 msgr2=0x55b6888865c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.439+0000 7fe59487a740 1 -- 172.21.15.44:0/1131107607 shutdown_connections 2024-08-20T03:03:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.439+0000 7fe59487a740 1 -- 172.21.15.44:0/1131107607 wait complete. 2024-08-20T03:03:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.439+0000 7fe59487a740 20 client.15819 trim_cache size 0 max 16384 2024-08-20T03:03:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.453+0000 7fe59487a740 1 Processor -- start 2024-08-20T03:03:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.453+0000 7fe59487a740 1 -- start start 2024-08-20T03:03:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.453+0000 7fe59487a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688882bf0 0x55b68889b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.453+0000 7fe59487a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b688883510 0x55b68888b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.453+0000 7fe59487a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b68888f4a0 0x55b6888a6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.453+0000 7fe59487a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1ae1d0 con 0x55b68888f4a0 2024-08-20T03:03:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.453+0000 7fe59487a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1afd10 con 0x55b688882bf0 2024-08-20T03:03:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.453+0000 7fe59487a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1ae8a0 con 0x55b688883510 2024-08-20T03:03:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.453+0000 7fe58b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b68888f4a0 0x55b6888a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.453+0000 7fe58b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b68888f4a0 0x55b6888a6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60160/0 (socket says 172.21.15.44:60160) 2024-08-20T03:03:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.453+0000 7fe58b7fe640 1 -- 172.21.15.44:0/528309996 learned_addr learned my addr 172.21.15.44:0/528309996 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.454+0000 7fe58a7fc640 1 --2- 172.21.15.44:0/528309996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b688883510 0x55b68888b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.454+0000 7fe58affd640 1 --2- 172.21.15.44:0/528309996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688882bf0 0x55b68889b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.454+0000 7fe58b7fe640 1 -- 172.21.15.44:0/528309996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b688883510 msgr2=0x55b68888b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.454+0000 7fe58b7fe640 1 --2- 172.21.15.44:0/528309996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b688883510 0x55b68888b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.454+0000 7fe58b7fe640 1 -- 172.21.15.44:0/528309996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688882bf0 msgr2=0x55b68889b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.454+0000 7fe58b7fe640 1 --2- 172.21.15.44:0/528309996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688882bf0 0x55b68889b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.454+0000 7fe58b7fe640 1 -- 172.21.15.44:0/528309996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6888838f0 con 0x55b68888f4a0 2024-08-20T03:03:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.454+0000 7fe58affd640 1 --2- 172.21.15.44:0/528309996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688882bf0 0x55b68889b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.454+0000 7fe58b7fe640 1 --2- 172.21.15.44:0/528309996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b68888f4a0 0x55b6888a6710 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe584103020 tx=0x7fe5840cbbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69141c732a60c645 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.454+0000 7fe5817fa640 1 -- 172.21.15.44:0/528309996 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5840cc3a0 con 0x55b68888f4a0 2024-08-20T03:03:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.454+0000 7fe5817fa640 1 -- 172.21.15.44:0/528309996 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe584036030 con 0x55b68888f4a0 2024-08-20T03:03:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.454+0000 7fe5817fa640 1 -- 172.21.15.44:0/528309996 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5840d4fc0 con 0x55b68888f4a0 2024-08-20T03:03:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.455+0000 7fe59487a740 1 -- 172.21.15.44:0/528309996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b68888f4a0 msgr2=0x55b6888a6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.455+0000 7fe59487a740 1 --2- 172.21.15.44:0/528309996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b68888f4a0 0x55b6888a6710 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe584103020 tx=0x7fe5840cbbd0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.455+0000 7fe59487a740 1 -- 172.21.15.44:0/528309996 shutdown_connections 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.455+0000 7fe59487a740 1 --2- 172.21.15.44:0/528309996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b68888f4a0 0x55b6888a6710 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.455+0000 7fe59487a740 1 --2- 172.21.15.44:0/528309996 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b688883510 0x55b68888b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.455+0000 7fe59487a740 1 --2- 172.21.15.44:0/528309996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688882bf0 0x55b68889b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.455+0000 7fe59487a740 1 -- 172.21.15.44:0/528309996 >> 172.21.15.44:0/528309996 conn(0x55b688880a20 msgr2=0x55b68888a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.455+0000 7fe59487a740 1 -- 172.21.15.44:0/528309996 shutdown_connections 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.455+0000 7fe59487a740 1 -- 172.21.15.44:0/528309996 wait complete. 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.455+0000 7fe59487a740 1 -- start start 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.455+0000 7fe59487a740 1 Processor -- start 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe59487a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 0x55b688897b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe59487a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688883510 0x55b688892fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe59487a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 0x55b6888935a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe59487a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1af640 con 0x55b688882bf0 2024-08-20T03:03:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe59487a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1aef70 con 0x55b688883510 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe59487a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe58c1ac1b0 con 0x55b68888f4a0 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe59487a740 10 client.0 _ll_register_callbacks cb 0x7fe57c00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe58affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 0x55b688897b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe58affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 0x55b688897b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60172/0 (socket says 172.21.15.44:60172) 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe58affd640 1 -- 172.21.15.44:0/456994804 learned_addr learned my addr 172.21.15.44:0/456994804 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe58b7fe640 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 0x55b6888935a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe58a7fc640 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688883510 0x55b688892fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe58affd640 1 -- 172.21.15.44:0/456994804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 msgr2=0x55b6888935a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe58affd640 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 0x55b6888935a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe58affd640 1 -- 172.21.15.44:0/456994804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688883510 msgr2=0x55b688892fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe58affd640 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688883510 0x55b688892fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe58affd640 1 -- 172.21.15.44:0/456994804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b688893dd0 con 0x55b688882bf0 2024-08-20T03:03:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe58affd640 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 0x55b688897b20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55b6888a1680 tx=0x7fe57c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4369ff4d3a497f01 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe56affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe56affd640 1 -- 172.21.15.44:0/456994804 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c005220 con 0x55b688882bf0 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe56affd640 1 -- 172.21.15.44:0/456994804 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe57c0053c0 con 0x55b688882bf0 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.456+0000 7fe59487a740 1 -- 172.21.15.44:0/456994804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe584036030 con 0x55b688882bf0 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.457+0000 7fe56affd640 1 -- 172.21.15.44:0/456994804 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe57c039050 con 0x55b688882bf0 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.457+0000 7fe59487a740 10 client.15825 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.457+0000 7fe59487a740 1 -- 172.21.15.44:0/456994804 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6889097a0 con 0x55b688882bf0 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.457+0000 7fe56affd640 1 -- 172.21.15.44:0/456994804 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe584036030 con 0x55b688882bf0 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.457+0000 7fe56affd640 1 -- 172.21.15.44:0/456994804 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe57c029800 con 0x55b688882bf0 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.457+0000 7fe59487a740 10 client.15825 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.457+0000 7fe59487a740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.457+0000 7fe59487a740 20 client.15825 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.457+0000 7fe59487a740 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b688909ed0 0x55b68890c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.457+0000 7fe59487a740 4 client.15825 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:03:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.457+0000 7fe59487a740 1 -- 172.21.15.44:0/456994804 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6887f5d50 con 0x55b688909ed0 2024-08-20T03:03:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.457+0000 7fe58a7fc640 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b688909ed0 0x55b68890c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.458+0000 7fe56affd640 10 client.15825 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.458+0000 7fe58a7fc640 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b688909ed0 0x55b68890c2f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be4221fed686210f server_cookie=4c9fea4c7b2034cf in_seq=0 out_seq=0 2024-08-20T03:03:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.461+0000 7fe56affd640 1 -- 172.21.15.44:0/456994804 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6887f5d50 con 0x55b688909ed0 2024-08-20T03:03:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.461+0000 7fe56affd640 10 client.15825 handle_command_reply: tid=0 2024-08-20T03:03:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.461+0000 7fe59487a740 1 client.15825 shutdown 2024-08-20T03:03:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe59487a740 1 -- 172.21.15.44:0/456994804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 msgr2=0x55b688897b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe59487a740 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 0x55b688897b20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55b6888a1680 tx=0x7fe57c0046c0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe59487a740 1 -- 172.21.15.44:0/456994804 shutdown_connections 2024-08-20T03:03:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe59487a740 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b688909ed0 0x55b68890c2f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe59487a740 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b68888f4a0 0x55b6888935a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe59487a740 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b688883510 0x55b688892fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe56affd640 0 client.15825 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe59487a740 1 --2- 172.21.15.44:0/456994804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b688882bf0 0x55b688897b20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe59487a740 1 -- 172.21.15.44:0/456994804 >> 172.21.15.44:0/456994804 conn(0x55b688880a20 msgr2=0x55b6888865c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe59487a740 1 -- 172.21.15.44:0/456994804 shutdown_connections 2024-08-20T03:03:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe59487a740 1 -- 172.21.15.44:0/456994804 wait complete. 2024-08-20T03:03:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe59487a740 20 client.15825 trim_cache size 0 max 16384 2024-08-20T03:03:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:03:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:03:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:03:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:03:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe5919aa640 1 -- 172.21.15.44:0/3517966055 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe5780b3dd0 msgr2=0x7fe5780b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe5919aa640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe5780b3dd0 0x7fe5780b6290 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55b688885d70 tx=0x7fe57c00b5a0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe5919aa640 1 -- 172.21.15.44:0/3517966055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe58c107c70 msgr2=0x7fe58c19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe5919aa640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe58c107c70 0x7fe58c19e7b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe584005cc0 tx=0x7fe58402f380 comp rx=0 tx=0).stop 2024-08-20T03:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe5919aa640 1 -- 172.21.15.44:0/3517966055 shutdown_connections 2024-08-20T03:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe5919aa640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe5780b3dd0 0x7fe5780b6290 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe5919aa640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe58c107c70 0x7fe58c19e7b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe5919aa640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe58c1072f0 0x7fe58c199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe5919aa640 1 --2- 172.21.15.44:0/3517966055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe58c1018f0 0x7fe58c198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.462+0000 7fe5919aa640 1 -- 172.21.15.44:0/3517966055 >> 172.21.15.44:0/3517966055 conn(0x7fe58c0fd350 msgr2=0x7fe58c06b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.463+0000 7fe5919aa640 1 -- 172.21.15.44:0/3517966055 shutdown_connections 2024-08-20T03:03:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:17.463+0000 7fe5919aa640 1 -- 172.21.15.44:0/3517966055 wait complete. 2024-08-20T03:03:17.722 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-20T03:03:17.725 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T03:03:17.783 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-20T03:03:17.824 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-20T03:03:17.825 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-20T03:03:17.825 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-20T03:03:17.864 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-20T03:03:17.868 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T03:03:17.868 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-08-20T03:03:17.925 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-08-20T03:03:17.927 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-08-20T03:03:17.928 DEBUG:teuthology.orchestra.run.smithi044:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ef4275410d8483698016fe484b2a3c664080b4c1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-08-20T03:03:17.987 INFO:tasks.workunit.client.0.smithi044.stdout:getting blogbench 2024-08-20T03:03:17.988 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 'getting blogbench' 2024-08-20T03:03:17.988 INFO:tasks.workunit.client.0.smithi044.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-08-20T03:03:18.004 INFO:tasks.workunit.client.0.smithi044.stderr:--2024-08-20 03:03:18-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-08-20T03:03:18.005 INFO:tasks.workunit.client.0.smithi044.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-08-20T03:03:18.020 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:18.026 INFO:tasks.workunit.client.0.smithi044.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-08-20T03:03:18.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:17 smithi044 ceph-mon[18790]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-08-20T03:03:18.047 INFO:tasks.workunit.client.0.smithi044.stderr:HTTP request sent, awaiting response... 200 OK 2024-08-20T03:03:18.047 INFO:tasks.workunit.client.0.smithi044.stderr:Length: 103355 (101K) [application/octet-stream] 2024-08-20T03:03:18.048 INFO:tasks.workunit.client.0.smithi044.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-08-20T03:03:18.050 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-08-20T03:03:18.088 INFO:tasks.workunit.client.0.smithi044.stderr: 0K .......... .......... .......... .......... .......... 49% 1.23M 0s 2024-08-20T03:03:18.129 INFO:tasks.workunit.client.0.smithi044.stderr: 50K .......... .......... .......... .......... .......... 99% 2.38M 0s 2024-08-20T03:03:18.129 INFO:tasks.workunit.client.0.smithi044.stderr: 100K 100% 45.7K=0.08s 2024-08-20T03:03:18.130 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-08-20T03:03:18.130 INFO:tasks.workunit.client.0.smithi044.stderr:2024-08-20 03:03:18 (1.22 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-08-20T03:03:18.130 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-08-20T03:03:18.130 INFO:tasks.workunit.client.0.smithi044.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-08-20T03:03:18.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:17 smithi165 ceph-mon[22357]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-08-20T03:03:18.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:17 smithi099 ceph-mon[21984]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 938 B/s wr, 13 op/s 2024-08-20T03:03:18.331 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/ 2024-08-20T03:03:18.332 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/m4/ 2024-08-20T03:03:18.333 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-08-20T03:03:18.334 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/man/ 2024-08-20T03:03:18.335 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/man/blogbench.8 2024-08-20T03:03:18.336 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/man/Makefile.am 2024-08-20T03:03:18.336 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/man/Makefile.in 2024-08-20T03:03:18.337 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/ 2024-08-20T03:03:18.338 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/mysnprintf.c 2024-08-20T03:03:18.339 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/mysnprintf.h 2024-08-20T03:03:18.340 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/blogbench.c 2024-08-20T03:03:18.341 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/blogbench.h 2024-08-20T03:03:18.341 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/fakesnprintf.c 2024-08-20T03:03:18.342 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/fakesnprintf.h 2024-08-20T03:03:18.343 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/blogbench_p.h 2024-08-20T03:03:18.344 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-08-20T03:03:18.345 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-08-20T03:03:18.346 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/Makefile.am 2024-08-20T03:03:18.346 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/Makefile.in 2024-08-20T03:03:18.347 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/process.c 2024-08-20T03:03:18.348 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/globals.h 2024-08-20T03:03:18.349 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/commenter.c 2024-08-20T03:03:18.350 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/rewriter.c 2024-08-20T03:03:18.351 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/reader.c 2024-08-20T03:03:18.351 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/writer.c 2024-08-20T03:03:18.352 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/src/helpers.c 2024-08-20T03:03:18.353 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/NEWS 2024-08-20T03:03:18.354 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/compile 2024-08-20T03:03:18.355 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/depcomp 2024-08-20T03:03:18.356 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/aclocal.m4 2024-08-20T03:03:18.357 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/README 2024-08-20T03:03:18.358 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/configure 2024-08-20T03:03:18.361 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/configure.ac 2024-08-20T03:03:18.362 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/config.guess 2024-08-20T03:03:18.363 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/install-sh 2024-08-20T03:03:18.364 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/config.sub 2024-08-20T03:03:18.365 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/missing 2024-08-20T03:03:18.366 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/mkinstalldirs 2024-08-20T03:03:18.367 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/Makefile.am 2024-08-20T03:03:18.368 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/Makefile.in 2024-08-20T03:03:18.369 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/config.h.in 2024-08-20T03:03:18.370 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/AUTHORS 2024-08-20T03:03:18.371 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/INSTALL 2024-08-20T03:03:18.372 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/ChangeLog 2024-08-20T03:03:18.372 INFO:tasks.workunit.client.0.smithi044.stdout:blogbench-1.0/COPYING 2024-08-20T03:03:18.374 INFO:tasks.workunit.client.0.smithi044.stderr:+ cd blogbench-1.0/ 2024-08-20T03:03:18.374 INFO:tasks.workunit.client.0.smithi044.stdout:making blogbench 2024-08-20T03:03:18.374 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 'making blogbench' 2024-08-20T03:03:18.374 INFO:tasks.workunit.client.0.smithi044.stderr:+ ./configure 2024-08-20T03:03:18.504 INFO:tasks.workunit.client.0.smithi044.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-08-20T03:03:18.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.673+0000 7f4dce445640 1 -- 172.21.15.44:0/2893555584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc81039e0 msgr2=0x7f4dc8103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:18.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.673+0000 7f4dce445640 1 --2- 172.21.15.44:0/2893555584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc81039e0 0x7f4dc8103dc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009900 tx=0x7f4db802a410 comp rx=0 tx=0).stop 2024-08-20T03:03:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.674+0000 7f4dce445640 1 -- 172.21.15.44:0/2893555584 shutdown_connections 2024-08-20T03:03:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.674+0000 7f4dce445640 1 --2- 172.21.15.44:0/2893555584 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dc810ce40 0x7f4dc810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.674+0000 7f4dce445640 1 --2- 172.21.15.44:0/2893555584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc8104300 0x7f4dc810c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.674+0000 7f4dce445640 1 --2- 172.21.15.44:0/2893555584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc81039e0 0x7f4dc8103dc0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.674+0000 7f4dce445640 1 -- 172.21.15.44:0/2893555584 >> 172.21.15.44:0/2893555584 conn(0x7f4dc80fd440 msgr2=0x7f4dc80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.674+0000 7f4dce445640 1 -- 172.21.15.44:0/2893555584 shutdown_connections 2024-08-20T03:03:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.674+0000 7f4dce445640 1 -- 172.21.15.44:0/2893555584 wait complete. 2024-08-20T03:03:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.674+0000 7f4dce445640 1 Processor -- start 2024-08-20T03:03:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.674+0000 7f4dce445640 1 -- start start 2024-08-20T03:03:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.674+0000 7f4dce445640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc81039e0 0x7f4dc8199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dce445640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dc8104300 0x7f4dc8199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dce445640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc810ce40 0x7f4dc819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dce445640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8112ab0 con 0x7f4dc810ce40 2024-08-20T03:03:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dce445640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8112930 con 0x7f4dc81039e0 2024-08-20T03:03:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dce445640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8112c30 con 0x7f4dc8104300 2024-08-20T03:03:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dcc9e2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc810ce40 0x7f4dc819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dcc9e2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc810ce40 0x7f4dc819e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60192/0 (socket says 172.21.15.44:60192) 2024-08-20T03:03:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dcc9e2640 1 -- 172.21.15.44:0/151916367 learned_addr learned my addr 172.21.15.44:0/151916367 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dc8104300 0x7f4dc8199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dc7fff640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc81039e0 0x7f4dc8199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dcc9e2640 1 -- 172.21.15.44:0/151916367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dc8104300 msgr2=0x7f4dc8199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dcc9e2640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dc8104300 0x7f4dc8199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dcc9e2640 1 -- 172.21.15.44:0/151916367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc81039e0 msgr2=0x7f4dc8199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dcc9e2640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc81039e0 0x7f4dc8199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dcc9e2640 1 -- 172.21.15.44:0/151916367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc819f0d0 con 0x7f4dc810ce40 2024-08-20T03:03:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dc7fff640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc81039e0 0x7f4dc8199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dcc9e2640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc810ce40 0x7f4dc819e900 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000eaa0 tx=0x7f4dc000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dc57fa640 1 -- 172.21.15.44:0/151916367 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc000cdb0 con 0x7f4dc810ce40 2024-08-20T03:03:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dce445640 1 -- 172.21.15.44:0/151916367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc819f3c0 con 0x7f4dc810ce40 2024-08-20T03:03:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dc57fa640 1 -- 172.21.15.44:0/151916367 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc0005020 con 0x7f4dc810ce40 2024-08-20T03:03:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dc57fa640 1 -- 172.21.15.44:0/151916367 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00116d0 con 0x7f4dc810ce40 2024-08-20T03:03:18.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.675+0000 7f4dce445640 1 -- 172.21.15.44:0/151916367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc81a5c50 con 0x7f4dc810ce40 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dd083e740 1 Processor -- start 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dc57fa640 1 -- 172.21.15.44:0/151916367 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f4dc00054a0 con 0x7f4dc810ce40 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dd083e740 1 -- start start 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dd083e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a15ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dd083e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a15b1a0 0x563c2a166bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dd083e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a167130 0x563c2a169520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dd083e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563c2a0d1930 con 0x563c2a15a880 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dd083e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563c2a0c0e30 con 0x563c2a167130 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dd083e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563c2a0c4e70 con 0x563c2a15b1a0 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a15ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a15ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60208/0 (socket says 172.21.15.44:60208) 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dc77fe640 1 -- 172.21.15.44:0/2382432121 learned_addr learned my addr 172.21.15.44:0/2382432121 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dc7fff640 1 --2- 172.21.15.44:0/2382432121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a15b1a0 0x563c2a166bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dc57fa640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4db00b3dd0 0x7f4db00b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dc77fe640 1 -- 172.21.15.44:0/2382432121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a15b1a0 msgr2=0x563c2a166bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/2382432121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a15b1a0 0x563c2a166bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dc77fe640 1 -- 172.21.15.44:0/2382432121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a167130 msgr2=0x563c2a169520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.678+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/2382432121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a167130 0x563c2a169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.679+0000 7f4dc77fe640 1 -- 172.21.15.44:0/2382432121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c29e20650 con 0x563c2a15a880 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.679+0000 7f4dcc9e2640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4db00b3dd0 0x7f4db00b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.679+0000 7f4dc57fa640 1 -- 172.21.15.44:0/151916367 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4dc00e4320 con 0x7f4dc810ce40 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.679+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/2382432121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a15ac60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00cc50 tx=0x7f4dbc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29de9b0283abc478 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.679+0000 7f4dae7fc640 1 -- 172.21.15.44:0/2382432121 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc016070 con 0x563c2a15a880 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.679+0000 7f4dae7fc640 1 -- 172.21.15.44:0/2382432121 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dbc012b30 con 0x563c2a15a880 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.679+0000 7f4dae7fc640 1 -- 172.21.15.44:0/2382432121 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc011c20 con 0x563c2a15a880 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.680+0000 7f4dd083e740 1 -- 172.21.15.44:0/2382432121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 msgr2=0x563c2a15ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.680+0000 7f4dcc9e2640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4db00b3dd0 0x7f4db00b6290 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00e1a80 tx=0x7f4dc0000f40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:18.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.680+0000 7f4dd083e740 1 --2- 172.21.15.44:0/2382432121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a15ac60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00cc50 tx=0x7f4dbc010740 comp rx=0 tx=0).stop 2024-08-20T03:03:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.680+0000 7f4dd083e740 1 -- 172.21.15.44:0/2382432121 shutdown_connections 2024-08-20T03:03:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.680+0000 7f4dd083e740 1 --2- 172.21.15.44:0/2382432121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a167130 0x563c2a169520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.680+0000 7f4dd083e740 1 --2- 172.21.15.44:0/2382432121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a15b1a0 0x563c2a166bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.680+0000 7f4dd083e740 1 --2- 172.21.15.44:0/2382432121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a15ac60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.680+0000 7f4dd083e740 1 -- 172.21.15.44:0/2382432121 >> 172.21.15.44:0/2382432121 conn(0x563c2a1586b0 msgr2=0x563c2a159850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.680+0000 7f4dd083e740 1 -- 172.21.15.44:0/2382432121 shutdown_connections 2024-08-20T03:03:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.680+0000 7f4dd083e740 1 -- 172.21.15.44:0/2382432121 wait complete. 2024-08-20T03:03:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.680+0000 7f4dd083e740 1 -- start start 2024-08-20T03:03:18.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.680+0000 7f4dd083e740 1 Processor -- start 2024-08-20T03:03:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dd083e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a164110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dd083e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a164650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dd083e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 0x563c2a164b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dd083e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563c29f23700 con 0x563c2a15a880 2024-08-20T03:03:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dd083e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563c2a047810 con 0x563c2a15b1a0 2024-08-20T03:03:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dd083e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563c2a169de0 con 0x563c2a167130 2024-08-20T03:03:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dd083e740 10 client.0 _ll_register_callbacks cb 0x563c2a0ba700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a164110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a164110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60210/0 (socket says 172.21.15.44:60210) 2024-08-20T03:03:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dc77fe640 1 -- 172.21.15.44:0/3999407959 learned_addr learned my addr 172.21.15.44:0/3999407959 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:18.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dc7fff640 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a164650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 0x563c2a164b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dc7fff640 1 -- 172.21.15.44:0/3999407959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 msgr2=0x563c2a164b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dc7fff640 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 0x563c2a164b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dc7fff640 1 -- 172.21.15.44:0/3999407959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 msgr2=0x563c2a164110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dc7fff640 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a164110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dc7fff640 1 -- 172.21.15.44:0/3999407959 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2a1731e0 con 0x563c2a15b1a0 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.681+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a164110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.682+0000 7f4dc7fff640 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a164650 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005ab0 tx=0x7f4db8032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb078d2f8d734a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.682+0000 7f4da77fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.682+0000 7f4da77fe640 1 -- 172.21.15.44:0/3999407959 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8035070 con 0x563c2a15b1a0 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.682+0000 7f4dd083e740 1 -- 172.21.15.44:0/3999407959 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563c2a070150 con 0x563c2a15b1a0 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.682+0000 7f4da77fe640 1 -- 172.21.15.44:0/3999407959 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4db8031520 con 0x563c2a15b1a0 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.682+0000 7f4da77fe640 1 -- 172.21.15.44:0/3999407959 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4db8041e20 con 0x563c2a15b1a0 2024-08-20T03:03:18.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.682+0000 7f4dd083e740 10 client.25370 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:18.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.682+0000 7f4dd083e740 1 -- 172.21.15.44:0/3999407959 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c2a175be0 con 0x563c2a15b1a0 2024-08-20T03:03:18.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.682+0000 7f4da77fe640 1 -- 172.21.15.44:0/3999407959 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563c2a070150 con 0x563c2a15b1a0 2024-08-20T03:03:18.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.683+0000 7f4da77fe640 1 -- 172.21.15.44:0/3999407959 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4db8052020 con 0x563c2a15b1a0 2024-08-20T03:03:18.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.683+0000 7f4dd083e740 10 client.25370 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:18.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.683+0000 7f4dd083e740 10 client.25370 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:03:18.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.683+0000 7f4dd083e740 20 client.25370 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:18.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.683+0000 7f4dd083e740 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x563c2a176ff0 0x563c2a1793e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.683+0000 7f4dd083e740 4 client.25370 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:18.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.683+0000 7f4dd083e740 1 -- 172.21.15.44:0/3999407959 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c2a0d3880 con 0x563c2a176ff0 2024-08-20T03:03:18.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.683+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x563c2a176ff0 0x563c2a1793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.683+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x563c2a176ff0 0x563c2a1793e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=417bd29eca934715 server_cookie=62ee77449ff8b86 in_seq=0 out_seq=0 2024-08-20T03:03:18.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.683+0000 7f4da77fe640 10 client.25370 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:03:18.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4da77fe640 1 -- 172.21.15.44:0/3999407959 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c2a0d3880 con 0x563c2a176ff0 2024-08-20T03:03:18.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4da77fe640 10 client.25370 handle_command_reply: tid=0 2024-08-20T03:03:18.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4dd083e740 1 client.25370 shutdown 2024-08-20T03:03:18.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4dd083e740 1 -- 172.21.15.44:0/3999407959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 msgr2=0x563c2a164650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:18.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4dd083e740 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a164650 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005ab0 tx=0x7f4db8032e90 comp rx=0 tx=0).stop 2024-08-20T03:03:18.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4dd083e740 1 -- 172.21.15.44:0/3999407959 shutdown_connections 2024-08-20T03:03:18.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4dd083e740 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x563c2a176ff0 0x563c2a1793e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4dd083e740 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 0x563c2a164b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4dd083e740 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a164650 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4da77fe640 0 client.25370 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:03:18.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4dd083e740 1 --2- 172.21.15.44:0/3999407959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a164110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4dd083e740 1 -- 172.21.15.44:0/3999407959 >> 172.21.15.44:0/3999407959 conn(0x563c2a1586b0 msgr2=0x563c2a15e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:18.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4dd083e740 1 -- 172.21.15.44:0/3999407959 shutdown_connections 2024-08-20T03:03:18.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.684+0000 7f4dd083e740 1 -- 172.21.15.44:0/3999407959 wait complete. 2024-08-20T03:03:18.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.685+0000 7f4dd083e740 20 client.25370 trim_cache size 0 max 16384 2024-08-20T03:03:18.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.697+0000 7f4dd083e740 1 Processor -- start 2024-08-20T03:03:18.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.697+0000 7f4dd083e740 1 -- start start 2024-08-20T03:03:18.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dd083e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a165e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dd083e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a17ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dd083e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 0x563c2a17f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dd083e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81ac110 con 0x563c2a15a880 2024-08-20T03:03:18.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dd083e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81adc50 con 0x563c2a15b1a0 2024-08-20T03:03:18.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dd083e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81ac7e0 con 0x563c2a167130 2024-08-20T03:03:18.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a165e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a165e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60212/0 (socket says 172.21.15.44:60212) 2024-08-20T03:03:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dc77fe640 1 -- 172.21.15.44:0/445528334 learned_addr learned my addr 172.21.15.44:0/445528334 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dc7fff640 1 --2- 172.21.15.44:0/445528334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a17ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dc77fe640 1 -- 172.21.15.44:0/445528334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 msgr2=0x563c2a17f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/445528334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 0x563c2a17f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dc77fe640 1 -- 172.21.15.44:0/445528334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 msgr2=0x563c2a17ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/445528334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a17ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dc77fe640 1 -- 172.21.15.44:0/445528334 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2a1584e0 con 0x563c2a15a880 2024-08-20T03:03:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dc7fff640 1 --2- 172.21.15.44:0/445528334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a17ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/445528334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a165e50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc010c40 tx=0x7f4dbc010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a632d22d0c407fad server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.698+0000 7f4dadffb640 1 -- 172.21.15.44:0/445528334 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc016070 con 0x563c2a15a880 2024-08-20T03:03:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.699+0000 7f4dadffb640 1 -- 172.21.15.44:0/445528334 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dbc004680 con 0x563c2a15a880 2024-08-20T03:03:18.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.699+0000 7f4dadffb640 1 -- 172.21.15.44:0/445528334 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc011ed0 con 0x563c2a15a880 2024-08-20T03:03:18.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.699+0000 7f4dd083e740 1 -- 172.21.15.44:0/445528334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 msgr2=0x563c2a165e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:18.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.699+0000 7f4dd083e740 1 --2- 172.21.15.44:0/445528334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a165e50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc010c40 tx=0x7f4dbc010f70 comp rx=0 tx=0).stop 2024-08-20T03:03:18.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.699+0000 7f4dd083e740 1 -- 172.21.15.44:0/445528334 shutdown_connections 2024-08-20T03:03:18.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.699+0000 7f4dd083e740 1 --2- 172.21.15.44:0/445528334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 0x563c2a17f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.699+0000 7f4dd083e740 1 --2- 172.21.15.44:0/445528334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a17ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.699+0000 7f4dd083e740 1 --2- 172.21.15.44:0/445528334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a165e50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.707 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.699+0000 7f4dd083e740 1 -- 172.21.15.44:0/445528334 >> 172.21.15.44:0/445528334 conn(0x563c2a1586b0 msgr2=0x563c2a165740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.699+0000 7f4dd083e740 1 -- 172.21.15.44:0/445528334 shutdown_connections 2024-08-20T03:03:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.699+0000 7f4dd083e740 1 -- 172.21.15.44:0/445528334 wait complete. 2024-08-20T03:03:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.699+0000 7f4dd083e740 1 -- start start 2024-08-20T03:03:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dd083e740 1 Processor -- start 2024-08-20T03:03:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dd083e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a16f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dd083e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a16a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dd083e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 0x563c2a16aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dd083e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81ad580 con 0x563c2a15a880 2024-08-20T03:03:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dd083e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81aceb0 con 0x563c2a15b1a0 2024-08-20T03:03:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dd083e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81aa110 con 0x563c2a167130 2024-08-20T03:03:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dd083e740 10 client.0 _ll_register_callbacks cb 0x7f4db8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:18.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a16f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a16f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60214/0 (socket says 172.21.15.44:60214) 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dc77fe640 1 -- 172.21.15.44:0/1247816351 learned_addr learned my addr 172.21.15.44:0/1247816351 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 0x563c2a16aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dc7fff640 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a16a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dc77fe640 1 -- 172.21.15.44:0/1247816351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 msgr2=0x563c2a16aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 0x563c2a16aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dc77fe640 1 -- 172.21.15.44:0/1247816351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 msgr2=0x563c2a16a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a16a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.700+0000 7f4dc77fe640 1 -- 172.21.15.44:0/1247816351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c2a16b470 con 0x563c2a15a880 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4dc77fe640 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a16f4a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0117d0 tx=0x7f4dbc011760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3bf4b4b78e6e78 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4da77fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4da77fe640 1 -- 172.21.15.44:0/1247816351 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc038020 con 0x563c2a15a880 2024-08-20T03:03:18.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4dd083e740 1 -- 172.21.15.44:0/1247816351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dbc004680 con 0x563c2a15a880 2024-08-20T03:03:18.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4da77fe640 1 -- 172.21.15.44:0/1247816351 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4dbc004cc0 con 0x563c2a15a880 2024-08-20T03:03:18.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4da77fe640 1 -- 172.21.15.44:0/1247816351 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4dbc016070 con 0x563c2a15a880 2024-08-20T03:03:18.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4dd083e740 10 client.15843 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:18.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4dd083e740 1 -- 172.21.15.44:0/1247816351 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563c2a1e1970 con 0x563c2a15a880 2024-08-20T03:03:18.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4da77fe640 1 -- 172.21.15.44:0/1247816351 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dbc004680 con 0x563c2a15a880 2024-08-20T03:03:18.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4da77fe640 1 -- 172.21.15.44:0/1247816351 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4dbc0315a0 con 0x563c2a15a880 2024-08-20T03:03:18.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4dd083e740 10 client.15843 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:18.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4dd083e740 10 client.15843 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:03:18.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4dd083e740 20 client.15843 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:18.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4dd083e740 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x563c2a1e1f80 0x563c2a1e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:18.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4dd083e740 4 client.15843 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:18.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.701+0000 7f4dd083e740 1 -- 172.21.15.44:0/1247816351 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c2a0cf320 con 0x563c2a1e1f80 2024-08-20T03:03:18.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4dc7fff640 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x563c2a1e1f80 0x563c2a1e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:18.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4dc7fff640 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x563c2a1e1f80 0x563c2a1e43a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb8b954f43c3f71c server_cookie=2ddda26fac30b513 in_seq=0 out_seq=0 2024-08-20T03:03:18.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4da77fe640 10 client.15843 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:03:18.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4da77fe640 1 -- 172.21.15.44:0/1247816351 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c2a0cf320 con 0x563c2a1e1f80 2024-08-20T03:03:18.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4da77fe640 10 client.15843 handle_command_reply: tid=0 2024-08-20T03:03:18.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4dd083e740 1 client.15843 shutdown 2024-08-20T03:03:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4dd083e740 1 -- 172.21.15.44:0/1247816351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 msgr2=0x563c2a16f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4dd083e740 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a16f4a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0117d0 tx=0x7f4dbc011760 comp rx=0 tx=0).stop 2024-08-20T03:03:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4dd083e740 1 -- 172.21.15.44:0/1247816351 shutdown_connections 2024-08-20T03:03:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4dd083e740 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x563c2a1e1f80 0x563c2a1e43a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4dd083e740 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563c2a167130 0x563c2a16aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4dd083e740 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c2a15b1a0 0x563c2a16a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4dd083e740 1 --2- 172.21.15.44:0/1247816351 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c2a15a880 0x563c2a16f4a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.702+0000 7f4da77fe640 0 client.15843 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:03:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dd083e740 1 -- 172.21.15.44:0/1247816351 >> 172.21.15.44:0/1247816351 conn(0x563c2a1586b0 msgr2=0x563c2a165f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dd083e740 1 -- 172.21.15.44:0/1247816351 shutdown_connections 2024-08-20T03:03:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dd083e740 1 -- 172.21.15.44:0/1247816351 wait complete. 2024-08-20T03:03:18.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dd083e740 20 client.15843 trim_cache size 0 max 16384 2024-08-20T03:03:18.714 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:18.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dce445640 1 -- 172.21.15.44:0/151916367 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4db00b3dd0 msgr2=0x7f4db00b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:18.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dce445640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4db00b3dd0 0x7f4db00b6290 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00e1a80 tx=0x7f4dc0000f40 comp rx=0 tx=0).stop 2024-08-20T03:03:18.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dce445640 1 -- 172.21.15.44:0/151916367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc810ce40 msgr2=0x7f4dc819e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:18.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dce445640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc810ce40 0x7f4dc819e900 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000eaa0 tx=0x7f4dc000ef60 comp rx=0 tx=0).stop 2024-08-20T03:03:18.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dce445640 1 -- 172.21.15.44:0/151916367 shutdown_connections 2024-08-20T03:03:18.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dce445640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4db00b3dd0 0x7f4db00b6290 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dce445640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dc810ce40 0x7f4dc819e900 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dce445640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4dc8104300 0x7f4dc8199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dce445640 1 --2- 172.21.15.44:0/151916367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dc81039e0 0x7f4dc8199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:18.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dce445640 1 -- 172.21.15.44:0/151916367 >> 172.21.15.44:0/151916367 conn(0x7f4dc80fd440 msgr2=0x7f4dc80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:18.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dce445640 1 -- 172.21.15.44:0/151916367 shutdown_connections 2024-08-20T03:03:18.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:18.703+0000 7f4dce445640 1 -- 172.21.15.44:0/151916367 wait complete. 2024-08-20T03:03:19.478 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T03:03:19.511 INFO:tasks.workunit.client.0.smithi044.stdout:checking whether build environment is sane... yes 2024-08-20T03:03:19.519 INFO:tasks.workunit.client.0.smithi044.stdout:checking for gawk... gawk 2024-08-20T03:03:19.534 INFO:tasks.workunit.client.0.smithi044.stdout:checking whether make sets $(MAKE)... yes 2024-08-20T03:03:19.547 INFO:tasks.workunit.client.0.smithi044.stdout:checking for gcc... gcc 2024-08-20T03:03:19.594 INFO:tasks.workunit.client.0.smithi044.stdout:checking for C compiler default output file name... a.out 2024-08-20T03:03:19.595 INFO:tasks.workunit.client.0.smithi044.stdout:checking whether the C compiler works... yes 2024-08-20T03:03:19.599 INFO:tasks.workunit.client.0.smithi044.stdout:checking whether we are cross compiling... no 2024-08-20T03:03:19.636 INFO:tasks.workunit.client.0.smithi044.stdout:checking for suffix of executables... 2024-08-20T03:03:19.672 INFO:tasks.workunit.client.0.smithi044.stdout:checking for suffix of object files... o 2024-08-20T03:03:19.702 INFO:tasks.workunit.client.0.smithi044.stdout:checking whether we are using the GNU C compiler... yes 2024-08-20T03:03:19.734 INFO:tasks.workunit.client.0.smithi044.stdout:checking whether gcc accepts -g... yes 2024-08-20T03:03:19.772 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:19.800 INFO:tasks.workunit.client.0.smithi044.stdout:checking for gcc option to accept ANSI C... none needed 2024-08-20T03:03:19.874 INFO:tasks.workunit.client.0.smithi044.stdout:checking for style of include used by make... GNU 2024-08-20T03:03:19.974 INFO:tasks.workunit.client.0.smithi044.stdout:checking dependency style of gcc... gcc3 2024-08-20T03:03:19.974 INFO:tasks.workunit.client.0.smithi044.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-08-20T03:03:20.065 INFO:tasks.workunit.client.0.smithi044.stdout:checking how to run the C preprocessor... gcc -E 2024-08-20T03:03:20.104 INFO:tasks.workunit.client.0.smithi044.stdout:checking for egrep... grep -E 2024-08-20T03:03:20.112 INFO:tasks.workunit.client.0.smithi044.stdout:checking for AIX... no 2024-08-20T03:03:20.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:19 smithi165 ceph-mon[22357]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 11 op/s 2024-08-20T03:03:20.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:19 smithi099 ceph-mon[21984]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 11 op/s 2024-08-20T03:03:20.176 INFO:tasks.workunit.client.0.smithi044.stdout:checking for library containing strerror... none required 2024-08-20T03:03:20.204 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:19 smithi044 ceph-mon[18790]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 341 B/s wr, 11 op/s 2024-08-20T03:03:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.286+0000 7fe7f74f1640 1 -- 172.21.15.44:0/1362307230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7f010ce40 msgr2=0x7fe7f010f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.286+0000 7fe7f74f1640 1 --2- 172.21.15.44:0/1362307230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7f010ce40 0x7fe7f010f2a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009ca0 tx=0x7fe7e802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.286+0000 7fe7f74f1640 1 -- 172.21.15.44:0/1362307230 shutdown_connections 2024-08-20T03:03:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.286+0000 7fe7f74f1640 1 --2- 172.21.15.44:0/1362307230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7f010ce40 0x7fe7f010f2a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.286+0000 7fe7f74f1640 1 --2- 172.21.15.44:0/1362307230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7f0104300 0x7fe7f010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.286+0000 7fe7f74f1640 1 --2- 172.21.15.44:0/1362307230 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7f01039e0 0x7fe7f0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.286+0000 7fe7f74f1640 1 -- 172.21.15.44:0/1362307230 >> 172.21.15.44:0/1362307230 conn(0x7fe7f00fd440 msgr2=0x7fe7f00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.287+0000 7fe7f74f1640 1 -- 172.21.15.44:0/1362307230 shutdown_connections 2024-08-20T03:03:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.287+0000 7fe7f74f1640 1 -- 172.21.15.44:0/1362307230 wait complete. 2024-08-20T03:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.287+0000 7fe7f74f1640 1 Processor -- start 2024-08-20T03:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.287+0000 7fe7f74f1640 1 -- start start 2024-08-20T03:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.287+0000 7fe7f74f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7f01039e0 0x7fe7f0101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.287+0000 7fe7f74f1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7f0104300 0x7fe7f0101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.287+0000 7fe7f74f1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7f010ce40 0x7fe7f0102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.287+0000 7fe7f74f1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0112ab0 con 0x7fe7f010ce40 2024-08-20T03:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.287+0000 7fe7f74f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0112930 con 0x7fe7f01039e0 2024-08-20T03:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.287+0000 7fe7f74f1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0112c30 con 0x7fe7f0104300 2024-08-20T03:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.287+0000 7fe7f5a8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7f010ce40 0x7fe7f0102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7f5a8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7f010ce40 0x7fe7f0102110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60218/0 (socket says 172.21.15.44:60218) 2024-08-20T03:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/2021990270 learned_addr learned my addr 172.21.15.44:0/2021990270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:20.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7f4a8c640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7f0104300 0x7fe7f0101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7f528d640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7f01039e0 0x7fe7f0101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/2021990270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7f0104300 msgr2=0x7fe7f0101bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7f5a8e640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7f0104300 0x7fe7f0101bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/2021990270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7f01039e0 msgr2=0x7fe7f0101690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7f5a8e640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7f01039e0 0x7fe7f0101690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/2021990270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f01a7400 con 0x7fe7f010ce40 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7f5a8e640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7f010ce40 0x7fe7f0102110 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8005cc0 tx=0x7fe7e8031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7e67fc640 1 -- 172.21.15.44:0/2021990270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8003b70 con 0x7fe7f010ce40 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7e67fc640 1 -- 172.21.15.44:0/2021990270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e80318f0 con 0x7fe7f010ce40 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7f74f1640 1 -- 172.21.15.44:0/2021990270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f01a7690 con 0x7fe7f010ce40 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7e67fc640 1 -- 172.21.15.44:0/2021990270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e802f950 con 0x7fe7f010ce40 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.288+0000 7fe7f74f1640 1 -- 172.21.15.44:0/2021990270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f01a7bd0 con 0x7fe7f010ce40 2024-08-20T03:03:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.291+0000 7fe7e67fc640 1 -- 172.21.15.44:0/2021990270 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7fe7e802fb50 con 0x7fe7f010ce40 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.291+0000 7fe7f98ea740 1 Processor -- start 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.291+0000 7fe7f98ea740 1 -- start start 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.291+0000 7fe7f98ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6b880 0x562e3ed6bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.291+0000 7fe7f98ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed6c1a0 0x562e3ed77bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.291+0000 7fe7f98ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed78130 0x562e3ed7a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.291+0000 7fe7f98ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562e3ece2930 con 0x562e3ed6c1a0 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.291+0000 7fe7f98ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562e3ecd1e30 con 0x562e3ed6b880 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.291+0000 7fe7f98ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562e3ecd5e70 con 0x562e3ed78130 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7f528d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed6c1a0 0x562e3ed77bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7f528d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed6c1a0 0x562e3ed77bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60228/0 (socket says 172.21.15.44:60228) 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7f528d640 1 -- 172.21.15.44:0/801603120 learned_addr learned my addr 172.21.15.44:0/801603120 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7f4a8c640 1 --2- 172.21.15.44:0/801603120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed78130 0x562e3ed7a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7f4a8c640 1 --2- 172.21.15.44:0/801603120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6b880 0x562e3ed6bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7f528d640 1 -- 172.21.15.44:0/801603120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed78130 msgr2=0x562e3ed7a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:20.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7f528d640 1 --2- 172.21.15.44:0/801603120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed78130 0x562e3ed7a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7f528d640 1 -- 172.21.15.44:0/801603120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6b880 msgr2=0x562e3ed6bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:20.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7f528d640 1 --2- 172.21.15.44:0/801603120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6b880 0x562e3ed6bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7f528d640 1 -- 172.21.15.44:0/801603120 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e3ea31650 con 0x562e3ed6c1a0 2024-08-20T03:03:20.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7f528d640 1 --2- 172.21.15.44:0/801603120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed6c1a0 0x562e3ed77bf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009d50 tx=0x7fe7ec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6c2d83dc92212d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:20.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7e67fc640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe7dc0b3dd0 0x7fe7dc0b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7db7fe640 1 -- 172.21.15.44:0/801603120 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec01e070 con 0x562e3ed6c1a0 2024-08-20T03:03:20.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7f4a8c640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe7dc0b3dd0 0x7fe7dc0b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7db7fe640 1 -- 172.21.15.44:0/801603120 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7ec019ae0 con 0x562e3ed6c1a0 2024-08-20T03:03:20.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7db7fe640 1 -- 172.21.15.44:0/801603120 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec00ea80 con 0x562e3ed6c1a0 2024-08-20T03:03:20.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.292+0000 7fe7e67fc640 1 -- 172.21.15.44:0/2021990270 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe7e8105390 con 0x7fe7f010ce40 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.293+0000 7fe7f98ea740 1 -- 172.21.15.44:0/801603120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed6c1a0 msgr2=0x562e3ed77bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.293+0000 7fe7f4a8c640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe7dc0b3dd0 0x7fe7dc0b6290 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x562e3ed6b830 tx=0x7fe7e0013040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.293+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/801603120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed6c1a0 0x562e3ed77bf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009d50 tx=0x7fe7ec011810 comp rx=0 tx=0).stop 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.293+0000 7fe7f98ea740 1 -- 172.21.15.44:0/801603120 shutdown_connections 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.293+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/801603120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed78130 0x562e3ed7a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.293+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/801603120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed6c1a0 0x562e3ed77bf0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.293+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/801603120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6b880 0x562e3ed6bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.293+0000 7fe7f98ea740 1 -- 172.21.15.44:0/801603120 >> 172.21.15.44:0/801603120 conn(0x562e3ed696b0 msgr2=0x562e3ed6a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.293+0000 7fe7f98ea740 1 -- 172.21.15.44:0/801603120 shutdown_connections 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.293+0000 7fe7f98ea740 1 -- 172.21.15.44:0/801603120 wait complete. 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f98ea740 1 -- start start 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f98ea740 1 Processor -- start 2024-08-20T03:03:20.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f98ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6b880 0x562e3ed75080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f98ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6c1a0 0x562e3ed755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f98ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3ed75b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f98ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562e3eb34700 con 0x562e3ed78130 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f98ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562e3ec58810 con 0x562e3ed6b880 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f98ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562e3ed7ade0 con 0x562e3ed6c1a0 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f98ea740 10 client.0 _ll_register_callbacks cb 0x562e3eccb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f5a8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3ed75b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f5a8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3ed75b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60232/0 (socket says 172.21.15.44:60232) 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f4a8c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6c1a0 0x562e3ed755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.294+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/1059413640 learned_addr learned my addr 172.21.15.44:0/1059413640 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7f528d640 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6b880 0x562e3ed75080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/1059413640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6c1a0 msgr2=0x562e3ed755c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:20.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7f5a8e640 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6c1a0 0x562e3ed755c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/1059413640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6b880 msgr2=0x562e3ed75080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7f5a8e640 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6b880 0x562e3ed75080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/1059413640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e3ed84170 con 0x562e3ed78130 2024-08-20T03:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7f4a8c640 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6c1a0 0x562e3ed755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7f5a8e640 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3ed75b00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80097b0 tx=0x7fe7e80cb7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b4c00aac87efdc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7d8ff9640 1 -- 172.21.15.44:0/1059413640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80cbfb0 con 0x562e3ed78130 2024-08-20T03:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7d8ff9640 1 -- 172.21.15.44:0/1059413640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe7e80c8240 con 0x562e3ed78130 2024-08-20T03:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7f98ea740 1 -- 172.21.15.44:0/1059413640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e3ec81150 con 0x562e3ed78130 2024-08-20T03:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7d8ff9640 1 -- 172.21.15.44:0/1059413640 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7e80d05b0 con 0x562e3ed78130 2024-08-20T03:03:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7f98ea740 10 client.15855 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.295+0000 7fe7f98ea740 1 -- 172.21.15.44:0/1059413640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e3ed86b20 con 0x562e3ed78130 2024-08-20T03:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.296+0000 7fe7d8ff9640 1 -- 172.21.15.44:0/1059413640 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562e3ec81150 con 0x562e3ed78130 2024-08-20T03:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.296+0000 7fe7d8ff9640 1 -- 172.21.15.44:0/1059413640 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe7e80d0070 con 0x562e3ed78130 2024-08-20T03:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.296+0000 7fe7f98ea740 10 client.15855 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.296+0000 7fe7f98ea740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.296+0000 7fe7f98ea740 20 client.15855 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.296+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562e3ed88070 0x562e3ed8a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.296+0000 7fe7f98ea740 4 client.15855 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.296+0000 7fe7f98ea740 1 -- 172.21.15.44:0/1059413640 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e3ece4880 con 0x562e3ed88070 2024-08-20T03:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.296+0000 7fe7f528d640 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562e3ed88070 0x562e3ed8a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7f528d640 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562e3ed88070 0x562e3ed8a4a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60ee0cf35571c2e9 server_cookie=ccfdbe5919c3f37c in_seq=0 out_seq=0 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7d8ff9640 10 client.15855 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7d8ff9640 1 -- 172.21.15.44:0/1059413640 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e3ece4880 con 0x562e3ed88070 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7d8ff9640 10 client.15855 handle_command_reply: tid=0 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7f98ea740 1 client.15855 shutdown 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7f98ea740 1 -- 172.21.15.44:0/1059413640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 msgr2=0x562e3ed75b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3ed75b00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80097b0 tx=0x7fe7e80cb7a0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7f98ea740 1 -- 172.21.15.44:0/1059413640 shutdown_connections 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562e3ed88070 0x562e3ed8a4a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3ed75b00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6c1a0 0x562e3ed755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/1059413640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6b880 0x562e3ed75080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7d8ff9640 0 client.15855 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.297+0000 7fe7f98ea740 1 -- 172.21.15.44:0/1059413640 >> 172.21.15.44:0/1059413640 conn(0x562e3ed696b0 msgr2=0x562e3ed6f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.298+0000 7fe7f98ea740 1 -- 172.21.15.44:0/1059413640 shutdown_connections 2024-08-20T03:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.298+0000 7fe7f98ea740 1 -- 172.21.15.44:0/1059413640 wait complete. 2024-08-20T03:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.298+0000 7fe7f98ea740 20 client.15855 trim_cache size 0 max 16384 2024-08-20T03:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.311+0000 7fe7f98ea740 1 Processor -- start 2024-08-20T03:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.311+0000 7fe7f98ea740 1 -- start start 2024-08-20T03:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.311+0000 7fe7f98ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6b880 0x562e3ed6fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.311+0000 7fe7f98ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6c1a0 0x562e3ed76dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.311+0000 7fe7f98ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3ed75580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.311+0000 7fe7f98ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01b04c0 con 0x562e3ed78130 2024-08-20T03:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.311+0000 7fe7f98ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01b2000 con 0x562e3ed6c1a0 2024-08-20T03:03:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.311+0000 7fe7f98ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01b0b90 con 0x562e3ed6b880 2024-08-20T03:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7f5a8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3ed75580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7f5a8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3ed75580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60236/0 (socket says 172.21.15.44:60236) 2024-08-20T03:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/550574031 learned_addr learned my addr 172.21.15.44:0/550574031 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7f528d640 1 --2- 172.21.15.44:0/550574031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6b880 0x562e3ed6fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7f4a8c640 1 --2- 172.21.15.44:0/550574031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6c1a0 0x562e3ed76dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/550574031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6b880 msgr2=0x562e3ed6fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7f5a8e640 1 --2- 172.21.15.44:0/550574031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6b880 0x562e3ed6fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/550574031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6c1a0 msgr2=0x562e3ed76dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7f5a8e640 1 --2- 172.21.15.44:0/550574031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6c1a0 0x562e3ed76dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/550574031 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e3ed67af0 con 0x562e3ed78130 2024-08-20T03:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7f4a8c640 1 --2- 172.21.15.44:0/550574031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6c1a0 0x562e3ed76dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:20.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7f5a8e640 1 --2- 172.21.15.44:0/550574031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3ed75580 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80c8030 tx=0x7fe7e80cbd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc64d99f50a997c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7daffd640 1 -- 172.21.15.44:0/550574031 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80cbfb0 con 0x562e3ed78130 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7daffd640 1 -- 172.21.15.44:0/550574031 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e8036040 con 0x562e3ed78130 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.312+0000 7fe7daffd640 1 -- 172.21.15.44:0/550574031 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80d4bd0 con 0x562e3ed78130 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.313+0000 7fe7f98ea740 1 -- 172.21.15.44:0/550574031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 msgr2=0x562e3ed75580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.313+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/550574031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3ed75580 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80c8030 tx=0x7fe7e80cbd80 comp rx=0 tx=0).stop 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.313+0000 7fe7f98ea740 1 -- 172.21.15.44:0/550574031 shutdown_connections 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.313+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/550574031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3ed75580 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.313+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/550574031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6c1a0 0x562e3ed76dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.313+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/550574031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6b880 0x562e3ed6fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.313+0000 7fe7f98ea740 1 -- 172.21.15.44:0/550574031 >> 172.21.15.44:0/550574031 conn(0x562e3ed696b0 msgr2=0x562e3ed766b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.313+0000 7fe7f98ea740 1 -- 172.21.15.44:0/550574031 shutdown_connections 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.313+0000 7fe7f98ea740 1 -- 172.21.15.44:0/550574031 wait complete. 2024-08-20T03:03:20.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.313+0000 7fe7f98ea740 1 -- start start 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.313+0000 7fe7f98ea740 1 Processor -- start 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f98ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6b880 0x562e3ed7b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f98ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6c1a0 0x562e3ed7b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f98ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3edf0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f98ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01b1930 con 0x562e3ed78130 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f98ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01b1260 con 0x562e3ed6c1a0 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f98ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f01ae4a0 con 0x562e3ed6b880 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f98ea740 10 client.0 _ll_register_callbacks cb 0x7fe7e8102a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f5a8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3edf0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f5a8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3edf0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60238/0 (socket says 172.21.15.44:60238) 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/2760644596 learned_addr learned my addr 172.21.15.44:0/2760644596 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f4a8c640 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6c1a0 0x562e3ed7b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f528d640 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6b880 0x562e3ed7b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/2760644596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6b880 msgr2=0x562e3ed7b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f5a8e640 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6b880 0x562e3ed7b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/2760644596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6c1a0 msgr2=0x562e3ed7b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f5a8e640 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6c1a0 0x562e3ed7b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f5a8e640 1 -- 172.21.15.44:0/2760644596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e3ed6a130 con 0x562e3ed78130 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f4a8c640 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6c1a0 0x562e3ed7b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7f5a8e640 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3edf0d30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009a60 tx=0x7fe7e80cb9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9df7bf0772c45a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.314+0000 7fe7d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7d8ff9640 1 -- 172.21.15.44:0/2760644596 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80cbfb0 con 0x562e3ed78130 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7d8ff9640 1 -- 172.21.15.44:0/2760644596 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe7e80d4bd0 con 0x562e3ed78130 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7f98ea740 1 -- 172.21.15.44:0/2760644596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e8036040 con 0x562e3ed78130 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7d8ff9640 1 -- 172.21.15.44:0/2760644596 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7e80d5060 con 0x562e3ed78130 2024-08-20T03:03:20.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7f98ea740 10 client.15861 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7f98ea740 1 -- 172.21.15.44:0/2760644596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e3ed6a130 con 0x562e3ed78130 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7d8ff9640 1 -- 172.21.15.44:0/2760644596 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e8036040 con 0x562e3ed78130 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7d8ff9640 1 -- 172.21.15.44:0/2760644596 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe7e80cb210 con 0x562e3ed78130 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7f98ea740 10 client.15861 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7f98ea740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7f98ea740 20 client.15861 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562e3edf2530 0x562e3edf4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7f98ea740 4 client.15861 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7f98ea740 1 -- 172.21.15.44:0/2760644596 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e3ece0320 con 0x562e3edf2530 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.315+0000 7fe7f528d640 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562e3edf2530 0x562e3edf4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7f528d640 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562e3edf2530 0x562e3edf4950 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8f0f27e21e43a5 server_cookie=6f13fc08d5c12fc9 in_seq=0 out_seq=0 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7d8ff9640 10 client.15861 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7d8ff9640 1 -- 172.21.15.44:0/2760644596 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e3ece0320 con 0x562e3edf2530 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7d8ff9640 10 client.15861 handle_command_reply: tid=0 2024-08-20T03:03:20.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7f98ea740 1 client.15861 shutdown 2024-08-20T03:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7f98ea740 1 -- 172.21.15.44:0/2760644596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 msgr2=0x562e3edf0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3edf0d30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009a60 tx=0x7fe7e80cb9a0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7f98ea740 1 -- 172.21.15.44:0/2760644596 shutdown_connections 2024-08-20T03:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562e3edf2530 0x562e3edf4950 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e3ed78130 0x562e3edf0d30 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562e3ed6c1a0 0x562e3ed7b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7d8ff9640 0 client.15861 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:20.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7f98ea740 1 --2- 172.21.15.44:0/2760644596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562e3ed6b880 0x562e3ed7b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7f98ea740 1 -- 172.21.15.44:0/2760644596 >> 172.21.15.44:0/2760644596 conn(0x562e3ed696b0 msgr2=0x562e3ed734d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7f98ea740 1 -- 172.21.15.44:0/2760644596 shutdown_connections 2024-08-20T03:03:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.316+0000 7fe7f98ea740 1 -- 172.21.15.44:0/2760644596 wait complete. 2024-08-20T03:03:20.347 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:20.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f98ea740 20 client.15861 trim_cache size 0 max 16384 2024-08-20T03:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f74f1640 1 -- 172.21.15.44:0/2021990270 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe7dc0b3dd0 msgr2=0x7fe7dc0b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f74f1640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe7dc0b3dd0 0x7fe7dc0b6290 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x562e3ed6b830 tx=0x7fe7e0013040 comp rx=0 tx=0).stop 2024-08-20T03:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f74f1640 1 -- 172.21.15.44:0/2021990270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7f010ce40 msgr2=0x7fe7f0102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f74f1640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7f010ce40 0x7fe7f0102110 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8005cc0 tx=0x7fe7e8031700 comp rx=0 tx=0).stop 2024-08-20T03:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f74f1640 1 -- 172.21.15.44:0/2021990270 shutdown_connections 2024-08-20T03:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f74f1640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe7dc0b3dd0 0x7fe7dc0b6290 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f74f1640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7f010ce40 0x7fe7f0102110 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f74f1640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7f0104300 0x7fe7f0101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f74f1640 1 --2- 172.21.15.44:0/2021990270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7f01039e0 0x7fe7f0101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f74f1640 1 -- 172.21.15.44:0/2021990270 >> 172.21.15.44:0/2021990270 conn(0x7fe7f00fd440 msgr2=0x7fe7f00fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f74f1640 1 -- 172.21.15.44:0/2021990270 shutdown_connections 2024-08-20T03:03:20.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:20.317+0000 7fe7f74f1640 1 -- 172.21.15.44:0/2021990270 wait complete. 2024-08-20T03:03:20.394 INFO:tasks.workunit.client.0.smithi044.stdout:checking for ANSI C header files... yes 2024-08-20T03:03:20.436 INFO:tasks.workunit.client.0.smithi044.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-08-20T03:03:20.488 INFO:tasks.workunit.client.0.smithi044.stdout:checking for sys/types.h... yes 2024-08-20T03:03:20.536 INFO:tasks.workunit.client.0.smithi044.stdout:checking for sys/stat.h... yes 2024-08-20T03:03:20.590 INFO:tasks.workunit.client.0.smithi044.stdout:checking for stdlib.h... yes 2024-08-20T03:03:20.647 INFO:tasks.workunit.client.0.smithi044.stdout:checking for string.h... yes 2024-08-20T03:03:20.730 INFO:tasks.workunit.client.0.smithi044.stdout:checking for memory.h... yes 2024-08-20T03:03:20.784 INFO:tasks.workunit.client.0.smithi044.stdout:checking for strings.h... yes 2024-08-20T03:03:20.835 INFO:tasks.workunit.client.0.smithi044.stdout:checking for inttypes.h... yes 2024-08-20T03:03:20.883 INFO:tasks.workunit.client.0.smithi044.stdout:checking for stdint.h... yes 2024-08-20T03:03:20.940 INFO:tasks.workunit.client.0.smithi044.stdout:checking for unistd.h... yes 2024-08-20T03:03:20.951 INFO:tasks.workunit.client.0.smithi044.stdout:checking for memory.h... (cached) yes 2024-08-20T03:03:20.956 INFO:tasks.workunit.client.0.smithi044.stdout:checking for strings.h... (cached) yes 2024-08-20T03:03:20.960 INFO:tasks.workunit.client.0.smithi044.stdout:checking for unistd.h... (cached) yes 2024-08-20T03:03:21.024 INFO:tasks.workunit.client.0.smithi044.stdout:checking locale.h usability... yes 2024-08-20T03:03:21.043 INFO:tasks.workunit.client.0.smithi044.stdout:checking locale.h presence... yes 2024-08-20T03:03:21.043 INFO:tasks.workunit.client.0.smithi044.stdout:checking for locale.h... yes 2024-08-20T03:03:21.061 INFO:tasks.workunit.client.0.smithi044.stdout:checking for stdlib.h... (cached) yes 2024-08-20T03:03:21.063 INFO:tasks.workunit.client.0.smithi044.stdout:checking for string.h... (cached) yes 2024-08-20T03:03:21.205 INFO:tasks.workunit.client.0.smithi044.stdout:checking sys/socket.h usability... yes 2024-08-20T03:03:21.226 INFO:tasks.workunit.client.0.smithi044.stdout:checking sys/socket.h presence... yes 2024-08-20T03:03:21.227 INFO:tasks.workunit.client.0.smithi044.stdout:checking for sys/socket.h... yes 2024-08-20T03:03:21.283 INFO:tasks.workunit.client.0.smithi044.stdout:checking sys/time.h usability... yes 2024-08-20T03:03:21.302 INFO:tasks.workunit.client.0.smithi044.stdout:checking sys/time.h presence... yes 2024-08-20T03:03:21.302 INFO:tasks.workunit.client.0.smithi044.stdout:checking for sys/time.h... yes 2024-08-20T03:03:21.356 INFO:tasks.workunit.client.0.smithi044.stdout:checking fcntl.h usability... yes 2024-08-20T03:03:21.375 INFO:tasks.workunit.client.0.smithi044.stdout:checking fcntl.h presence... yes 2024-08-20T03:03:21.376 INFO:tasks.workunit.client.0.smithi044.stdout:checking for fcntl.h... yes 2024-08-20T03:03:21.434 INFO:tasks.workunit.client.0.smithi044.stdout:checking sys/fcntl.h usability... yes 2024-08-20T03:03:21.454 INFO:tasks.workunit.client.0.smithi044.stdout:checking sys/fcntl.h presence... yes 2024-08-20T03:03:21.455 INFO:tasks.workunit.client.0.smithi044.stdout:checking for sys/fcntl.h... yes 2024-08-20T03:03:21.545 INFO:tasks.workunit.client.0.smithi044.stdout:checking sys/param.h usability... yes 2024-08-20T03:03:21.568 INFO:tasks.workunit.client.0.smithi044.stdout:checking sys/param.h presence... yes 2024-08-20T03:03:21.568 INFO:tasks.workunit.client.0.smithi044.stdout:checking for sys/param.h... yes 2024-08-20T03:03:21.614 INFO:tasks.workunit.client.0.smithi044.stdout:checking ioctl.h usability... no 2024-08-20T03:03:21.634 INFO:tasks.workunit.client.0.smithi044.stdout:checking ioctl.h presence... no 2024-08-20T03:03:21.635 INFO:tasks.workunit.client.0.smithi044.stdout:checking for ioctl.h... no 2024-08-20T03:03:21.684 INFO:tasks.workunit.client.0.smithi044.stdout:checking sys/ioctl.h usability... yes 2024-08-20T03:03:21.703 INFO:tasks.workunit.client.0.smithi044.stdout:checking sys/ioctl.h presence... yes 2024-08-20T03:03:21.703 INFO:tasks.workunit.client.0.smithi044.stdout:checking for sys/ioctl.h... yes 2024-08-20T03:03:21.752 INFO:tasks.workunit.client.0.smithi044.stdout:checking return type of signal handlers... void 2024-08-20T03:03:21.795 INFO:tasks.workunit.client.0.smithi044.stdout:checking for size_t... yes 2024-08-20T03:03:21.832 INFO:tasks.workunit.client.0.smithi044.stdout:checking for dev_t... yes 2024-08-20T03:03:21.874 INFO:tasks.workunit.client.0.smithi044.stdout:checking for short... yes 2024-08-20T03:03:21.951 INFO:tasks.workunit.client.0.smithi044.stdout:checking size of short... 2 2024-08-20T03:03:21.994 INFO:tasks.workunit.client.0.smithi044.stdout:checking for int... yes 2024-08-20T03:03:22.063 INFO:tasks.workunit.client.0.smithi044.stdout:checking size of int... 4 2024-08-20T03:03:22.110 INFO:tasks.workunit.client.0.smithi044.stdout:checking for long... yes 2024-08-20T03:03:22.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:21 smithi165 ceph-mon[22357]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-08-20T03:03:22.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:21 smithi099 ceph-mon[21984]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-08-20T03:03:22.179 INFO:tasks.workunit.client.0.smithi044.stdout:checking size of long... 8 2024-08-20T03:03:22.190 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T03:03:22.191 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T03:03:22.213 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:21 smithi044 ceph-mon[18790]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-08-20T03:03:22.221 INFO:tasks.workunit.client.0.smithi044.stdout:checking for long long... yes 2024-08-20T03:03:22.291 INFO:tasks.workunit.client.0.smithi044.stdout:checking size of long long... 8 2024-08-20T03:03:22.333 INFO:tasks.workunit.client.0.smithi044.stdout:checking for ssize_t... yes 2024-08-20T03:03:22.370 INFO:tasks.workunit.client.0.smithi044.stdout:checking for sig_atomic_t... yes 2024-08-20T03:03:22.431 INFO:tasks.workunit.client.0.smithi044.stdout:checking whether gcc needs -traditional... no 2024-08-20T03:03:22.459 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:22.465 INFO:tasks.workunit.client.0.smithi044.stdout:checking for an ANSI C-conforming const... yes 2024-08-20T03:03:22.489 INFO:tasks.workunit.client.0.smithi044.stdout:checking for inline... inline 2024-08-20T03:03:22.555 INFO:tasks.workunit.client.0.smithi044.stdout:checking for working alloca.h... yes 2024-08-20T03:03:22.601 INFO:tasks.workunit.client.0.smithi044.stdout:checking for alloca... yes 2024-08-20T03:03:22.675 INFO:tasks.workunit.client.0.smithi044.stdout:checking for working memcmp... yes 2024-08-20T03:03:22.730 INFO:tasks.workunit.client.0.smithi044.stdout:checking for strftime... yes 2024-08-20T03:03:22.828 INFO:tasks.workunit.client.0.smithi044.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-08-20T03:03:22.885 INFO:tasks.workunit.client.0.smithi044.stdout:checking whether stat accepts an empty string... no 2024-08-20T03:03:22.950 INFO:tasks.workunit.client.0.smithi044.stdout:checking for vprintf... yes 2024-08-20T03:03:23.012 INFO:tasks.workunit.client.0.smithi044.stdout:checking for _doprnt... no 2024-08-20T03:03:23.035 INFO:tasks.workunit.client.0.smithi044.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-08-20T03:03:23.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.017+0000 7fd850d6e640 1 -- 172.21.15.44:0/2325968031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd84c101990 msgr2=0x7fd84c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:23.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.017+0000 7fd850d6e640 1 --2- 172.21.15.44:0/2325968031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd84c101990 0x7fd84c101d70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009900 tx=0x7fd83c02a410 comp rx=0 tx=0).stop 2024-08-20T03:03:23.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.017+0000 7fd850d6e640 1 -- 172.21.15.44:0/2325968031 shutdown_connections 2024-08-20T03:03:23.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.017+0000 7fd850d6e640 1 --2- 172.21.15.44:0/2325968031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd84c107d20 0x7fd84c10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.017+0000 7fd850d6e640 1 --2- 172.21.15.44:0/2325968031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd84c1073a0 0x7fd84c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.017+0000 7fd850d6e640 1 --2- 172.21.15.44:0/2325968031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd84c101990 0x7fd84c101d70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.017+0000 7fd850d6e640 1 -- 172.21.15.44:0/2325968031 >> 172.21.15.44:0/2325968031 conn(0x7fd84c0fd3e0 msgr2=0x7fd84c0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:23.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.017+0000 7fd850d6e640 1 -- 172.21.15.44:0/2325968031 shutdown_connections 2024-08-20T03:03:23.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.017+0000 7fd850d6e640 1 -- 172.21.15.44:0/2325968031 wait complete. 2024-08-20T03:03:23.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd850d6e640 1 Processor -- start 2024-08-20T03:03:23.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd850d6e640 1 -- start start 2024-08-20T03:03:23.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd850d6e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd84c101990 0x7fd84c199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd850d6e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd84c1073a0 0x7fd84c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd850d6e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd84c107d20 0x7fd84c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd850d6e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1119e0 con 0x7fd84c107d20 2024-08-20T03:03:23.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd850d6e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c111860 con 0x7fd84c1073a0 2024-08-20T03:03:23.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd850d6e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c111b60 con 0x7fd84c101990 2024-08-20T03:03:23.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd84ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd84c107d20 0x7fd84c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd84ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd84c107d20 0x7fd84c19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60262/0 (socket says 172.21.15.44:60262) 2024-08-20T03:03:23.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd84ad9d640 1 -- 172.21.15.44:0/2030381594 learned_addr learned my addr 172.21.15.44:0/2030381594 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd84a59c640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd84c101990 0x7fd84c199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd849d9b640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd84c1073a0 0x7fd84c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd84ad9d640 1 -- 172.21.15.44:0/2030381594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd84c101990 msgr2=0x7fd84c199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd84ad9d640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd84c101990 0x7fd84c199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd84ad9d640 1 -- 172.21.15.44:0/2030381594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd84c1073a0 msgr2=0x7fd84c19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd84ad9d640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd84c1073a0 0x7fd84c19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.018+0000 7fd84ad9d640 1 -- 172.21.15.44:0/2030381594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c1a5210 con 0x7fd84c107d20 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.019+0000 7fd84ad9d640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd84c107d20 0x7fd84c19a8d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd84400cad0 tx=0x7fd84400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.019+0000 7fd83b7fe640 1 -- 172.21.15.44:0/2030381594 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84400ae20 con 0x7fd84c107d20 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.019+0000 7fd850d6e640 1 -- 172.21.15.44:0/2030381594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84c1a5500 con 0x7fd84c107d20 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.019+0000 7fd83b7fe640 1 -- 172.21.15.44:0/2030381594 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd844005020 con 0x7fd84c107d20 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.019+0000 7fd83b7fe640 1 -- 172.21.15.44:0/2030381594 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84400f6b0 con 0x7fd84c107d20 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.019+0000 7fd850d6e640 1 -- 172.21.15.44:0/2030381594 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84c1a5a40 con 0x7fd84c107d20 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.022+0000 7fd83b7fe640 1 -- 172.21.15.44:0/2030381594 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7fd8440051d0 con 0x7fd84c107d20 2024-08-20T03:03:23.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.022+0000 7fd853167740 1 Processor -- start 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.022+0000 7fd853167740 1 -- start start 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.022+0000 7fd853167740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b42b3b0 0x55ee1b42b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.022+0000 7fd853167740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b437720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.022+0000 7fd853167740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 0x55ee1b43a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.022+0000 7fd853167740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ee1b3a1940 con 0x55ee1b437c60 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.022+0000 7fd853167740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ee1b390e40 con 0x55ee1b42b3b0 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.022+0000 7fd853167740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ee1b394e80 con 0x55ee1b42bcd0 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.022+0000 7fd849d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 0x55ee1b43a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.022+0000 7fd849d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 0x55ee1b43a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60278/0 (socket says 172.21.15.44:60278) 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.022+0000 7fd849d9b640 1 -- 172.21.15.44:0/2096645137 learned_addr learned my addr 172.21.15.44:0/2096645137 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd84a59c640 1 --2- 172.21.15.44:0/2096645137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b437720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd83b7fe640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd8340b3dd0 0x7fd8340b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd849d9b640 1 --2- 172.21.15.44:0/2096645137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b42b3b0 0x55ee1b42b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd84ad9d640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd8340b3dd0 0x7fd8340b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd849d9b640 1 -- 172.21.15.44:0/2096645137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 msgr2=0x55ee1b437720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd849d9b640 1 --2- 172.21.15.44:0/2096645137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b437720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd849d9b640 1 -- 172.21.15.44:0/2096645137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b42b3b0 msgr2=0x55ee1b42b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd849d9b640 1 --2- 172.21.15.44:0/2096645137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b42b3b0 0x55ee1b42b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd83b7fe640 1 -- 172.21.15.44:0/2030381594 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd8440e40d0 con 0x7fd84c107d20 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd849d9b640 1 -- 172.21.15.44:0/2096645137 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee1b32d520 con 0x55ee1b437c60 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd84a59c640 1 --2- 172.21.15.44:0/2096645137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b437720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd849d9b640 1 --2- 172.21.15.44:0/2096645137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 0x55ee1b43a050 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd840013630 tx=0x7fd84000c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13ab9cfa813e5f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd84ad9d640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd8340b3dd0 0x7fd8340b6290 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd8440e1830 tx=0x7fd844092040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd838ff9640 1 -- 172.21.15.44:0/2096645137 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840019030 con 0x55ee1b437c60 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd838ff9640 1 -- 172.21.15.44:0/2096645137 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd840011a50 con 0x55ee1b437c60 2024-08-20T03:03:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.023+0000 7fd838ff9640 1 -- 172.21.15.44:0/2096645137 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84000fa60 con 0x55ee1b437c60 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.024+0000 7fd853167740 1 -- 172.21.15.44:0/2096645137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 msgr2=0x55ee1b43a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.024+0000 7fd853167740 1 --2- 172.21.15.44:0/2096645137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 0x55ee1b43a050 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd840013630 tx=0x7fd84000c6b0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.024+0000 7fd853167740 1 -- 172.21.15.44:0/2096645137 shutdown_connections 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.024+0000 7fd853167740 1 --2- 172.21.15.44:0/2096645137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 0x55ee1b43a050 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.024+0000 7fd853167740 1 --2- 172.21.15.44:0/2096645137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b437720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.024+0000 7fd853167740 1 --2- 172.21.15.44:0/2096645137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b42b3b0 0x55ee1b42b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.024+0000 7fd853167740 1 -- 172.21.15.44:0/2096645137 >> 172.21.15.44:0/2096645137 conn(0x55ee1b4291e0 msgr2=0x55ee1b42a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.024+0000 7fd853167740 1 -- 172.21.15.44:0/2096645137 shutdown_connections 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.024+0000 7fd853167740 1 -- 172.21.15.44:0/2096645137 wait complete. 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd853167740 1 -- start start 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd853167740 1 Processor -- start 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd853167740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b42b3b0 0x55ee1b434c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd853167740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b435190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd853167740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 0x55ee1b4356d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd853167740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ee1b1f3890 con 0x55ee1b437c60 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd853167740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ee1b317700 con 0x55ee1b42b3b0 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd853167740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ee1b43a9b0 con 0x55ee1b42bcd0 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd853167740 10 client.0 _ll_register_callbacks cb 0x55ee1b38c6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd849d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 0x55ee1b4356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd849d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 0x55ee1b4356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60294/0 (socket says 172.21.15.44:60294) 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd849d9b640 1 -- 172.21.15.44:0/3635818822 learned_addr learned my addr 172.21.15.44:0/3635818822 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd84a59c640 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b435190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.025+0000 7fd849d9b640 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b42b3b0 0x55ee1b434c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd849d9b640 1 -- 172.21.15.44:0/3635818822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 msgr2=0x55ee1b435190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd849d9b640 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b435190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd849d9b640 1 -- 172.21.15.44:0/3635818822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b42b3b0 msgr2=0x55ee1b434c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd849d9b640 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b42b3b0 0x55ee1b434c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd849d9b640 1 -- 172.21.15.44:0/3635818822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee1b0f0930 con 0x55ee1b437c60 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd84a59c640 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b435190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd849d9b640 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 0x55ee1b4356d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd840013f80 tx=0x7fd840015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9600b111b6d1ca9d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd831ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd831ffb640 1 -- 172.21.15.44:0/3635818822 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840019030 con 0x55ee1b437c60 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd831ffb640 1 -- 172.21.15.44:0/3635818822 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd840027040 con 0x55ee1b437c60 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd853167740 1 -- 172.21.15.44:0/3635818822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd840011a50 con 0x55ee1b437c60 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd831ffb640 1 -- 172.21.15.44:0/3635818822 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd84000f7e0 con 0x55ee1b437c60 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd853167740 10 client.15873 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd853167740 1 -- 172.21.15.44:0/3635818822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee1b446590 con 0x55ee1b437c60 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.026+0000 7fd831ffb640 1 -- 172.21.15.44:0/3635818822 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd840011a50 con 0x55ee1b437c60 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.027+0000 7fd831ffb640 1 -- 172.21.15.44:0/3635818822 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd840034050 con 0x55ee1b437c60 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.027+0000 7fd853167740 10 client.15873 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:23.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.027+0000 7fd853167740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.027+0000 7fd853167740 20 client.15873 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.027+0000 7fd853167740 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55ee1b447940 0x55ee1b449d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.027+0000 7fd853167740 4 client.15873 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.027+0000 7fd853167740 1 -- 172.21.15.44:0/3635818822 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee1b3a3890 con 0x55ee1b447940 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.027+0000 7fd84a59c640 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55ee1b447940 0x55ee1b449d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.027+0000 7fd84a59c640 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55ee1b447940 0x55ee1b449d30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe8c773975cd5a server_cookie=656f4157409a1870 in_seq=0 out_seq=0 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.027+0000 7fd831ffb640 10 client.15873 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd831ffb640 1 -- 172.21.15.44:0/3635818822 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee1b3a3890 con 0x55ee1b447940 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd831ffb640 10 client.15873 handle_command_reply: tid=0 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd853167740 1 client.15873 shutdown 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd853167740 1 -- 172.21.15.44:0/3635818822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 msgr2=0x55ee1b4356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd853167740 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 0x55ee1b4356d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd840013f80 tx=0x7fd840015040 comp rx=0 tx=0).stop 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd853167740 1 -- 172.21.15.44:0/3635818822 shutdown_connections 2024-08-20T03:03:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd853167740 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55ee1b447940 0x55ee1b449d30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd853167740 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b437c60 0x55ee1b4356d0 secure :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd840013f80 tx=0x7fd840015040 comp rx=0 tx=0).stop 2024-08-20T03:03:23.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd831ffb640 0 client.15873 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:23.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd853167740 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b435190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd853167740 1 --2- 172.21.15.44:0/3635818822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b42b3b0 0x55ee1b434c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd853167740 1 -- 172.21.15.44:0/3635818822 >> 172.21.15.44:0/3635818822 conn(0x55ee1b4291e0 msgr2=0x55ee1b42ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:23.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd853167740 1 -- 172.21.15.44:0/3635818822 shutdown_connections 2024-08-20T03:03:23.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd853167740 1 -- 172.21.15.44:0/3635818822 wait complete. 2024-08-20T03:03:23.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.028+0000 7fd853167740 20 client.15873 trim_cache size 0 max 16384 2024-08-20T03:03:23.054 INFO:tasks.workunit.client.0.smithi044.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-08-20T03:03:23.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.046+0000 7fd853167740 1 Processor -- start 2024-08-20T03:03:23.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.046+0000 7fd853167740 1 -- start start 2024-08-20T03:03:23.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd853167740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42b3b0 0x55ee1b42f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd853167740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42bcd0 0x55ee1b44f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd853167740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b44f9b0 0x55ee1b431e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd853167740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1ae2e0 con 0x55ee1b42bcd0 2024-08-20T03:03:23.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd853167740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1afe20 con 0x55ee1b44f9b0 2024-08-20T03:03:23.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd853167740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1ae9b0 con 0x55ee1b42b3b0 2024-08-20T03:03:23.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42bcd0 0x55ee1b44f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd84a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42bcd0 0x55ee1b44f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60298/0 (socket says 172.21.15.44:60298) 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd84a59c640 1 -- 172.21.15.44:0/323924092 learned_addr learned my addr 172.21.15.44:0/323924092 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd849d9b640 1 --2- 172.21.15.44:0/323924092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42b3b0 0x55ee1b42f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd849d9b640 1 --2- 172.21.15.44:0/323924092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b44f9b0 0x55ee1b431e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd84a59c640 1 -- 172.21.15.44:0/323924092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42b3b0 msgr2=0x55ee1b42f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd84a59c640 1 --2- 172.21.15.44:0/323924092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42b3b0 0x55ee1b42f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd84a59c640 1 -- 172.21.15.44:0/323924092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b44f9b0 msgr2=0x55ee1b431e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd84a59c640 1 --2- 172.21.15.44:0/323924092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b44f9b0 0x55ee1b431e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd84a59c640 1 -- 172.21.15.44:0/323924092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee1b435af0 con 0x55ee1b42bcd0 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd849d9b640 1 --2- 172.21.15.44:0/323924092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b44f9b0 0x55ee1b431e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd84a59c640 1 --2- 172.21.15.44:0/323924092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42bcd0 0x55ee1b44f470 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009900 tx=0x7fd83c005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad898ec30dd3cc90 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.047+0000 7fd833fff640 1 -- 172.21.15.44:0/323924092 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c035040 con 0x55ee1b42bcd0 2024-08-20T03:03:23.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.048+0000 7fd833fff640 1 -- 172.21.15.44:0/323924092 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83c04b070 con 0x55ee1b42bcd0 2024-08-20T03:03:23.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.048+0000 7fd833fff640 1 -- 172.21.15.44:0/323924092 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c0049b0 con 0x55ee1b42bcd0 2024-08-20T03:03:23.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.048+0000 7fd853167740 1 -- 172.21.15.44:0/323924092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42bcd0 msgr2=0x55ee1b44f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:23.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.048+0000 7fd853167740 1 --2- 172.21.15.44:0/323924092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42bcd0 0x55ee1b44f470 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009900 tx=0x7fd83c005e60 comp rx=0 tx=0).stop 2024-08-20T03:03:23.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.048+0000 7fd853167740 1 -- 172.21.15.44:0/323924092 shutdown_connections 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.048+0000 7fd853167740 1 --2- 172.21.15.44:0/323924092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b44f9b0 0x55ee1b431e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.048+0000 7fd853167740 1 --2- 172.21.15.44:0/323924092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42bcd0 0x55ee1b44f470 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.048+0000 7fd853167740 1 --2- 172.21.15.44:0/323924092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42b3b0 0x55ee1b42f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.048+0000 7fd853167740 1 -- 172.21.15.44:0/323924092 >> 172.21.15.44:0/323924092 conn(0x55ee1b4291e0 msgr2=0x55ee1b4353e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.048+0000 7fd853167740 1 -- 172.21.15.44:0/323924092 shutdown_connections 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.048+0000 7fd853167740 1 -- 172.21.15.44:0/323924092 wait complete. 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd853167740 1 -- start start 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd853167740 1 Processor -- start 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd853167740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42b3b0 0x55ee1b440520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd853167740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b43ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd853167740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b44f9b0 0x55ee1b43c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd853167740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1af750 con 0x55ee1b42b3b0 2024-08-20T03:03:23.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd853167740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1af080 con 0x55ee1b44f9b0 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd853167740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c1ac2c0 con 0x55ee1b42bcd0 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd853167740 10 client.0 _ll_register_callbacks cb 0x7fd83c005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd849d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42b3b0 0x55ee1b440520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd849d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42b3b0 0x55ee1b440520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60302/0 (socket says 172.21.15.44:60302) 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd849d9b640 1 -- 172.21.15.44:0/3069789054 learned_addr learned my addr 172.21.15.44:0/3069789054 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd84a59c640 1 --2- 172.21.15.44:0/3069789054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b43ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd849d9b640 1 -- 172.21.15.44:0/3069789054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 msgr2=0x55ee1b43ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd849d9b640 1 --2- 172.21.15.44:0/3069789054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b43ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd849d9b640 1 -- 172.21.15.44:0/3069789054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b44f9b0 msgr2=0x55ee1b43c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd849d9b640 1 --2- 172.21.15.44:0/3069789054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b44f9b0 0x55ee1b43c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.049+0000 7fd849d9b640 1 -- 172.21.15.44:0/3069789054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee1b4b1440 con 0x55ee1b42b3b0 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.050+0000 7fd849d9b640 1 --2- 172.21.15.44:0/3069789054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42b3b0 0x55ee1b440520 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd840013fd0 tx=0x7fd84000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ff3ec9d6ebecdf server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:23.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.050+0000 7fd831ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.050+0000 7fd831ffb640 1 -- 172.21.15.44:0/3069789054 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840005b90 con 0x55ee1b42b3b0 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.050+0000 7fd831ffb640 1 -- 172.21.15.44:0/3069789054 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fd840004040 con 0x55ee1b42b3b0 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.050+0000 7fd853167740 1 -- 172.21.15.44:0/3069789054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd83c04b070 con 0x55ee1b42b3b0 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.050+0000 7fd831ffb640 1 -- 172.21.15.44:0/3069789054 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd84000f500 con 0x55ee1b42b3b0 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.050+0000 7fd853167740 10 client.15879 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.050+0000 7fd853167740 1 -- 172.21.15.44:0/3069789054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee1b4b2730 con 0x55ee1b42b3b0 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.050+0000 7fd831ffb640 1 -- 172.21.15.44:0/3069789054 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83c04b070 con 0x55ee1b42b3b0 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.050+0000 7fd831ffb640 1 -- 172.21.15.44:0/3069789054 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd840048020 con 0x55ee1b42b3b0 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.051+0000 7fd853167740 10 client.15879 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.051+0000 7fd853167740 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.051+0000 7fd853167740 20 client.15879 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.051+0000 7fd853167740 1 --2- 172.21.15.44:0/3069789054 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55ee1b4b2bb0 0x55ee1b4b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.051+0000 7fd853167740 4 client.15879 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.051+0000 7fd853167740 1 -- 172.21.15.44:0/3069789054 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ee1b39f330 con 0x55ee1b4b2bb0 2024-08-20T03:03:23.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.051+0000 7fd84a59c640 1 --2- 172.21.15.44:0/3069789054 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55ee1b4b2bb0 0x55ee1b4b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.051+0000 7fd84a59c640 1 --2- 172.21.15.44:0/3069789054 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55ee1b4b2bb0 0x55ee1b4b4fa0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b663c283c61c38 server_cookie=8622aada83768e7 in_seq=0 out_seq=0 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.051+0000 7fd831ffb640 10 client.15879 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.052+0000 7fd831ffb640 1 -- 172.21.15.44:0/3069789054 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ee1b39f330 con 0x55ee1b4b2bb0 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.052+0000 7fd831ffb640 10 client.15879 handle_command_reply: tid=0 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.052+0000 7fd853167740 1 client.15879 shutdown 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.052+0000 7fd853167740 1 -- 172.21.15.44:0/3069789054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42b3b0 msgr2=0x55ee1b440520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.052+0000 7fd853167740 1 --2- 172.21.15.44:0/3069789054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42b3b0 0x55ee1b440520 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd840013fd0 tx=0x7fd84000cae0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.052+0000 7fd853167740 1 -- 172.21.15.44:0/3069789054 shutdown_connections 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.052+0000 7fd853167740 1 --2- 172.21.15.44:0/3069789054 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55ee1b4b2bb0 0x55ee1b4b4fa0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.052+0000 7fd853167740 1 --2- 172.21.15.44:0/3069789054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ee1b44f9b0 0x55ee1b43c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.052+0000 7fd853167740 1 --2- 172.21.15.44:0/3069789054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ee1b42bcd0 0x55ee1b43ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.052+0000 7fd853167740 1 --2- 172.21.15.44:0/3069789054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee1b42b3b0 0x55ee1b440520 secure :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd840013fd0 tx=0x7fd84000cae0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.052+0000 7fd831ffb640 0 client.15879 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:23.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.052+0000 7fd853167740 1 -- 172.21.15.44:0/3069789054 >> 172.21.15.44:0/3069789054 conn(0x55ee1b4291e0 msgr2=0x55ee1b427ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:23.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd853167740 1 -- 172.21.15.44:0/3069789054 shutdown_connections 2024-08-20T03:03:23.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd853167740 1 -- 172.21.15.44:0/3069789054 wait complete. 2024-08-20T03:03:23.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd853167740 20 client.15879 trim_cache size 0 max 16384 2024-08-20T03:03:23.065 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:03:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-08-20T03:03:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "434a6185-30aa-41a4-a5d0-f5ef27a93b88", 2024-08-20T03:03:23.065 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-08-20T03:03:23.065 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:03:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd850d6e640 1 -- 172.21.15.44:0/2030381594 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd8340b3dd0 msgr2=0x7fd8340b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:23.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd850d6e640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd8340b3dd0 0x7fd8340b6290 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd8440e1830 tx=0x7fd844092040 comp rx=0 tx=0).stop 2024-08-20T03:03:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd850d6e640 1 -- 172.21.15.44:0/2030381594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd84c107d20 msgr2=0x7fd84c19a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd850d6e640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd84c107d20 0x7fd84c19a8d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd84400cad0 tx=0x7fd84400cf90 comp rx=0 tx=0).stop 2024-08-20T03:03:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd850d6e640 1 -- 172.21.15.44:0/2030381594 shutdown_connections 2024-08-20T03:03:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd850d6e640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd8340b3dd0 0x7fd8340b6290 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd850d6e640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd84c107d20 0x7fd84c19a8d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd850d6e640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd84c1073a0 0x7fd84c19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd850d6e640 1 --2- 172.21.15.44:0/2030381594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd84c101990 0x7fd84c199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.053+0000 7fd850d6e640 1 -- 172.21.15.44:0/2030381594 >> 172.21.15.44:0/2030381594 conn(0x7fd84c0fd3e0 msgr2=0x7fd84c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.054+0000 7fd850d6e640 1 -- 172.21.15.44:0/2030381594 shutdown_connections 2024-08-20T03:03:23.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:23.054+0000 7fd850d6e640 1 -- 172.21.15.44:0/2030381594 wait complete. 2024-08-20T03:03:23.097 INFO:tasks.workunit.client.0.smithi044.stdout:checking for the pthreads library -lpthreads... no 2024-08-20T03:03:23.179 INFO:tasks.workunit.client.0.smithi044.stdout:checking whether pthreads work without any flags... yes 2024-08-20T03:03:23.246 INFO:tasks.workunit.client.0.smithi044.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-08-20T03:03:23.246 INFO:tasks.workunit.client.0.smithi044.stdout:checking if more special flags are required for pthreads... no 2024-08-20T03:03:23.246 INFO:tasks.workunit.client.0.smithi044.stdout:checking for cc_r... gcc 2024-08-20T03:03:23.330 INFO:tasks.workunit.client.0.smithi044.stdout:checking for setrlimit... yes 2024-08-20T03:03:23.402 INFO:tasks.workunit.client.0.smithi044.stdout:checking for setproctitle... no 2024-08-20T03:03:23.470 INFO:tasks.workunit.client.0.smithi044.stdout:checking for getopt_long... yes 2024-08-20T03:03:23.533 INFO:tasks.workunit.client.0.smithi044.stdout:checking for strtoull... yes 2024-08-20T03:03:23.591 INFO:tasks.workunit.client.0.smithi044.stdout:checking for strtoq... yes 2024-08-20T03:03:23.657 INFO:tasks.workunit.client.0.smithi044.stdout:checking for snprintf... yes 2024-08-20T03:03:23.727 INFO:tasks.workunit.client.0.smithi044.stdout:checking for vsnprintf... yes 2024-08-20T03:03:23.805 INFO:tasks.workunit.client.0.smithi044.stdout:checking for vfprintf... yes 2024-08-20T03:03:23.865 INFO:tasks.workunit.client.0.smithi044.stdout:checking for gethostname... yes 2024-08-20T03:03:23.922 INFO:tasks.workunit.client.0.smithi044.stdout:checking for setlocale... yes 2024-08-20T03:03:23.979 INFO:tasks.workunit.client.0.smithi044.stdout:checking for sync... yes 2024-08-20T03:03:24.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:23 smithi044 ceph-mon[18790]: pgmap v347: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 77 KiB/s wr, 15 op/s 2024-08-20T03:03:24.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:23 smithi044 ceph-mon[18790]: scrub queued for path: / 2024-08-20T03:03:24.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:23 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:24.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:23 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:03:24.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:23 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:24.061 INFO:tasks.workunit.client.0.smithi044.stdout:checking whether snprintf is C99 conformant... done 2024-08-20T03:03:24.072 INFO:tasks.workunit.client.0.smithi044.stdout:configure: creating ./config.status 2024-08-20T03:03:24.101 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:03:24.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:23 smithi165 ceph-mon[22357]: pgmap v347: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 77 KiB/s wr, 15 op/s 2024-08-20T03:03:24.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:23 smithi165 ceph-mon[22357]: scrub queued for path: / 2024-08-20T03:03:24.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:23 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:24.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:23 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:03:24.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:23 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:24.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:23 smithi099 ceph-mon[21984]: pgmap v347: 97 pgs: 97 active+clean; 2.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 77 KiB/s wr, 15 op/s 2024-08-20T03:03:24.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:23 smithi099 ceph-mon[21984]: scrub queued for path: / 2024-08-20T03:03:24.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:23 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:24.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:23 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:03:24.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:23 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:24.217 INFO:tasks.workunit.client.0.smithi044.stdout:config.status: creating Makefile 2024-08-20T03:03:24.233 INFO:tasks.workunit.client.0.smithi044.stdout:config.status: creating src/Makefile 2024-08-20T03:03:24.251 INFO:tasks.workunit.client.0.smithi044.stdout:config.status: creating man/Makefile 2024-08-20T03:03:24.262 INFO:tasks.workunit.client.0.smithi044.stdout:config.status: creating config.h 2024-08-20T03:03:24.317 INFO:tasks.workunit.client.0.smithi044.stdout:config.status: executing depfiles commands 2024-08-20T03:03:24.410 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:24.447 INFO:tasks.workunit.client.0.smithi044.stderr:+ make 2024-08-20T03:03:24.457 INFO:tasks.workunit.client.0.smithi044.stdout:make all-recursive 2024-08-20T03:03:24.459 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-20T03:03:24.462 INFO:tasks.workunit.client.0.smithi044.stdout:Making all in src 2024-08-20T03:03:24.465 INFO:tasks.workunit.client.0.smithi044.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-08-20T03:03:24.465 INFO:tasks.workunit.client.0.smithi044.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-08-20T03:03:24.465 INFO:tasks.workunit.client.0.smithi044.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-08-20T03:03:24.620 INFO:tasks.workunit.client.0.smithi044.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-08-20T03:03:24.620 INFO:tasks.workunit.client.0.smithi044.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-08-20T03:03:24.691 INFO:tasks.workunit.client.0.smithi044.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-08-20T03:03:24.691 INFO:tasks.workunit.client.0.smithi044.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-08-20T03:03:24.750 INFO:tasks.workunit.client.0.smithi044.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-08-20T03:03:24.750 INFO:tasks.workunit.client.0.smithi044.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-08-20T03:03:24.803 INFO:tasks.workunit.client.0.smithi044.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-08-20T03:03:24.803 INFO:tasks.workunit.client.0.smithi044.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-08-20T03:03:24.854 INFO:tasks.workunit.client.0.smithi044.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-08-20T03:03:24.854 INFO:tasks.workunit.client.0.smithi044.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-08-20T03:03:24.915 INFO:tasks.workunit.client.0.smithi044.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-08-20T03:03:24.915 INFO:tasks.workunit.client.0.smithi044.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-08-20T03:03:24.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.927+0000 7f280df21640 1 -- 172.21.15.44:0/2123764178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f280810cde0 msgr2=0x7f280810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:24.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.927+0000 7f280df21640 1 --2- 172.21.15.44:0/2123764178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f280810cde0 0x7f280810f250 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2800009ca0 tx=0x7f280002a8c0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.927+0000 7f280df21640 1 -- 172.21.15.44:0/2123764178 shutdown_connections 2024-08-20T03:03:24.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.927+0000 7f280df21640 1 --2- 172.21.15.44:0/2123764178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f280810cde0 0x7f280810f250 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.927+0000 7f280df21640 1 --2- 172.21.15.44:0/2123764178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2808104230 0x7f280810c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.927+0000 7f280df21640 1 --2- 172.21.15.44:0/2123764178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2808103830 0x7f2808103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.927+0000 7f280df21640 1 -- 172.21.15.44:0/2123764178 >> 172.21.15.44:0/2123764178 conn(0x7f28080fd400 msgr2=0x7f28080ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.927+0000 7f280df21640 1 -- 172.21.15.44:0/2123764178 shutdown_connections 2024-08-20T03:03:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.927+0000 7f280df21640 1 -- 172.21.15.44:0/2123764178 wait complete. 2024-08-20T03:03:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.928+0000 7f280df21640 1 Processor -- start 2024-08-20T03:03:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.928+0000 7f280df21640 1 -- start start 2024-08-20T03:03:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.928+0000 7f280df21640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2808103830 0x7f28081016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.928+0000 7f280df21640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2808104230 0x7f2808101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.928+0000 7f280df21640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f280810cde0 0x7f2808102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.928+0000 7f280df21640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2808112ab0 con 0x7f2808104230 2024-08-20T03:03:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.928+0000 7f280df21640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2808112930 con 0x7f280810cde0 2024-08-20T03:03:24.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.928+0000 7f280df21640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2808112c30 con 0x7f2808103830 2024-08-20T03:03:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2808104230 0x7f2808101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2808104230 0x7f2808101bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60328/0 (socket says 172.21.15.44:60328) 2024-08-20T03:03:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2806ffd640 1 -- 172.21.15.44:0/359804024 learned_addr learned my addr 172.21.15.44:0/359804024 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2807fff640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f280810cde0 0x7f2808102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f28077fe640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2808103830 0x7f28081016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2806ffd640 1 -- 172.21.15.44:0/359804024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2808103830 msgr2=0x7f28081016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2806ffd640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2808103830 0x7f28081016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2806ffd640 1 -- 172.21.15.44:0/359804024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f280810cde0 msgr2=0x7f2808102130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2806ffd640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f280810cde0 0x7f2808102130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2806ffd640 1 -- 172.21.15.44:0/359804024 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28081a72d0 con 0x7f2808104230 2024-08-20T03:03:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2806ffd640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2808104230 0x7f2808101bf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00c920 tx=0x7f27fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:24.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2804ff9640 1 -- 172.21.15.44:0/359804024 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc007c70 con 0x7f2808104230 2024-08-20T03:03:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f280df21640 1 -- 172.21.15.44:0/359804024 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28081a75c0 con 0x7f2808104230 2024-08-20T03:03:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2804ff9640 1 -- 172.21.15.44:0/359804024 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27fc007e10 con 0x7f2808104230 2024-08-20T03:03:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f2804ff9640 1 -- 172.21.15.44:0/359804024 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc018820 con 0x7f2808104230 2024-08-20T03:03:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.929+0000 7f280df21640 1 -- 172.21.15.44:0/359804024 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28081a7b50 con 0x7f2808104230 2024-08-20T03:03:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.932+0000 7f2804ff9640 1 -- 172.21.15.44:0/359804024 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f27fc00f450 con 0x7f2808104230 2024-08-20T03:03:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.933+0000 7f2804ff9640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f27f00b3dd0 0x7f27f00b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.933+0000 7f28077fe640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f27f00b3dd0 0x7f27f00b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.933+0000 7f281031a740 1 Processor -- start 2024-08-20T03:03:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.933+0000 7f2804ff9640 1 -- 172.21.15.44:0/359804024 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f27fc0e41b0 con 0x7f2808104230 2024-08-20T03:03:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.933+0000 7f281031a740 1 -- start start 2024-08-20T03:03:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.933+0000 7f281031a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 0x56167a8adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.933+0000 7f281031a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ae150 0x56167a8b9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.933+0000 7f281031a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ba0e0 0x56167a8bc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.933+0000 7f281031a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56167a8248e0 con 0x56167a8ad830 2024-08-20T03:03:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.933+0000 7f281031a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56167a813de0 con 0x56167a8ba0e0 2024-08-20T03:03:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.933+0000 7f281031a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56167a817e20 con 0x56167a8ae150 2024-08-20T03:03:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f28077fe640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f27f00b3dd0 0x7f27f00b6290 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f27f8005d00 tx=0x7f27f800b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f28077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 0x56167a8adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f28077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 0x56167a8adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60340/0 (socket says 172.21.15.44:60340) 2024-08-20T03:03:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f28077fe640 1 -- 172.21.15.44:0/2827950470 learned_addr learned my addr 172.21.15.44:0/2827950470 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f2807fff640 1 --2- 172.21.15.44:0/2827950470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ba0e0 0x56167a8bc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f2806ffd640 1 --2- 172.21.15.44:0/2827950470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ae150 0x56167a8b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f28077fe640 1 -- 172.21.15.44:0/2827950470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ae150 msgr2=0x56167a8b9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:24.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f28077fe640 1 --2- 172.21.15.44:0/2827950470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ae150 0x56167a8b9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f28077fe640 1 -- 172.21.15.44:0/2827950470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ba0e0 msgr2=0x56167a8bc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f28077fe640 1 --2- 172.21.15.44:0/2827950470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ba0e0 0x56167a8bc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f28077fe640 1 -- 172.21.15.44:0/2827950470 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56167a573650 con 0x56167a8ad830 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f2807fff640 1 --2- 172.21.15.44:0/2827950470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ba0e0 0x56167a8bc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f28077fe640 1 --2- 172.21.15.44:0/2827950470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 0x56167a8adc10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f27f80114a0 tx=0x7f27f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1001b82183b263 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f27edffb640 1 -- 172.21.15.44:0/2827950470 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f801e620 con 0x56167a8ad830 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f27edffb640 1 -- 172.21.15.44:0/2827950470 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f801ec70 con 0x56167a8ad830 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.934+0000 7f27edffb640 1 -- 172.21.15.44:0/2827950470 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f801dcb0 con 0x56167a8ad830 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.935+0000 7f281031a740 1 -- 172.21.15.44:0/2827950470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 msgr2=0x56167a8adc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.935+0000 7f281031a740 1 --2- 172.21.15.44:0/2827950470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 0x56167a8adc10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f27f80114a0 tx=0x7f27f8013000 comp rx=0 tx=0).stop 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.935+0000 7f281031a740 1 -- 172.21.15.44:0/2827950470 shutdown_connections 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.935+0000 7f281031a740 1 --2- 172.21.15.44:0/2827950470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ba0e0 0x56167a8bc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.935+0000 7f281031a740 1 --2- 172.21.15.44:0/2827950470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ae150 0x56167a8b9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.935+0000 7f281031a740 1 --2- 172.21.15.44:0/2827950470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 0x56167a8adc10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.935+0000 7f281031a740 1 -- 172.21.15.44:0/2827950470 >> 172.21.15.44:0/2827950470 conn(0x56167a8ab660 msgr2=0x56167a8ac800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.936+0000 7f281031a740 1 -- 172.21.15.44:0/2827950470 shutdown_connections 2024-08-20T03:03:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.936+0000 7f281031a740 1 -- 172.21.15.44:0/2827950470 wait complete. 2024-08-20T03:03:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.936+0000 7f281031a740 1 -- start start 2024-08-20T03:03:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.936+0000 7f281031a740 1 Processor -- start 2024-08-20T03:03:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.936+0000 7f281031a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ad830 0x56167a8b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.936+0000 7f281031a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 0x56167a8b7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.936+0000 7f281031a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 0x56167a8b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f281031a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56167a676700 con 0x56167a8ae150 2024-08-20T03:03:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f281031a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56167a79a7c0 con 0x56167a8ad830 2024-08-20T03:03:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f281031a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56167a8bcd90 con 0x56167a8ba0e0 2024-08-20T03:03:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f281031a740 10 client.0 _ll_register_callbacks cb 0x56167a80d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 0x56167a8b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 0x56167a8b7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60350/0 (socket says 172.21.15.44:60350) 2024-08-20T03:03:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f2806ffd640 1 -- 172.21.15.44:0/963870054 learned_addr learned my addr 172.21.15.44:0/963870054 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f28077fe640 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ad830 0x56167a8b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f2807fff640 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 0x56167a8b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f2806ffd640 1 -- 172.21.15.44:0/963870054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 msgr2=0x56167a8b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f2806ffd640 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 0x56167a8b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.959 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f2806ffd640 1 -- 172.21.15.44:0/963870054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ad830 msgr2=0x56167a8b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f2806ffd640 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ad830 0x56167a8b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f2806ffd640 1 -- 172.21.15.44:0/963870054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56167a8c6110 con 0x56167a8ae150 2024-08-20T03:03:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f2807fff640 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 0x56167a8b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f2806ffd640 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 0x56167a8b7590 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f27fc007590 tx=0x7f27fc0a64b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b18023d1e167bb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f27e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f27e6ffd640 1 -- 172.21.15.44:0/963870054 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0aac60 con 0x56167a8ae150 2024-08-20T03:03:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f27e6ffd640 1 -- 172.21.15.44:0/963870054 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f27fc0e1b00 con 0x56167a8ae150 2024-08-20T03:03:24.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f281031a740 1 -- 172.21.15.44:0/963870054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56167a7c3100 con 0x56167a8ae150 2024-08-20T03:03:24.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f27e6ffd640 1 -- 172.21.15.44:0/963870054 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27fc0e1d00 con 0x56167a8ae150 2024-08-20T03:03:24.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f281031a740 10 client.15891 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:24.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.937+0000 7f281031a740 1 -- 172.21.15.44:0/963870054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56167a8c8b70 con 0x56167a8ae150 2024-08-20T03:03:24.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.938+0000 7f27e6ffd640 1 -- 172.21.15.44:0/963870054 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56167a7c3100 con 0x56167a8ae150 2024-08-20T03:03:24.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.938+0000 7f27e6ffd640 1 -- 172.21.15.44:0/963870054 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27fc0920e0 con 0x56167a8ae150 2024-08-20T03:03:24.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.938+0000 7f281031a740 10 client.15891 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:24.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.938+0000 7f281031a740 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:24.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.938+0000 7f281031a740 20 client.15891 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:24.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.938+0000 7f281031a740 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56167a8ca030 0x56167a8cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.938+0000 7f281031a740 4 client.15891 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:24.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.938+0000 7f281031a740 1 -- 172.21.15.44:0/963870054 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56167a826830 con 0x56167a8ca030 2024-08-20T03:03:24.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.938+0000 7f28077fe640 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56167a8ca030 0x56167a8cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f28077fe640 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56167a8ca030 0x56167a8cc420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7cc9e2816179d68 server_cookie=d89abd24cb880a5e in_seq=0 out_seq=0 2024-08-20T03:03:24.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f27e6ffd640 10 client.15891 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:24.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f27e6ffd640 1 -- 172.21.15.44:0/963870054 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56167a826830 con 0x56167a8ca030 2024-08-20T03:03:24.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f27e6ffd640 10 client.15891 handle_command_reply: tid=0 2024-08-20T03:03:24.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f281031a740 1 client.15891 shutdown 2024-08-20T03:03:24.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f281031a740 1 -- 172.21.15.44:0/963870054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 msgr2=0x56167a8b7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:24.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f281031a740 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 0x56167a8b7590 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f27fc007590 tx=0x7f27fc0a64b0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f281031a740 1 -- 172.21.15.44:0/963870054 shutdown_connections 2024-08-20T03:03:24.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f281031a740 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56167a8ca030 0x56167a8cc420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f281031a740 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 0x56167a8b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f281031a740 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 0x56167a8b7590 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f281031a740 1 --2- 172.21.15.44:0/963870054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ad830 0x56167a8b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f281031a740 1 -- 172.21.15.44:0/963870054 >> 172.21.15.44:0/963870054 conn(0x56167a8ab660 msgr2=0x56167a8b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:24.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.939+0000 7f27e6ffd640 0 client.15891 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:24.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.940+0000 7f281031a740 1 -- 172.21.15.44:0/963870054 shutdown_connections 2024-08-20T03:03:24.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.940+0000 7f281031a740 1 -- 172.21.15.44:0/963870054 wait complete. 2024-08-20T03:03:24.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.940+0000 7f281031a740 20 client.15891 trim_cache size 0 max 16384 2024-08-20T03:03:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.957+0000 7f281031a740 1 Processor -- start 2024-08-20T03:03:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.957+0000 7f281031a740 1 -- start start 2024-08-20T03:03:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f281031a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 0x56167a8cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f281031a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ae150 0x56167a8aa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f281031a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 0x56167a8d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f281031a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b03b0 con 0x56167a8ad830 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f281031a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b1ef0 con 0x56167a8ae150 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f281031a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b0a80 con 0x56167a8ba0e0 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f28077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 0x56167a8cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f28077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 0x56167a8cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60356/0 (socket says 172.21.15.44:60356) 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f28077fe640 1 -- 172.21.15.44:0/4190188901 learned_addr learned my addr 172.21.15.44:0/4190188901 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f2806ffd640 1 --2- 172.21.15.44:0/4190188901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ae150 0x56167a8aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f2806ffd640 1 -- 172.21.15.44:0/4190188901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 msgr2=0x56167a8d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f2806ffd640 1 --2- 172.21.15.44:0/4190188901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 0x56167a8d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f2806ffd640 1 -- 172.21.15.44:0/4190188901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 msgr2=0x56167a8cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f2806ffd640 1 --2- 172.21.15.44:0/4190188901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 0x56167a8cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f2806ffd640 1 -- 172.21.15.44:0/4190188901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56167a8cca00 con 0x56167a8ae150 2024-08-20T03:03:24.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f2806ffd640 1 --2- 172.21.15.44:0/4190188901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ae150 0x56167a8aa610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00c8f0 tx=0x7f27fc0abf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c1041006e31654 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.958+0000 7f27ed7fa640 1 -- 172.21.15.44:0/4190188901 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0a6b20 con 0x56167a8ae150 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.959+0000 7f27ed7fa640 1 -- 172.21.15.44:0/4190188901 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27fc0aac60 con 0x56167a8ae150 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.959+0000 7f27ed7fa640 1 -- 172.21.15.44:0/4190188901 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0b3d10 con 0x56167a8ae150 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.959+0000 7f281031a740 1 -- 172.21.15.44:0/4190188901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ae150 msgr2=0x56167a8aa610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.959+0000 7f281031a740 1 --2- 172.21.15.44:0/4190188901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ae150 0x56167a8aa610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00c8f0 tx=0x7f27fc0abf90 comp rx=0 tx=0).stop 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.959+0000 7f281031a740 1 -- 172.21.15.44:0/4190188901 shutdown_connections 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.959+0000 7f281031a740 1 --2- 172.21.15.44:0/4190188901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 0x56167a8d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.959+0000 7f281031a740 1 --2- 172.21.15.44:0/4190188901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ae150 0x56167a8aa610 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.959+0000 7f281031a740 1 --2- 172.21.15.44:0/4190188901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ad830 0x56167a8cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.959+0000 7f281031a740 1 -- 172.21.15.44:0/4190188901 >> 172.21.15.44:0/4190188901 conn(0x56167a8ab660 msgr2=0x56167a8a9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.959+0000 7f281031a740 1 -- 172.21.15.44:0/4190188901 shutdown_connections 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.959+0000 7f281031a740 1 -- 172.21.15.44:0/4190188901 wait complete. 2024-08-20T03:03:24.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f281031a740 1 -- start start 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f281031a740 1 Processor -- start 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f281031a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ad830 0x56167a8bf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f281031a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 0x56167a8bfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f281031a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 0x56167a92cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f281031a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b1820 con 0x56167a8ae150 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f281031a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f28081b1150 con 0x56167a8ad830 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f281031a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f28081ae390 con 0x56167a8ba0e0 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f281031a740 10 client.0 _ll_register_callbacks cb 0x7f27fc0aa3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 0x56167a8bfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 0x56167a8bfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60368/0 (socket says 172.21.15.44:60368) 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f2806ffd640 1 -- 172.21.15.44:0/1140084736 learned_addr learned my addr 172.21.15.44:0/1140084736 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.960+0000 7f28077fe640 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ad830 0x56167a8bf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f2807fff640 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 0x56167a92cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f2806ffd640 1 -- 172.21.15.44:0/1140084736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 msgr2=0x56167a92cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f2806ffd640 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 0x56167a92cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f2806ffd640 1 -- 172.21.15.44:0/1140084736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ad830 msgr2=0x56167a8bf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f2806ffd640 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ad830 0x56167a8bf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f2806ffd640 1 -- 172.21.15.44:0/1140084736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56167a92d360 con 0x56167a8ae150 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f2806ffd640 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 0x56167a8bfde0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0a6560 tx=0x7f27fc0b3b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6034cf97d5803338 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f27e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f27e6ffd640 1 -- 172.21.15.44:0/1140084736 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0a6b20 con 0x56167a8ae150 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f27e6ffd640 1 -- 172.21.15.44:0/1140084736 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f27fc0a68f0 con 0x56167a8ae150 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f281031a740 1 -- 172.21.15.44:0/1140084736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27fc0aac60 con 0x56167a8ae150 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f27e6ffd640 1 -- 172.21.15.44:0/1140084736 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27fc0b3d10 con 0x56167a8ae150 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f281031a740 10 client.15897 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f281031a740 1 -- 172.21.15.44:0/1140084736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56167a92d360 con 0x56167a8ae150 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.961+0000 7f27e6ffd640 1 -- 172.21.15.44:0/1140084736 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27fc0aac60 con 0x56167a8ae150 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.962+0000 7f27e6ffd640 1 -- 172.21.15.44:0/1140084736 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27fc0b03f0 con 0x56167a8ae150 2024-08-20T03:03:24.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.962+0000 7f281031a740 10 client.15897 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.962+0000 7f281031a740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.962+0000 7f281031a740 20 client.15897 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.962+0000 7f281031a740 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56167a934990 0x56167a936db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.962+0000 7f281031a740 4 client.15897 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.962+0000 7f281031a740 1 -- 172.21.15.44:0/1140084736 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56167a8222d0 con 0x56167a934990 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.962+0000 7f28077fe640 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56167a934990 0x56167a936db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f27e6ffd640 10 client.15897 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f28077fe640 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56167a934990 0x56167a936db0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7090faeeba7ce1d server_cookie=6ef3703124c93b37 in_seq=0 out_seq=0 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f27e6ffd640 1 -- 172.21.15.44:0/1140084736 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56167a8222d0 con 0x56167a934990 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f27e6ffd640 10 client.15897 handle_command_reply: tid=0 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f281031a740 1 client.15897 shutdown 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f281031a740 1 -- 172.21.15.44:0/1140084736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 msgr2=0x56167a8bfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:24.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f281031a740 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 0x56167a8bfde0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0a6560 tx=0x7f27fc0b3b30 comp rx=0 tx=0).stop 2024-08-20T03:03:24.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f281031a740 1 -- 172.21.15.44:0/1140084736 shutdown_connections 2024-08-20T03:03:24.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f281031a740 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56167a934990 0x56167a936db0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f281031a740 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56167a8ba0e0 0x56167a92cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f281031a740 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56167a8ae150 0x56167a8bfde0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f27e6ffd640 0 client.15897 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:24.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f281031a740 1 --2- 172.21.15.44:0/1140084736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56167a8ad830 0x56167a8bf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f281031a740 1 -- 172.21.15.44:0/1140084736 >> 172.21.15.44:0/1140084736 conn(0x56167a8ab660 msgr2=0x56167a8aa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:24.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f281031a740 1 -- 172.21.15.44:0/1140084736 shutdown_connections 2024-08-20T03:03:24.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f281031a740 1 -- 172.21.15.44:0/1140084736 wait complete. 2024-08-20T03:03:24.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.963+0000 7f281031a740 20 client.15897 trim_cache size 0 max 16384 2024-08-20T03:03:24.974 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:03:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-08-20T03:03:24.975 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-08-20T03:03:24.975 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:03:24.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.964+0000 7f280df21640 1 -- 172.21.15.44:0/359804024 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f27f00b3dd0 msgr2=0x7f27f00b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:24.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.964+0000 7f280df21640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f27f00b3dd0 0x7f27f00b6290 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f27f8005d00 tx=0x7f27f800b410 comp rx=0 tx=0).stop 2024-08-20T03:03:24.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.964+0000 7f280df21640 1 -- 172.21.15.44:0/359804024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2808104230 msgr2=0x7f2808101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:24.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.964+0000 7f280df21640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2808104230 0x7f2808101bf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00c920 tx=0x7f27fc00cde0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.964+0000 7f280df21640 1 -- 172.21.15.44:0/359804024 shutdown_connections 2024-08-20T03:03:24.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.964+0000 7f280df21640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f27f00b3dd0 0x7f27f00b6290 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.964+0000 7f280df21640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f280810cde0 0x7f2808102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.964+0000 7f280df21640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2808104230 0x7f2808101bf0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.964+0000 7f280df21640 1 --2- 172.21.15.44:0/359804024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2808103830 0x7f28081016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:24.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.964+0000 7f280df21640 1 -- 172.21.15.44:0/359804024 >> 172.21.15.44:0/359804024 conn(0x7f28080fd400 msgr2=0x7f28080fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:24.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.964+0000 7f280df21640 1 -- 172.21.15.44:0/359804024 shutdown_connections 2024-08-20T03:03:24.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:24.964+0000 7f280df21640 1 -- 172.21.15.44:0/359804024 wait complete. 2024-08-20T03:03:24.999 INFO:tasks.workunit.client.0.smithi044.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-08-20T03:03:24.999 INFO:tasks.workunit.client.0.smithi044.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-08-20T03:03:25.032 INFO:tasks.workunit.client.0.smithi044.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-08-20T03:03:25.032 INFO:tasks.workunit.client.0.smithi044.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-08-20T03:03:25.090 INFO:tasks.workunit.client.0.smithi044.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-08-20T03:03:25.091 INFO:tasks.workunit.client.0.smithi044.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-08-20T03:03:25.127 INFO:tasks.workunit.client.0.smithi044.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-08-20T03:03:25.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:24 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:03:25.145 INFO:tasks.workunit.client.0.smithi044.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-08-20T03:03:25.145 INFO:tasks.workunit.client.0.smithi044.stdout:Making all in man 2024-08-20T03:03:25.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:24 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:03:25.167 INFO:tasks.workunit.client.0.smithi044.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-08-20T03:03:25.167 INFO:tasks.workunit.client.0.smithi044.stdout:make[2]: Nothing to be done for 'all'. 2024-08-20T03:03:25.167 INFO:tasks.workunit.client.0.smithi044.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-08-20T03:03:25.168 INFO:tasks.workunit.client.0.smithi044.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-20T03:03:25.169 INFO:tasks.workunit.client.0.smithi044.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-20T03:03:25.170 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-20T03:03:25.170 INFO:tasks.workunit.client.0.smithi044.stderr:+ cd src 2024-08-20T03:03:25.170 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir blogtest_in 2024-08-20T03:03:25.172 INFO:tasks.workunit.client.0.smithi044.stdout:running blogbench 2024-08-20T03:03:25.172 INFO:tasks.workunit.client.0.smithi044.stderr:+ echo 'running blogbench' 2024-08-20T03:03:25.172 INFO:tasks.workunit.client.0.smithi044.stderr:+ ./blogbench -d blogtest_in 2024-08-20T03:03:25.173 INFO:tasks.workunit.client.0.smithi044.stdout: 2024-08-20T03:03:25.173 INFO:tasks.workunit.client.0.smithi044.stdout:Frequency = 10 secs 2024-08-20T03:03:25.173 INFO:tasks.workunit.client.0.smithi044.stdout:Scratch dir = [blogtest_in] 2024-08-20T03:03:25.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:24 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:03:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:25 smithi165 ceph-mon[22357]: pgmap v348: 97 pgs: 97 active+clean; 2.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 122 KiB/s wr, 23 op/s 2024-08-20T03:03:26.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:25 smithi099 ceph-mon[21984]: pgmap v348: 97 pgs: 97 active+clean; 2.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 122 KiB/s wr, 23 op/s 2024-08-20T03:03:26.262 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:25 smithi044 ceph-mon[18790]: pgmap v348: 97 pgs: 97 active+clean; 2.6 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 122 KiB/s wr, 23 op/s 2024-08-20T03:03:27.471 INFO:tasks.workunit.client.0.smithi044.stdout:Spawning 3 writers... 2024-08-20T03:03:27.471 INFO:tasks.workunit.client.0.smithi044.stdout:Spawning 1 rewriters... 2024-08-20T03:03:27.471 INFO:tasks.workunit.client.0.smithi044.stdout:Spawning 5 commenters... 2024-08-20T03:03:27.471 INFO:tasks.workunit.client.0.smithi044.stdout:Spawning 100 readers... 2024-08-20T03:03:27.475 INFO:tasks.workunit.client.0.smithi044.stdout:Benchmarking for 30 iterations. 2024-08-20T03:03:27.475 INFO:tasks.workunit.client.0.smithi044.stdout:The test will run during 5 minutes. 2024-08-20T03:03:27.475 INFO:tasks.workunit.client.0.smithi044.stdout: 2024-08-20T03:03:27.475 INFO:tasks.workunit.client.0.smithi044.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-08-20T03:03:27.895 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T03:03:27.895 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:03:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:27 smithi165 ceph-mon[22357]: pgmap v349: 97 pgs: 97 active+clean; 4.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 325 KiB/s wr, 38 op/s 2024-08-20T03:03:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:03:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:27 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:27 smithi099 ceph-mon[21984]: pgmap v349: 97 pgs: 97 active+clean; 4.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 325 KiB/s wr, 38 op/s 2024-08-20T03:03:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:03:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:28.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:27 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:28.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:27 smithi044 ceph-mon[18790]: pgmap v349: 97 pgs: 97 active+clean; 4.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 325 KiB/s wr, 38 op/s 2024-08-20T03:03:28.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:03:28.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:28.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:28.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:28.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:27 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:28.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.667+0000 7f5143aa6640 1 -- 172.21.15.44:0/3983834385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f513c107d20 msgr2=0x7f513c10e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:28.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.667+0000 7f5143aa6640 1 --2- 172.21.15.44:0/3983834385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f513c107d20 0x7f513c10e520 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5134009ca0 tx=0x7f513402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:03:28.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.667+0000 7f5143aa6640 1 -- 172.21.15.44:0/3983834385 shutdown_connections 2024-08-20T03:03:28.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.667+0000 7f5143aa6640 1 --2- 172.21.15.44:0/3983834385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f513c107d20 0x7f513c10e520 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:28.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.667+0000 7f5143aa6640 1 --2- 172.21.15.44:0/3983834385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f513c1073a0 0x7f513c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.667+0000 7f5143aa6640 1 --2- 172.21.15.44:0/3983834385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513c101990 0x7f513c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.667+0000 7f5143aa6640 1 -- 172.21.15.44:0/3983834385 >> 172.21.15.44:0/3983834385 conn(0x7f513c0fd400 msgr2=0x7f513c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.667+0000 7f5143aa6640 1 -- 172.21.15.44:0/3983834385 shutdown_connections 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.667+0000 7f5143aa6640 1 -- 172.21.15.44:0/3983834385 wait complete. 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.668+0000 7f5143aa6640 1 Processor -- start 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.668+0000 7f5143aa6640 1 -- start start 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.668+0000 7f5143aa6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513c101990 0x7f513c19b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.668+0000 7f5143aa6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f513c1073a0 0x7f513c19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.668+0000 7f5143aa6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f513c107d20 0x7f513c1a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.668+0000 7f5143aa6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f513c1119e0 con 0x7f513c1073a0 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.668+0000 7f5143aa6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f513c111860 con 0x7f513c107d20 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.668+0000 7f5143aa6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f513c111b60 con 0x7f513c101990 2024-08-20T03:03:28.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5141041640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f513c1073a0 0x7f513c19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5141041640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f513c1073a0 0x7f513c19b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37230/0 (socket says 172.21.15.44:37230) 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5141041640 1 -- 172.21.15.44:0/1632817283 learned_addr learned my addr 172.21.15.44:0/1632817283 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5141842640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513c101990 0x7f513c19b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5142043640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f513c107d20 0x7f513c1a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5141041640 1 -- 172.21.15.44:0/1632817283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513c101990 msgr2=0x7f513c19b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5141041640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513c101990 0x7f513c19b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5141041640 1 -- 172.21.15.44:0/1632817283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f513c107d20 msgr2=0x7f513c1a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5141041640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f513c107d20 0x7f513c1a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5141041640 1 -- 172.21.15.44:0/1632817283 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513c1a11d0 con 0x7f513c1073a0 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5141041640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f513c1073a0 0x7f513c19b660 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f512c00c920 tx=0x7f512c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5132ffd640 1 -- 172.21.15.44:0/1632817283 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c007c70 con 0x7f513c1073a0 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5132ffd640 1 -- 172.21.15.44:0/1632817283 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f512c007e10 con 0x7f513c1073a0 2024-08-20T03:03:28.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5143aa6640 1 -- 172.21.15.44:0/1632817283 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f513c1a14c0 con 0x7f513c1073a0 2024-08-20T03:03:28.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5132ffd640 1 -- 172.21.15.44:0/1632817283 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c018820 con 0x7f513c1073a0 2024-08-20T03:03:28.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.669+0000 7f5143aa6640 1 -- 172.21.15.44:0/1632817283 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f513c1a7b80 con 0x7f513c1073a0 2024-08-20T03:03:28.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.670+0000 7f5143aa6640 1 -- 172.21.15.44:0/1632817283 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f513c101d70 con 0x7f513c1073a0 2024-08-20T03:03:28.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.674+0000 7f5132ffd640 1 -- 172.21.15.44:0/1632817283 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f512c00f450 con 0x7f513c1073a0 2024-08-20T03:03:28.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.675+0000 7f5132ffd640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f51280b3dd0 0x7f51280b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:28.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.675+0000 7f5132ffd640 1 -- 172.21.15.44:0/1632817283 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f512c0e4490 con 0x7f513c1073a0 2024-08-20T03:03:28.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.678+0000 7f5141842640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f51280b3dd0 0x7f51280b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:28.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.678+0000 7f5141842640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f51280b3dd0 0x7f51280b6290 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5138009bd0 tx=0x7f5138005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:28.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.679+0000 7f5132ffd640 1 -- 172.21.15.44:0/1632817283 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f512c014030 con 0x7f513c1073a0 2024-08-20T03:03:28.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.918+0000 7f5143aa6640 1 -- 172.21.15.44:0/1632817283 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f513c1a7f60 con 0x7f513c1073a0 2024-08-20T03:03:28.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.919+0000 7f5132ffd640 1 -- 172.21.15.44:0/1632817283 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11965 (secure 0 0 0) 0x7f512c0ab730 con 0x7f513c1073a0 2024-08-20T03:03:28.923 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:03:28.923 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":28,"btime":"2024-08-20T03:02:12:953534+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:02:12.906757+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24569,24583,24596,24589,24604]},"id":1}]} 2024-08-20T03:03:28.923 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 28 2024-08-20T03:03:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.922+0000 7f5143aa6640 1 -- 172.21.15.44:0/1632817283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f51280b3dd0 msgr2=0x7f51280b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.922+0000 7f5143aa6640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f51280b3dd0 0x7f51280b6290 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5138009bd0 tx=0x7f5138005f90 comp rx=0 tx=0).stop 2024-08-20T03:03:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.922+0000 7f5143aa6640 1 -- 172.21.15.44:0/1632817283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f513c1073a0 msgr2=0x7f513c19b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.922+0000 7f5143aa6640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f513c1073a0 0x7f513c19b660 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f512c00c920 tx=0x7f512c00cde0 comp rx=0 tx=0).stop 2024-08-20T03:03:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.922+0000 7f5143aa6640 1 -- 172.21.15.44:0/1632817283 shutdown_connections 2024-08-20T03:03:28.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.922+0000 7f5143aa6640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f51280b3dd0 0x7f51280b6290 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:28.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.922+0000 7f5143aa6640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f513c107d20 0x7f513c1a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:28.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.922+0000 7f5143aa6640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f513c1073a0 0x7f513c19b660 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:28.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.922+0000 7f5143aa6640 1 --2- 172.21.15.44:0/1632817283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f513c101990 0x7f513c19b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:28.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.922+0000 7f5143aa6640 1 -- 172.21.15.44:0/1632817283 >> 172.21.15.44:0/1632817283 conn(0x7f513c0fd400 msgr2=0x7f513c06a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:28.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.922+0000 7f5143aa6640 1 -- 172.21.15.44:0/1632817283 shutdown_connections 2024-08-20T03:03:28.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:28.922+0000 7f5143aa6640 1 -- 172.21.15.44:0/1632817283 wait complete. 2024-08-20T03:03:29.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:29.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:29.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:03:29.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:03:29.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:28 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:29.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:28 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1632817283' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:03:29.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:29.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:29.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:03:29.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:03:29.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:28 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:29.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:28 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1632817283' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:03:29.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:29.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:29.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:03:29.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:03:29.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:28 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:03:29.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:28 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1632817283' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:03:29.673 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-20T03:03:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:29.965 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:29 smithi044 ceph-mon[18790]: pgmap v350: 97 pgs: 97 active+clean; 5.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 412 KiB/s wr, 54 op/s 2024-08-20T03:03:30.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:29 smithi165 ceph-mon[22357]: pgmap v350: 97 pgs: 97 active+clean; 5.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 412 KiB/s wr, 54 op/s 2024-08-20T03:03:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:29 smithi099 ceph-mon[21984]: pgmap v350: 97 pgs: 97 active+clean; 5.6 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 412 KiB/s wr, 54 op/s 2024-08-20T03:03:30.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.454+0000 7f71ecf23640 1 -- 172.21.15.44:0/1660526487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e8107d20 msgr2=0x7f71e810e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.454+0000 7f71ecf23640 1 --2- 172.21.15.44:0/1660526487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e8107d20 0x7f71e810e520 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009ca0 tx=0x7f71e002a8c0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.454+0000 7f71ecf23640 1 -- 172.21.15.44:0/1660526487 shutdown_connections 2024-08-20T03:03:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.454+0000 7f71ecf23640 1 --2- 172.21.15.44:0/1660526487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e8107d20 0x7f71e810e520 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.454+0000 7f71ecf23640 1 --2- 172.21.15.44:0/1660526487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81073a0 0x7f71e81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.454+0000 7f71ecf23640 1 --2- 172.21.15.44:0/1660526487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71e8101990 0x7f71e8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.454+0000 7f71ecf23640 1 -- 172.21.15.44:0/1660526487 >> 172.21.15.44:0/1660526487 conn(0x7f71e80fd400 msgr2=0x7f71e80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.454+0000 7f71ecf23640 1 -- 172.21.15.44:0/1660526487 shutdown_connections 2024-08-20T03:03:30.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.454+0000 7f71ecf23640 1 -- 172.21.15.44:0/1660526487 wait complete. 2024-08-20T03:03:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71ecf23640 1 Processor -- start 2024-08-20T03:03:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71ecf23640 1 -- start start 2024-08-20T03:03:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71ecf23640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e8101990 0x7f71e8198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71ecf23640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71e81073a0 0x7f71e8199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71ecf23640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e8107d20 0x7f71e819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71ecf23640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71e81119e0 con 0x7f71e8107d20 2024-08-20T03:03:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71ecf23640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8111860 con 0x7f71e8101990 2024-08-20T03:03:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71ecf23640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8111b60 con 0x7f71e81073a0 2024-08-20T03:03:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71e6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e8107d20 0x7f71e819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71e6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e8107d20 0x7f71e819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37250/0 (socket says 172.21.15.44:37250) 2024-08-20T03:03:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71e6d9d640 1 -- 172.21.15.44:0/2442152263 learned_addr learned my addr 172.21.15.44:0/2442152263 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:30.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71e5d9b640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71e81073a0 0x7f71e8199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71e659c640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e8101990 0x7f71e8198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71e6d9d640 1 -- 172.21.15.44:0/2442152263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71e81073a0 msgr2=0x7f71e8199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71e6d9d640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71e81073a0 0x7f71e8199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71e6d9d640 1 -- 172.21.15.44:0/2442152263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e8101990 msgr2=0x7f71e8198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71e6d9d640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e8101990 0x7f71e8198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71e6d9d640 1 -- 172.21.15.44:0/2442152263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e819eff0 con 0x7f71e8107d20 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71e659c640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e8101990 0x7f71e8198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.455+0000 7f71e5d9b640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71e81073a0 0x7f71e8199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.456+0000 7f71e6d9d640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e8107d20 0x7f71e819e820 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f71e0005cc0 tx=0x7f71e0031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.456+0000 7f71d77fe640 1 -- 172.21.15.44:0/2442152263 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0003b70 con 0x7f71e8107d20 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.456+0000 7f71d77fe640 1 -- 172.21.15.44:0/2442152263 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e00318f0 con 0x7f71e8107d20 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.456+0000 7f71ecf23640 1 -- 172.21.15.44:0/2442152263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e819f280 con 0x7f71e8107d20 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.456+0000 7f71d77fe640 1 -- 172.21.15.44:0/2442152263 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e002f950 con 0x7f71e8107d20 2024-08-20T03:03:30.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.456+0000 7f71ecf23640 1 -- 172.21.15.44:0/2442152263 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e81a59a0 con 0x7f71e8107d20 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.458+0000 7f71ef31c740 1 Processor -- start 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.458+0000 7f71d77fe640 1 -- 172.21.15.44:0/2442152263 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f71e002fb30 con 0x7f71e8107d20 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71ef31c740 1 -- start start 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71ef31c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b76b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71ef31c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b77050 0x55f719b86eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71ef31c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b873f0 0x55f719b897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71ef31c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f719aed930 con 0x55f719b873f0 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71ef31c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f719adce30 con 0x55f719b76730 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71ef31c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f719ae0e70 con 0x55f719b77050 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71e5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b873f0 0x55f719b897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71e659c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b77050 0x55f719b86eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71e659c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b77050 0x55f719b86eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:50836/0 (socket says 172.21.15.44:50836) 2024-08-20T03:03:30.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71e659c640 1 -- 172.21.15.44:0/3481036262 learned_addr learned my addr 172.21.15.44:0/3481036262 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71e5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b873f0 0x55f719b897e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37264/0 (socket says 172.21.15.44:37264) 2024-08-20T03:03:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71e5d9b640 1 --2- 172.21.15.44:0/3481036262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b76b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71e5d9b640 1 -- 172.21.15.44:0/3481036262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b77050 msgr2=0x55f719b86eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71e5d9b640 1 --2- 172.21.15.44:0/3481036262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b77050 0x55f719b86eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71e5d9b640 1 -- 172.21.15.44:0/3481036262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 msgr2=0x55f719b76b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71e5d9b640 1 --2- 172.21.15.44:0/3481036262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b76b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71e5d9b640 1 -- 172.21.15.44:0/3481036262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71983c650 con 0x55f719b873f0 2024-08-20T03:03:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.459+0000 7f71d77fe640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f71d00b3e50 0x7f71d00b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.460+0000 7f71e5d9b640 1 --2- 172.21.15.44:0/3481036262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b873f0 0x55f719b897e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f71dc011b00 tx=0x7f71dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99a2ee0ec916b33 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:30.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.460+0000 7f71e659c640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f71d00b3e50 0x7f71d00b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.460+0000 7f71d77fe640 1 -- 172.21.15.44:0/2442152263 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f71e0105370 con 0x7f71e8107d20 2024-08-20T03:03:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.460+0000 7f71d4ff9640 1 -- 172.21.15.44:0/3481036262 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc011050 con 0x55f719b873f0 2024-08-20T03:03:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.460+0000 7f71d4ff9640 1 -- 172.21.15.44:0/3481036262 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71dc0111f0 con 0x55f719b873f0 2024-08-20T03:03:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.460+0000 7f71d4ff9640 1 -- 172.21.15.44:0/3481036262 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc003850 con 0x55f719b873f0 2024-08-20T03:03:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71ef31c740 1 -- 172.21.15.44:0/3481036262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b873f0 msgr2=0x55f719b897e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71e659c640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f71d00b3e50 0x7f71d00b6310 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55f719b7a970 tx=0x7f71d8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71ef31c740 1 --2- 172.21.15.44:0/3481036262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b873f0 0x55f719b897e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f71dc011b00 tx=0x7f71dc004730 comp rx=0 tx=0).stop 2024-08-20T03:03:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71ef31c740 1 -- 172.21.15.44:0/3481036262 shutdown_connections 2024-08-20T03:03:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71ef31c740 1 --2- 172.21.15.44:0/3481036262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b873f0 0x55f719b897e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71ef31c740 1 --2- 172.21.15.44:0/3481036262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b77050 0x55f719b86eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71ef31c740 1 --2- 172.21.15.44:0/3481036262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b76b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71ef31c740 1 -- 172.21.15.44:0/3481036262 >> 172.21.15.44:0/3481036262 conn(0x55f719b74560 msgr2=0x55f719b75700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71ef31c740 1 -- 172.21.15.44:0/3481036262 shutdown_connections 2024-08-20T03:03:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71ef31c740 1 -- 172.21.15.44:0/3481036262 wait complete. 2024-08-20T03:03:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71ef31c740 1 -- start start 2024-08-20T03:03:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71ef31c740 1 Processor -- start 2024-08-20T03:03:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.461+0000 7f71ef31c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b84410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71ef31c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b84950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71ef31c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b873f0 0x55f719b84e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71ef31c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f71993f700 con 0x55f719b77050 2024-08-20T03:03:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71ef31c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f719a63810 con 0x55f719b76730 2024-08-20T03:03:30.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71ef31c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f719b8a0a0 con 0x55f719b873f0 2024-08-20T03:03:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71ef31c740 10 client.0 _ll_register_callbacks cb 0x55f719ad6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71e5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b84950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71e6d9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b873f0 0x55f719b84e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71e5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b84950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37268/0 (socket says 172.21.15.44:37268) 2024-08-20T03:03:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71e5d9b640 1 -- 172.21.15.44:0/20284186 learned_addr learned my addr 172.21.15.44:0/20284186 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71e659c640 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b84410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71e5d9b640 1 -- 172.21.15.44:0/20284186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b873f0 msgr2=0x55f719b84e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71e5d9b640 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b873f0 0x55f719b84e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71e5d9b640 1 -- 172.21.15.44:0/20284186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 msgr2=0x55f719b84410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71e5d9b640 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b84410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71e5d9b640 1 -- 172.21.15.44:0/20284186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f719b934d0 con 0x55f719b77050 2024-08-20T03:03:30.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71e6d9d640 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b873f0 0x55f719b84e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71e5d9b640 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b84950 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f71dc005bc0 tx=0x7f71dc012fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e47447858a8c4d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71cdffb640 1 -- 172.21.15.44:0/20284186 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc00fe70 con 0x55f719b77050 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71cdffb640 1 -- 172.21.15.44:0/20284186 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f71dc004030 con 0x55f719b77050 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.462+0000 7f71ef31c740 1 -- 172.21.15.44:0/20284186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f719a8c150 con 0x55f719b77050 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.463+0000 7f71cdffb640 1 -- 172.21.15.44:0/20284186 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71dc030050 con 0x55f719b77050 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.463+0000 7f71ef31c740 10 client.15915 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.463+0000 7f71ef31c740 1 -- 172.21.15.44:0/20284186 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f719b95e80 con 0x55f719b77050 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.463+0000 7f71cdffb640 1 -- 172.21.15.44:0/20284186 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f719a8c150 con 0x55f719b77050 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.463+0000 7f71cdffb640 1 -- 172.21.15.44:0/20284186 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f71dc01d020 con 0x55f719b77050 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.463+0000 7f71ef31c740 10 client.15915 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.463+0000 7f71ef31c740 10 client.15915 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:03:30.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.463+0000 7f71ef31c740 20 client.15915 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.463+0000 7f71ef31c740 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55f719b971d0 0x55f719b995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.463+0000 7f71ef31c740 4 client.15915 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.463+0000 7f71ef31c740 1 -- 172.21.15.44:0/20284186 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f719aef880 con 0x55f719b971d0 2024-08-20T03:03:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.463+0000 7f71e659c640 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55f719b971d0 0x55f719b995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.464+0000 7f71e659c640 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55f719b971d0 0x55f719b995c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d256a66024329ffe server_cookie=650889d192ffbaf0 in_seq=0 out_seq=0 2024-08-20T03:03:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.464+0000 7f71cdffb640 10 client.15915 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:03:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71cdffb640 1 -- 172.21.15.44:0/20284186 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f719aef880 con 0x55f719b971d0 2024-08-20T03:03:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71cdffb640 10 client.15915 handle_command_reply: tid=0 2024-08-20T03:03:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71ef31c740 1 client.15915 shutdown 2024-08-20T03:03:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71ef31c740 1 -- 172.21.15.44:0/20284186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 msgr2=0x55f719b84950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71ef31c740 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b84950 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f71dc005bc0 tx=0x7f71dc012fd0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71ef31c740 1 -- 172.21.15.44:0/20284186 shutdown_connections 2024-08-20T03:03:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71ef31c740 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b873f0 0x55f719b84e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71ef31c740 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b84950 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71ef31c740 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55f719b971d0 0x55f719b995c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71cdffb640 0 client.15915 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:03:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71ef31c740 1 --2- 172.21.15.44:0/20284186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b84410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71ef31c740 1 -- 172.21.15.44:0/20284186 >> 172.21.15.44:0/20284186 conn(0x55f719b74560 msgr2=0x55f719b7a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71ef31c740 1 -- 172.21.15.44:0/20284186 shutdown_connections 2024-08-20T03:03:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71ef31c740 1 -- 172.21.15.44:0/20284186 wait complete. 2024-08-20T03:03:30.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.465+0000 7f71ef31c740 20 client.15915 trim_cache size 0 max 16384 2024-08-20T03:03:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71ef31c740 1 Processor -- start 2024-08-20T03:03:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71ef31c740 1 -- start start 2024-08-20T03:03:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71ef31c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b81e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71ef31c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71ef31c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b873f0 0x55f719b9ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71ef31c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71e81abfc0 con 0x55f719b77050 2024-08-20T03:03:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71ef31c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71e81adb00 con 0x55f719b76730 2024-08-20T03:03:30.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71ef31c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f71e81ac690 con 0x55f719b873f0 2024-08-20T03:03:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71e5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71e659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b81e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71e659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b81e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:39432/0 (socket says 172.21.15.44:39432) 2024-08-20T03:03:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71e5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37274/0 (socket says 172.21.15.44:37274) 2024-08-20T03:03:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71e659c640 1 -- 172.21.15.44:0/4173568900 learned_addr learned my addr 172.21.15.44:0/4173568900 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71e659c640 1 -- 172.21.15.44:0/4173568900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b873f0 msgr2=0x55f719b9ed30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:03:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71e659c640 1 --2- 172.21.15.44:0/4173568900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b873f0 0x55f719b9ed30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71e659c640 1 -- 172.21.15.44:0/4173568900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 msgr2=0x55f719b734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71e659c640 1 --2- 172.21.15.44:0/4173568900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.481+0000 7f71e659c640 1 -- 172.21.15.44:0/4173568900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f719b73a30 con 0x55f719b76730 2024-08-20T03:03:30.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.482+0000 7f71e659c640 1 --2- 172.21.15.44:0/4173568900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b81e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f71d801f780 tx=0x7f71d801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdd038dceb4c56c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.482+0000 7f71cffff640 1 -- 172.21.15.44:0/4173568900 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d802b420 con 0x55f719b76730 2024-08-20T03:03:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.482+0000 7f71cffff640 1 -- 172.21.15.44:0/4173568900 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d802ba70 con 0x55f719b76730 2024-08-20T03:03:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.482+0000 7f71cffff640 1 -- 172.21.15.44:0/4173568900 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d801ec60 con 0x55f719b76730 2024-08-20T03:03:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.483+0000 7f71ef31c740 1 -- 172.21.15.44:0/4173568900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 msgr2=0x55f719b81e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.483+0000 7f71ef31c740 1 --2- 172.21.15.44:0/4173568900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b81e80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f71d801f780 tx=0x7f71d801a000 comp rx=0 tx=0).stop 2024-08-20T03:03:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.483+0000 7f71ef31c740 1 -- 172.21.15.44:0/4173568900 shutdown_connections 2024-08-20T03:03:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.483+0000 7f71ef31c740 1 --2- 172.21.15.44:0/4173568900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b873f0 0x55f719b9ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.483+0000 7f71ef31c740 1 --2- 172.21.15.44:0/4173568900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.483+0000 7f71ef31c740 1 --2- 172.21.15.44:0/4173568900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b76730 0x55f719b81e80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.483+0000 7f71ef31c740 1 -- 172.21.15.44:0/4173568900 >> 172.21.15.44:0/4173568900 conn(0x55f719b74560 msgr2=0x55f719b84cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:30.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.484+0000 7f71ef31c740 1 -- 172.21.15.44:0/4173568900 shutdown_connections 2024-08-20T03:03:30.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.484+0000 7f71ef31c740 1 -- 172.21.15.44:0/4173568900 wait complete. 2024-08-20T03:03:30.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.484+0000 7f71ef31c740 1 -- start start 2024-08-20T03:03:30.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.485+0000 7f71ef31c740 1 Processor -- start 2024-08-20T03:03:30.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.485+0000 7f71ef31c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b76730 0x55f719b8caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.485+0000 7f71ef31c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b8d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.485+0000 7f71ef31c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b873f0 0x55f719bffd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.485+0000 7f71ef31c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f71e81ad430 con 0x55f719b77050 2024-08-20T03:03:30.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.485+0000 7f71ef31c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71e81acd60 con 0x55f719b873f0 2024-08-20T03:03:30.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.485+0000 7f71ef31c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f71e81a9fa0 con 0x55f719b76730 2024-08-20T03:03:30.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.485+0000 7f71ef31c740 10 client.0 _ll_register_callbacks cb 0x7f71dc0115f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.485+0000 7f71e5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b8d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.486+0000 7f71e5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b8d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37288/0 (socket says 172.21.15.44:37288) 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.486+0000 7f71e5d9b640 1 -- 172.21.15.44:0/2452225174 learned_addr learned my addr 172.21.15.44:0/2452225174 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.486+0000 7f71e6d9d640 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b873f0 0x55f719bffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.486+0000 7f71e659c640 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b76730 0x55f719b8caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.486+0000 7f71e5d9b640 1 -- 172.21.15.44:0/2452225174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b76730 msgr2=0x55f719b8caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.486+0000 7f71e5d9b640 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b76730 0x55f719b8caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.486+0000 7f71e5d9b640 1 -- 172.21.15.44:0/2452225174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b873f0 msgr2=0x55f719bffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.486+0000 7f71e5d9b640 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b873f0 0x55f719bffd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.486+0000 7f71e5d9b640 1 -- 172.21.15.44:0/2452225174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f719c003c0 con 0x55f719b77050 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.486+0000 7f71e5d9b640 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b8d030 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f71dc003fd0 tx=0x7f71dc009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08a32043f919f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.486+0000 7f71cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.486+0000 7f71cdffb640 1 -- 172.21.15.44:0/2452225174 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc039060 con 0x55f719b77050 2024-08-20T03:03:30.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.487+0000 7f71cdffb640 1 -- 172.21.15.44:0/2452225174 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f71dc01f4a0 con 0x55f719b77050 2024-08-20T03:03:30.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.487+0000 7f71ef31c740 1 -- 172.21.15.44:0/2452225174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d802ba70 con 0x55f719b77050 2024-08-20T03:03:30.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.487+0000 7f71cdffb640 1 -- 172.21.15.44:0/2452225174 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71dc00fe50 con 0x55f719b77050 2024-08-20T03:03:30.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.493+0000 7f71ef31c740 10 client.15921 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:30.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.493+0000 7f71ef31c740 1 -- 172.21.15.44:0/2452225174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f719c016a0 con 0x55f719b77050 2024-08-20T03:03:30.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.493+0000 7f71cdffb640 1 -- 172.21.15.44:0/2452225174 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d802ba70 con 0x55f719b77050 2024-08-20T03:03:30.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.493+0000 7f71cdffb640 1 -- 172.21.15.44:0/2452225174 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f71dc01d040 con 0x55f719b77050 2024-08-20T03:03:30.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.493+0000 7f71ef31c740 10 client.15921 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:30.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.493+0000 7f71ef31c740 10 client.15921 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:03:30.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.493+0000 7f71ef31c740 20 client.15921 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:30.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.493+0000 7f71ef31c740 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55f719b971d0 0x55f719c03d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.493+0000 7f71ef31c740 4 client.15921 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:30.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.493+0000 7f71ef31c740 1 -- 172.21.15.44:0/2452225174 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f719aeb320 con 0x55f719b971d0 2024-08-20T03:03:30.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.494+0000 7f71e659c640 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55f719b971d0 0x55f719c03d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.495+0000 7f71e659c640 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55f719b971d0 0x55f719c03d50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=63faed62ce03b4c5 server_cookie=73baa6b3f7440afc in_seq=0 out_seq=0 2024-08-20T03:03:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.495+0000 7f71cdffb640 10 client.15921 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:03:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.495+0000 7f71cdffb640 1 -- 172.21.15.44:0/2452225174 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f719aeb320 con 0x55f719b971d0 2024-08-20T03:03:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.495+0000 7f71cdffb640 10 client.15921 handle_command_reply: tid=0 2024-08-20T03:03:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.495+0000 7f71ef31c740 1 client.15921 shutdown 2024-08-20T03:03:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.496+0000 7f71ef31c740 1 -- 172.21.15.44:0/2452225174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 msgr2=0x55f719b8d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.496+0000 7f71ef31c740 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b8d030 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f71dc003fd0 tx=0x7f71dc009510 comp rx=0 tx=0).stop 2024-08-20T03:03:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.496+0000 7f71ef31c740 1 -- 172.21.15.44:0/2452225174 shutdown_connections 2024-08-20T03:03:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.496+0000 7f71ef31c740 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55f719b971d0 0x55f719c03d50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.496+0000 7f71ef31c740 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f719b873f0 0x55f719bffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.496+0000 7f71cdffb640 0 client.15921 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:03:30.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.496+0000 7f71ef31c740 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f719b77050 0x55f719b8d030 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.496+0000 7f71ef31c740 1 --2- 172.21.15.44:0/2452225174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f719b76730 0x55f719b8caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.496+0000 7f71ef31c740 1 -- 172.21.15.44:0/2452225174 >> 172.21.15.44:0/2452225174 conn(0x55f719b74560 msgr2=0x55f719b79870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:30.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.497+0000 7f71ef31c740 1 -- 172.21.15.44:0/2452225174 shutdown_connections 2024-08-20T03:03:30.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.497+0000 7f71ef31c740 1 -- 172.21.15.44:0/2452225174 wait complete. 2024-08-20T03:03:30.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.497+0000 7f71ef31c740 20 client.15921 trim_cache size 0 max 16384 2024-08-20T03:03:30.501 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:30.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.498+0000 7f71ecf23640 1 -- 172.21.15.44:0/2442152263 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f71d00b3e50 msgr2=0x7f71d00b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.498+0000 7f71ecf23640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f71d00b3e50 0x7f71d00b6310 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55f719b7a970 tx=0x7f71d8005f90 comp rx=0 tx=0).stop 2024-08-20T03:03:30.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.498+0000 7f71ecf23640 1 -- 172.21.15.44:0/2442152263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e8107d20 msgr2=0x7f71e819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.498+0000 7f71ecf23640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e8107d20 0x7f71e819e820 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f71e0005cc0 tx=0x7f71e0031700 comp rx=0 tx=0).stop 2024-08-20T03:03:30.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.498+0000 7f71ecf23640 1 -- 172.21.15.44:0/2442152263 shutdown_connections 2024-08-20T03:03:30.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.498+0000 7f71ecf23640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f71d00b3e50 0x7f71d00b6310 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.498+0000 7f71ecf23640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71e8107d20 0x7f71e819e820 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.498+0000 7f71ecf23640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71e81073a0 0x7f71e8199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.498+0000 7f71ecf23640 1 --2- 172.21.15.44:0/2442152263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e8101990 0x7f71e8198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.498+0000 7f71ecf23640 1 -- 172.21.15.44:0/2442152263 >> 172.21.15.44:0/2442152263 conn(0x7f71e80fd400 msgr2=0x7f71e806a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:30.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.498+0000 7f71ecf23640 1 -- 172.21.15.44:0/2442152263 shutdown_connections 2024-08-20T03:03:30.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:30.498+0000 7f71ecf23640 1 -- 172.21.15.44:0/2442152263 wait complete. 2024-08-20T03:03:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.569+0000 7f653f060640 1 -- 172.21.15.44:0/1512886317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65381029e0 msgr2=0x7f6538102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.569+0000 7f653f060640 1 --2- 172.21.15.44:0/1512886317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65381029e0 0x7f6538102dc0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f652c009900 tx=0x7f652c02a410 comp rx=0 tx=0).stop 2024-08-20T03:03:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.570+0000 7f653f060640 1 -- 172.21.15.44:0/1512886317 shutdown_connections 2024-08-20T03:03:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.570+0000 7f653f060640 1 --2- 172.21.15.44:0/1512886317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6538108d60 0x7f653810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.570+0000 7f653f060640 1 --2- 172.21.15.44:0/1512886317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65381083e0 0x7f6538108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.570+0000 7f653f060640 1 --2- 172.21.15.44:0/1512886317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65381029e0 0x7f6538102dc0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.570+0000 7f653f060640 1 -- 172.21.15.44:0/1512886317 >> 172.21.15.44:0/1512886317 conn(0x7f65380fe420 msgr2=0x7f6538100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.570+0000 7f653f060640 1 -- 172.21.15.44:0/1512886317 shutdown_connections 2024-08-20T03:03:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.570+0000 7f653f060640 1 -- 172.21.15.44:0/1512886317 wait complete. 2024-08-20T03:03:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.570+0000 7f653f060640 1 Processor -- start 2024-08-20T03:03:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.570+0000 7f653f060640 1 -- start start 2024-08-20T03:03:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.570+0000 7f653f060640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65381029e0 0x7f65380751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f653f060640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65381083e0 0x7f6538075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f653f060640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6538108d60 0x7f6538075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f653f060640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6538112a20 con 0x7f65381083e0 2024-08-20T03:03:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f653f060640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f65381128a0 con 0x7f65381029e0 2024-08-20T03:03:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f653f060640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6538112ba0 con 0x7f6538108d60 2024-08-20T03:03:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f6537fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65381083e0 0x7f6538075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f6537fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65381083e0 0x7f6538075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37304/0 (socket says 172.21.15.44:37304) 2024-08-20T03:03:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f6537fff640 1 -- 172.21.15.44:0/642288586 learned_addr learned my addr 172.21.15.44:0/642288586 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f653d5fd640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6538108d60 0x7f6538075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f653cdfc640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65381029e0 0x7f65380751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f6537fff640 1 -- 172.21.15.44:0/642288586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6538108d60 msgr2=0x7f6538075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f6537fff640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6538108d60 0x7f6538075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f6537fff640 1 -- 172.21.15.44:0/642288586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65381029e0 msgr2=0x7f65380751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f6537fff640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65381029e0 0x7f65380751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f6537fff640 1 -- 172.21.15.44:0/642288586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65381a7380 con 0x7f65381083e0 2024-08-20T03:03:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f653d5fd640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6538108d60 0x7f6538075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f653cdfc640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65381029e0 0x7f65380751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f6537fff640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65381083e0 0x7f6538075710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f652400e9f0 tx=0x7f652400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f6535ffb640 1 -- 172.21.15.44:0/642288586 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652400cd80 con 0x7f65381083e0 2024-08-20T03:03:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f6535ffb640 1 -- 172.21.15.44:0/642288586 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65240108e0 con 0x7f65381083e0 2024-08-20T03:03:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f653f060640 1 -- 172.21.15.44:0/642288586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65381a7670 con 0x7f65381083e0 2024-08-20T03:03:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.571+0000 7f6535ffb640 1 -- 172.21.15.44:0/642288586 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524018980 con 0x7f65381083e0 2024-08-20T03:03:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.572+0000 7f653f060640 1 -- 172.21.15.44:0/642288586 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65381a7bc0 con 0x7f65381083e0 2024-08-20T03:03:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.574+0000 7f6541e5c740 1 Processor -- start 2024-08-20T03:03:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.574+0000 7f6541e5c740 1 -- start start 2024-08-20T03:03:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f6541e5c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dcbf0 0x5580c65dcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f6541e5c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dd510 0x5580c65e8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f6541e5c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65e94a0 0x5580c65eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f6541e5c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5580c6552360 con 0x5580c65dcbf0 2024-08-20T03:03:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f6541e5c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5580c6541860 con 0x5580c65dd510 2024-08-20T03:03:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f6541e5c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5580c65458a0 con 0x5580c65e94a0 2024-08-20T03:03:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f6535ffb640 1 -- 172.21.15.44:0/642288586 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 26) ==== 151241+0+0 (secure 0 0 0) 0x7f6524010430 con 0x7f65381083e0 2024-08-20T03:03:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f653d5fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dcbf0 0x5580c65dcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f653d5fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dcbf0 0x5580c65dcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37314/0 (socket says 172.21.15.44:37314) 2024-08-20T03:03:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f653d5fd640 1 -- 172.21.15.44:0/2993548640 learned_addr learned my addr 172.21.15.44:0/2993548640 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f653cdfc640 1 --2- 172.21.15.44:0/2993548640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dd510 0x5580c65e8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f6537fff640 1 --2- 172.21.15.44:0/2993548640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65e94a0 0x5580c65eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f653d5fd640 1 -- 172.21.15.44:0/2993548640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65e94a0 msgr2=0x5580c65eb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f653d5fd640 1 --2- 172.21.15.44:0/2993548640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65e94a0 0x5580c65eb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f653d5fd640 1 -- 172.21.15.44:0/2993548640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dd510 msgr2=0x5580c65e8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f653d5fd640 1 --2- 172.21.15.44:0/2993548640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dd510 0x5580c65e8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.575+0000 7f653d5fd640 1 -- 172.21.15.44:0/2993548640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c64e8e40 con 0x5580c65dcbf0 2024-08-20T03:03:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.576+0000 7f653d5fd640 1 --2- 172.21.15.44:0/2993548640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dcbf0 0x5580c65dcfd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f65300117e0 tx=0x7f6530011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864b5e85e329f862 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.576+0000 7f6522ffd640 1 -- 172.21.15.44:0/2993548640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530011310 con 0x5580c65dcbf0 2024-08-20T03:03:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.576+0000 7f6522ffd640 1 -- 172.21.15.44:0/2993548640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6530003e20 con 0x5580c65dcbf0 2024-08-20T03:03:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.576+0000 7f6522ffd640 1 -- 172.21.15.44:0/2993548640 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300044f0 con 0x5580c65dcbf0 2024-08-20T03:03:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.577+0000 7f6541e5c740 1 -- 172.21.15.44:0/2993548640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dcbf0 msgr2=0x5580c65dcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.577+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2993548640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dcbf0 0x5580c65dcfd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f65300117e0 tx=0x7f6530011ca0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.577+0000 7f6535ffb640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f65280b3dd0 0x7f65280b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.577+0000 7f6541e5c740 1 -- 172.21.15.44:0/2993548640 shutdown_connections 2024-08-20T03:03:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.577+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2993548640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65e94a0 0x5580c65eb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.577+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2993548640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dd510 0x5580c65e8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.577+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2993548640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dcbf0 0x5580c65dcfd0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.577+0000 7f6541e5c740 1 -- 172.21.15.44:0/2993548640 >> 172.21.15.44:0/2993548640 conn(0x5580c65daa20 msgr2=0x5580c65dbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.577+0000 7f653d5fd640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f65280b3dd0 0x7f65280b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.577+0000 7f6535ffb640 1 -- 172.21.15.44:0/642288586 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f65240e4440 con 0x7f65381083e0 2024-08-20T03:03:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.577+0000 7f6541e5c740 1 -- 172.21.15.44:0/2993548640 shutdown_connections 2024-08-20T03:03:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.577+0000 7f6541e5c740 1 -- 172.21.15.44:0/2993548640 wait complete. 2024-08-20T03:03:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.578+0000 7f6541e5c740 1 -- start start 2024-08-20T03:03:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.578+0000 7f653d5fd640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f65280b3dd0 0x7f65280b6290 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6530011620 tx=0x7f6530005bd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.578+0000 7f6541e5c740 1 Processor -- start 2024-08-20T03:03:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.578+0000 7f6541e5c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dcbf0 0x5580c65e8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.578+0000 7f6541e5c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dd510 0x5580c65e6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.578+0000 7f6541e5c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65e94a0 0x5580c65e68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.578+0000 7f6541e5c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5580c647aab0 con 0x5580c65dd510 2024-08-20T03:03:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.578+0000 7f6541e5c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5580c64c8780 con 0x5580c65e94a0 2024-08-20T03:03:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.578+0000 7f6541e5c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5580c65ec180 con 0x5580c65dcbf0 2024-08-20T03:03:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.578+0000 7f6541e5c740 10 client.0 _ll_register_callbacks cb 0x5580c6250a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f6537fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dd510 0x5580c65e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f653cdfc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dcbf0 0x5580c65e8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f653cdfc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dcbf0 0x5580c65e8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:50876/0 (socket says 172.21.15.44:50876) 2024-08-20T03:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f653cdfc640 1 -- 172.21.15.44:0/2839070007 learned_addr learned my addr 172.21.15.44:0/2839070007 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f653cdfc640 1 -- 172.21.15.44:0/2839070007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65e94a0 msgr2=0x5580c65e68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f653cdfc640 1 --2- 172.21.15.44:0/2839070007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65e94a0 0x5580c65e68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f653cdfc640 1 -- 172.21.15.44:0/2839070007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dd510 msgr2=0x5580c65e6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f653cdfc640 1 --2- 172.21.15.44:0/2839070007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dd510 0x5580c65e6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f653cdfc640 1 -- 172.21.15.44:0/2839070007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c62a4790 con 0x5580c65dcbf0 2024-08-20T03:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f6537fff640 1 --2- 172.21.15.44:0/2839070007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dd510 0x5580c65e6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f653cdfc640 1 --2- 172.21.15.44:0/2839070007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dcbf0 0x5580c65e8de0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f652c02a930 tx=0x7f652c031740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1231f226d190e7c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f6517fff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.579+0000 7f6517fff640 1 -- 172.21.15.44:0/2839070007 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c035040 con 0x5580c65dcbf0 2024-08-20T03:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6517fff640 1 -- 172.21.15.44:0/2839070007 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f652c004240 con 0x5580c65dcbf0 2024-08-20T03:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6541e5c740 1 -- 172.21.15.44:0/2839070007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5580c64f0e00 con 0x5580c65dcbf0 2024-08-20T03:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6517fff640 1 -- 172.21.15.44:0/2839070007 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f652c004440 con 0x5580c65dcbf0 2024-08-20T03:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6541e5c740 10 client.25399 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6541e5c740 1 -- 172.21.15.44:0/2839070007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5580c631d620 con 0x5580c65dcbf0 2024-08-20T03:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6517fff640 1 -- 172.21.15.44:0/2839070007 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5580c64f0e00 con 0x5580c65dcbf0 2024-08-20T03:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6517fff640 1 -- 172.21.15.44:0/2839070007 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f652c04e050 con 0x5580c65dcbf0 2024-08-20T03:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6541e5c740 10 client.25399 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6541e5c740 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6541e5c740 20 client.25399 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2839070007 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5580c65f8e20 0x5580c65fb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6541e5c740 4 client.25399 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6541e5c740 1 -- 172.21.15.44:0/2839070007 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580c65542b0 con 0x5580c65f8e20 2024-08-20T03:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.580+0000 7f6537fff640 1 --2- 172.21.15.44:0/2839070007 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5580c65f8e20 0x5580c65fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.581+0000 7f6517fff640 10 client.25399 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.581+0000 7f6537fff640 1 --2- 172.21.15.44:0/2839070007 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5580c65f8e20 0x5580c65fb210 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f07edfc5f132ca0f server_cookie=e1bc6d212c532839 in_seq=0 out_seq=0 2024-08-20T03:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6517fff640 1 -- 172.21.15.44:0/2839070007 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580c65542b0 con 0x5580c65f8e20 2024-08-20T03:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6517fff640 10 client.25399 handle_command_reply: tid=0 2024-08-20T03:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6541e5c740 1 client.25399 shutdown 2024-08-20T03:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6541e5c740 1 -- 172.21.15.44:0/2839070007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dcbf0 msgr2=0x5580c65e8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2839070007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dcbf0 0x5580c65e8de0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f652c02a930 tx=0x7f652c031740 comp rx=0 tx=0).stop 2024-08-20T03:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6541e5c740 1 -- 172.21.15.44:0/2839070007 shutdown_connections 2024-08-20T03:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2839070007 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5580c65f8e20 0x5580c65fb210 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2839070007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65e94a0 0x5580c65e68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6517fff640 0 client.25399 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2839070007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65dd510 0x5580c65e6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2839070007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dcbf0 0x5580c65e8de0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6541e5c740 1 -- 172.21.15.44:0/2839070007 >> 172.21.15.44:0/2839070007 conn(0x5580c65daa20 msgr2=0x5580c65e0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6541e5c740 1 -- 172.21.15.44:0/2839070007 shutdown_connections 2024-08-20T03:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.583+0000 7f6541e5c740 1 -- 172.21.15.44:0/2839070007 wait complete. 2024-08-20T03:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.584+0000 7f6541e5c740 20 client.25399 trim_cache size 0 max 16384 2024-08-20T03:03:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.615+0000 7f6541e5c740 1 Processor -- start 2024-08-20T03:03:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.615+0000 7f6541e5c740 1 -- start start 2024-08-20T03:03:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.615+0000 7f6541e5c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dcbf0 0x5580c65f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.615+0000 7f6541e5c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dd510 0x5580c65e8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.628 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.615+0000 7f6541e5c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 0x5580c65e8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.615+0000 7f6541e5c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65381b03d0 con 0x5580c65e94a0 2024-08-20T03:03:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.615+0000 7f6541e5c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f65381b1f10 con 0x5580c65dcbf0 2024-08-20T03:03:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.615+0000 7f6541e5c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f65381b0aa0 con 0x5580c65dd510 2024-08-20T03:03:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.615+0000 7f653d5fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 0x5580c65e8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f653d5fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 0x5580c65e8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37324/0 (socket says 172.21.15.44:37324) 2024-08-20T03:03:30.629 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f653d5fd640 1 -- 172.21.15.44:0/84633218 learned_addr learned my addr 172.21.15.44:0/84633218 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f6537fff640 1 --2- 172.21.15.44:0/84633218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dd510 0x5580c65e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f653cdfc640 1 --2- 172.21.15.44:0/84633218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dcbf0 0x5580c65f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f653d5fd640 1 -- 172.21.15.44:0/84633218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dd510 msgr2=0x5580c65e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f653d5fd640 1 --2- 172.21.15.44:0/84633218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dd510 0x5580c65e8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.630 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f653d5fd640 1 -- 172.21.15.44:0/84633218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dcbf0 msgr2=0x5580c65f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f653d5fd640 1 --2- 172.21.15.44:0/84633218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dcbf0 0x5580c65f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f653d5fd640 1 -- 172.21.15.44:0/84633218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c65fb580 con 0x5580c65e94a0 2024-08-20T03:03:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f653cdfc640 1 --2- 172.21.15.44:0/84633218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dcbf0 0x5580c65f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f653d5fd640 1 --2- 172.21.15.44:0/84633218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 0x5580c65e8da0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f65300040a0 tx=0x7f6530009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0032c5be0b7290 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f65227fc640 1 -- 172.21.15.44:0/84633218 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530003b70 con 0x5580c65e94a0 2024-08-20T03:03:30.631 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f65227fc640 1 -- 172.21.15.44:0/84633218 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6530030070 con 0x5580c65e94a0 2024-08-20T03:03:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.616+0000 7f65227fc640 1 -- 172.21.15.44:0/84633218 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653001a8a0 con 0x5580c65e94a0 2024-08-20T03:03:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.617+0000 7f6541e5c740 1 -- 172.21.15.44:0/84633218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 msgr2=0x5580c65e8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.617+0000 7f6541e5c740 1 --2- 172.21.15.44:0/84633218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 0x5580c65e8da0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f65300040a0 tx=0x7f6530009500 comp rx=0 tx=0).stop 2024-08-20T03:03:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.617+0000 7f6541e5c740 1 -- 172.21.15.44:0/84633218 shutdown_connections 2024-08-20T03:03:30.632 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.617+0000 7f6541e5c740 1 --2- 172.21.15.44:0/84633218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 0x5580c65e8da0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.617+0000 7f6541e5c740 1 --2- 172.21.15.44:0/84633218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dd510 0x5580c65e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.617+0000 7f6541e5c740 1 --2- 172.21.15.44:0/84633218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dcbf0 0x5580c65f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.617+0000 7f6541e5c740 1 -- 172.21.15.44:0/84633218 >> 172.21.15.44:0/84633218 conn(0x5580c65daa20 msgr2=0x5580c65e1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.617+0000 7f6541e5c740 1 -- 172.21.15.44:0/84633218 shutdown_connections 2024-08-20T03:03:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.617+0000 7f6541e5c740 1 -- 172.21.15.44:0/84633218 wait complete. 2024-08-20T03:03:30.633 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.617+0000 7f6541e5c740 1 -- start start 2024-08-20T03:03:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.617+0000 7f6541e5c740 1 Processor -- start 2024-08-20T03:03:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.618+0000 7f6541e5c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dcbf0 0x5580c65f1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.618+0000 7f6541e5c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dd510 0x5580c65f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.618+0000 7f6541e5c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 0x5580c65ea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.618+0000 7f6541e5c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f65381b1840 con 0x5580c65e94a0 2024-08-20T03:03:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.618+0000 7f6541e5c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f65381b1170 con 0x5580c65dd510 2024-08-20T03:03:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.618+0000 7f6541e5c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f65381ae3b0 con 0x5580c65dcbf0 2024-08-20T03:03:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.618+0000 7f6541e5c740 10 client.0 _ll_register_callbacks cb 0x7f652c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.618+0000 7f653d5fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 0x5580c65ea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.618+0000 7f653d5fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 0x5580c65ea0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37328/0 (socket says 172.21.15.44:37328) 2024-08-20T03:03:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.618+0000 7f653d5fd640 1 -- 172.21.15.44:0/2956694101 learned_addr learned my addr 172.21.15.44:0/2956694101 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.618+0000 7f653cdfc640 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dcbf0 0x5580c65f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.618+0000 7f6537fff640 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dd510 0x5580c65f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f653d5fd640 1 -- 172.21.15.44:0/2956694101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dcbf0 msgr2=0x5580c65f1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f653d5fd640 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dcbf0 0x5580c65f1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f653d5fd640 1 -- 172.21.15.44:0/2956694101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dd510 msgr2=0x5580c65f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:30.636 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f653d5fd640 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dd510 0x5580c65f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f653d5fd640 1 -- 172.21.15.44:0/2956694101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c65ea8d0 con 0x5580c65e94a0 2024-08-20T03:03:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f653d5fd640 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 0x5580c65ea0a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6530004300 tx=0x7f6530011250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234bb48b7145f8ae server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f6517fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f6517fff640 1 -- 172.21.15.44:0/2956694101 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653000f9a0 con 0x5580c65e94a0 2024-08-20T03:03:30.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f6517fff640 1 -- 172.21.15.44:0/2956694101 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f653000fb40 con 0x5580c65e94a0 2024-08-20T03:03:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f6541e5c740 1 -- 172.21.15.44:0/2956694101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6530030070 con 0x5580c65e94a0 2024-08-20T03:03:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f6517fff640 1 -- 172.21.15.44:0/2956694101 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f653000fd40 con 0x5580c65e94a0 2024-08-20T03:03:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f6541e5c740 10 client.15939 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f6541e5c740 1 -- 172.21.15.44:0/2956694101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5580c65f1120 con 0x5580c65e94a0 2024-08-20T03:03:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.619+0000 7f6517fff640 1 -- 172.21.15.44:0/2956694101 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6530030070 con 0x5580c65e94a0 2024-08-20T03:03:30.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.620+0000 7f6517fff640 1 -- 172.21.15.44:0/2956694101 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6530044050 con 0x5580c65e94a0 2024-08-20T03:03:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.620+0000 7f6541e5c740 10 client.15939 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.620+0000 7f6541e5c740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.620+0000 7f6541e5c740 20 client.15939 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.620+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5580c6663ca0 0x5580c6666090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.620+0000 7f6541e5c740 4 client.15939 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:03:30.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.620+0000 7f6541e5c740 1 -- 172.21.15.44:0/2956694101 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580c654fd50 con 0x5580c6663ca0 2024-08-20T03:03:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.620+0000 7f653cdfc640 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5580c6663ca0 0x5580c6666090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.620+0000 7f6517fff640 10 client.15939 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.620+0000 7f653cdfc640 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5580c6663ca0 0x5580c6666090 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e409ec86fef3233 server_cookie=ea583ac56397d01b in_seq=0 out_seq=0 2024-08-20T03:03:30.982 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:30.978+0000 7f6535ffb640 1 -- 172.21.15.44:0/642288586 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f65240e1df0 con 0x7f65381083e0 2024-08-20T03:03:31.316 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-20T03:03:31.681 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:32.094 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:31 smithi044 ceph-mon[18790]: pgmap v351: 97 pgs: 97 active+clean; 7.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 596 KiB/s wr, 60 op/s 2024-08-20T03:03:32.094 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:31 smithi044 ceph-mon[18790]: mgrmap e27: x(active, since 9m), standbys: z, y 2024-08-20T03:03:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.177+0000 7f32c5349640 1 -- 172.21.15.44:0/1685049831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c0107d60 msgr2=0x7f32c010e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.177+0000 7f32c5349640 1 --2- 172.21.15.44:0/1685049831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c0107d60 0x7f32c010e560 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009900 tx=0x7f32b002a410 comp rx=0 tx=0).stop 2024-08-20T03:03:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.178+0000 7f32c5349640 1 -- 172.21.15.44:0/1685049831 shutdown_connections 2024-08-20T03:03:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.178+0000 7f32c5349640 1 --2- 172.21.15.44:0/1685049831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c0107d60 0x7f32c010e560 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.178+0000 7f32c5349640 1 --2- 172.21.15.44:0/1685049831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c01073e0 0x7f32c0107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.178+0000 7f32c5349640 1 --2- 172.21.15.44:0/1685049831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32c01019d0 0x7f32c0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.178+0000 7f32c5349640 1 -- 172.21.15.44:0/1685049831 >> 172.21.15.44:0/1685049831 conn(0x7f32c00fd460 msgr2=0x7f32c00ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.178+0000 7f32c5349640 1 -- 172.21.15.44:0/1685049831 shutdown_connections 2024-08-20T03:03:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.178+0000 7f32c5349640 1 -- 172.21.15.44:0/1685049831 wait complete. 2024-08-20T03:03:32.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.178+0000 7f32c5349640 1 Processor -- start 2024-08-20T03:03:32.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.178+0000 7f32c5349640 1 -- start start 2024-08-20T03:03:32.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32c5349640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c01019d0 0x7f32c0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32c5349640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32c01073e0 0x7f32c0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32c5349640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c0107d60 0x7f32c01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32c5349640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0111a20 con 0x7f32c01019d0 2024-08-20T03:03:32.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32c5349640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01118a0 con 0x7f32c0107d60 2024-08-20T03:03:32.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32c5349640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0111ba0 con 0x7f32c01073e0 2024-08-20T03:03:32.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c01019d0 0x7f32c0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c01019d0 0x7f32c0197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37352/0 (socket says 172.21.15.44:37352) 2024-08-20T03:03:32.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32beffd640 1 -- 172.21.15.44:0/4200973329 learned_addr learned my addr 172.21.15.44:0/4200973329 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:32.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32be7fc640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32c01073e0 0x7f32c0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32bf7fe640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c0107d60 0x7f32c01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32beffd640 1 -- 172.21.15.44:0/4200973329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32c01073e0 msgr2=0x7f32c0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32beffd640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32c01073e0 0x7f32c0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32beffd640 1 -- 172.21.15.44:0/4200973329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c0107d60 msgr2=0x7f32c01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32beffd640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c0107d60 0x7f32c01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32beffd640 1 -- 172.21.15.44:0/4200973329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c01a7320 con 0x7f32c01019d0 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.179+0000 7f32beffd640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c01019d0 0x7f32c0197b40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f32b800eb90 tx=0x7f32b800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.180+0000 7f32b7fff640 1 -- 172.21.15.44:0/4200973329 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8004b20 con 0x7f32c01019d0 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.180+0000 7f32b7fff640 1 -- 172.21.15.44:0/4200973329 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32b8004cc0 con 0x7f32c01019d0 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.180+0000 7f32b7fff640 1 -- 172.21.15.44:0/4200973329 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b8010690 con 0x7f32c01019d0 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.180+0000 7f32c5349640 1 -- 172.21.15.44:0/4200973329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c01a7610 con 0x7f32c01019d0 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.180+0000 7f32c5349640 1 -- 172.21.15.44:0/4200973329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c01a7b00 con 0x7f32c01019d0 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.182+0000 7f32c7742740 1 Processor -- start 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.182+0000 7f32b7fff640 1 -- 172.21.15.44:0/4200973329 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f32b8005020 con 0x7f32c01019d0 2024-08-20T03:03:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.182+0000 7f32c7742740 1 -- start start 2024-08-20T03:03:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.182+0000 7f32c7742740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467a880 0x55ebf467ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32c7742740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf467b1a0 0x55ebf4686bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32c7742740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf4687130 0x55ebf4689520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32c7742740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ebf45f1930 con 0x55ebf467b1a0 2024-08-20T03:03:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32c7742740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ebf45e0e30 con 0x55ebf4687130 2024-08-20T03:03:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32c7742740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ebf45e4e70 con 0x55ebf467a880 2024-08-20T03:03:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32bf7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467a880 0x55ebf467ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf467b1a0 0x55ebf4686bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32bf7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467a880 0x55ebf467ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:50930/0 (socket says 172.21.15.44:50930) 2024-08-20T03:03:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32bf7fe640 1 -- 172.21.15.44:0/4117834154 learned_addr learned my addr 172.21.15.44:0/4117834154 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32be7fc640 1 --2- 172.21.15.44:0/4117834154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf4687130 0x55ebf4689520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32bf7fe640 1 -- 172.21.15.44:0/4117834154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf4687130 msgr2=0x55ebf4689520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32bf7fe640 1 --2- 172.21.15.44:0/4117834154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf4687130 0x55ebf4689520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32bf7fe640 1 -- 172.21.15.44:0/4117834154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf467b1a0 msgr2=0x55ebf4686bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32bf7fe640 1 --2- 172.21.15.44:0/4117834154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf467b1a0 0x55ebf4686bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32bf7fe640 1 -- 172.21.15.44:0/4117834154 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf4340650 con 0x55ebf467a880 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32beffd640 1 --2- 172.21.15.44:0/4117834154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf467b1a0 0x55ebf4686bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32b7fff640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f32ac0b3d40 0x7f32ac0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32bf7fe640 1 --2- 172.21.15.44:0/4117834154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467a880 0x55ebf467ac60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f32b0033000 tx=0x7f32b0032dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca9be37dd67c9cbd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.183+0000 7f32beffd640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f32ac0b3d40 0x7f32ac0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.184+0000 7f32b7fff640 1 -- 172.21.15.44:0/4200973329 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f32b80e4680 con 0x7f32c01019d0 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.184+0000 7f32b57fa640 1 -- 172.21.15.44:0/4117834154 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0035030 con 0x55ebf467a880 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.184+0000 7f32b57fa640 1 -- 172.21.15.44:0/4117834154 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32b00316a0 con 0x55ebf467a880 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.184+0000 7f32b57fa640 1 -- 172.21.15.44:0/4117834154 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0005630 con 0x55ebf467a880 2024-08-20T03:03:32.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.184+0000 7f32c7742740 1 -- 172.21.15.44:0/4117834154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467a880 msgr2=0x55ebf467ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.184+0000 7f32beffd640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f32ac0b3d40 0x7f32ac0b6200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f32b80e1ef0 tx=0x7f32b80ea550 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.184+0000 7f32c7742740 1 --2- 172.21.15.44:0/4117834154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467a880 0x55ebf467ac60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f32b0033000 tx=0x7f32b0032dc0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 -- 172.21.15.44:0/4117834154 shutdown_connections 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 --2- 172.21.15.44:0/4117834154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf4687130 0x55ebf4689520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 --2- 172.21.15.44:0/4117834154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf467b1a0 0x55ebf4686bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 --2- 172.21.15.44:0/4117834154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467a880 0x55ebf467ac60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 -- 172.21.15.44:0/4117834154 >> 172.21.15.44:0/4117834154 conn(0x55ebf46786b0 msgr2=0x55ebf4679850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 -- 172.21.15.44:0/4117834154 shutdown_connections 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 -- 172.21.15.44:0/4117834154 wait complete. 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 -- start start 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 Processor -- start 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467a880 0x55ebf4686be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467b1a0 0x55ebf4684070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf4687130 0x55ebf4684800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55ebf4443700 con 0x55ebf4687130 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ebf4567810 con 0x55ebf467a880 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55ebf4689d80 con 0x55ebf467b1a0 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32c7742740 10 client.0 _ll_register_callbacks cb 0x55ebf45da700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32be7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467b1a0 0x55ebf4684070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.185+0000 7f32bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf4687130 0x55ebf4684800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32bf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467a880 0x55ebf4686be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32be7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467b1a0 0x55ebf4684070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:50940/0 (socket says 172.21.15.44:50940) 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32be7fc640 1 -- 172.21.15.44:0/1361589456 learned_addr learned my addr 172.21.15.44:0/1361589456 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32bf7fe640 1 -- 172.21.15.44:0/1361589456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467b1a0 msgr2=0x55ebf4684070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32bf7fe640 1 --2- 172.21.15.44:0/1361589456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467b1a0 0x55ebf4684070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32bf7fe640 1 -- 172.21.15.44:0/1361589456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf4687130 msgr2=0x55ebf4684800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:32.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32bf7fe640 1 --2- 172.21.15.44:0/1361589456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf4687130 0x55ebf4684800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32bf7fe640 1 -- 172.21.15.44:0/1361589456 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf46930b0 con 0x55ebf467a880 2024-08-20T03:03:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32be7fc640 1 --2- 172.21.15.44:0/1361589456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467b1a0 0x55ebf4684070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32bf7fe640 1 --2- 172.21.15.44:0/1361589456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467a880 0x55ebf4686be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009850 tx=0x7f32b0031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=668c43e3eb4afd57 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:03:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32a67fc640 1 -- 172.21.15.44:0/1361589456 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0035040 con 0x55ebf467a880 2024-08-20T03:03:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32c7742740 1 -- 172.21.15.44:0/1361589456 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ebf4590150 con 0x55ebf467a880 2024-08-20T03:03:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32a67fc640 1 -- 172.21.15.44:0/1361589456 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f32b0004220 con 0x55ebf467a880 2024-08-20T03:03:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32a67fc640 1 -- 172.21.15.44:0/1361589456 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32b0004420 con 0x55ebf467a880 2024-08-20T03:03:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32c7742740 10 client.25466 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.186+0000 7f32c7742740 1 -- 172.21.15.44:0/1361589456 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ebf4695a60 con 0x55ebf467a880 2024-08-20T03:03:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.187+0000 7f32a67fc640 1 -- 172.21.15.44:0/1361589456 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ebf4590150 con 0x55ebf467a880 2024-08-20T03:03:32.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.187+0000 7f32a67fc640 1 -- 172.21.15.44:0/1361589456 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32b0003710 con 0x55ebf467a880 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.187+0000 7f32c7742740 10 client.25466 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.187+0000 7f32c7742740 10 client.25466 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.187+0000 7f32c7742740 20 client.25466 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.187+0000 7f32c7742740 1 --2- 172.21.15.44:0/1361589456 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55ebf4696db0 0x55ebf46991a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.187+0000 7f32c7742740 4 client.25466 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.187+0000 7f32c7742740 1 -- 172.21.15.44:0/1361589456 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebf45f3880 con 0x55ebf4696db0 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.187+0000 7f32be7fc640 1 --2- 172.21.15.44:0/1361589456 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55ebf4696db0 0x55ebf46991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.187+0000 7f32a67fc640 10 client.25466 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.187+0000 7f32be7fc640 1 --2- 172.21.15.44:0/1361589456 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55ebf4696db0 0x55ebf46991a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a3e70d0e9cb311 server_cookie=7aa02a8ce81439c9 in_seq=0 out_seq=0 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32a67fc640 1 -- 172.21.15.44:0/1361589456 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ebf45f3880 con 0x55ebf4696db0 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32a67fc640 10 client.25466 handle_command_reply: tid=0 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32c7742740 1 client.25466 shutdown 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32c7742740 1 -- 172.21.15.44:0/1361589456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467a880 msgr2=0x55ebf4686be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:32.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32c7742740 1 --2- 172.21.15.44:0/1361589456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467a880 0x55ebf4686be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009850 tx=0x7f32b0031400 comp rx=0 tx=0).stop 2024-08-20T03:03:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32c7742740 1 -- 172.21.15.44:0/1361589456 shutdown_connections 2024-08-20T03:03:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32c7742740 1 --2- 172.21.15.44:0/1361589456 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55ebf4696db0 0x55ebf46991a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32c7742740 1 --2- 172.21.15.44:0/1361589456 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf4687130 0x55ebf4684800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32c7742740 1 --2- 172.21.15.44:0/1361589456 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467b1a0 0x55ebf4684070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32a67fc640 0 client.25466 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:03:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32c7742740 1 --2- 172.21.15.44:0/1361589456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467a880 0x55ebf4686be0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32c7742740 1 -- 172.21.15.44:0/1361589456 >> 172.21.15.44:0/1361589456 conn(0x55ebf46786b0 msgr2=0x55ebf467e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32c7742740 1 -- 172.21.15.44:0/1361589456 shutdown_connections 2024-08-20T03:03:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32c7742740 1 -- 172.21.15.44:0/1361589456 wait complete. 2024-08-20T03:03:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.188+0000 7f32c7742740 20 client.25466 trim_cache size 0 max 16384 2024-08-20T03:03:32.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32c7742740 1 Processor -- start 2024-08-20T03:03:32.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32c7742740 1 -- start start 2024-08-20T03:03:32.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32c7742740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf467a880 0x55ebf46838c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32c7742740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467b1a0 0x55ebf4685d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32c7742740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf4687130 0x55ebf469c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32c7742740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01b03d0 con 0x55ebf467a880 2024-08-20T03:03:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32c7742740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01b1f10 con 0x55ebf467b1a0 2024-08-20T03:03:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32c7742740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01b0aa0 con 0x55ebf4687130 2024-08-20T03:03:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf467a880 0x55ebf46838c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf467a880 0x55ebf46838c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37386/0 (socket says 172.21.15.44:37386) 2024-08-20T03:03:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32bf7fe640 1 -- 172.21.15.44:0/2238079103 learned_addr learned my addr 172.21.15.44:0/2238079103 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32be7fc640 1 --2- 172.21.15.44:0/2238079103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467b1a0 0x55ebf4685d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32bf7fe640 1 --2- 172.21.15.44:0/2238079103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf4687130 0x55ebf469c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32be7fc640 1 -- 172.21.15.44:0/2238079103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf4687130 msgr2=0x55ebf469c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32be7fc640 1 --2- 172.21.15.44:0/2238079103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf4687130 0x55ebf469c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32be7fc640 1 -- 172.21.15.44:0/2238079103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf467a880 msgr2=0x55ebf46838c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32be7fc640 1 --2- 172.21.15.44:0/2238079103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf467a880 0x55ebf46838c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32be7fc640 1 -- 172.21.15.44:0/2238079103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf467afd0 con 0x55ebf467b1a0 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.202+0000 7f32bf7fe640 1 --2- 172.21.15.44:0/2238079103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf4687130 0x55ebf469c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.203+0000 7f32be7fc640 1 --2- 172.21.15.44:0/2238079103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467b1a0 0x55ebf4685d70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55ebf4686920 tx=0x7f32a800a2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=867203af34d9f309 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.203+0000 7f32b4ff9640 1 -- 172.21.15.44:0/2238079103 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a800fad0 con 0x55ebf467b1a0 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.203+0000 7f32b4ff9640 1 -- 172.21.15.44:0/2238079103 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a800fc70 con 0x55ebf467b1a0 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.203+0000 7f32b4ff9640 1 -- 172.21.15.44:0/2238079103 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a801d650 con 0x55ebf467b1a0 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 -- 172.21.15.44:0/2238079103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467b1a0 msgr2=0x55ebf4685d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 --2- 172.21.15.44:0/2238079103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467b1a0 0x55ebf4685d70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55ebf4686920 tx=0x7f32a800a2d0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 -- 172.21.15.44:0/2238079103 shutdown_connections 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 --2- 172.21.15.44:0/2238079103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf4687130 0x55ebf469c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 --2- 172.21.15.44:0/2238079103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467b1a0 0x55ebf4685d70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 --2- 172.21.15.44:0/2238079103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf467a880 0x55ebf46838c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 -- 172.21.15.44:0/2238079103 >> 172.21.15.44:0/2238079103 conn(0x55ebf46786b0 msgr2=0x55ebf4685660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 -- 172.21.15.44:0/2238079103 shutdown_connections 2024-08-20T03:03:32.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 -- 172.21.15.44:0/2238079103 wait complete. 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 -- start start 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 Processor -- start 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467a880 0x55ebf4699990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467b1a0 0x55ebf4699ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf4687130 0x55ebf46ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01b1840 con 0x55ebf4687130 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01b1170 con 0x55ebf467a880 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f32c01ae3b0 con 0x55ebf467b1a0 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.204+0000 7f32c7742740 10 client.0 _ll_register_callbacks cb 0x7f32b80ea420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32bf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467a880 0x55ebf4699990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf4687130 0x55ebf46ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf4687130 0x55ebf46ff6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37388/0 (socket says 172.21.15.44:37388) 2024-08-20T03:03:32.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32bf7fe640 1 -- 172.21.15.44:0/448985826 learned_addr learned my addr 172.21.15.44:0/448985826 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32be7fc640 1 --2- 172.21.15.44:0/448985826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467b1a0 0x55ebf4699ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32bf7fe640 1 -- 172.21.15.44:0/448985826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467b1a0 msgr2=0x55ebf4699ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32bf7fe640 1 --2- 172.21.15.44:0/448985826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467b1a0 0x55ebf4699ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32bf7fe640 1 -- 172.21.15.44:0/448985826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467a880 msgr2=0x55ebf4699990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32bf7fe640 1 --2- 172.21.15.44:0/448985826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467a880 0x55ebf4699990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32bf7fe640 1 -- 172.21.15.44:0/448985826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf46ffcf0 con 0x55ebf4687130 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32bf7fe640 1 --2- 172.21.15.44:0/448985826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf4687130 0x55ebf46ff6e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f32b0003ca0 tx=0x7f32b0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32666cc1fb436d13 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32a67fc640 1 -- 172.21.15.44:0/448985826 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0058070 con 0x55ebf4687130 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32a67fc640 1 -- 172.21.15.44:0/448985826 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f32b0003ff0 con 0x55ebf4687130 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32c7742740 1 -- 172.21.15.44:0/448985826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a800fc70 con 0x55ebf4687130 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.205+0000 7f32a67fc640 1 -- 172.21.15.44:0/448985826 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32b0035040 con 0x55ebf4687130 2024-08-20T03:03:32.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.206+0000 7f32c7742740 10 client.15957 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.206+0000 7f32c7742740 1 -- 172.21.15.44:0/448985826 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ebf4700ff0 con 0x55ebf4687130 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.206+0000 7f32a67fc640 1 -- 172.21.15.44:0/448985826 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a800fc70 con 0x55ebf4687130 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.206+0000 7f32a67fc640 1 -- 172.21.15.44:0/448985826 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32b0069050 con 0x55ebf4687130 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.206+0000 7f32c7742740 10 client.15957 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.206+0000 7f32c7742740 10 client.15957 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.206+0000 7f32c7742740 20 client.15957 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.206+0000 7f32c7742740 1 --2- 172.21.15.44:0/448985826 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55ebf4701330 0x55ebf4703720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.206+0000 7f32c7742740 4 client.15957 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.206+0000 7f32c7742740 1 -- 172.21.15.44:0/448985826 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ebf4673880 con 0x55ebf4701330 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.206+0000 7f32be7fc640 1 --2- 172.21.15.44:0/448985826 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55ebf4701330 0x55ebf4703720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32be7fc640 1 --2- 172.21.15.44:0/448985826 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55ebf4701330 0x55ebf4703720 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c54186230c9809f1 server_cookie=67991420c78e2567 in_seq=0 out_seq=0 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32a67fc640 10 client.15957 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:03:32.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32a67fc640 1 -- 172.21.15.44:0/448985826 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ebf4673880 con 0x55ebf4701330 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32a67fc640 10 client.15957 handle_command_reply: tid=0 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32c7742740 1 client.15957 shutdown 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32c7742740 1 -- 172.21.15.44:0/448985826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf4687130 msgr2=0x55ebf46ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32c7742740 1 --2- 172.21.15.44:0/448985826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf4687130 0x55ebf46ff6e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f32b0003ca0 tx=0x7f32b0005d50 comp rx=0 tx=0).stop 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32c7742740 1 -- 172.21.15.44:0/448985826 shutdown_connections 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32c7742740 1 --2- 172.21.15.44:0/448985826 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55ebf4701330 0x55ebf4703720 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32c7742740 1 --2- 172.21.15.44:0/448985826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebf4687130 0x55ebf46ff6e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32c7742740 1 --2- 172.21.15.44:0/448985826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ebf467b1a0 0x55ebf4699ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32c7742740 1 --2- 172.21.15.44:0/448985826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ebf467a880 0x55ebf4699990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32c7742740 1 -- 172.21.15.44:0/448985826 >> 172.21.15.44:0/448985826 conn(0x55ebf46786b0 msgr2=0x55ebf4685e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32a67fc640 0 client.15957 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32c7742740 1 -- 172.21.15.44:0/448985826 shutdown_connections 2024-08-20T03:03:32.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32c7742740 1 -- 172.21.15.44:0/448985826 wait complete. 2024-08-20T03:03:32.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.207+0000 7f32c7742740 20 client.15957 trim_cache size 0 max 16384 2024-08-20T03:03:32.216 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.208+0000 7f32c5349640 1 -- 172.21.15.44:0/4200973329 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f32ac0b3d40 msgr2=0x7f32ac0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.208+0000 7f32c5349640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f32ac0b3d40 0x7f32ac0b6200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f32b80e1ef0 tx=0x7f32b80ea550 comp rx=0 tx=0).stop 2024-08-20T03:03:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.208+0000 7f32c5349640 1 -- 172.21.15.44:0/4200973329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c01019d0 msgr2=0x7f32c0197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.208+0000 7f32c5349640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c01019d0 0x7f32c0197b40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f32b800eb90 tx=0x7f32b800ef90 comp rx=0 tx=0).stop 2024-08-20T03:03:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.208+0000 7f32c5349640 1 -- 172.21.15.44:0/4200973329 shutdown_connections 2024-08-20T03:03:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.208+0000 7f32c5349640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f32ac0b3d40 0x7f32ac0b6200 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.208+0000 7f32c5349640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32c0107d60 0x7f32c01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.208+0000 7f32c5349640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32c01073e0 0x7f32c0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.208+0000 7f32c5349640 1 --2- 172.21.15.44:0/4200973329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32c01019d0 0x7f32c0197b40 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.208+0000 7f32c5349640 1 -- 172.21.15.44:0/4200973329 >> 172.21.15.44:0/4200973329 conn(0x7f32c00fd460 msgr2=0x7f32c0103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:32.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.208+0000 7f32c5349640 1 -- 172.21.15.44:0/4200973329 shutdown_connections 2024-08-20T03:03:32.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:32.208+0000 7f32c5349640 1 -- 172.21.15.44:0/4200973329 wait complete. 2024-08-20T03:03:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:31 smithi165 ceph-mon[22357]: pgmap v351: 97 pgs: 97 active+clean; 7.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 596 KiB/s wr, 60 op/s 2024-08-20T03:03:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:31 smithi165 ceph-mon[22357]: mgrmap e27: x(active, since 9m), standbys: z, y 2024-08-20T03:03:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:31 smithi099 ceph-mon[21984]: pgmap v351: 97 pgs: 97 active+clean; 7.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 596 KiB/s wr, 60 op/s 2024-08-20T03:03:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:31 smithi099 ceph-mon[21984]: mgrmap e27: x(active, since 9m), standbys: z, y 2024-08-20T03:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.673+0000 7f6517fff640 1 -- 172.21.15.44:0/2956694101 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580c654fd50 con 0x5580c6663ca0 2024-08-20T03:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.673+0000 7f6517fff640 10 client.15939 handle_command_reply: tid=0 2024-08-20T03:03:32.676 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.673+0000 7f6541e5c740 1 client.15939 shutdown 2024-08-20T03:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.674+0000 7f6541e5c740 1 -- 172.21.15.44:0/2956694101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 msgr2=0x5580c65ea0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.674+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 0x5580c65ea0a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6530004300 tx=0x7f6530011250 comp rx=0 tx=0).stop 2024-08-20T03:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.674+0000 7f6541e5c740 1 -- 172.21.15.44:0/2956694101 shutdown_connections 2024-08-20T03:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.674+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5580c6663ca0 0x5580c6666090 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.674+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580c65e94a0 0x5580c65ea0a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.674+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580c65dd510 0x5580c65f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.674+0000 7f6541e5c740 1 --2- 172.21.15.44:0/2956694101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5580c65dcbf0 0x5580c65f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.674+0000 7f6541e5c740 1 -- 172.21.15.44:0/2956694101 >> 172.21.15.44:0/2956694101 conn(0x5580c65daa20 msgr2=0x5580c65d92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.674+0000 7f6517fff640 0 client.15939 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.674+0000 7f6541e5c740 1 -- 172.21.15.44:0/2956694101 shutdown_connections 2024-08-20T03:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.674+0000 7f6541e5c740 1 -- 172.21.15.44:0/2956694101 wait complete. 2024-08-20T03:03:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.674+0000 7f6541e5c740 20 client.15939 trim_cache size 0 max 16384 2024-08-20T03:03:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:03:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:03:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:03:32.678 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:03:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.675+0000 7f653f060640 1 -- 172.21.15.44:0/642288586 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f65280b3dd0 msgr2=0x7f65280b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.675+0000 7f653f060640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f65280b3dd0 0x7f65280b6290 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6530011620 tx=0x7f6530005bd0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.675+0000 7f653f060640 1 -- 172.21.15.44:0/642288586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65381083e0 msgr2=0x7f6538075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.675+0000 7f653f060640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65381083e0 0x7f6538075710 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f652400e9f0 tx=0x7f652400eeb0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.675+0000 7f653f060640 1 -- 172.21.15.44:0/642288586 shutdown_connections 2024-08-20T03:03:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.675+0000 7f653f060640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f65280b3dd0 0x7f65280b6290 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.675+0000 7f653f060640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6538108d60 0x7f6538075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.676+0000 7f653f060640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65381083e0 0x7f6538075710 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.676+0000 7f653f060640 1 --2- 172.21.15.44:0/642288586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65381029e0 0x7f65380751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.676+0000 7f653f060640 1 -- 172.21.15.44:0/642288586 >> 172.21.15.44:0/642288586 conn(0x7f65380fe420 msgr2=0x7f65380ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.676+0000 7f653f060640 1 -- 172.21.15.44:0/642288586 shutdown_connections 2024-08-20T03:03:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:32.676+0000 7f653f060640 1 -- 172.21.15.44:0/642288586 wait complete. 2024-08-20T03:03:33.086 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T03:03:33.441 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:33.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.979+0000 7f6360407640 1 -- 172.21.15.44:0/3928835272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f635810f160 msgr2=0x7f6358111550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:33.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.979+0000 7f6360407640 1 --2- 172.21.15.44:0/3928835272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f635810f160 0x7f6358111550 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6350009ca0 tx=0x7f635002a950 comp rx=0 tx=0).stop 2024-08-20T03:03:33.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.979+0000 7f6360407640 1 -- 172.21.15.44:0/3928835272 shutdown_connections 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.979+0000 7f6360407640 1 --2- 172.21.15.44:0/3928835272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f635810f160 0x7f6358111550 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.979+0000 7f6360407640 1 --2- 172.21.15.44:0/3928835272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6358101ed0 0x7f635810ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.979+0000 7f6360407640 1 --2- 172.21.15.44:0/3928835272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63581015b0 0x7f6358101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.979+0000 7f6360407640 1 -- 172.21.15.44:0/3928835272 >> 172.21.15.44:0/3928835272 conn(0x7f63580fd400 msgr2=0x7f63580ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.980+0000 7f6360407640 1 -- 172.21.15.44:0/3928835272 shutdown_connections 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.980+0000 7f6360407640 1 -- 172.21.15.44:0/3928835272 wait complete. 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.980+0000 7f6360407640 1 Processor -- start 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.980+0000 7f6360407640 1 -- start start 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f6360407640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63581015b0 0x7f635819b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f6360407640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6358101ed0 0x7f635819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f6360407640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f635810f160 0x7f63581a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f6360407640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6358114c70 con 0x7f635810f160 2024-08-20T03:03:33.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f6360407640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6358114af0 con 0x7f63581015b0 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f6360407640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6358114df0 con 0x7f6358101ed0 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f635e9a4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f635810f160 0x7f63581a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f635e9a4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f635810f160 0x7f63581a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37408/0 (socket says 172.21.15.44:37408) 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f635e9a4640 1 -- 172.21.15.44:0/1134622388 learned_addr learned my addr 172.21.15.44:0/1134622388 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f635d9a2640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6358101ed0 0x7f635819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f635e1a3640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63581015b0 0x7f635819b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f635e9a4640 1 -- 172.21.15.44:0/1134622388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6358101ed0 msgr2=0x7f635819b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f635e9a4640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6358101ed0 0x7f635819b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f635e9a4640 1 -- 172.21.15.44:0/1134622388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63581015b0 msgr2=0x7f635819b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f635e9a4640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63581015b0 0x7f635819b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f635e9a4640 1 -- 172.21.15.44:0/1134622388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63581a1190 con 0x7f635810f160 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f635e9a4640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f635810f160 0x7f63581a09c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6350004af0 tx=0x7f635002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f634f7fe640 1 -- 172.21.15.44:0/1134622388 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350034040 con 0x7f635810f160 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f634f7fe640 1 -- 172.21.15.44:0/1134622388 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f635002fe80 con 0x7f635810f160 2024-08-20T03:03:33.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.981+0000 7f6360407640 1 -- 172.21.15.44:0/1134622388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63581a1420 con 0x7f635810f160 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.982+0000 7f634f7fe640 1 -- 172.21.15.44:0/1134622388 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500046a0 con 0x7f635810f160 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.982+0000 7f6360407640 1 -- 172.21.15.44:0/1134622388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63581a7b40 con 0x7f635810f160 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.984+0000 7f634f7fe640 1 -- 172.21.15.44:0/1134622388 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6350037020 con 0x7f635810f160 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.984+0000 7f6362800740 1 Processor -- start 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.984+0000 7f6362800740 1 -- start start 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f6362800740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f6362800740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f6362800740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507d6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f6362800740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d75073e930 con 0x55d7507c81a0 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f6362800740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d75072de30 con 0x55d7507c7880 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f6362800740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d750731e70 con 0x55d7507d4130 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635e1a3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635e1a3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37420/0 (socket says 172.21.15.44:37420) 2024-08-20T03:03:34.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635e1a3640 1 -- 172.21.15.44:0/1739301851 learned_addr learned my addr 172.21.15.44:0/1739301851 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f634f7fe640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f63440b3d40 0x7f63440b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635d9a2640 1 --2- 172.21.15.44:0/1739301851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507d6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635d9a2640 1 --2- 172.21.15.44:0/1739301851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635e9a4640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f63440b3d40 0x7f63440b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635e1a3640 1 -- 172.21.15.44:0/1739301851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 msgr2=0x55d7507d6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635e1a3640 1 --2- 172.21.15.44:0/1739301851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507d6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635e1a3640 1 -- 172.21.15.44:0/1739301851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 msgr2=0x55d7507c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635e1a3640 1 --2- 172.21.15.44:0/1739301851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635e1a3640 1 -- 172.21.15.44:0/1739301851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75048d650 con 0x55d7507c81a0 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f634f7fe640 1 -- 172.21.15.44:0/1134622388 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6350104d40 con 0x7f635810f160 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635e1a3640 1 --2- 172.21.15.44:0/1739301851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d3bf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6354009d20 tx=0x7f6354011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef16bd05bf52cc0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.985+0000 7f635e9a4640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f63440b3d40 0x7f63440b6200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f63500c7aa0 tx=0x7f63500991c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:34.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.986+0000 7f634cde6640 1 -- 172.21.15.44:0/1739301851 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354004230 con 0x55d7507c81a0 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.986+0000 7f634cde6640 1 -- 172.21.15.44:0/1739301851 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63540043d0 con 0x55d7507c81a0 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.986+0000 7f634cde6640 1 -- 172.21.15.44:0/1739301851 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63540197c0 con 0x55d7507c81a0 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.986+0000 7f6362800740 1 -- 172.21.15.44:0/1739301851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 msgr2=0x55d7507d3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.986+0000 7f6362800740 1 --2- 172.21.15.44:0/1739301851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d3bf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6354009d20 tx=0x7f6354011810 comp rx=0 tx=0).stop 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.986+0000 7f6362800740 1 -- 172.21.15.44:0/1739301851 shutdown_connections 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.986+0000 7f6362800740 1 --2- 172.21.15.44:0/1739301851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507d6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.986+0000 7f6362800740 1 --2- 172.21.15.44:0/1739301851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d3bf0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.986+0000 7f6362800740 1 --2- 172.21.15.44:0/1739301851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.986+0000 7f6362800740 1 -- 172.21.15.44:0/1739301851 >> 172.21.15.44:0/1739301851 conn(0x55d7507c56b0 msgr2=0x55d7507c6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.986+0000 7f6362800740 1 -- 172.21.15.44:0/1739301851 shutdown_connections 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.986+0000 7f6362800740 1 -- 172.21.15.44:0/1739301851 wait complete. 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f6362800740 1 -- start start 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f6362800740 1 Processor -- start 2024-08-20T03:03:34.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f6362800740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f6362800740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f6362800740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f6362800740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d750590700 con 0x55d7507c81a0 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f6362800740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d7506b4810 con 0x55d7507c7880 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f6362800740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d7507d6de0 con 0x55d7507d4130 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f6362800740 10 client.0 _ll_register_callbacks cb 0x55d750727700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f635e1a3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f635e1a3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37422/0 (socket says 172.21.15.44:37422) 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f635e1a3640 1 -- 172.21.15.44:0/542146659 learned_addr learned my addr 172.21.15.44:0/542146659 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.987+0000 7f635d9a2640 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f635d9a2640 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f635e1a3640 1 -- 172.21.15.44:0/542146659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 msgr2=0x55d7507d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:34.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f635e1a3640 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f635e1a3640 1 -- 172.21.15.44:0/542146659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 msgr2=0x55d7507d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f635e1a3640 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507d1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f635e1a3640 1 -- 172.21.15.44:0/542146659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7507e0140 con 0x55d7507c81a0 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f635d9a2640 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f635e1a3640 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d15c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f63540052f0 tx=0x7f6354005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d43e39d534e67 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f6341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f6341ffb640 1 -- 172.21.15.44:0/542146659 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354027070 con 0x55d7507c81a0 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f6362800740 1 -- 172.21.15.44:0/542146659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7506dd150 con 0x55d7507c81a0 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f6341ffb640 1 -- 172.21.15.44:0/542146659 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f63540037f0 con 0x55d7507c81a0 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f6341ffb640 1 -- 172.21.15.44:0/542146659 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6354033050 con 0x55d7507c81a0 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f6362800740 10 client.15969 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f6362800740 1 -- 172.21.15.44:0/542146659 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7507e2af0 con 0x55d7507c81a0 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.988+0000 7f6341ffb640 1 -- 172.21.15.44:0/542146659 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d7506dd150 con 0x55d7507c81a0 2024-08-20T03:03:34.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.989+0000 7f6341ffb640 1 -- 172.21.15.44:0/542146659 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f63540048d0 con 0x55d7507c81a0 2024-08-20T03:03:34.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.989+0000 7f6362800740 10 client.15969 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:34.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.989+0000 7f6362800740 10 client.15969 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:03:34.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.989+0000 7f6362800740 20 client.15969 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:34.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.989+0000 7f6362800740 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d7507e3f00 0x55d7507e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.989+0000 7f6362800740 4 client.15969 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:34.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.989+0000 7f6362800740 1 -- 172.21.15.44:0/542146659 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d750740880 con 0x55d7507e3f00 2024-08-20T03:03:34.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.989+0000 7f635d9a2640 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d7507e3f00 0x55d7507e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.989+0000 7f635d9a2640 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d7507e3f00 0x55d7507e62f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1354cf3c7c903278 server_cookie=e5339924284b6da5 in_seq=0 out_seq=0 2024-08-20T03:03:34.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.989+0000 7f6341ffb640 10 client.15969 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:03:34.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6341ffb640 1 -- 172.21.15.44:0/542146659 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d750740880 con 0x55d7507e3f00 2024-08-20T03:03:34.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6341ffb640 10 client.15969 handle_command_reply: tid=0 2024-08-20T03:03:34.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6362800740 1 client.15969 shutdown 2024-08-20T03:03:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6362800740 1 -- 172.21.15.44:0/542146659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 msgr2=0x55d7507d15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6362800740 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d15c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f63540052f0 tx=0x7f6354005320 comp rx=0 tx=0).stop 2024-08-20T03:03:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6362800740 1 -- 172.21.15.44:0/542146659 shutdown_connections 2024-08-20T03:03:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6362800740 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d7507e3f00 0x55d7507e62f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6341ffb640 0 client.15969 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:03:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6362800740 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6362800740 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d15c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6362800740 1 --2- 172.21.15.44:0/542146659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6362800740 1 -- 172.21.15.44:0/542146659 >> 172.21.15.44:0/542146659 conn(0x55d7507c56b0 msgr2=0x55d7507cb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6362800740 1 -- 172.21.15.44:0/542146659 shutdown_connections 2024-08-20T03:03:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6362800740 1 -- 172.21.15.44:0/542146659 wait complete. 2024-08-20T03:03:34.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:33.990+0000 7f6362800740 20 client.15969 trim_cache size 0 max 16384 2024-08-20T03:03:34.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.003+0000 7f6362800740 1 Processor -- start 2024-08-20T03:03:34.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.003+0000 7f6362800740 1 -- start start 2024-08-20T03:03:34.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f6362800740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f6362800740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f6362800740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507ec050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f6362800740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f63581b03c0 con 0x55d7507c81a0 2024-08-20T03:03:34.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f6362800740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63581b1f00 con 0x55d7507c7880 2024-08-20T03:03:34.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f6362800740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f63581b0a90 con 0x55d7507d4130 2024-08-20T03:03:34.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f635d9a2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507ec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f635d9a2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507ec050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:50998/0 (socket says 172.21.15.44:50998) 2024-08-20T03:03:34.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f635d9a2640 1 -- 172.21.15.44:0/3043035608 learned_addr learned my addr 172.21.15.44:0/3043035608 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:34.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f635e1a3640 1 --2- 172.21.15.44:0/3043035608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f635d9a2640 1 --2- 172.21.15.44:0/3043035608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f635e1a3640 1 -- 172.21.15.44:0/3043035608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 msgr2=0x55d7507ec050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:34.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f635e1a3640 1 --2- 172.21.15.44:0/3043035608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507ec050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f635e1a3640 1 -- 172.21.15.44:0/3043035608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 msgr2=0x55d7507d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:34.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f635e1a3640 1 --2- 172.21.15.44:0/3043035608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507d1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f635e1a3640 1 -- 172.21.15.44:0/3043035608 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7507c7fd0 con 0x55d7507c81a0 2024-08-20T03:03:34.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f635d9a2640 1 --2- 172.21.15.44:0/3043035608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:34.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.004+0000 7f635e1a3640 1 --2- 172.21.15.44:0/3043035608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507ebb10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6354001eb0 tx=0x7f6354009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90d3a3742a10c25 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:34.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.005+0000 7f6343fff640 1 -- 172.21.15.44:0/3043035608 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354030020 con 0x55d7507c81a0 2024-08-20T03:03:34.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.005+0000 7f6343fff640 1 -- 172.21.15.44:0/3043035608 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6354009910 con 0x55d7507c81a0 2024-08-20T03:03:34.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.005+0000 7f6343fff640 1 -- 172.21.15.44:0/3043035608 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354027070 con 0x55d7507c81a0 2024-08-20T03:03:34.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.005+0000 7f6362800740 1 -- 172.21.15.44:0/3043035608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 msgr2=0x55d7507ebb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:34.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.005+0000 7f6362800740 1 --2- 172.21.15.44:0/3043035608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507ebb10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6354001eb0 tx=0x7f6354009640 comp rx=0 tx=0).stop 2024-08-20T03:03:34.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.005+0000 7f6362800740 1 -- 172.21.15.44:0/3043035608 shutdown_connections 2024-08-20T03:03:34.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.005+0000 7f6362800740 1 --2- 172.21.15.44:0/3043035608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507ec050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.005+0000 7f6362800740 1 --2- 172.21.15.44:0/3043035608 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507ebb10 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.005+0000 7f6362800740 1 --2- 172.21.15.44:0/3043035608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.005+0000 7f6362800740 1 -- 172.21.15.44:0/3043035608 >> 172.21.15.44:0/3043035608 conn(0x55d7507c56b0 msgr2=0x55d7507d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:34.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.005+0000 7f6362800740 1 -- 172.21.15.44:0/3043035608 shutdown_connections 2024-08-20T03:03:34.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.005+0000 7f6362800740 1 -- 172.21.15.44:0/3043035608 wait complete. 2024-08-20T03:03:34.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f6362800740 1 -- start start 2024-08-20T03:03:34.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f6362800740 1 Processor -- start 2024-08-20T03:03:34.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f6362800740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507dc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f6362800740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f6362800740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507d77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f6362800740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f63581b1830 con 0x55d7507c81a0 2024-08-20T03:03:34.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f6362800740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63581b1160 con 0x55d7507c7880 2024-08-20T03:03:34.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f6362800740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f63581ae3a0 con 0x55d7507d4130 2024-08-20T03:03:34.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f6362800740 10 client.0 _ll_register_callbacks cb 0x7f634800b7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:34.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f635e1a3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f635d9a2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507d77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f635e1a3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37442/0 (socket says 172.21.15.44:37442) 2024-08-20T03:03:34.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f635e1a3640 1 -- 172.21.15.44:0/2171916744 learned_addr learned my addr 172.21.15.44:0/2171916744 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:34.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f635d9a2640 1 --2- 172.21.15.44:0/2171916744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507dc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f635e1a3640 1 -- 172.21.15.44:0/2171916744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 msgr2=0x55d7507d77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:34.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f635e1a3640 1 --2- 172.21.15.44:0/2171916744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507d77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f635e1a3640 1 -- 172.21.15.44:0/2171916744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 msgr2=0x55d7507dc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:34.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f635e1a3640 1 --2- 172.21.15.44:0/2171916744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507dc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.006+0000 7f635e1a3640 1 -- 172.21.15.44:0/2171916744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75084cad0 con 0x55d7507c81a0 2024-08-20T03:03:34.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f635e1a3640 1 --2- 172.21.15.44:0/2171916744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d7030 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6354000c00 tx=0x7f6354009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e3acb355301e7e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:34.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:34.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6341ffb640 1 -- 172.21.15.44:0/2171916744 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354027040 con 0x55d7507c81a0 2024-08-20T03:03:34.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6341ffb640 1 -- 172.21.15.44:0/2171916744 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6354019630 con 0x55d7507c81a0 2024-08-20T03:03:34.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6362800740 1 -- 172.21.15.44:0/2171916744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6354009910 con 0x55d7507c81a0 2024-08-20T03:03:34.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6341ffb640 1 -- 172.21.15.44:0/2171916744 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f63540198a0 con 0x55d7507c81a0 2024-08-20T03:03:34.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6362800740 10 client.15975 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:34.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6362800740 1 -- 172.21.15.44:0/2171916744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d75084dcc0 con 0x55d7507c81a0 2024-08-20T03:03:34.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6341ffb640 1 -- 172.21.15.44:0/2171916744 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6354009910 con 0x55d7507c81a0 2024-08-20T03:03:34.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6341ffb640 1 -- 172.21.15.44:0/2171916744 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6354046020 con 0x55d7507c81a0 2024-08-20T03:03:34.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6362800740 10 client.15975 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:34.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6362800740 10 client.15975 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:03:34.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6362800740 20 client.15975 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:34.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6362800740 1 --2- 172.21.15.44:0/2171916744 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d7507e3f00 0x55d7508502b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:34.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6362800740 4 client.15975 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:34.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.007+0000 7f6362800740 1 -- 172.21.15.44:0/2171916744 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d75073c320 con 0x55d7507e3f00 2024-08-20T03:03:34.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f635d9a2640 1 --2- 172.21.15.44:0/2171916744 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d7507e3f00 0x55d7508502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:34.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6341ffb640 10 client.15975 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:03:34.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f635d9a2640 1 --2- 172.21.15.44:0/2171916744 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d7507e3f00 0x55d7508502b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=590f2529fe669e17 server_cookie=97be614648d19913 in_seq=0 out_seq=0 2024-08-20T03:03:34.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6341ffb640 1 -- 172.21.15.44:0/2171916744 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d75073c320 con 0x55d7507e3f00 2024-08-20T03:03:34.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6341ffb640 10 client.15975 handle_command_reply: tid=0 2024-08-20T03:03:34.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6362800740 1 client.15975 shutdown 2024-08-20T03:03:34.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6362800740 1 -- 172.21.15.44:0/2171916744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 msgr2=0x55d7507d7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:34.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6362800740 1 --2- 172.21.15.44:0/2171916744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d7030 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6354000c00 tx=0x7f6354009cb0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6362800740 1 -- 172.21.15.44:0/2171916744 shutdown_connections 2024-08-20T03:03:34.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6362800740 1 --2- 172.21.15.44:0/2171916744 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55d7507e3f00 0x55d7508502b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6362800740 1 --2- 172.21.15.44:0/2171916744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7507d4130 0x55d7507d77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6362800740 1 --2- 172.21.15.44:0/2171916744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7507c81a0 0x55d7507d7030 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6362800740 1 --2- 172.21.15.44:0/2171916744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7507c7880 0x55d7507dc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6362800740 1 -- 172.21.15.44:0/2171916744 >> 172.21.15.44:0/2171916744 conn(0x55d7507c56b0 msgr2=0x55d7507cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:34.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6341ffb640 0 client.15975 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:03:34.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6362800740 1 -- 172.21.15.44:0/2171916744 shutdown_connections 2024-08-20T03:03:34.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6362800740 1 -- 172.21.15.44:0/2171916744 wait complete. 2024-08-20T03:03:34.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.008+0000 7f6362800740 20 client.15975 trim_cache size 0 max 16384 2024-08-20T03:03:34.019 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:34.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.009+0000 7f6360407640 1 -- 172.21.15.44:0/1134622388 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f63440b3d40 msgr2=0x7f63440b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:34.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.009+0000 7f6360407640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f63440b3d40 0x7f63440b6200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f63500c7aa0 tx=0x7f63500991c0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.009+0000 7f6360407640 1 -- 172.21.15.44:0/1134622388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f635810f160 msgr2=0x7f63581a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:34.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.009+0000 7f6360407640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f635810f160 0x7f63581a09c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6350004af0 tx=0x7f635002f5d0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.009+0000 7f6360407640 1 -- 172.21.15.44:0/1134622388 shutdown_connections 2024-08-20T03:03:34.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.009+0000 7f6360407640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f63440b3d40 0x7f63440b6200 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.009+0000 7f6360407640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f635810f160 0x7f63581a09c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.009+0000 7f6360407640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6358101ed0 0x7f635819b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.009+0000 7f6360407640 1 --2- 172.21.15.44:0/1134622388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63581015b0 0x7f635819b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:34.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.009+0000 7f6360407640 1 -- 172.21.15.44:0/1134622388 >> 172.21.15.44:0/1134622388 conn(0x7f63580fd400 msgr2=0x7f6358111400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:34.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.009+0000 7f6360407640 1 -- 172.21.15.44:0/1134622388 shutdown_connections 2024-08-20T03:03:34.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:34.009+0000 7f6360407640 1 -- 172.21.15.44:0/1134622388 wait complete. 2024-08-20T03:03:34.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:33 smithi044 ceph-mon[18790]: pgmap v352: 97 pgs: 97 active+clean; 11 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 951 KiB/s wr, 131 op/s 2024-08-20T03:03:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:33 smithi165 ceph-mon[22357]: pgmap v352: 97 pgs: 97 active+clean; 11 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 951 KiB/s wr, 131 op/s 2024-08-20T03:03:34.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:33 smithi099 ceph-mon[21984]: pgmap v352: 97 pgs: 97 active+clean; 11 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 951 KiB/s wr, 131 op/s 2024-08-20T03:03:34.898 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T03:03:35.238 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.175+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/3762742098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a4107d20 msgr2=0x7ff0a410e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.175+0000 7ff0a9dd4640 1 --2- 172.21.15.44:0/3762742098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a4107d20 0x7ff0a410e520 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009ca0 tx=0x7ff09c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.175+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/3762742098 shutdown_connections 2024-08-20T03:03:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.175+0000 7ff0a9dd4640 1 --2- 172.21.15.44:0/3762742098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a4107d20 0x7ff0a410e520 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.175+0000 7ff0a9dd4640 1 --2- 172.21.15.44:0/3762742098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0a41073a0 0x7ff0a41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.175+0000 7ff0a9dd4640 1 --2- 172.21.15.44:0/3762742098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0a4101990 0x7ff0a4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.175+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/3762742098 >> 172.21.15.44:0/3762742098 conn(0x7ff0a40fd400 msgr2=0x7ff0a40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.175+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/3762742098 shutdown_connections 2024-08-20T03:03:36.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.175+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/3762742098 wait complete. 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.176+0000 7ff0a9dd4640 1 Processor -- start 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.176+0000 7ff0a9dd4640 1 -- start start 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.176+0000 7ff0a9dd4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a4101990 0x7ff0a4198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.176+0000 7ff0a9dd4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0a41073a0 0x7ff0a4199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.176+0000 7ff0a9dd4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0a4107d20 0x7ff0a419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.176+0000 7ff0a9dd4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41119e0 con 0x7ff0a4101990 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.176+0000 7ff0a9dd4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4111860 con 0x7ff0a41073a0 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.176+0000 7ff0a9dd4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4111b60 con 0x7ff0a4107d20 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.176+0000 7ff0a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a4101990 0x7ff0a4198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.176+0000 7ff0a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a4101990 0x7ff0a4198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37450/0 (socket says 172.21.15.44:37450) 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.176+0000 7ff0a37fe640 1 -- 172.21.15.44:0/4213417596 learned_addr learned my addr 172.21.15.44:0/4213417596 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.176+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0a4107d20 0x7ff0a419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.177+0000 7ff0a2ffd640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0a41073a0 0x7ff0a4199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.177+0000 7ff0a37fe640 1 -- 172.21.15.44:0/4213417596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0a4107d20 msgr2=0x7ff0a419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.177+0000 7ff0a37fe640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0a4107d20 0x7ff0a419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.177+0000 7ff0a37fe640 1 -- 172.21.15.44:0/4213417596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0a41073a0 msgr2=0x7ff0a4199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.177+0000 7ff0a37fe640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0a41073a0 0x7ff0a4199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.177+0000 7ff0a37fe640 1 -- 172.21.15.44:0/4213417596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a419eff0 con 0x7ff0a4101990 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.177+0000 7ff0a37fe640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a4101990 0x7ff0a4198f40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff09400c890 tx=0x7ff09400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.177+0000 7ff0a0ff9640 1 -- 172.21.15.44:0/4213417596 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094004220 con 0x7ff0a4101990 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.177+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/4213417596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a419f2e0 con 0x7ff0a4101990 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.177+0000 7ff0a0ff9640 1 -- 172.21.15.44:0/4213417596 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0940043c0 con 0x7ff0a4101990 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.177+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/4213417596 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a41a5b70 con 0x7ff0a4101990 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.177+0000 7ff0a0ff9640 1 -- 172.21.15.44:0/4213417596 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094004a00 con 0x7ff0a4101990 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0ac1cd740 1 Processor -- start 2024-08-20T03:03:36.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0ac1cd740 1 -- start start 2024-08-20T03:03:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0ac1cd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea340880 0x559eea340c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0ac1cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea3411a0 0x559eea351000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0ac1cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea351540 0x559eea353930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0ac1cd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559eea2b7930 con 0x559eea340880 2024-08-20T03:03:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0ac1cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559eea2a6e30 con 0x559eea3411a0 2024-08-20T03:03:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0ac1cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559eea2aae70 con 0x559eea351540 2024-08-20T03:03:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea340880 0x559eea340c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea340880 0x559eea340c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37466/0 (socket says 172.21.15.44:37466) 2024-08-20T03:03:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0a3fff640 1 -- 172.21.15.44:0/3710450926 learned_addr learned my addr 172.21.15.44:0/3710450926 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0a2ffd640 1 --2- 172.21.15.44:0/3710450926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea351540 0x559eea353930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0a3fff640 1 -- 172.21.15.44:0/3710450926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea351540 msgr2=0x559eea353930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:36.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.180+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/3710450926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea351540 0x559eea353930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.181+0000 7ff0a3fff640 1 -- 172.21.15.44:0/3710450926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea3411a0 msgr2=0x559eea351000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:03:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.181+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/3710450926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea3411a0 0x559eea351000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.181+0000 7ff0a3fff640 1 -- 172.21.15.44:0/3710450926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eea006650 con 0x559eea340880 2024-08-20T03:03:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.181+0000 7ff0a0ff9640 1 -- 172.21.15.44:0/4213417596 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff0940079b0 con 0x7ff0a4101990 2024-08-20T03:03:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.181+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/3710450926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea340880 0x559eea340c60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0098e0 tx=0x7ff09c009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc606d45cbd1511 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.181+0000 7ff099ffb640 1 -- 172.21.15.44:0/3710450926 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c031710 con 0x559eea340880 2024-08-20T03:03:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.181+0000 7ff099ffb640 1 -- 172.21.15.44:0/3710450926 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff09c036070 con 0x559eea340880 2024-08-20T03:03:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.181+0000 7ff099ffb640 1 -- 172.21.15.44:0/3710450926 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0047a0 con 0x559eea340880 2024-08-20T03:03:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0a0ff9640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff0900b3d40 0x7ff0900b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/3710450926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea340880 msgr2=0x559eea340c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/3710450926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea340880 0x559eea340c60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0098e0 tx=0x7ff09c009910 comp rx=0 tx=0).stop 2024-08-20T03:03:36.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0a37fe640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff0900b3d40 0x7ff0900b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/3710450926 shutdown_connections 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/3710450926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea351540 0x559eea353930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/3710450926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea3411a0 0x559eea351000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/3710450926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea340880 0x559eea340c60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/3710450926 >> 172.21.15.44:0/3710450926 conn(0x559eea33e6b0 msgr2=0x559eea33f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0a0ff9640 1 -- 172.21.15.44:0/4213417596 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff0940e3cb0 con 0x7ff0a4101990 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/3710450926 shutdown_connections 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/3710450926 wait complete. 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0a37fe640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff0900b3d40 0x7ff0900b6200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x559eea343a00 tx=0x7ff094092040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.182+0000 7ff0ac1cd740 1 -- start start 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0ac1cd740 1 Processor -- start 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0ac1cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea340880 0x559eea34e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0ac1cd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 0x559eea34ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0ac1cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea351540 0x559eea34efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0ac1cd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559eea109700 con 0x559eea3411a0 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0ac1cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559eea22d810 con 0x559eea351540 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0ac1cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559eea354010 con 0x559eea340880 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0ac1cd740 10 client.0 _ll_register_callbacks cb 0x559eea2a0700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0a2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 0x559eea34ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea340880 0x559eea34e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0a2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 0x559eea34ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37482/0 (socket says 172.21.15.44:37482) 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0a2ffd640 1 -- 172.21.15.44:0/348479533 learned_addr learned my addr 172.21.15.44:0/348479533 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea351540 0x559eea34efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0a2ffd640 1 -- 172.21.15.44:0/348479533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea340880 msgr2=0x559eea34e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0a2ffd640 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea340880 0x559eea34e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0a2ffd640 1 -- 172.21.15.44:0/348479533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea351540 msgr2=0x559eea34efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0a2ffd640 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea351540 0x559eea34efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0a2ffd640 1 -- 172.21.15.44:0/348479533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eea35d440 con 0x559eea3411a0 2024-08-20T03:03:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.183+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea351540 0x559eea34efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff0a2ffd640 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 0x559eea34ea60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff08c011470 tx=0x7ff08c011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f00b5bc53726e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff082ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff082ffd640 1 -- 172.21.15.44:0/348479533 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c004020 con 0x559eea3411a0 2024-08-20T03:03:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff082ffd640 1 -- 172.21.15.44:0/348479533 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff08c0041c0 con 0x559eea3411a0 2024-08-20T03:03:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/348479533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559eea256150 con 0x559eea3411a0 2024-08-20T03:03:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff082ffd640 1 -- 172.21.15.44:0/348479533 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff08c0043c0 con 0x559eea3411a0 2024-08-20T03:03:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff0ac1cd740 10 client.15987 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/348479533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559eea35fdf0 con 0x559eea3411a0 2024-08-20T03:03:36.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff082ffd640 1 -- 172.21.15.44:0/348479533 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559eea256150 con 0x559eea3411a0 2024-08-20T03:03:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff082ffd640 1 -- 172.21.15.44:0/348479533 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff08c022030 con 0x559eea3411a0 2024-08-20T03:03:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff0ac1cd740 10 client.15987 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff0ac1cd740 10 client.15987 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:03:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff0ac1cd740 20 client.15987 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x559eea3613a0 0x559eea3637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff0ac1cd740 4 client.15987 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.184+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/348479533 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559eea2b9880 con 0x559eea3613a0 2024-08-20T03:03:36.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.185+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x559eea3613a0 0x559eea3637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.185+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x559eea3613a0 0x559eea3637c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f9986a31248b2f5 server_cookie=c2159145a395e629 in_seq=0 out_seq=0 2024-08-20T03:03:36.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.185+0000 7ff082ffd640 10 client.15987 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:03:36.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff082ffd640 1 -- 172.21.15.44:0/348479533 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559eea2b9880 con 0x559eea3613a0 2024-08-20T03:03:36.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff082ffd640 10 client.15987 handle_command_reply: tid=0 2024-08-20T03:03:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff0ac1cd740 1 client.15987 shutdown 2024-08-20T03:03:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/348479533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 msgr2=0x559eea34ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 0x559eea34ea60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff08c011470 tx=0x7ff08c011930 comp rx=0 tx=0).stop 2024-08-20T03:03:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/348479533 shutdown_connections 2024-08-20T03:03:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x559eea3613a0 0x559eea3637c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea351540 0x559eea34efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 0x559eea34ea60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff082ffd640 0 client.15987 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:03:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/348479533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea340880 0x559eea34e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/348479533 >> 172.21.15.44:0/348479533 conn(0x559eea33e6b0 msgr2=0x559eea344250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/348479533 shutdown_connections 2024-08-20T03:03:36.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/348479533 wait complete. 2024-08-20T03:03:36.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.186+0000 7ff0ac1cd740 20 client.15987 trim_cache size 0 max 16384 2024-08-20T03:03:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0ac1cd740 1 Processor -- start 2024-08-20T03:03:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0ac1cd740 1 -- start start 2024-08-20T03:03:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0ac1cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea340880 0x559eea34bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0ac1cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea3411a0 0x559eea350260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0ac1cd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea351540 0x559eea369090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0ac1cd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41ac080 con 0x559eea351540 2024-08-20T03:03:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0ac1cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41adbc0 con 0x559eea3411a0 2024-08-20T03:03:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0ac1cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41ac750 con 0x559eea340880 2024-08-20T03:03:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea351540 0x559eea369090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea351540 0x559eea369090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37494/0 (socket says 172.21.15.44:37494) 2024-08-20T03:03:36.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0a3fff640 1 -- 172.21.15.44:0/22316264 learned_addr learned my addr 172.21.15.44:0/22316264 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/22316264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea340880 0x559eea34bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0a2ffd640 1 --2- 172.21.15.44:0/22316264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea3411a0 0x559eea350260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0a3fff640 1 -- 172.21.15.44:0/22316264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea340880 msgr2=0x559eea34bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/22316264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea340880 0x559eea34bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0a3fff640 1 -- 172.21.15.44:0/22316264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea3411a0 msgr2=0x559eea350260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/22316264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea3411a0 0x559eea350260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.200+0000 7ff0a3fff640 1 -- 172.21.15.44:0/22316264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eea34c510 con 0x559eea351540 2024-08-20T03:03:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.201+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/22316264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea351540 0x559eea369090 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009a60 tx=0x7ff09c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fbedfb679ad253 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.201+0000 7ff0997fa640 1 -- 172.21.15.44:0/22316264 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c031920 con 0x559eea351540 2024-08-20T03:03:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.201+0000 7ff0997fa640 1 -- 172.21.15.44:0/22316264 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff09c036040 con 0x559eea351540 2024-08-20T03:03:36.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.201+0000 7ff0997fa640 1 -- 172.21.15.44:0/22316264 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c02f8c0 con 0x559eea351540 2024-08-20T03:03:36.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/22316264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea351540 msgr2=0x559eea369090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:36.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/22316264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea351540 0x559eea369090 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009a60 tx=0x7ff09c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/22316264 shutdown_connections 2024-08-20T03:03:36.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/22316264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea351540 0x559eea369090 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/22316264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea3411a0 0x559eea350260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/22316264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea340880 0x559eea34bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/22316264 >> 172.21.15.44:0/22316264 conn(0x559eea33e6b0 msgr2=0x559eea34fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:36.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/22316264 shutdown_connections 2024-08-20T03:03:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/22316264 wait complete. 2024-08-20T03:03:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 -- start start 2024-08-20T03:03:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 Processor -- start 2024-08-20T03:03:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea340880 0x559eea3596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 0x559eea356bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.202+0000 7ff0ac1cd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea351540 0x559eea3570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0ac1cd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41ad4f0 con 0x559eea3411a0 2024-08-20T03:03:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0ac1cd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41ace20 con 0x559eea340880 2024-08-20T03:03:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0ac1cd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41aa060 con 0x559eea351540 2024-08-20T03:03:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0ac1cd740 10 client.0 _ll_register_callbacks cb 0x7ff08c00bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:36.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0a2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 0x559eea356bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0a2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 0x559eea356bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37506/0 (socket says 172.21.15.44:37506) 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0a3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea351540 0x559eea3570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0a2ffd640 1 -- 172.21.15.44:0/2205552111 learned_addr learned my addr 172.21.15.44:0/2205552111 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/2205552111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea340880 0x559eea3596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0a2ffd640 1 -- 172.21.15.44:0/2205552111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea351540 msgr2=0x559eea3570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0a2ffd640 1 --2- 172.21.15.44:0/2205552111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea351540 0x559eea3570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0a2ffd640 1 -- 172.21.15.44:0/2205552111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea340880 msgr2=0x559eea3596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0a2ffd640 1 --2- 172.21.15.44:0/2205552111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea340880 0x559eea3596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0a2ffd640 1 -- 172.21.15.44:0/2205552111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eea3576b0 con 0x559eea3411a0 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0a2ffd640 1 --2- 172.21.15.44:0/2205552111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 0x559eea356bb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x559eea350e10 tx=0x7ff08c0045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107c62706a545895 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff082ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff082ffd640 1 -- 172.21.15.44:0/2205552111 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c031060 con 0x559eea3411a0 2024-08-20T03:03:36.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff082ffd640 1 -- 172.21.15.44:0/2205552111 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff08c005040 con 0x559eea3411a0 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.203+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/2205552111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff09c036040 con 0x559eea3411a0 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.204+0000 7ff082ffd640 1 -- 172.21.15.44:0/2205552111 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff08c00b480 con 0x559eea3411a0 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.204+0000 7ff0ac1cd740 10 client.15993 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.204+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/2205552111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559eea3cbe20 con 0x559eea3411a0 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.204+0000 7ff082ffd640 1 -- 172.21.15.44:0/2205552111 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff09c036040 con 0x559eea3411a0 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.204+0000 7ff082ffd640 1 -- 172.21.15.44:0/2205552111 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff08c034030 con 0x559eea3411a0 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.204+0000 7ff0ac1cd740 10 client.15993 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.204+0000 7ff0ac1cd740 10 client.15993 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.204+0000 7ff0ac1cd740 20 client.15993 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.204+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/2205552111 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x559eea3cc570 0x559eea3ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.204+0000 7ff0ac1cd740 4 client.15993 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.204+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/2205552111 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559eea2b5320 con 0x559eea3cc570 2024-08-20T03:03:36.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.204+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/2205552111 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x559eea3cc570 0x559eea3ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0a3fff640 1 --2- 172.21.15.44:0/2205552111 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x559eea3cc570 0x559eea3ce990 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58aa5308c9bd8db server_cookie=60ce640818daaddb in_seq=0 out_seq=0 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff082ffd640 10 client.15993 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff082ffd640 1 -- 172.21.15.44:0/2205552111 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559eea2b5320 con 0x559eea3cc570 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff082ffd640 10 client.15993 handle_command_reply: tid=0 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0ac1cd740 1 client.15993 shutdown 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/2205552111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 msgr2=0x559eea356bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/2205552111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 0x559eea356bb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x559eea350e10 tx=0x7ff08c0045d0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/2205552111 shutdown_connections 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/2205552111 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x559eea3cc570 0x559eea3ce990 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/2205552111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559eea351540 0x559eea3570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/2205552111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eea3411a0 0x559eea356bb0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff082ffd640 0 client.15993 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0ac1cd740 1 --2- 172.21.15.44:0/2205552111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559eea340880 0x559eea3596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/2205552111 >> 172.21.15.44:0/2205552111 conn(0x559eea33e6b0 msgr2=0x559eea344250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:36.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/2205552111 shutdown_connections 2024-08-20T03:03:36.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0ac1cd740 1 -- 172.21.15.44:0/2205552111 wait complete. 2024-08-20T03:03:36.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.205+0000 7ff0ac1cd740 20 client.15993 trim_cache size 0 max 16384 2024-08-20T03:03:36.216 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.206+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/4213417596 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff0900b3d40 msgr2=0x7ff0900b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.206+0000 7ff0a9dd4640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff0900b3d40 0x7ff0900b6200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x559eea343a00 tx=0x7ff094092040 comp rx=0 tx=0).stop 2024-08-20T03:03:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.206+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/4213417596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a4101990 msgr2=0x7ff0a4198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.206+0000 7ff0a9dd4640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a4101990 0x7ff0a4198f40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff09400c890 tx=0x7ff09400cd50 comp rx=0 tx=0).stop 2024-08-20T03:03:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.206+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/4213417596 shutdown_connections 2024-08-20T03:03:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.206+0000 7ff0a9dd4640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff0900b3d40 0x7ff0900b6200 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.206+0000 7ff0a9dd4640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0a4107d20 0x7ff0a419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.206+0000 7ff0a9dd4640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0a41073a0 0x7ff0a4199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.206+0000 7ff0a9dd4640 1 --2- 172.21.15.44:0/4213417596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0a4101990 0x7ff0a4198f40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.206+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/4213417596 >> 172.21.15.44:0/4213417596 conn(0x7ff0a40fd400 msgr2=0x7ff0a406aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.206+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/4213417596 shutdown_connections 2024-08-20T03:03:36.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:36.206+0000 7ff0a9dd4640 1 -- 172.21.15.44:0/4213417596 wait complete. 2024-08-20T03:03:36.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:35 smithi044 ceph-mon[18790]: pgmap v353: 97 pgs: 97 active+clean; 28 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-08-20T03:03:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:35 smithi165 ceph-mon[22357]: pgmap v353: 97 pgs: 97 active+clean; 28 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-08-20T03:03:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:35 smithi099 ceph-mon[21984]: pgmap v353: 97 pgs: 97 active+clean; 28 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-08-20T03:03:37.134 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T03:03:37.421 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:37.475 INFO:tasks.workunit.client.0.smithi044.stdout: 19 432497 977 280695 1224 68786 849 2024-08-20T03:03:37.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.968+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/1884143464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd781042c0 msgr2=0x7fdd7810c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.968+0000 7fdd7eeb3640 1 --2- 172.21.15.44:0/1884143464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd781042c0 0x7fdd7810c7b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009900 tx=0x7fdd6402a3f0 comp rx=0 tx=0).stop 2024-08-20T03:03:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.969+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/1884143464 shutdown_connections 2024-08-20T03:03:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.969+0000 7fdd7eeb3640 1 --2- 172.21.15.44:0/1884143464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd7810ce00 0x7fdd7810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.969+0000 7fdd7eeb3640 1 --2- 172.21.15.44:0/1884143464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd781042c0 0x7fdd7810c7b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.969+0000 7fdd7eeb3640 1 --2- 172.21.15.44:0/1884143464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd781039a0 0x7fdd78103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.969+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/1884143464 >> 172.21.15.44:0/1884143464 conn(0x7fdd780fd440 msgr2=0x7fdd780ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.969+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/1884143464 shutdown_connections 2024-08-20T03:03:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.969+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/1884143464 wait complete. 2024-08-20T03:03:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.970+0000 7fdd7eeb3640 1 Processor -- start 2024-08-20T03:03:37.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.970+0000 7fdd7eeb3640 1 -- start start 2024-08-20T03:03:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.970+0000 7fdd7eeb3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd781039a0 0x7fdd78101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.970+0000 7fdd7eeb3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd781042c0 0x7fdd78101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.970+0000 7fdd7eeb3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd7810ce00 0x7fdd781020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.970+0000 7fdd7eeb3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78112ac0 con 0x7fdd781039a0 2024-08-20T03:03:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.970+0000 7fdd7eeb3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78112940 con 0x7fdd7810ce00 2024-08-20T03:03:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.970+0000 7fdd7eeb3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78112c40 con 0x7fdd781042c0 2024-08-20T03:03:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.970+0000 7fdd7cc4f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd781039a0 0x7fdd78101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.970+0000 7fdd7cc4f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd781039a0 0x7fdd78101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46852/0 (socket says 172.21.15.44:46852) 2024-08-20T03:03:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.970+0000 7fdd7cc4f640 1 -- 172.21.15.44:0/4006619192 learned_addr learned my addr 172.21.15.44:0/4006619192 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.970+0000 7fdd77fff640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd781042c0 0x7fdd78101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:37.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.971+0000 7fdd7d450640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd7810ce00 0x7fdd781020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.971+0000 7fdd7cc4f640 1 -- 172.21.15.44:0/4006619192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd781042c0 msgr2=0x7fdd78101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.971+0000 7fdd7cc4f640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd781042c0 0x7fdd78101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.971+0000 7fdd7cc4f640 1 -- 172.21.15.44:0/4006619192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd7810ce00 msgr2=0x7fdd781020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.971+0000 7fdd7cc4f640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd7810ce00 0x7fdd781020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.971+0000 7fdd7cc4f640 1 -- 172.21.15.44:0/4006619192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd781a7430 con 0x7fdd781039a0 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.971+0000 7fdd7d450640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd7810ce00 0x7fdd781020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.971+0000 7fdd7cc4f640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd781039a0 0x7fdd78101650 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00e9f0 tx=0x7fdd6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.971+0000 7fdd75ffb640 1 -- 172.21.15.44:0/4006619192 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c00cd80 con 0x7fdd781039a0 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.972+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/4006619192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd781a7720 con 0x7fdd781039a0 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.972+0000 7fdd75ffb640 1 -- 172.21.15.44:0/4006619192 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd6c0108e0 con 0x7fdd781039a0 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.972+0000 7fdd75ffb640 1 -- 172.21.15.44:0/4006619192 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c018980 con 0x7fdd781039a0 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.972+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/4006619192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd781a7c80 con 0x7fdd781039a0 2024-08-20T03:03:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd75ffb640 1 -- 172.21.15.44:0/4006619192 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fdd6c010430 con 0x7fdd781039a0 2024-08-20T03:03:37.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd812ac740 1 Processor -- start 2024-08-20T03:03:37.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd812ac740 1 -- start start 2024-08-20T03:03:37.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd812ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc21763880 0x55cc21763c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:37.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd812ac740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 0x55cc2176fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:37.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd812ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21770130 0x55cc21772520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:37.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd812ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cc216da930 con 0x55cc21763880 2024-08-20T03:03:37.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd812ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cc216c9e30 con 0x55cc21770130 2024-08-20T03:03:37.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd812ac740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cc216cde70 con 0x55cc217641a0 2024-08-20T03:03:37.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd7cc4f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 0x55cc2176fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:37.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd7d450640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc21763880 0x55cc21763c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:37.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd7cc4f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 0x55cc2176fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45882/0 (socket says 172.21.15.44:45882) 2024-08-20T03:03:37.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd7cc4f640 1 -- 172.21.15.44:0/2413184099 learned_addr learned my addr 172.21.15.44:0/2413184099 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd77fff640 1 --2- 172.21.15.44:0/2413184099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21770130 0x55cc21772520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd7cc4f640 1 -- 172.21.15.44:0/2413184099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21770130 msgr2=0x55cc21772520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd7cc4f640 1 --2- 172.21.15.44:0/2413184099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21770130 0x55cc21772520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd7cc4f640 1 -- 172.21.15.44:0/2413184099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc21763880 msgr2=0x55cc21763c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.975+0000 7fdd7cc4f640 1 --2- 172.21.15.44:0/2413184099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc21763880 0x55cc21763c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.976+0000 7fdd7cc4f640 1 -- 172.21.15.44:0/2413184099 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc21429650 con 0x55cc217641a0 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.976+0000 7fdd75ffb640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdd680b3dc0 0x7fdd680b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.976+0000 7fdd7cc4f640 1 --2- 172.21.15.44:0/2413184099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 0x55cc2176fbf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0e19e0 tx=0x7fdd6c0ec9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=259eb73b17a6df3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.976+0000 7fdd7d450640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdd680b3dc0 0x7fdd680b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.976+0000 7fdd75ffb640 1 -- 172.21.15.44:0/4006619192 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdd6c0e42e0 con 0x7fdd781039a0 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.976+0000 7fdd62ffd640 1 -- 172.21.15.44:0/2413184099 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0a6e20 con 0x55cc217641a0 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.976+0000 7fdd62ffd640 1 -- 172.21.15.44:0/2413184099 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd6c010720 con 0x55cc217641a0 2024-08-20T03:03:37.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.977+0000 7fdd62ffd640 1 -- 172.21.15.44:0/2413184099 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0aa6d0 con 0x55cc217641a0 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.977+0000 7fdd812ac740 1 -- 172.21.15.44:0/2413184099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 msgr2=0x55cc2176fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.977+0000 7fdd7d450640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdd680b3dc0 0x7fdd680b6280 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd700099a0 tx=0x7fdd70009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.977+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2413184099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 0x55cc2176fbf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0e19e0 tx=0x7fdd6c0ec9d0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.977+0000 7fdd812ac740 1 -- 172.21.15.44:0/2413184099 shutdown_connections 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.977+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2413184099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21770130 0x55cc21772520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.977+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2413184099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 0x55cc2176fbf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.977+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2413184099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc21763880 0x55cc21763c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.977+0000 7fdd812ac740 1 -- 172.21.15.44:0/2413184099 >> 172.21.15.44:0/2413184099 conn(0x55cc217616b0 msgr2=0x55cc21762850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.978+0000 7fdd812ac740 1 -- 172.21.15.44:0/2413184099 shutdown_connections 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.978+0000 7fdd812ac740 1 -- 172.21.15.44:0/2413184099 wait complete. 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.978+0000 7fdd812ac740 1 -- start start 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.978+0000 7fdd812ac740 1 Processor -- start 2024-08-20T03:03:38.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.979+0000 7fdd812ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 0x55cc2176faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.979+0000 7fdd812ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc217641a0 0x55cc2176cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.979+0000 7fdd7cc4f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 0x55cc2176faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.979+0000 7fdd7cc4f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 0x55cc2176faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57770/0 (socket says 172.21.15.44:57770) 2024-08-20T03:03:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.979+0000 7fdd812ac740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc21770130 0x55cc2176d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.979+0000 7fdd812ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55cc2152c700 con 0x55cc217641a0 2024-08-20T03:03:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.979+0000 7fdd812ac740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cc21650810 con 0x55cc21763880 2024-08-20T03:03:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.979+0000 7fdd812ac740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55cc21772e80 con 0x55cc21770130 2024-08-20T03:03:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.979+0000 7fdd7cc4f640 1 -- 172.21.15.44:0/2854365294 learned_addr learned my addr 172.21.15.44:0/2854365294 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.979+0000 7fdd7d450640 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc21770130 0x55cc2176d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.979+0000 7fdd77fff640 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc217641a0 0x55cc2176cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:38.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.980+0000 7fdd7d450640 1 -- 172.21.15.44:0/2854365294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 msgr2=0x55cc2176faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.980+0000 7fdd7d450640 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 0x55cc2176faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.980+0000 7fdd7d450640 1 -- 172.21.15.44:0/2854365294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc217641a0 msgr2=0x55cc2176cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.980+0000 7fdd7d450640 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc217641a0 0x55cc2176cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.980+0000 7fdd7d450640 1 -- 172.21.15.44:0/2854365294 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc2177c200 con 0x55cc21770130 2024-08-20T03:03:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.980+0000 7fdd812ac740 10 client.0 _ll_register_callbacks cb 0x55cc216c3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.980+0000 7fdd7d450640 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc21770130 0x55cc2176d610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd70005fd0 tx=0x7fdd70004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a557891416759292 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.980+0000 7fdd57fff640 10 client.25471 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:03:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.980+0000 7fdd812ac740 1 -- 172.21.15.44:0/2854365294 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cc21679150 con 0x55cc21770130 2024-08-20T03:03:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.982+0000 7fdd57fff640 1 -- 172.21.15.44:0/2854365294 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7002c060 con 0x55cc21770130 2024-08-20T03:03:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.983+0000 7fdd57fff640 1 -- 172.21.15.44:0/2854365294 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdd70017030 con 0x55cc21770130 2024-08-20T03:03:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.985+0000 7fdd57fff640 1 -- 172.21.15.44:0/2854365294 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd7001fbc0 con 0x55cc21770130 2024-08-20T03:03:38.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.985+0000 7fdd812ac740 10 client.25471 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.985+0000 7fdd812ac740 1 -- 172.21.15.44:0/2854365294 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55cc2177ea40 con 0x55cc21770130 2024-08-20T03:03:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.986+0000 7fdd57fff640 1 -- 172.21.15.44:0/2854365294 <== mon.1 v2:172.21.15.165:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd70027410 con 0x55cc21770130 2024-08-20T03:03:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.986+0000 7fdd812ac740 10 client.25471 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.986+0000 7fdd812ac740 10 client.25471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.986+0000 7fdd812ac740 20 client.25471 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.986+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55cc2177fe00 0x55cc21782290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.986+0000 7fdd812ac740 4 client.25471 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.986+0000 7fdd812ac740 1 -- 172.21.15.44:0/2854365294 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc216dc880 con 0x55cc2177fe00 2024-08-20T03:03:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.990+0000 7fdd7cc4f640 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55cc2177fe00 0x55cc21782290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.991+0000 7fdd57fff640 10 client.25471 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:38.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.991+0000 7fdd7cc4f640 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55cc2177fe00 0x55cc21782290 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dfc865863c08cae server_cookie=4479acc20e9482ea in_seq=0 out_seq=0 2024-08-20T03:03:38.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.994+0000 7fdd57fff640 1 -- 172.21.15.44:0/2854365294 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc216dc880 con 0x55cc2177fe00 2024-08-20T03:03:38.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.994+0000 7fdd57fff640 10 client.25471 handle_command_reply: tid=0 2024-08-20T03:03:38.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.994+0000 7fdd812ac740 1 client.25471 shutdown 2024-08-20T03:03:38.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.996+0000 7fdd812ac740 1 -- 172.21.15.44:0/2854365294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc21770130 msgr2=0x55cc2176d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:38.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.996+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc21770130 0x55cc2176d610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd70005fd0 tx=0x7fdd70004ab0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.996+0000 7fdd812ac740 1 -- 172.21.15.44:0/2854365294 shutdown_connections 2024-08-20T03:03:38.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.996+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55cc2177fe00 0x55cc21782290 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.996+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc21770130 0x55cc2176d610 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.996+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc217641a0 0x55cc2176cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.996+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2854365294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 0x55cc2176faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.996+0000 7fdd812ac740 1 -- 172.21.15.44:0/2854365294 >> 172.21.15.44:0/2854365294 conn(0x55cc217616b0 msgr2=0x55cc21767250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:38.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:37.997+0000 7fdd57fff640 0 client.25471 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:38.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.000+0000 7fdd812ac740 1 -- 172.21.15.44:0/2854365294 shutdown_connections 2024-08-20T03:03:38.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.000+0000 7fdd812ac740 1 -- 172.21.15.44:0/2854365294 wait complete. 2024-08-20T03:03:38.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.003+0000 7fdd812ac740 20 client.25471 trim_cache size 0 max 16384 2024-08-20T03:03:38.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.036+0000 7fdd812ac740 1 Processor -- start 2024-08-20T03:03:38.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.036+0000 7fdd812ac740 1 -- start start 2024-08-20T03:03:38.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.037+0000 7fdd812ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 0x55cc217678f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:38.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.037+0000 7fdd812ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc217641a0 0x55cc2176c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:38.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.037+0000 7fdd812ac740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc21770130 0x55cc21787bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:38.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.037+0000 7fdd77fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc217641a0 0x55cc2176c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:38.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.037+0000 7fdd77fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc217641a0 0x55cc2176c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46860/0 (socket says 172.21.15.44:46860) 2024-08-20T03:03:38.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.037+0000 7fdd77fff640 1 -- 172.21.15.44:0/2938562470 learned_addr learned my addr 172.21.15.44:0/2938562470 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:38.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.037+0000 7fdd812ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781b0520 con 0x55cc217641a0 2024-08-20T03:03:38.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.037+0000 7fdd812ac740 1 -- 172.21.15.44:0/2938562470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781b2060 con 0x55cc21763880 2024-08-20T03:03:38.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.037+0000 7fdd812ac740 1 -- 172.21.15.44:0/2938562470 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781b0bf0 con 0x55cc21770130 2024-08-20T03:03:38.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.038+0000 7fdd77fff640 1 -- 172.21.15.44:0/2938562470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc21770130 msgr2=0x55cc21787bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:03:38.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.038+0000 7fdd77fff640 1 --2- 172.21.15.44:0/2938562470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc21770130 0x55cc21787bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.038+0000 7fdd77fff640 1 -- 172.21.15.44:0/2938562470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 msgr2=0x55cc217678f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:03:38.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.038+0000 7fdd77fff640 1 --2- 172.21.15.44:0/2938562470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 0x55cc217678f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.038+0000 7fdd77fff640 1 -- 172.21.15.44:0/2938562470 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc2176fe60 con 0x55cc217641a0 2024-08-20T03:03:38.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.039+0000 7fdd77fff640 1 --2- 172.21.15.44:0/2938562470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc217641a0 0x55cc2176c450 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009900 tx=0x7fdd64004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6edbd6528db1b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:38.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.040+0000 7fdd627fc640 1 -- 172.21.15.44:0/2938562470 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64035040 con 0x55cc217641a0 2024-08-20T03:03:38.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.040+0000 7fdd627fc640 1 -- 172.21.15.44:0/2938562470 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd640044c0 con 0x55cc217641a0 2024-08-20T03:03:38.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.040+0000 7fdd627fc640 1 -- 172.21.15.44:0/2938562470 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64003850 con 0x55cc217641a0 2024-08-20T03:03:38.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.041+0000 7fdd812ac740 1 -- 172.21.15.44:0/2938562470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc217641a0 msgr2=0x55cc2176c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:38.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.041+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2938562470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc217641a0 0x55cc2176c450 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009900 tx=0x7fdd64004310 comp rx=0 tx=0).stop 2024-08-20T03:03:38.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.047+0000 7fdd812ac740 1 -- 172.21.15.44:0/2938562470 shutdown_connections 2024-08-20T03:03:38.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.047+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2938562470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc21770130 0x55cc21787bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.047+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2938562470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc217641a0 0x55cc2176c450 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.047+0000 7fdd812ac740 1 --2- 172.21.15.44:0/2938562470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 0x55cc217678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.047+0000 7fdd812ac740 1 -- 172.21.15.44:0/2938562470 >> 172.21.15.44:0/2938562470 conn(0x55cc217616b0 msgr2=0x55cc2176ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:38.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.050+0000 7fdd812ac740 1 -- 172.21.15.44:0/2938562470 shutdown_connections 2024-08-20T03:03:38.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.050+0000 7fdd812ac740 1 -- 172.21.15.44:0/2938562470 wait complete. 2024-08-20T03:03:38.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.051+0000 7fdd812ac740 1 -- start start 2024-08-20T03:03:38.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.051+0000 7fdd812ac740 1 Processor -- start 2024-08-20T03:03:38.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.052+0000 7fdd812ac740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 0x55cc21771f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:38.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.052+0000 7fdd812ac740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 0x55cc21778190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:38.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.052+0000 7fdd812ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc21770130 0x55cc21778920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:38.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.052+0000 7fdd77fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 0x55cc21778190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:38.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.052+0000 7fdd77fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 0x55cc21778190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45900/0 (socket says 172.21.15.44:45900) 2024-08-20T03:03:38.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.052+0000 7fdd77fff640 1 -- 172.21.15.44:0/3437053656 learned_addr learned my addr 172.21.15.44:0/3437053656 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:38.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.052+0000 7fdd812ac740 1 -- 172.21.15.44:0/3437053656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781b1990 con 0x55cc21770130 2024-08-20T03:03:38.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.053+0000 7fdd812ac740 1 -- 172.21.15.44:0/3437053656 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781b12c0 con 0x55cc21763880 2024-08-20T03:03:38.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.053+0000 7fdd812ac740 1 -- 172.21.15.44:0/3437053656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fdd781ae500 con 0x55cc217641a0 2024-08-20T03:03:38.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.053+0000 7fdd7cc4f640 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 0x55cc21771f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:38.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.053+0000 7fdd812ac740 10 client.0 _ll_register_callbacks cb 0x7fdd700109e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:38.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.053+0000 7fdd7d450640 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc21770130 0x55cc21778920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:38.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.053+0000 7fdd7d450640 1 -- 172.21.15.44:0/3437053656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 msgr2=0x55cc21778190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:38.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.053+0000 7fdd7d450640 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 0x55cc21778190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.053+0000 7fdd7d450640 1 -- 172.21.15.44:0/3437053656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 msgr2=0x55cc21771f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:38.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.053+0000 7fdd7d450640 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 0x55cc21771f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.053+0000 7fdd7d450640 1 -- 172.21.15.44:0/3437053656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc217732a0 con 0x55cc21770130 2024-08-20T03:03:38.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.054+0000 7fdd77fff640 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 0x55cc21778190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:38.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.054+0000 7fdd7d450640 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc21770130 0x55cc21778920 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdd70005fd0 tx=0x7fdd70018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4362896e66d04f83 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:38.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.054+0000 7fdd57fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:38.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.055+0000 7fdd57fff640 1 -- 172.21.15.44:0/3437053656 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7002c040 con 0x55cc21770130 2024-08-20T03:03:38.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.055+0000 7fdd812ac740 1 -- 172.21.15.44:0/3437053656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cc216d8320 con 0x55cc21770130 2024-08-20T03:03:38.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.055+0000 7fdd57fff640 1 -- 172.21.15.44:0/3437053656 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fdd70003970 con 0x55cc21770130 2024-08-20T03:03:38.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.057+0000 7fdd57fff640 1 -- 172.21.15.44:0/3437053656 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd7001f800 con 0x55cc21770130 2024-08-20T03:03:38.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.059+0000 7fdd812ac740 10 client.16011 fetch_fsmap learned FSMap version 28 2024-08-20T03:03:38.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.059+0000 7fdd812ac740 1 -- 172.21.15.44:0/3437053656 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc217f3800 con 0x55cc21770130 2024-08-20T03:03:38.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.059+0000 7fdd57fff640 1 -- 172.21.15.44:0/3437053656 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cc216d8320 con 0x55cc21770130 2024-08-20T03:03:38.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.060+0000 7fdd57fff640 1 -- 172.21.15.44:0/3437053656 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdd70038030 con 0x55cc21770130 2024-08-20T03:03:38.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.060+0000 7fdd812ac740 10 client.16011 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T03:03:38.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.060+0000 7fdd812ac740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:38.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.060+0000 7fdd812ac740 20 client.16011 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:38.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.060+0000 7fdd812ac740 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55cc217f3f10 0x55cc217f6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:38.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.060+0000 7fdd812ac740 4 client.16011 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:03:38.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.060+0000 7fdd812ac740 1 -- 172.21.15.44:0/3437053656 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc216d5a60 con 0x55cc217f3f10 2024-08-20T03:03:38.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.061+0000 7fdd7cc4f640 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55cc217f3f10 0x55cc217f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:38.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.062+0000 7fdd7cc4f640 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55cc217f3f10 0x55cc217f6330 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d2e7a51184218c server_cookie=8649140bb664031c in_seq=0 out_seq=0 2024-08-20T03:03:38.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.062+0000 7fdd57fff640 10 client.16011 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:38.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.066+0000 7fdd57fff640 1 -- 172.21.15.44:0/3437053656 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc216d5a60 con 0x55cc217f3f10 2024-08-20T03:03:38.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.066+0000 7fdd57fff640 10 client.16011 handle_command_reply: tid=0 2024-08-20T03:03:38.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.067+0000 7fdd812ac740 1 client.16011 shutdown 2024-08-20T03:03:38.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.067+0000 7fdd812ac740 1 -- 172.21.15.44:0/3437053656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc21770130 msgr2=0x55cc21778920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:38.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.067+0000 7fdd812ac740 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc21770130 0x55cc21778920 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdd70005fd0 tx=0x7fdd70018000 comp rx=0 tx=0).stop 2024-08-20T03:03:38.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.067+0000 7fdd812ac740 1 -- 172.21.15.44:0/3437053656 shutdown_connections 2024-08-20T03:03:38.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.067+0000 7fdd812ac740 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55cc217f3f10 0x55cc217f6330 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.067+0000 7fdd812ac740 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc21770130 0x55cc21778920 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.067+0000 7fdd812ac740 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cc217641a0 0x55cc21778190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.067+0000 7fdd812ac740 1 --2- 172.21.15.44:0/3437053656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cc21763880 0x55cc21771f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.067+0000 7fdd812ac740 1 -- 172.21.15.44:0/3437053656 >> 172.21.15.44:0/3437053656 conn(0x55cc217616b0 msgr2=0x55cc2175ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:38.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.067+0000 7fdd57fff640 0 client.16011 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:38.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.067+0000 7fdd812ac740 1 -- 172.21.15.44:0/3437053656 shutdown_connections 2024-08-20T03:03:38.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.068+0000 7fdd812ac740 1 -- 172.21.15.44:0/3437053656 wait complete. 2024-08-20T03:03:38.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.068+0000 7fdd812ac740 20 client.16011 trim_cache size 0 max 16384 2024-08-20T03:03:38.071 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:03:38.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.069+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/4006619192 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdd680b3dc0 msgr2=0x7fdd680b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:38.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.069+0000 7fdd7eeb3640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdd680b3dc0 0x7fdd680b6280 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd700099a0 tx=0x7fdd70009500 comp rx=0 tx=0).stop 2024-08-20T03:03:38.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.069+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/4006619192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd781039a0 msgr2=0x7fdd78101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:38.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.069+0000 7fdd7eeb3640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd781039a0 0x7fdd78101650 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00e9f0 tx=0x7fdd6c00eeb0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.069+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/4006619192 shutdown_connections 2024-08-20T03:03:38.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.069+0000 7fdd7eeb3640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fdd680b3dc0 0x7fdd680b6280 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.069+0000 7fdd7eeb3640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd7810ce00 0x7fdd781020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.069+0000 7fdd7eeb3640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd781042c0 0x7fdd78101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.069+0000 7fdd7eeb3640 1 --2- 172.21.15.44:0/4006619192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdd781039a0 0x7fdd78101650 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:38.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.070+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/4006619192 >> 172.21.15.44:0/4006619192 conn(0x7fdd780fd440 msgr2=0x7fdd780fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:38.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.071+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/4006619192 shutdown_connections 2024-08-20T03:03:38.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:38.071+0000 7fdd7eeb3640 1 -- 172.21.15.44:0/4006619192 wait complete. 2024-08-20T03:03:38.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:37 smithi044 ceph-mon[18790]: pgmap v354: 97 pgs: 97 active+clean; 40 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 4.0 MiB/s wr, 402 op/s 2024-08-20T03:03:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:37 smithi165 ceph-mon[22357]: pgmap v354: 97 pgs: 97 active+clean; 40 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 4.0 MiB/s wr, 402 op/s 2024-08-20T03:03:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:37 smithi099 ceph-mon[21984]: pgmap v354: 97 pgs: 97 active+clean; 40 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 4.0 MiB/s wr, 402 op/s 2024-08-20T03:03:39.741 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T03:03:39.742 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T03:03:40.021 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:40.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:39 smithi044 ceph-mon[18790]: pgmap v355: 97 pgs: 97 active+clean; 57 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 460 op/s 2024-08-20T03:03:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:39 smithi165 ceph-mon[22357]: pgmap v355: 97 pgs: 97 active+clean; 57 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 460 op/s 2024-08-20T03:03:40.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:39 smithi099 ceph-mon[21984]: pgmap v355: 97 pgs: 97 active+clean; 57 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 460 op/s 2024-08-20T03:03:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.536+0000 7f0e2772a640 1 -- 172.21.15.44:0/2466674803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e20108e20 msgr2=0x7f0e2010f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:40.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.536+0000 7f0e2772a640 1 --2- 172.21.15.44:0/2466674803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e20108e20 0x7f0e2010f620 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009900 tx=0x7f0e1802a410 comp rx=0 tx=0).stop 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.536+0000 7f0e2772a640 1 -- 172.21.15.44:0/2466674803 shutdown_connections 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.536+0000 7f0e2772a640 1 --2- 172.21.15.44:0/2466674803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e20108e20 0x7f0e2010f620 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.536+0000 7f0e2772a640 1 --2- 172.21.15.44:0/2466674803 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e201084a0 0x7f0e201088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.536+0000 7f0e2772a640 1 --2- 172.21.15.44:0/2466674803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e20102a90 0x7f0e20102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.536+0000 7f0e2772a640 1 -- 172.21.15.44:0/2466674803 >> 172.21.15.44:0/2466674803 conn(0x7f0e200fe540 msgr2=0x7f0e20100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.536+0000 7f0e2772a640 1 -- 172.21.15.44:0/2466674803 shutdown_connections 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.536+0000 7f0e2772a640 1 -- 172.21.15.44:0/2466674803 wait complete. 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.537+0000 7f0e2772a640 1 Processor -- start 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.537+0000 7f0e2772a640 1 -- start start 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.537+0000 7f0e2772a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e20102a90 0x7f0e20076c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.537+0000 7f0e2772a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e201084a0 0x7f0e200751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.537+0000 7f0e2772a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e20108e20 0x7f0e20075800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.537+0000 7f0e2772a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20112aa0 con 0x7f0e20102a90 2024-08-20T03:03:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.537+0000 7f0e2772a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20112920 con 0x7f0e201084a0 2024-08-20T03:03:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.537+0000 7f0e2772a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20112c20 con 0x7f0e20108e20 2024-08-20T03:03:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e254c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e20102a90 0x7f0e20076c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e254c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e20102a90 0x7f0e20076c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46896/0 (socket says 172.21.15.44:46896) 2024-08-20T03:03:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e254c6640 1 -- 172.21.15.44:0/9382685 learned_addr learned my addr 172.21.15.44:0/9382685 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e201084a0 0x7f0e200751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e20108e20 0x7f0e20075800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e254c6640 1 -- 172.21.15.44:0/9382685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e20108e20 msgr2=0x7f0e20075800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e254c6640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e20108e20 0x7f0e20075800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e254c6640 1 -- 172.21.15.44:0/9382685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e201084a0 msgr2=0x7f0e200751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:40.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e254c6640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e201084a0 0x7f0e200751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e254c6640 1 -- 172.21.15.44:0/9382685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e201ab8b0 con 0x7f0e20102a90 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e201084a0 0x7f0e200751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e254c6640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e20102a90 0x7f0e20076c80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00ebc0 tx=0x7f0e1c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e167fc640 1 -- 172.21.15.44:0/9382685 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c010630 con 0x7f0e20102a90 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e167fc640 1 -- 172.21.15.44:0/9382685 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e1c010c80 con 0x7f0e20102a90 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e2772a640 1 -- 172.21.15.44:0/9382685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e201aba80 con 0x7f0e20102a90 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e167fc640 1 -- 172.21.15.44:0/9382685 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c018b60 con 0x7f0e20102a90 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.538+0000 7f0e2772a640 1 -- 172.21.15.44:0/9382685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e201abf90 con 0x7f0e20102a90 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.541+0000 7f0e29b23740 1 Processor -- start 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.541+0000 7f0e167fc640 1 -- 172.21.15.44:0/9382685 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0e1c018d00 con 0x7f0e20102a90 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.541+0000 7f0e29b23740 1 -- start start 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.541+0000 7f0e29b23740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 0x559d42891790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.541+0000 7f0e29b23740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d42891cd0 0x559d4289d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.541+0000 7f0e29b23740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d4289dc60 0x559d428a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.541+0000 7f0e29b23740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559d42807940 con 0x559d428913b0 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.541+0000 7f0e29b23740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559d427f6e40 con 0x559d4289dc60 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.541+0000 7f0e29b23740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559d427fae80 con 0x559d42891cd0 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e25cc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 0x559d42891790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e25cc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 0x559d42891790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46910/0 (socket says 172.21.15.44:46910) 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e25cc7640 1 -- 172.21.15.44:0/765011317 learned_addr learned my addr 172.21.15.44:0/765011317 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/765011317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d4289dc60 0x559d428a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e254c6640 1 --2- 172.21.15.44:0/765011317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d42891cd0 0x559d4289d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e25cc7640 1 -- 172.21.15.44:0/765011317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d42891cd0 msgr2=0x559d4289d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/765011317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d42891cd0 0x559d4289d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e25cc7640 1 -- 172.21.15.44:0/765011317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d4289dc60 msgr2=0x559d428a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/765011317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d4289dc60 0x559d428a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e25cc7640 1 -- 172.21.15.44:0/765011317 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d42793520 con 0x559d428913b0 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e254c6640 1 --2- 172.21.15.44:0/765011317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d42891cd0 0x559d4289d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/765011317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d4289dc60 0x559d428a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e167fc640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0e0c0b3d40 0x7f0e0c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/765011317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 0x559d42891790 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009900 tx=0x7f0e18032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3ad2b056d1992e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e254c6640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0e0c0b3d40 0x7f0e0c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e0b7fe640 1 -- 172.21.15.44:0/765011317 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18035040 con 0x559d428913b0 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e0b7fe640 1 -- 172.21.15.44:0/765011317 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e18004030 con 0x559d428913b0 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e0b7fe640 1 -- 172.21.15.44:0/765011317 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e180054d0 con 0x559d428913b0 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.542+0000 7f0e167fc640 1 -- 172.21.15.44:0/9382685 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0e1c0e4560 con 0x7f0e20102a90 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.543+0000 7f0e29b23740 1 -- 172.21.15.44:0/765011317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 msgr2=0x559d42891790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.543+0000 7f0e254c6640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0e0c0b3d40 0x7f0e0c0b6200 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0e1dd0 tx=0x7f0e1c0e9040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.543+0000 7f0e29b23740 1 --2- 172.21.15.44:0/765011317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 0x559d42891790 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009900 tx=0x7f0e18032b20 comp rx=0 tx=0).stop 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.543+0000 7f0e29b23740 1 -- 172.21.15.44:0/765011317 shutdown_connections 2024-08-20T03:03:40.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.543+0000 7f0e29b23740 1 --2- 172.21.15.44:0/765011317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d4289dc60 0x559d428a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.543+0000 7f0e29b23740 1 --2- 172.21.15.44:0/765011317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d42891cd0 0x559d4289d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.543+0000 7f0e29b23740 1 --2- 172.21.15.44:0/765011317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 0x559d42891790 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.543+0000 7f0e29b23740 1 -- 172.21.15.44:0/765011317 >> 172.21.15.44:0/765011317 conn(0x559d4288f1e0 msgr2=0x559d42890380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:40.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.543+0000 7f0e29b23740 1 -- 172.21.15.44:0/765011317 shutdown_connections 2024-08-20T03:03:40.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.543+0000 7f0e29b23740 1 -- 172.21.15.44:0/765011317 wait complete. 2024-08-20T03:03:40.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e29b23740 1 -- start start 2024-08-20T03:03:40.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e29b23740 1 Processor -- start 2024-08-20T03:03:40.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e29b23740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 0x559d4289ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e29b23740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d42891cd0 0x559d4289b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e29b23740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d4289dc60 0x559d4289b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e29b23740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559d42659890 con 0x559d428913b0 2024-08-20T03:03:40.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e29b23740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559d4277d700 con 0x559d42891cd0 2024-08-20T03:03:40.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e29b23740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559d428a0950 con 0x559d4289dc60 2024-08-20T03:03:40.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e29b23740 10 client.0 _ll_register_callbacks cb 0x559d427f26f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:40.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e25cc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 0x559d4289ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e25cc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 0x559d4289ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46924/0 (socket says 172.21.15.44:46924) 2024-08-20T03:03:40.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e25cc7640 1 -- 172.21.15.44:0/854302788 learned_addr learned my addr 172.21.15.44:0/854302788 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:40.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d4289dc60 0x559d4289b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d42891cd0 0x559d4289b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.544+0000 7f0e25cc7640 1 -- 172.21.15.44:0/854302788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d4289dc60 msgr2=0x559d4289b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:40.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d4289dc60 0x559d4289b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e25cc7640 1 -- 172.21.15.44:0/854302788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d42891cd0 msgr2=0x559d4289b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:40.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d42891cd0 0x559d4289b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e25cc7640 1 -- 172.21.15.44:0/854302788 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d42556930 con 0x559d428913b0 2024-08-20T03:03:40.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d42891cd0 0x559d4289b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:03:40.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 0x559d4289ac90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0e1802a930 tx=0x7f0e18005190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29fe1bfb2bfa864 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:40.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:40.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e08ff9640 1 -- 172.21.15.44:0/854302788 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18035070 con 0x559d428913b0 2024-08-20T03:03:40.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e08ff9640 1 -- 172.21.15.44:0/854302788 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0e180314a0 con 0x559d428913b0 2024-08-20T03:03:40.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e29b23740 1 -- 172.21.15.44:0/854302788 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e18004030 con 0x559d428913b0 2024-08-20T03:03:40.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e08ff9640 1 -- 172.21.15.44:0/854302788 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0e1805b050 con 0x559d428913b0 2024-08-20T03:03:40.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e29b23740 10 client.16023 fetch_fsmap learned FSMap version 29 2024-08-20T03:03:40.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e29b23740 1 -- 172.21.15.44:0/854302788 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d428ac570 con 0x559d428913b0 2024-08-20T03:03:40.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e08ff9640 1 -- 172.21.15.44:0/854302788 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e18004030 con 0x559d428913b0 2024-08-20T03:03:40.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.545+0000 7f0e08ff9640 1 -- 172.21.15.44:0/854302788 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f0e180035f0 con 0x559d428913b0 2024-08-20T03:03:40.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.546+0000 7f0e29b23740 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T03:03:40.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.546+0000 7f0e29b23740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:40.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.546+0000 7f0e29b23740 20 client.16023 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:40.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.546+0000 7f0e29b23740 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559d428ad8c0 0x559d428afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.546+0000 7f0e29b23740 4 client.16023 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:40.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.546+0000 7f0e29b23740 1 -- 172.21.15.44:0/854302788 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d42809890 con 0x559d428ad8c0 2024-08-20T03:03:40.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.546+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559d428ad8c0 0x559d428afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.546+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559d428ad8c0 0x559d428afcb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24fb83c565601ad server_cookie=8dce3c49867af0b in_seq=0 out_seq=0 2024-08-20T03:03:40.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.546+0000 7f0e08ff9640 10 client.16023 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:40.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e08ff9640 1 -- 172.21.15.44:0/854302788 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d42809890 con 0x559d428ad8c0 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e08ff9640 10 client.16023 handle_command_reply: tid=0 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e29b23740 1 client.16023 shutdown 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e29b23740 1 -- 172.21.15.44:0/854302788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 msgr2=0x559d4289ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e29b23740 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 0x559d4289ac90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0e1802a930 tx=0x7f0e18005190 comp rx=0 tx=0).stop 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e29b23740 1 -- 172.21.15.44:0/854302788 shutdown_connections 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e29b23740 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559d428ad8c0 0x559d428afcb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e29b23740 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d4289dc60 0x559d4289b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e29b23740 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d42891cd0 0x559d4289b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e29b23740 1 --2- 172.21.15.44:0/854302788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d428913b0 0x559d4289ac90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e29b23740 1 -- 172.21.15.44:0/854302788 >> 172.21.15.44:0/854302788 conn(0x559d4288f1e0 msgr2=0x559d42894d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e08ff9640 0 client.16023 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e29b23740 1 -- 172.21.15.44:0/854302788 shutdown_connections 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.548+0000 7f0e29b23740 1 -- 172.21.15.44:0/854302788 wait complete. 2024-08-20T03:03:40.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.549+0000 7f0e29b23740 20 client.16023 trim_cache size 0 max 16384 2024-08-20T03:03:40.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.567+0000 7f0e29b23740 1 Processor -- start 2024-08-20T03:03:40.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.567+0000 7f0e29b23740 1 -- start start 2024-08-20T03:03:40.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.567+0000 7f0e29b23740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d428913b0 0x559d4289c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e29b23740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d42891cd0 0x559d428b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e29b23740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d4289dc60 0x559d428b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e29b23740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0e201b4810 con 0x559d42891cd0 2024-08-20T03:03:40.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e29b23740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0e201b6380 con 0x559d4289dc60 2024-08-20T03:03:40.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e29b23740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0e201b4ee0 con 0x559d428913b0 2024-08-20T03:03:40.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e24cc5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d42891cd0 0x559d428b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e25cc7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d428913b0 0x559d4289c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e24cc5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d42891cd0 0x559d428b52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46940/0 (socket says 172.21.15.44:46940) 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e24cc5640 1 -- 172.21.15.44:0/1815103072 learned_addr learned my addr 172.21.15.44:0/1815103072 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/1815103072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d4289dc60 0x559d428b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e24cc5640 1 -- 172.21.15.44:0/1815103072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d428913b0 msgr2=0x559d4289c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/1815103072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d428913b0 0x559d4289c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e24cc5640 1 -- 172.21.15.44:0/1815103072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d4289dc60 msgr2=0x559d428b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/1815103072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d4289dc60 0x559d428b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e24cc5640 1 -- 172.21.15.44:0/1815103072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d428a0440 con 0x559d42891cd0 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/1815103072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d428913b0 0x559d4289c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/1815103072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d42891cd0 0x559d428b52e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0e20104f00 tx=0x7f0e10012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1c48ce99716483 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e0affd640 1 -- 172.21.15.44:0/1815103072 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10012390 con 0x559d42891cd0 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e0affd640 1 -- 172.21.15.44:0/1815103072 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e10010650 con 0x559d42891cd0 2024-08-20T03:03:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.568+0000 7f0e0affd640 1 -- 172.21.15.44:0/1815103072 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10003850 con 0x559d42891cd0 2024-08-20T03:03:40.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.569+0000 7f0e29b23740 1 -- 172.21.15.44:0/1815103072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d42891cd0 msgr2=0x559d428b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:40.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.569+0000 7f0e29b23740 1 --2- 172.21.15.44:0/1815103072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d42891cd0 0x559d428b52e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0e20104f00 tx=0x7f0e10012b00 comp rx=0 tx=0).stop 2024-08-20T03:03:40.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.569+0000 7f0e29b23740 1 -- 172.21.15.44:0/1815103072 shutdown_connections 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.569+0000 7f0e29b23740 1 --2- 172.21.15.44:0/1815103072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d4289dc60 0x559d428b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.569+0000 7f0e29b23740 1 --2- 172.21.15.44:0/1815103072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d42891cd0 0x559d428b52e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.569+0000 7f0e29b23740 1 --2- 172.21.15.44:0/1815103072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d428913b0 0x559d4289c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.569+0000 7f0e29b23740 1 -- 172.21.15.44:0/1815103072 >> 172.21.15.44:0/1815103072 conn(0x559d4288f1e0 msgr2=0x559d4289c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.569+0000 7f0e29b23740 1 -- 172.21.15.44:0/1815103072 shutdown_connections 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.569+0000 7f0e29b23740 1 -- 172.21.15.44:0/1815103072 wait complete. 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e29b23740 1 -- start start 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e29b23740 1 Processor -- start 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e29b23740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d428913b0 0x559d428a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e29b23740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d42891cd0 0x559d428a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e29b23740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d4289dc60 0x559d428a2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e29b23740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0e201b5c80 con 0x559d4289dc60 2024-08-20T03:03:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e29b23740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0e201b55b0 con 0x559d428913b0 2024-08-20T03:03:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e29b23740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0e201b27f0 con 0x559d42891cd0 2024-08-20T03:03:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e29b23740 10 client.0 _ll_register_callbacks cb 0x7f0e10009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e25cc7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d4289dc60 0x559d428a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e25cc7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d428913b0 0x559d428a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e25cc7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d428913b0 0x559d428a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57832/0 (socket says 172.21.15.44:57832) 2024-08-20T03:03:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e25cc7640 1 -- 172.21.15.44:0/477272246 learned_addr learned my addr 172.21.15.44:0/477272246 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.570+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/477272246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d42891cd0 0x559d428a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e25cc7640 1 -- 172.21.15.44:0/477272246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d42891cd0 msgr2=0x559d428a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/477272246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d42891cd0 0x559d428a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e25cc7640 1 -- 172.21.15.44:0/477272246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d4289dc60 msgr2=0x559d428a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:40.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/477272246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d4289dc60 0x559d428a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e25cc7640 1 -- 172.21.15.44:0/477272246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d429172b0 con 0x559d428913b0 2024-08-20T03:03:40.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e25cc7640 1 --2- 172.21.15.44:0/477272246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d428913b0 0x559d428a19f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0e18001eb0 tx=0x7f0e18000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f0bbb841a5a7a4e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:40.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:03:40.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e08ff9640 1 -- 172.21.15.44:0/477272246 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18035070 con 0x559d428913b0 2024-08-20T03:03:40.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e08ff9640 1 -- 172.21.15.44:0/477272246 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0e18003e40 con 0x559d428913b0 2024-08-20T03:03:40.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e29b23740 1 -- 172.21.15.44:0/477272246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e10010650 con 0x559d428913b0 2024-08-20T03:03:40.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e08ff9640 1 -- 172.21.15.44:0/477272246 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0e18004040 con 0x559d428913b0 2024-08-20T03:03:40.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e29b23740 10 client.25526 fetch_fsmap learned FSMap version 29 2024-08-20T03:03:40.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.571+0000 7f0e29b23740 1 -- 172.21.15.44:0/477272246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d429185f0 con 0x559d428913b0 2024-08-20T03:03:40.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.572+0000 7f0e08ff9640 1 -- 172.21.15.44:0/477272246 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e10010650 con 0x559d428913b0 2024-08-20T03:03:40.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.572+0000 7f0e08ff9640 1 -- 172.21.15.44:0/477272246 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f0e18004af0 con 0x559d428913b0 2024-08-20T03:03:40.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.572+0000 7f0e29b23740 10 client.25526 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T03:03:40.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.572+0000 7f0e29b23740 10 client.25526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:40.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.572+0000 7f0e29b23740 20 client.25526 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:40.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.572+0000 7f0e29b23740 1 --2- 172.21.15.44:0/477272246 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559d42918cf0 0x559d4291b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:40.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.572+0000 7f0e29b23740 4 client.25526 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T03:03:40.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.572+0000 7f0e29b23740 1 -- 172.21.15.44:0/477272246 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559d42805330 con 0x559d42918cf0 2024-08-20T03:03:40.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.572+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/477272246 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559d42918cf0 0x559d4291b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:40.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.572+0000 7f0e24cc5640 1 --2- 172.21.15.44:0/477272246 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559d42918cf0 0x559d4291b110 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e67f11fba79d6e server_cookie=a4cad0916b5d3323 in_seq=0 out_seq=0 2024-08-20T03:03:40.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.572+0000 7f0e08ff9640 10 client.25526 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:40.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e08ff9640 1 -- 172.21.15.44:0/477272246 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559d42805330 con 0x559d42918cf0 2024-08-20T03:03:40.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e08ff9640 10 client.25526 handle_command_reply: tid=0 2024-08-20T03:03:40.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e29b23740 1 client.25526 shutdown 2024-08-20T03:03:40.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e29b23740 1 -- 172.21.15.44:0/477272246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d428913b0 msgr2=0x559d428a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:40.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e29b23740 1 --2- 172.21.15.44:0/477272246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d428913b0 0x559d428a19f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0e18001eb0 tx=0x7f0e18000bc0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e29b23740 1 -- 172.21.15.44:0/477272246 shutdown_connections 2024-08-20T03:03:40.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e29b23740 1 --2- 172.21.15.44:0/477272246 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559d42918cf0 0x559d4291b110 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e29b23740 1 --2- 172.21.15.44:0/477272246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d4289dc60 0x559d428a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e29b23740 1 --2- 172.21.15.44:0/477272246 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559d42891cd0 0x559d428a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e08ff9640 0 client.25526 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:40.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e29b23740 1 --2- 172.21.15.44:0/477272246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d428913b0 0x559d428a19f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e29b23740 1 -- 172.21.15.44:0/477272246 >> 172.21.15.44:0/477272246 conn(0x559d4288f1e0 msgr2=0x559d42894cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:40.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e29b23740 1 -- 172.21.15.44:0/477272246 shutdown_connections 2024-08-20T03:03:40.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.574+0000 7f0e29b23740 1 -- 172.21.15.44:0/477272246 wait complete. 2024-08-20T03:03:40.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e29b23740 20 client.25526 trim_cache size 0 max 16384 2024-08-20T03:03:40.581 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:03:40.581 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-08-20T03:03:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "f8648551-42f6-4dc0-a845-e3dbb09ee940", 2024-08-20T03:03:40.582 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-08-20T03:03:40.582 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:03:40.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e2772a640 1 -- 172.21.15.44:0/9382685 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0e0c0b3d40 msgr2=0x7f0e0c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:40.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e2772a640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0e0c0b3d40 0x7f0e0c0b6200 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0e1dd0 tx=0x7f0e1c0e9040 comp rx=0 tx=0).stop 2024-08-20T03:03:40.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e2772a640 1 -- 172.21.15.44:0/9382685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e20102a90 msgr2=0x7f0e20076c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:40.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e2772a640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e20102a90 0x7f0e20076c80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00ebc0 tx=0x7f0e1c00c6a0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e2772a640 1 -- 172.21.15.44:0/9382685 shutdown_connections 2024-08-20T03:03:40.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e2772a640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0e0c0b3d40 0x7f0e0c0b6200 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e2772a640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e20108e20 0x7f0e20075800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e2772a640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e201084a0 0x7f0e200751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e2772a640 1 --2- 172.21.15.44:0/9382685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e20102a90 0x7f0e20076c80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:40.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e2772a640 1 -- 172.21.15.44:0/9382685 >> 172.21.15.44:0/9382685 conn(0x7f0e200fe540 msgr2=0x7f0e20100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:40.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e2772a640 1 -- 172.21.15.44:0/9382685 shutdown_connections 2024-08-20T03:03:40.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:40.575+0000 7f0e2772a640 1 -- 172.21.15.44:0/9382685 wait complete. 2024-08-20T03:03:41.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:40 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:03:41.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:40 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:03:41.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:40 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:03:41.929 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:03:42.200 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:03:42.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:41 smithi044 ceph-mon[18790]: pgmap v356: 97 pgs: 97 active+clean; 60 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 457 op/s 2024-08-20T03:03:42.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:41 smithi044 ceph-mon[18790]: scrub queued for path: / 2024-08-20T03:03:42.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:41 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:42.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:41 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:03:42.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:41 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:03:42.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:41 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:41 smithi165 ceph-mon[22357]: pgmap v356: 97 pgs: 97 active+clean; 60 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 457 op/s 2024-08-20T03:03:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:41 smithi165 ceph-mon[22357]: scrub queued for path: / 2024-08-20T03:03:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:41 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:41 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:03:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:41 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:03:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:41 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:42.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:41 smithi099 ceph-mon[21984]: pgmap v356: 97 pgs: 97 active+clean; 60 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 457 op/s 2024-08-20T03:03:42.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:41 smithi099 ceph-mon[21984]: scrub queued for path: / 2024-08-20T03:03:42.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:41 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:42.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:41 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:03:42.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:41 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:03:42.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:41 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:42.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.669+0000 7fb192f71640 1 -- 172.21.15.44:0/2675144543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb18c1073a0 msgr2=0x7fb18c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.669+0000 7fb192f71640 1 --2- 172.21.15.44:0/2675144543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb18c1073a0 0x7fb18c1077e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb178009900 tx=0x7fb17802a410 comp rx=0 tx=0).stop 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.669+0000 7fb192f71640 1 -- 172.21.15.44:0/2675144543 shutdown_connections 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.669+0000 7fb192f71640 1 --2- 172.21.15.44:0/2675144543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb18c107d20 0x7fb18c10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.669+0000 7fb192f71640 1 --2- 172.21.15.44:0/2675144543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb18c1073a0 0x7fb18c1077e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.669+0000 7fb192f71640 1 --2- 172.21.15.44:0/2675144543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18c101990 0x7fb18c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.669+0000 7fb192f71640 1 -- 172.21.15.44:0/2675144543 >> 172.21.15.44:0/2675144543 conn(0x7fb18c0fd400 msgr2=0x7fb18c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.669+0000 7fb192f71640 1 -- 172.21.15.44:0/2675144543 shutdown_connections 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.669+0000 7fb192f71640 1 -- 172.21.15.44:0/2675144543 wait complete. 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.669+0000 7fb192f71640 1 Processor -- start 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.669+0000 7fb192f71640 1 -- start start 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb192f71640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18c101990 0x7fb18c198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb192f71640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb18c1073a0 0x7fb18c199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb192f71640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb18c107d20 0x7fb18c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb192f71640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c112ae0 con 0x7fb18c1073a0 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb192f71640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c112960 con 0x7fb18c107d20 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb192f71640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c112c60 con 0x7fb18c101990 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb18bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb18c1073a0 0x7fb18c199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb18bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb18c1073a0 0x7fb18c199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46970/0 (socket says 172.21.15.44:46970) 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb18bfff640 1 -- 172.21.15.44:0/4290071228 learned_addr learned my addr 172.21.15.44:0/4290071228 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb190d0d640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18c101990 0x7fb18c198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb19150e640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb18c107d20 0x7fb18c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb18bfff640 1 -- 172.21.15.44:0/4290071228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18c101990 msgr2=0x7fb18c198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb18bfff640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18c101990 0x7fb18c198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb18bfff640 1 -- 172.21.15.44:0/4290071228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb18c107d20 msgr2=0x7fb18c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb18bfff640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb18c107d20 0x7fb18c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb18bfff640 1 -- 172.21.15.44:0/4290071228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18c19f080 con 0x7fb18c1073a0 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb18bfff640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb18c1073a0 0x7fb18c199510 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb178005940 tx=0x7fb1780040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb189ffb640 1 -- 172.21.15.44:0/4290071228 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17803f030 con 0x7fb18c1073a0 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb189ffb640 1 -- 172.21.15.44:0/4290071228 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1780044c0 con 0x7fb18c1073a0 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb192f71640 1 -- 172.21.15.44:0/4290071228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18c19f310 con 0x7fb18c1073a0 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.670+0000 7fb189ffb640 1 -- 172.21.15.44:0/4290071228 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178035070 con 0x7fb18c1073a0 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.671+0000 7fb192f71640 1 -- 172.21.15.44:0/4290071228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb18c1a5a30 con 0x7fb18c1073a0 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.673+0000 7fb189ffb640 1 -- 172.21.15.44:0/4290071228 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb178042030 con 0x7fb18c1073a0 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.673+0000 7fb19536a740 1 Processor -- start 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.673+0000 7fb19536a740 1 -- start start 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb19536a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afdc830 0x56061afdcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb19536a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdd150 0x56061afe8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb19536a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afe90e0 0x56061afeb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb19536a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56061af538e0 con 0x56061afdd150 2024-08-20T03:03:42.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb19536a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56061af42de0 con 0x56061afdc830 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb19536a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56061af46e20 con 0x56061afe90e0 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb190d0d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdd150 0x56061afe8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb190d0d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdd150 0x56061afe8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46976/0 (socket says 172.21.15.44:46976) 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb190d0d640 1 -- 172.21.15.44:0/687804035 learned_addr learned my addr 172.21.15.44:0/687804035 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb18bfff640 1 --2- 172.21.15.44:0/687804035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afe90e0 0x56061afeb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb19150e640 1 --2- 172.21.15.44:0/687804035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afdc830 0x56061afdcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb190d0d640 1 -- 172.21.15.44:0/687804035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afe90e0 msgr2=0x56061afeb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb190d0d640 1 --2- 172.21.15.44:0/687804035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afe90e0 0x56061afeb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb190d0d640 1 -- 172.21.15.44:0/687804035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afdc830 msgr2=0x56061afdcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb190d0d640 1 --2- 172.21.15.44:0/687804035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afdc830 0x56061afdcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb190d0d640 1 -- 172.21.15.44:0/687804035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56061aca2650 con 0x56061afdd150 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb189ffb640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb17c0b3d40 0x7fb17c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb18bfff640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb17c0b3d40 0x7fb17c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.674+0000 7fb190d0d640 1 --2- 172.21.15.44:0/687804035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdd150 0x56061afe8ba0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb180009d50 tx=0x7fb180011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03501e16f4ce98 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.675+0000 7fb189ffb640 1 -- 172.21.15.44:0/4290071228 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb1781042b0 con 0x7fb18c1073a0 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.675+0000 7fb176ffd640 1 -- 172.21.15.44:0/687804035 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18001e070 con 0x56061afdd150 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.675+0000 7fb176ffd640 1 -- 172.21.15.44:0/687804035 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb180019ae0 con 0x56061afdd150 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.675+0000 7fb176ffd640 1 -- 172.21.15.44:0/687804035 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18000ea80 con 0x56061afdd150 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.675+0000 7fb19536a740 1 -- 172.21.15.44:0/687804035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdd150 msgr2=0x56061afe8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.675+0000 7fb18bfff640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb17c0b3d40 0x7fb17c0b6200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb178101b20 tx=0x7fb178107e00 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.675+0000 7fb19536a740 1 --2- 172.21.15.44:0/687804035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdd150 0x56061afe8ba0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb180009d50 tx=0x7fb180011810 comp rx=0 tx=0).stop 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.676+0000 7fb19536a740 1 -- 172.21.15.44:0/687804035 shutdown_connections 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.676+0000 7fb19536a740 1 --2- 172.21.15.44:0/687804035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afe90e0 0x56061afeb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.676+0000 7fb19536a740 1 --2- 172.21.15.44:0/687804035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdd150 0x56061afe8ba0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.676+0000 7fb19536a740 1 --2- 172.21.15.44:0/687804035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afdc830 0x56061afdcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.676+0000 7fb19536a740 1 -- 172.21.15.44:0/687804035 >> 172.21.15.44:0/687804035 conn(0x56061afda660 msgr2=0x56061afdb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:42.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.676+0000 7fb19536a740 1 -- 172.21.15.44:0/687804035 shutdown_connections 2024-08-20T03:03:42.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.676+0000 7fb19536a740 1 -- 172.21.15.44:0/687804035 wait complete. 2024-08-20T03:03:42.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.676+0000 7fb19536a740 1 -- start start 2024-08-20T03:03:42.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.676+0000 7fb19536a740 1 Processor -- start 2024-08-20T03:03:42.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19536a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 0x56061afe6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19536a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afe6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19536a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afe90e0 0x56061afe6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19536a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56061ada5700 con 0x56061afdc830 2024-08-20T03:03:42.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19536a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56061aec97c0 con 0x56061afe90e0 2024-08-20T03:03:42.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19536a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56061afebd90 con 0x56061afdd150 2024-08-20T03:03:42.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19536a740 10 client.0 _ll_register_callbacks cb 0x56061af3c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19150e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 0x56061afe6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19150e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 0x56061afe6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46992/0 (socket says 172.21.15.44:46992) 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19150e640 1 -- 172.21.15.44:0/3218943510 learned_addr learned my addr 172.21.15.44:0/3218943510 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb190d0d640 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afe6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19150e640 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afe90e0 0x56061afe6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19150e640 1 -- 172.21.15.44:0/3218943510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 msgr2=0x56061afe6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19150e640 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afe6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19150e640 1 -- 172.21.15.44:0/3218943510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afe90e0 msgr2=0x56061afe6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19150e640 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afe90e0 0x56061afe6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19150e640 1 -- 172.21.15.44:0/3218943510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56061aff5110 con 0x56061afdc830 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19150e640 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 0x56061afe6050 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb184011e50 tx=0x7fb184009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f92beb0b8011acc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb16bfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb16bfff640 1 -- 172.21.15.44:0/3218943510 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18400e880 con 0x56061afdc830 2024-08-20T03:03:42.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb16bfff640 1 -- 172.21.15.44:0/3218943510 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb18400ea20 con 0x56061afdc830 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.677+0000 7fb19536a740 1 -- 172.21.15.44:0/3218943510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56061aef2100 con 0x56061afdc830 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.678+0000 7fb16bfff640 1 -- 172.21.15.44:0/3218943510 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb18400ec20 con 0x56061afdc830 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.678+0000 7fb19536a740 10 client.16041 fetch_fsmap learned FSMap version 29 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.678+0000 7fb19536a740 1 -- 172.21.15.44:0/3218943510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56061aff7b20 con 0x56061afdc830 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.678+0000 7fb16bfff640 1 -- 172.21.15.44:0/3218943510 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56061aef2100 con 0x56061afdc830 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.679+0000 7fb16bfff640 1 -- 172.21.15.44:0/3218943510 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb184028020 con 0x56061afdc830 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.679+0000 7fb19536a740 10 client.16041 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.679+0000 7fb19536a740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.679+0000 7fb19536a740 20 client.16041 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.679+0000 7fb19536a740 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56061aff9030 0x56061affb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.679+0000 7fb19536a740 4 client.16041 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.679+0000 7fb19536a740 1 -- 172.21.15.44:0/3218943510 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56061af55830 con 0x56061aff9030 2024-08-20T03:03:42.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.679+0000 7fb190d0d640 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56061aff9030 0x56061affb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.679+0000 7fb190d0d640 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56061aff9030 0x56061affb420 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb8d398abb7e926 server_cookie=6b167f38779c65ec in_seq=0 out_seq=0 2024-08-20T03:03:42.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.679+0000 7fb16bfff640 10 client.16041 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:42.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.683+0000 7fb16bfff640 1 -- 172.21.15.44:0/3218943510 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56061af55830 con 0x56061aff9030 2024-08-20T03:03:42.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.683+0000 7fb16bfff640 10 client.16041 handle_command_reply: tid=0 2024-08-20T03:03:42.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.683+0000 7fb19536a740 1 client.16041 shutdown 2024-08-20T03:03:42.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.683+0000 7fb19536a740 1 -- 172.21.15.44:0/3218943510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 msgr2=0x56061afe6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:42.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.683+0000 7fb19536a740 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 0x56061afe6050 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb184011e50 tx=0x7fb184009db0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.683+0000 7fb19536a740 1 -- 172.21.15.44:0/3218943510 shutdown_connections 2024-08-20T03:03:42.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.683+0000 7fb19536a740 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56061aff9030 0x56061affb420 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.683+0000 7fb16bfff640 0 client.16041 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:42.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.683+0000 7fb19536a740 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afe90e0 0x56061afe6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.683+0000 7fb19536a740 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afe6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.683+0000 7fb19536a740 1 --2- 172.21.15.44:0/3218943510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 0x56061afe6050 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.684+0000 7fb19536a740 1 -- 172.21.15.44:0/3218943510 >> 172.21.15.44:0/3218943510 conn(0x56061afda660 msgr2=0x56061afe0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:42.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.684+0000 7fb19536a740 1 -- 172.21.15.44:0/3218943510 shutdown_connections 2024-08-20T03:03:42.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.684+0000 7fb19536a740 1 -- 172.21.15.44:0/3218943510 wait complete. 2024-08-20T03:03:42.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.684+0000 7fb19536a740 20 client.16041 trim_cache size 0 max 16384 2024-08-20T03:03:42.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.721+0000 7fb19536a740 1 Processor -- start 2024-08-20T03:03:42.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.721+0000 7fb19536a740 1 -- start start 2024-08-20T03:03:42.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb19536a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 0x56061afe09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb19536a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afe4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb19536a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afe90e0 0x56061afe7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb19536a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c1ae290 con 0x56061afdc830 2024-08-20T03:03:42.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb19536a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c1afe00 con 0x56061afe90e0 2024-08-20T03:03:42.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb19536a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c1ae960 con 0x56061afdd150 2024-08-20T03:03:42.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb19150e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 0x56061afe09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb19150e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 0x56061afe09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47006/0 (socket says 172.21.15.44:47006) 2024-08-20T03:03:42.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb19150e640 1 -- 172.21.15.44:0/4179848027 learned_addr learned my addr 172.21.15.44:0/4179848027 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:42.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb190d0d640 1 --2- 172.21.15.44:0/4179848027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afe4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb19150e640 1 --2- 172.21.15.44:0/4179848027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afe90e0 0x56061afe7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb190d0d640 1 -- 172.21.15.44:0/4179848027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afe90e0 msgr2=0x56061afe7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:42.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb190d0d640 1 --2- 172.21.15.44:0/4179848027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afe90e0 0x56061afe7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb190d0d640 1 -- 172.21.15.44:0/4179848027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 msgr2=0x56061afe09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:42.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb190d0d640 1 --2- 172.21.15.44:0/4179848027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 0x56061afe09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.722+0000 7fb190d0d640 1 -- 172.21.15.44:0/4179848027 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56061afe4550 con 0x56061afdd150 2024-08-20T03:03:42.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.723+0000 7fb190d0d640 1 --2- 172.21.15.44:0/4179848027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afe4f40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb18001c040 tx=0x7fb18000e450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fa9cc1e0a8b0ba3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:42.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.723+0000 7fb1767fc640 1 -- 172.21.15.44:0/4179848027 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18001e040 con 0x56061afdd150 2024-08-20T03:03:42.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.723+0000 7fb1767fc640 1 -- 172.21.15.44:0/4179848027 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb18001f040 con 0x56061afdd150 2024-08-20T03:03:42.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.723+0000 7fb1767fc640 1 -- 172.21.15.44:0/4179848027 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18000ee10 con 0x56061afdd150 2024-08-20T03:03:42.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.724+0000 7fb19536a740 1 -- 172.21.15.44:0/4179848027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 msgr2=0x56061afe4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:42.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.724+0000 7fb19536a740 1 --2- 172.21.15.44:0/4179848027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afe4f40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb18001c040 tx=0x7fb18000e450 comp rx=0 tx=0).stop 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.724+0000 7fb19536a740 1 -- 172.21.15.44:0/4179848027 shutdown_connections 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.724+0000 7fb19536a740 1 --2- 172.21.15.44:0/4179848027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afe90e0 0x56061afe7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.724+0000 7fb19536a740 1 --2- 172.21.15.44:0/4179848027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afe4f40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.724+0000 7fb19536a740 1 --2- 172.21.15.44:0/4179848027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afdc830 0x56061afe09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.724+0000 7fb19536a740 1 -- 172.21.15.44:0/4179848027 >> 172.21.15.44:0/4179848027 conn(0x56061afda660 msgr2=0x56061afe7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.724+0000 7fb19536a740 1 -- 172.21.15.44:0/4179848027 shutdown_connections 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.724+0000 7fb19536a740 1 -- 172.21.15.44:0/4179848027 wait complete. 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.724+0000 7fb19536a740 1 -- start start 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.724+0000 7fb19536a740 1 Processor -- start 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb19536a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afdc830 0x56061afec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb19536a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb19536a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afe90e0 0x56061b0660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb19536a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c1af700 con 0x56061afe90e0 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb19536a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c1af030 con 0x56061afdc830 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb19536a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c1ac270 con 0x56061afdd150 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb19536a740 10 client.0 _ll_register_callbacks cb 0x7fb184009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb190d0d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb19150e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afe90e0 0x56061b0660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb19150e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afe90e0 0x56061b0660c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47008/0 (socket says 172.21.15.44:47008) 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb190d0d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afec950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45982/0 (socket says 172.21.15.44:45982) 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb19150e640 1 -- 172.21.15.44:0/215489423 learned_addr learned my addr 172.21.15.44:0/215489423 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb19150e640 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afdc830 0x56061afec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb190d0d640 1 -- 172.21.15.44:0/215489423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afdc830 msgr2=0x56061afec410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb190d0d640 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afdc830 0x56061afec410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb190d0d640 1 -- 172.21.15.44:0/215489423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afe90e0 msgr2=0x56061b0660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb190d0d640 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afe90e0 0x56061b0660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb190d0d640 1 -- 172.21.15.44:0/215489423 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56061b066600 con 0x56061afdd150 2024-08-20T03:03:42.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.725+0000 7fb19150e640 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afe90e0 0x56061b0660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:42.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb190d0d640 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afec950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb180011850 tx=0x7fb18000e550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b83e200e8c5f79de server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:42.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb16bfff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:03:42.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb16bfff640 1 -- 172.21.15.44:0/215489423 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18001e040 con 0x56061afdd150 2024-08-20T03:03:42.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb19536a740 1 -- 172.21.15.44:0/215489423 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb18001f040 con 0x56061afdd150 2024-08-20T03:03:42.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb16bfff640 1 -- 172.21.15.44:0/215489423 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb1800052b0 con 0x56061afdd150 2024-08-20T03:03:42.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb16bfff640 1 -- 172.21.15.44:0/215489423 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb18003d050 con 0x56061afdd150 2024-08-20T03:03:42.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb19536a740 10 client.25510 fetch_fsmap learned FSMap version 29 2024-08-20T03:03:42.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb19536a740 1 -- 172.21.15.44:0/215489423 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56061b067830 con 0x56061afdd150 2024-08-20T03:03:42.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb16bfff640 1 -- 172.21.15.44:0/215489423 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb18001f040 con 0x56061afdd150 2024-08-20T03:03:42.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb16bfff640 1 -- 172.21.15.44:0/215489423 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb180003680 con 0x56061afdd150 2024-08-20T03:03:42.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb19536a740 10 client.25510 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T03:03:42.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb19536a740 10 client.25510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:42.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb19536a740 20 client.25510 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:42.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb19536a740 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56061aff9030 0x56061b069de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:42.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb19536a740 4 client.25510 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:03:42.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.726+0000 7fb19536a740 1 -- 172.21.15.44:0/215489423 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56061af512d0 con 0x56061aff9030 2024-08-20T03:03:42.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.727+0000 7fb19150e640 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56061aff9030 0x56061b069de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:42.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.727+0000 7fb16bfff640 10 client.25510 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:42.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.727+0000 7fb19150e640 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56061aff9030 0x56061b069de0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8158f943ebd2e6 server_cookie=7a5073528647dd0 in_seq=0 out_seq=0 2024-08-20T03:03:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb16bfff640 1 -- 172.21.15.44:0/215489423 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x56061af512d0 con 0x56061aff9030 2024-08-20T03:03:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb16bfff640 10 client.25510 handle_command_reply: tid=0 2024-08-20T03:03:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb19536a740 1 client.25510 shutdown 2024-08-20T03:03:42.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb19536a740 1 -- 172.21.15.44:0/215489423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 msgr2=0x56061afec950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb19536a740 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afec950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb180011850 tx=0x7fb18000e550 comp rx=0 tx=0).stop 2024-08-20T03:03:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb19536a740 1 -- 172.21.15.44:0/215489423 shutdown_connections 2024-08-20T03:03:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb19536a740 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56061aff9030 0x56061b069de0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb19536a740 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56061afe90e0 0x56061b0660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb19536a740 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56061afdd150 0x56061afec950 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb16bfff640 0 client.25510 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb19536a740 1 --2- 172.21.15.44:0/215489423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56061afdc830 0x56061afec410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb19536a740 1 -- 172.21.15.44:0/215489423 >> 172.21.15.44:0/215489423 conn(0x56061afda660 msgr2=0x56061afe7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb19536a740 1 -- 172.21.15.44:0/215489423 shutdown_connections 2024-08-20T03:03:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb19536a740 1 -- 172.21.15.44:0/215489423 wait complete. 2024-08-20T03:03:42.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.734+0000 7fb19536a740 20 client.25510 trim_cache size 0 max 16384 2024-08-20T03:03:42.738 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:03:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "scrub active (2861 inodes in the stack)", 2024-08-20T03:03:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-08-20T03:03:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: "f8648551-42f6-4dc0-a845-e3dbb09ee940": { 2024-08-20T03:03:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-08-20T03:03:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "f8648551-42f6-4dc0-a845-e3dbb09ee940", 2024-08-20T03:03:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-08-20T03:03:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:03:42.738 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:03:42.738 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:03:42.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.735+0000 7fb192f71640 1 -- 172.21.15.44:0/4290071228 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb17c0b3d40 msgr2=0x7fb17c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:42.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.735+0000 7fb192f71640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb17c0b3d40 0x7fb17c0b6200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb178101b20 tx=0x7fb178107e00 comp rx=0 tx=0).stop 2024-08-20T03:03:42.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.735+0000 7fb192f71640 1 -- 172.21.15.44:0/4290071228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb18c1073a0 msgr2=0x7fb18c199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:42.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.735+0000 7fb192f71640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb18c1073a0 0x7fb18c199510 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb178005940 tx=0x7fb1780040c0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.735+0000 7fb192f71640 1 -- 172.21.15.44:0/4290071228 shutdown_connections 2024-08-20T03:03:42.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.735+0000 7fb192f71640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb17c0b3d40 0x7fb17c0b6200 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.735+0000 7fb192f71640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb18c107d20 0x7fb18c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.735+0000 7fb192f71640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb18c1073a0 0x7fb18c199510 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.735+0000 7fb192f71640 1 --2- 172.21.15.44:0/4290071228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18c101990 0x7fb18c198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:42.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.735+0000 7fb192f71640 1 -- 172.21.15.44:0/4290071228 >> 172.21.15.44:0/4290071228 conn(0x7fb18c0fd400 msgr2=0x7fb18c0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:42.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.735+0000 7fb192f71640 1 -- 172.21.15.44:0/4290071228 shutdown_connections 2024-08-20T03:03:42.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:03:42.735+0000 7fb192f71640 1 -- 172.21.15.44:0/4290071228 wait complete. 2024-08-20T03:03:43.526 INFO:tasks.cephfs.filesystem:scrub status for tag:f8648551-42f6-4dc0-a845-e3dbb09ee940 - {'path': '/', 'tag': 'f8648551-42f6-4dc0-a845-e3dbb09ee940', 'options': 'recursive,force'} 2024-08-20T03:03:44.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:43 smithi044 ceph-mon[18790]: pgmap v357: 97 pgs: 97 active+clean; 66 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 476 op/s 2024-08-20T03:03:44.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:43 smithi165 ceph-mon[22357]: pgmap v357: 97 pgs: 97 active+clean; 66 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 476 op/s 2024-08-20T03:03:44.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:43 smithi099 ceph-mon[21984]: pgmap v357: 97 pgs: 97 active+clean; 66 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.1 MiB/s wr, 476 op/s 2024-08-20T03:03:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.776+0000 7f3c621fd640 1 -- 172.21.15.44:0/3374819335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c5c107c50 msgr2=0x7f3c5c10e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.776+0000 7f3c621fd640 1 --2- 172.21.15.44:0/3374819335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c5c107c50 0x7f3c5c10e450 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009ca0 tx=0x7f3c5402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.776+0000 7f3c621fd640 1 -- 172.21.15.44:0/3374819335 shutdown_connections 2024-08-20T03:03:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.776+0000 7f3c621fd640 1 --2- 172.21.15.44:0/3374819335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c5c107c50 0x7f3c5c10e450 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.776+0000 7f3c621fd640 1 --2- 172.21.15.44:0/3374819335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c5c1072d0 0x7f3c5c107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.776+0000 7f3c621fd640 1 --2- 172.21.15.44:0/3374819335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c5c101920 0x7f3c5c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.776+0000 7f3c621fd640 1 -- 172.21.15.44:0/3374819335 >> 172.21.15.44:0/3374819335 conn(0x7f3c5c078620 msgr2=0x7f3c5c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.776+0000 7f3c621fd640 1 -- 172.21.15.44:0/3374819335 shutdown_connections 2024-08-20T03:03:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.776+0000 7f3c621fd640 1 -- 172.21.15.44:0/3374819335 wait complete. 2024-08-20T03:03:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c621fd640 1 Processor -- start 2024-08-20T03:03:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c621fd640 1 -- start start 2024-08-20T03:03:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c621fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c5c101920 0x7f3c5c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c621fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c5c1072d0 0x7f3c5c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c621fd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c5c107c50 0x7f3c5c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c621fd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c111910 con 0x7f3c5c1072d0 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c621fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c111790 con 0x7f3c5c101920 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c621fd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c111a90 con 0x7f3c5c107c50 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c5c1072d0 0x7f3c5c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c5c1072d0 0x7f3c5c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47024/0 (socket says 172.21.15.44:47024) 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c5affd640 1 -- 172.21.15.44:0/1452816982 learned_addr learned my addr 172.21.15.44:0/1452816982 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c5b7fe640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c5c101920 0x7f3c5c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.777+0000 7f3c5bfff640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c5c107c50 0x7f3c5c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.778+0000 7f3c5affd640 1 -- 172.21.15.44:0/1452816982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c5c107c50 msgr2=0x7f3c5c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.778+0000 7f3c5affd640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c5c107c50 0x7f3c5c19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.778+0000 7f3c5affd640 1 -- 172.21.15.44:0/1452816982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c5c101920 msgr2=0x7f3c5c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.778+0000 7f3c5affd640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c5c101920 0x7f3c5c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.778+0000 7f3c5affd640 1 -- 172.21.15.44:0/1452816982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5c19de90 con 0x7f3c5c1072d0 2024-08-20T03:03:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.778+0000 7f3c5b7fe640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c5c101920 0x7f3c5c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.778+0000 7f3c5affd640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c5c1072d0 0x7f3c5c198330 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000e9f0 tx=0x7f3c5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.778+0000 7f3c58ff9640 1 -- 172.21.15.44:0/1452816982 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5000cd80 con 0x7f3c5c1072d0 2024-08-20T03:03:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.778+0000 7f3c621fd640 1 -- 172.21.15.44:0/1452816982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5c19e180 con 0x7f3c5c1072d0 2024-08-20T03:03:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.778+0000 7f3c58ff9640 1 -- 172.21.15.44:0/1452816982 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c500108e0 con 0x7f3c5c1072d0 2024-08-20T03:03:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.778+0000 7f3c58ff9640 1 -- 172.21.15.44:0/1452816982 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50018980 con 0x7f3c5c1072d0 2024-08-20T03:03:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.778+0000 7f3c621fd640 1 -- 172.21.15.44:0/1452816982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c5c1a4a20 con 0x7f3c5c1072d0 2024-08-20T03:03:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.781+0000 7f3c64fec740 1 Processor -- start 2024-08-20T03:03:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.781+0000 7f3c64fec740 1 -- start start 2024-08-20T03:03:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.781+0000 7f3c58ff9640 1 -- 172.21.15.44:0/1452816982 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3c50010430 con 0x7f3c5c1072d0 2024-08-20T03:03:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.781+0000 7f3c64fec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ed8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.781+0000 7f3c64fec740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 0x55d629ee6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.781+0000 7f3c64fec740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ee7410 0x55d629ee9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.781+0000 7f3c64fec740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d629e4e360 con 0x55d629ed9270 2024-08-20T03:03:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.781+0000 7f3c64fec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d629e3d860 con 0x55d629ed8950 2024-08-20T03:03:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.781+0000 7f3c64fec740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d629e418a0 con 0x55d629ee7410 2024-08-20T03:03:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.781+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ee7410 0x55d629ee9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.781+0000 7f3c5b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 0x55d629ee6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ee7410 0x55d629ee9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46016/0 (socket says 172.21.15.44:46016) 2024-08-20T03:03:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c5affd640 1 -- 172.21.15.44:0/2323757672 learned_addr learned my addr 172.21.15.44:0/2323757672 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c5b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 0x55d629ee6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47030/0 (socket says 172.21.15.44:47030) 2024-08-20T03:03:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.781+0000 7f3c5bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ed8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c5bfff640 1 -- 172.21.15.44:0/2323757672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ee7410 msgr2=0x55d629ee9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c5bfff640 1 --2- 172.21.15.44:0/2323757672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ee7410 0x55d629ee9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c5bfff640 1 -- 172.21.15.44:0/2323757672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 msgr2=0x55d629ee6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c5bfff640 1 --2- 172.21.15.44:0/2323757672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 0x55d629ee6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c5bfff640 1 -- 172.21.15.44:0/2323757672 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d629de4e40 con 0x55d629ed8950 2024-08-20T03:03:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c5b7fe640 1 --2- 172.21.15.44:0/2323757672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 0x55d629ee6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c58ff9640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3c440b3d40 0x7f3c440b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c5bfff640 1 --2- 172.21.15.44:0/2323757672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ed8d30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3c54005e30 tx=0x7f3c54005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39b74e30fc006baa server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c5b7fe640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3c440b3d40 0x7f3c440b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c58ff9640 1 -- 172.21.15.44:0/1452816982 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3c500e42a0 con 0x7f3c5c1072d0 2024-08-20T03:03:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c41ffb640 1 -- 172.21.15.44:0/2323757672 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54031cf0 con 0x55d629ed8950 2024-08-20T03:03:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c41ffb640 1 -- 172.21.15.44:0/2323757672 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c54036030 con 0x55d629ed8950 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.782+0000 7f3c41ffb640 1 -- 172.21.15.44:0/2323757672 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54004040 con 0x55d629ed8950 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.783+0000 7f3c64fec740 1 -- 172.21.15.44:0/2323757672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 msgr2=0x55d629ed8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.783+0000 7f3c64fec740 1 --2- 172.21.15.44:0/2323757672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ed8d30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3c54005e30 tx=0x7f3c54005ef0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.783+0000 7f3c5b7fe640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3c440b3d40 0x7f3c440b6200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009fd0 tx=0x7f3c4c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.783+0000 7f3c64fec740 1 -- 172.21.15.44:0/2323757672 shutdown_connections 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.783+0000 7f3c64fec740 1 --2- 172.21.15.44:0/2323757672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ee7410 0x55d629ee9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.783+0000 7f3c64fec740 1 --2- 172.21.15.44:0/2323757672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 0x55d629ee6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.783+0000 7f3c64fec740 1 --2- 172.21.15.44:0/2323757672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ed8d30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.783+0000 7f3c64fec740 1 -- 172.21.15.44:0/2323757672 >> 172.21.15.44:0/2323757672 conn(0x55d629ed6730 msgr2=0x55d629ed7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.783+0000 7f3c64fec740 1 -- 172.21.15.44:0/2323757672 shutdown_connections 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.783+0000 7f3c64fec740 1 -- 172.21.15.44:0/2323757672 wait complete. 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c64fec740 1 -- start start 2024-08-20T03:03:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c64fec740 1 Processor -- start 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c64fec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ee6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c64fec740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ed9270 0x55d629ee4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c64fec740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ee7410 0x55d629ee4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c64fec740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d629d76ab0 con 0x55d629ee7410 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c64fec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d629dc4780 con 0x55d629ed8950 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c64fec740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d629ee9f10 con 0x55d629ed9270 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c64fec740 10 client.0 _ll_register_callbacks cb 0x55d629b4ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c5bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ee7410 0x55d629ee4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c5b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ee6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ed9270 0x55d629ee4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c5b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ee6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:57928/0 (socket says 172.21.15.44:57928) 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.784+0000 7f3c5b7fe640 1 -- 172.21.15.44:0/2867459960 learned_addr learned my addr 172.21.15.44:0/2867459960 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c5b7fe640 1 -- 172.21.15.44:0/2867459960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ed9270 msgr2=0x55d629ee4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c5b7fe640 1 --2- 172.21.15.44:0/2867459960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ed9270 0x55d629ee4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c5b7fe640 1 -- 172.21.15.44:0/2867459960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ee7410 msgr2=0x55d629ee4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c5b7fe640 1 --2- 172.21.15.44:0/2867459960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ee7410 0x55d629ee4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c5b7fe640 1 -- 172.21.15.44:0/2867459960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d629ba0790 con 0x55d629ed8950 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c5bfff640 1 --2- 172.21.15.44:0/2867459960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ee7410 0x55d629ee4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c5b7fe640 1 --2- 172.21.15.44:0/2867459960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ee6d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c012e50 tx=0x7f3c4c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc6c668a90098b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c3affd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c3affd640 1 -- 172.21.15.44:0/2867459960 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c004050 con 0x55d629ed8950 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c3affd640 1 -- 172.21.15.44:0/2867459960 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3c4c0041f0 con 0x55d629ed8950 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c64fec740 1 -- 172.21.15.44:0/2867459960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d629dece00 con 0x55d629ed8950 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c3affd640 1 -- 172.21.15.44:0/2867459960 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c4c0043f0 con 0x55d629ed8950 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c64fec740 10 client.25556 fetch_fsmap learned FSMap version 30 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c64fec740 1 -- 172.21.15.44:0/2867459960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d629c19620 con 0x55d629ed8950 2024-08-20T03:03:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.785+0000 7f3c3affd640 1 -- 172.21.15.44:0/2867459960 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d629dece00 con 0x55d629ed8950 2024-08-20T03:03:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.786+0000 7f3c3affd640 1 -- 172.21.15.44:0/2867459960 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3c4c031020 con 0x55d629ed8950 2024-08-20T03:03:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.786+0000 7f3c64fec740 10 client.25556 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:03:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.786+0000 7f3c64fec740 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.786+0000 7f3c64fec740 20 client.25556 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.786+0000 7f3c64fec740 1 --2- 172.21.15.44:0/2867459960 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d629ef6c20 0x55d629ef9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.786+0000 7f3c64fec740 4 client.25556 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:03:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.786+0000 7f3c64fec740 1 -- 172.21.15.44:0/2867459960 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d629e502b0 con 0x55d629ef6c20 2024-08-20T03:03:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.786+0000 7f3c5affd640 1 --2- 172.21.15.44:0/2867459960 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d629ef6c20 0x55d629ef9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.786+0000 7f3c3affd640 10 client.25556 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.786+0000 7f3c5affd640 1 --2- 172.21.15.44:0/2867459960 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d629ef6c20 0x55d629ef9080 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc4c05ba14eb89df server_cookie=22a553739d7ee0b9 in_seq=0 out_seq=0 2024-08-20T03:03:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.791+0000 7f3c3affd640 1 -- 172.21.15.44:0/2867459960 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d629e502b0 con 0x55d629ef6c20 2024-08-20T03:03:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.791+0000 7f3c3affd640 10 client.25556 handle_command_reply: tid=0 2024-08-20T03:03:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.791+0000 7f3c64fec740 1 client.25556 shutdown 2024-08-20T03:03:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.791+0000 7f3c64fec740 1 -- 172.21.15.44:0/2867459960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 msgr2=0x55d629ee6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.791+0000 7f3c64fec740 1 --2- 172.21.15.44:0/2867459960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ee6d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c012e50 tx=0x7f3c4c004730 comp rx=0 tx=0).stop 2024-08-20T03:03:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.791+0000 7f3c64fec740 1 -- 172.21.15.44:0/2867459960 shutdown_connections 2024-08-20T03:03:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.791+0000 7f3c64fec740 1 --2- 172.21.15.44:0/2867459960 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d629ef6c20 0x55d629ef9080 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.791+0000 7f3c64fec740 1 --2- 172.21.15.44:0/2867459960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ee7410 0x55d629ee4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.791+0000 7f3c3affd640 0 client.25556 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.791+0000 7f3c64fec740 1 --2- 172.21.15.44:0/2867459960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ed9270 0x55d629ee4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.791+0000 7f3c64fec740 1 --2- 172.21.15.44:0/2867459960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ee6d40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.792+0000 7f3c64fec740 1 -- 172.21.15.44:0/2867459960 >> 172.21.15.44:0/2867459960 conn(0x55d629ed6730 msgr2=0x55d629edc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.792+0000 7f3c64fec740 1 -- 172.21.15.44:0/2867459960 shutdown_connections 2024-08-20T03:03:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.792+0000 7f3c64fec740 1 -- 172.21.15.44:0/2867459960 wait complete. 2024-08-20T03:03:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.792+0000 7f3c64fec740 20 client.25556 trim_cache size 0 max 16384 2024-08-20T03:03:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.823+0000 7f3c64fec740 1 Processor -- start 2024-08-20T03:03:45.828 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.827+0000 7f3c64fec740 1 -- start start 2024-08-20T03:03:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.827+0000 7f3c64fec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ee1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.827+0000 7f3c64fec740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 0x55d629ef3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.827+0000 7f3c64fec740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ee7410 0x55d629ee38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.827+0000 7f3c64fec740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1abf40 con 0x55d629ed9270 2024-08-20T03:03:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.827+0000 7f3c64fec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1adb40 con 0x55d629ed8950 2024-08-20T03:03:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.827+0000 7f3c64fec740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1ac610 con 0x55d629ee7410 2024-08-20T03:03:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.827+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 0x55d629ef3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.827+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 0x55d629ef3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47044/0 (socket says 172.21.15.44:47044) 2024-08-20T03:03:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.828+0000 7f3c5affd640 1 -- 172.21.15.44:0/1902675695 learned_addr learned my addr 172.21.15.44:0/1902675695 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.828+0000 7f3c5affd640 1 -- 172.21.15.44:0/1902675695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ee7410 msgr2=0x55d629ee38c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:03:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.828+0000 7f3c5affd640 1 --2- 172.21.15.44:0/1902675695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ee7410 0x55d629ee38c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.828+0000 7f3c5affd640 1 -- 172.21.15.44:0/1902675695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 msgr2=0x55d629ee1ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:03:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.828+0000 7f3c5affd640 1 --2- 172.21.15.44:0/1902675695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ee1ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.828+0000 7f3c5affd640 1 -- 172.21.15.44:0/1902675695 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d629ed90a0 con 0x55d629ed9270 2024-08-20T03:03:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.829+0000 7f3c5affd640 1 --2- 172.21.15.44:0/1902675695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 0x55d629ef3030 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3c500aacd0 tx=0x7f3c500e7df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7c348d3ecebc0c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.861+0000 7f3c417fa640 1 -- 172.21.15.44:0/1902675695 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50010720 con 0x55d629ed9270 2024-08-20T03:03:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.861+0000 7f3c417fa640 1 -- 172.21.15.44:0/1902675695 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c500aaae0 con 0x55d629ed9270 2024-08-20T03:03:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.861+0000 7f3c417fa640 1 -- 172.21.15.44:0/1902675695 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50010bb0 con 0x55d629ed9270 2024-08-20T03:03:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.862+0000 7f3c64fec740 1 -- 172.21.15.44:0/1902675695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 msgr2=0x55d629ef3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.862+0000 7f3c64fec740 1 --2- 172.21.15.44:0/1902675695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 0x55d629ef3030 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3c500aacd0 tx=0x7f3c500e7df0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.862+0000 7f3c64fec740 1 -- 172.21.15.44:0/1902675695 shutdown_connections 2024-08-20T03:03:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.862+0000 7f3c64fec740 1 --2- 172.21.15.44:0/1902675695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ee7410 0x55d629ee38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.862+0000 7f3c64fec740 1 --2- 172.21.15.44:0/1902675695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed9270 0x55d629ef3030 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.862+0000 7f3c64fec740 1 --2- 172.21.15.44:0/1902675695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ed8950 0x55d629ee1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.862+0000 7f3c64fec740 1 -- 172.21.15.44:0/1902675695 >> 172.21.15.44:0/1902675695 conn(0x55d629ed6730 msgr2=0x55d629ee57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.862+0000 7f3c64fec740 1 -- 172.21.15.44:0/1902675695 shutdown_connections 2024-08-20T03:03:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.862+0000 7f3c64fec740 1 -- 172.21.15.44:0/1902675695 wait complete. 2024-08-20T03:03:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.863+0000 7f3c64fec740 1 -- start start 2024-08-20T03:03:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.863+0000 7f3c64fec740 1 Processor -- start 2024-08-20T03:03:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c64fec740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed8950 0x55d629eefa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c64fec740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ed9270 0x55d629efe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c64fec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ee7410 0x55d629efed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c64fec740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1ad3b0 con 0x55d629ed8950 2024-08-20T03:03:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c64fec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1acce0 con 0x55d629ee7410 2024-08-20T03:03:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c64fec740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1a9f20 con 0x55d629ed9270 2024-08-20T03:03:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c64fec740 10 client.0 _ll_register_callbacks cb 0x7f3c4c009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:03:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c5b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed8950 0x55d629eefa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ed9270 0x55d629efe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ed9270 0x55d629efe5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46024/0 (socket says 172.21.15.44:46024) 2024-08-20T03:03:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c5affd640 1 -- 172.21.15.44:0/3844972819 learned_addr learned my addr 172.21.15.44:0/3844972819 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:03:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c5bfff640 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ee7410 0x55d629efed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c5affd640 1 -- 172.21.15.44:0/3844972819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ee7410 msgr2=0x55d629efed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c5affd640 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ee7410 0x55d629efed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c5affd640 1 -- 172.21.15.44:0/3844972819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed8950 msgr2=0x55d629eefa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:03:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c5affd640 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed8950 0x55d629eefa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.864+0000 7f3c5affd640 1 -- 172.21.15.44:0/3844972819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d629eee6c0 con 0x55d629ed9270 2024-08-20T03:03:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.865+0000 7f3c5b7fe640 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed8950 0x55d629eefa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:03:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.865+0000 7f3c5affd640 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ed9270 0x55d629efe5d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000c6a0 tx=0x7f3c500e1f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bdc387ebe93bd21 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:03:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.872+0000 7f3c3affd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:03:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.872+0000 7f3c3affd640 1 -- 172.21.15.44:0/3844972819 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c500bd020 con 0x55d629ed9270 2024-08-20T03:03:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.872+0000 7f3c64fec740 1 -- 172.21.15.44:0/3844972819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c500aaae0 con 0x55d629ed9270 2024-08-20T03:03:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.872+0000 7f3c3affd640 1 -- 172.21.15.44:0/3844972819 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3c500aa230 con 0x55d629ed9270 2024-08-20T03:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.872+0000 7f3c3affd640 1 -- 172.21.15.44:0/3844972819 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c50010b90 con 0x55d629ed9270 2024-08-20T03:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.873+0000 7f3c64fec740 10 client.25525 fetch_fsmap learned FSMap version 30 2024-08-20T03:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.873+0000 7f3c64fec740 1 -- 172.21.15.44:0/3844972819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d629f68430 con 0x55d629ed9270 2024-08-20T03:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.873+0000 7f3c3affd640 1 -- 172.21.15.44:0/3844972819 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c500aaae0 con 0x55d629ed9270 2024-08-20T03:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.873+0000 7f3c3affd640 1 -- 172.21.15.44:0/3844972819 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3c500a6940 con 0x55d629ed9270 2024-08-20T03:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.873+0000 7f3c64fec740 10 client.25525 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.873+0000 7f3c64fec740 10 client.25525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:03:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.873+0000 7f3c64fec740 20 client.25525 populate_metadata read hostname 'smithi044' 2024-08-20T03:03:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.873+0000 7f3c64fec740 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d629f68c10 0x55d629f6b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:03:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.873+0000 7f3c64fec740 4 client.25525 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:03:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.873+0000 7f3c64fec740 1 -- 172.21.15.44:0/3844972819 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d629e4bd50 con 0x55d629f68c10 2024-08-20T03:03:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.873+0000 7f3c5b7fe640 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d629f68c10 0x55d629f6b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:03:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.874+0000 7f3c5b7fe640 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d629f68c10 0x55d629f6b030 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22b0db5d0760a6c4 server_cookie=a4ab397937ea3e13 in_seq=0 out_seq=0 2024-08-20T03:03:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:45.875+0000 7f3c3affd640 10 client.25525 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:46.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:46 smithi044 ceph-mon[18790]: pgmap v358: 97 pgs: 97 active+clean; 66 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 452 op/s 2024-08-20T03:03:46.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:46 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:03:46.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:46 smithi165 ceph-mon[22357]: pgmap v358: 97 pgs: 97 active+clean; 66 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 452 op/s 2024-08-20T03:03:46.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:46 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:03:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:46 smithi099 ceph-mon[21984]: pgmap v358: 97 pgs: 97 active+clean; 66 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.8 MiB/s wr, 452 op/s 2024-08-20T03:03:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:46 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:03:47.475 INFO:tasks.workunit.client.0.smithi044.stdout: 33 760896 772 554207 826 307479 1719 2024-08-20T03:03:48.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:48 smithi044 ceph-mon[18790]: pgmap v359: 97 pgs: 97 active+clean; 68 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 290 op/s 2024-08-20T03:03:48.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:48 smithi165 ceph-mon[22357]: pgmap v359: 97 pgs: 97 active+clean; 68 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 290 op/s 2024-08-20T03:03:48.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:48 smithi099 ceph-mon[21984]: pgmap v359: 97 pgs: 97 active+clean; 68 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 290 op/s 2024-08-20T03:03:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:50 smithi044 ceph-mon[18790]: pgmap v360: 97 pgs: 97 active+clean; 75 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 302 op/s 2024-08-20T03:03:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:50 smithi165 ceph-mon[22357]: pgmap v360: 97 pgs: 97 active+clean; 75 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 302 op/s 2024-08-20T03:03:50.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:50 smithi099 ceph-mon[21984]: pgmap v360: 97 pgs: 97 active+clean; 75 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 302 op/s 2024-08-20T03:03:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c3affd640 1 -- 172.21.15.44:0/3844972819 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d629e4bd50 con 0x55d629f68c10 2024-08-20T03:03:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c3affd640 10 client.25525 handle_command_reply: tid=0 2024-08-20T03:03:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c64fec740 1 client.25525 shutdown 2024-08-20T03:03:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c64fec740 1 -- 172.21.15.44:0/3844972819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ed9270 msgr2=0x55d629efe5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c64fec740 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ed9270 0x55d629efe5d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000c6a0 tx=0x7f3c500e1f10 comp rx=0 tx=0).stop 2024-08-20T03:03:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c64fec740 1 -- 172.21.15.44:0/3844972819 shutdown_connections 2024-08-20T03:03:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c64fec740 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d629f68c10 0x55d629f6b030 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c64fec740 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d629ee7410 0x55d629efed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c64fec740 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d629ed9270 0x55d629efe5d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c64fec740 1 --2- 172.21.15.44:0/3844972819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d629ed8950 0x55d629eefa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c3affd640 0 client.25525 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:03:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c64fec740 1 -- 172.21.15.44:0/3844972819 >> 172.21.15.44:0/3844972819 conn(0x55d629ed6730 msgr2=0x55d629ee5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c64fec740 1 -- 172.21.15.44:0/3844972819 shutdown_connections 2024-08-20T03:03:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c64fec740 1 -- 172.21.15.44:0/3844972819 wait complete. 2024-08-20T03:03:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.684+0000 7f3c64fec740 20 client.25525 trim_cache size 0 max 16384 2024-08-20T03:03:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:03:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:03:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:03:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:03:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.685+0000 7f3c621fd640 1 -- 172.21.15.44:0/1452816982 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3c440b3d40 msgr2=0x7f3c440b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:50.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.685+0000 7f3c621fd640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3c440b3d40 0x7f3c440b6200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009fd0 tx=0x7f3c4c005f90 comp rx=0 tx=0).stop 2024-08-20T03:03:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.685+0000 7f3c621fd640 1 -- 172.21.15.44:0/1452816982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c5c1072d0 msgr2=0x7f3c5c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:03:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.685+0000 7f3c621fd640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c5c1072d0 0x7f3c5c198330 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000e9f0 tx=0x7f3c5000eeb0 comp rx=0 tx=0).stop 2024-08-20T03:03:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.685+0000 7f3c621fd640 1 -- 172.21.15.44:0/1452816982 shutdown_connections 2024-08-20T03:03:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.685+0000 7f3c621fd640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3c440b3d40 0x7f3c440b6200 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.685+0000 7f3c621fd640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c5c107c50 0x7f3c5c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.685+0000 7f3c621fd640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c5c1072d0 0x7f3c5c198330 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.685+0000 7f3c621fd640 1 --2- 172.21.15.44:0/1452816982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c5c101920 0x7f3c5c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:03:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.685+0000 7f3c621fd640 1 -- 172.21.15.44:0/1452816982 >> 172.21.15.44:0/1452816982 conn(0x7f3c5c078620 msgr2=0x7f3c5c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:03:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.685+0000 7f3c621fd640 1 -- 172.21.15.44:0/1452816982 shutdown_connections 2024-08-20T03:03:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:03:50.685+0000 7f3c621fd640 1 -- 172.21.15.44:0/1452816982 wait complete. 2024-08-20T03:03:52.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:52 smithi044 ceph-mon[18790]: pgmap v361: 97 pgs: 97 active+clean; 75 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 573 op/s 2024-08-20T03:03:52.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:52 smithi165 ceph-mon[22357]: pgmap v361: 97 pgs: 97 active+clean; 75 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 573 op/s 2024-08-20T03:03:52.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:52 smithi099 ceph-mon[21984]: pgmap v361: 97 pgs: 97 active+clean; 75 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 573 op/s 2024-08-20T03:03:54.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:54 smithi044 ceph-mon[18790]: pgmap v362: 97 pgs: 97 active+clean; 56 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 939 op/s 2024-08-20T03:03:54.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:54 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:54.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:54 smithi165 ceph-mon[22357]: pgmap v362: 97 pgs: 97 active+clean; 56 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 939 op/s 2024-08-20T03:03:54.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:54 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:54.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:54 smithi099 ceph-mon[21984]: pgmap v362: 97 pgs: 97 active+clean; 56 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 939 op/s 2024-08-20T03:03:54.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:54 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:03:55.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:55 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:03:55.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:55 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:03:55.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:55 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:03:56.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:56 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:03:56.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:56 smithi044 ceph-mon[18790]: pgmap v363: 97 pgs: 97 active+clean; 59 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 1.05k op/s 2024-08-20T03:03:56.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:56 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:03:56.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:56 smithi165 ceph-mon[22357]: pgmap v363: 97 pgs: 97 active+clean; 59 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 1.05k op/s 2024-08-20T03:03:56.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:56 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:03:56.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:56 smithi099 ceph-mon[21984]: pgmap v363: 97 pgs: 97 active+clean; 59 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 1.05k op/s 2024-08-20T03:03:57.476 INFO:tasks.workunit.client.0.smithi044.stdout: 45 545472 619 369697 522 264904 1419 2024-08-20T03:03:58.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:03:58 smithi044 ceph-mon[18790]: pgmap v364: 97 pgs: 97 active+clean; 59 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 1.16k op/s 2024-08-20T03:03:58.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:03:58 smithi165 ceph-mon[22357]: pgmap v364: 97 pgs: 97 active+clean; 59 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 1.16k op/s 2024-08-20T03:03:58.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:03:58 smithi099 ceph-mon[21984]: pgmap v364: 97 pgs: 97 active+clean; 59 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 1.16k op/s 2024-08-20T03:04:00.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:00 smithi044 ceph-mon[18790]: pgmap v365: 97 pgs: 97 active+clean; 60 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.1 MiB/s wr, 1.19k op/s 2024-08-20T03:04:00.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:00 smithi165 ceph-mon[22357]: pgmap v365: 97 pgs: 97 active+clean; 60 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.1 MiB/s wr, 1.19k op/s 2024-08-20T03:04:00.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:00 smithi099 ceph-mon[21984]: pgmap v365: 97 pgs: 97 active+clean; 60 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.1 MiB/s wr, 1.19k op/s 2024-08-20T03:04:02.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:02 smithi044 ceph-mon[18790]: pgmap v366: 97 pgs: 97 active+clean; 93 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.3 MiB/s wr, 1.22k op/s 2024-08-20T03:04:02.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:02 smithi165 ceph-mon[22357]: pgmap v366: 97 pgs: 97 active+clean; 93 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.3 MiB/s wr, 1.22k op/s 2024-08-20T03:04:02.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:02 smithi099 ceph-mon[21984]: pgmap v366: 97 pgs: 97 active+clean; 93 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.3 MiB/s wr, 1.22k op/s 2024-08-20T03:04:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.838+0000 7f87bddbf640 1 -- 172.21.15.44:0/4024205953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b810ef40 msgr2=0x7f87b810f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.839+0000 7f87bddbf640 1 --2- 172.21.15.44:0/4024205953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b810ef40 0x7f87b810f380 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009860 tx=0x7f87a802a360 comp rx=0 tx=0).stop 2024-08-20T03:04:03.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.839+0000 7f87bddbf640 1 -- 172.21.15.44:0/4024205953 shutdown_connections 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.839+0000 7f87bddbf640 1 --2- 172.21.15.44:0/4024205953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b810f8c0 0x7f87b8115ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.839+0000 7f87bddbf640 1 --2- 172.21.15.44:0/4024205953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b810ef40 0x7f87b810f380 secure :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009860 tx=0x7f87a802a360 comp rx=0 tx=0).stop 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.839+0000 7f87bddbf640 1 --2- 172.21.15.44:0/4024205953 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b810a0f0 0x7f87b810a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.839+0000 7f87bddbf640 1 -- 172.21.15.44:0/4024205953 >> 172.21.15.44:0/4024205953 conn(0x7f87b806dec0 msgr2=0x7f87b806e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.839+0000 7f87bddbf640 1 -- 172.21.15.44:0/4024205953 shutdown_connections 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.839+0000 7f87bddbf640 1 -- 172.21.15.44:0/4024205953 wait complete. 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.839+0000 7f87bddbf640 1 Processor -- start 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87bddbf640 1 -- start start 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87bddbf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b810a0f0 0x7f87b81a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87bddbf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b810f8c0 0x7f87b81a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87bddbf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b81a7340 0x7f87b81a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87bddbf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8119030 con 0x7f87b810f8c0 2024-08-20T03:04:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87bddbf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8118eb0 con 0x7f87b810a0f0 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87bddbf640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81191b0 con 0x7f87b81a7340 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b810f8c0 0x7f87b81a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b810a0f0 0x7f87b81a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87b6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b810f8c0 0x7f87b81a1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58010/0 (socket says 172.21.15.44:58010) 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87b6ffd640 1 -- 172.21.15.44:0/1349633081 learned_addr learned my addr 172.21.15.44:0/1349633081 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87b7fff640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b81a7340 0x7f87b81a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87b6ffd640 1 -- 172.21.15.44:0/1349633081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b81a7340 msgr2=0x7f87b81a77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87b6ffd640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b81a7340 0x7f87b81a77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87b6ffd640 1 -- 172.21.15.44:0/1349633081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b810a0f0 msgr2=0x7f87b81a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87b6ffd640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b810a0f0 0x7f87b81a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87b6ffd640 1 -- 172.21.15.44:0/1349633081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b81a7f20 con 0x7f87b810f8c0 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.840+0000 7f87b77fe640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b810a0f0 0x7f87b81a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.841+0000 7f87b6ffd640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b810f8c0 0x7f87b81a1fb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f87a80057c0 tx=0x7f87a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:03.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.841+0000 7f87b4ff9640 1 -- 172.21.15.44:0/1349633081 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8037050 con 0x7f87b810f8c0 2024-08-20T03:04:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.841+0000 7f87bddbf640 1 -- 172.21.15.44:0/1349633081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b8113fe0 con 0x7f87b810f8c0 2024-08-20T03:04:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.841+0000 7f87b4ff9640 1 -- 172.21.15.44:0/1349633081 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a8038040 con 0x7f87b810f8c0 2024-08-20T03:04:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.841+0000 7f87b4ff9640 1 -- 172.21.15.44:0/1349633081 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8030a10 con 0x7f87b810f8c0 2024-08-20T03:04:03.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.841+0000 7f87bddbf640 1 -- 172.21.15.44:0/1349633081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b8114520 con 0x7f87b810f8c0 2024-08-20T03:04:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.843+0000 7f87c0c90740 1 Processor -- start 2024-08-20T03:04:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.843+0000 7f87c0c90740 1 -- start start 2024-08-20T03:04:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87c0c90740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f0b950 0x559c39f0bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b4ff9640 1 -- 172.21.15.44:0/1349633081 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f87a8045020 con 0x7f87b810f8c0 2024-08-20T03:04:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87c0c90740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0c270 0x559c39f17cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87c0c90740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f18200 0x559c39f1a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.850 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87c0c90740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559c39e81360 con 0x559c39f0c270 2024-08-20T03:04:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87c0c90740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559c39e70860 con 0x559c39f0b950 2024-08-20T03:04:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87c0c90740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559c39e748a0 con 0x559c39f18200 2024-08-20T03:04:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0c270 0x559c39f17cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.851 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f0b950 0x559c39f0bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.853 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f0b950 0x559c39f0bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54566/0 (socket says 172.21.15.44:54566) 2024-08-20T03:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0c270 0x559c39f17cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58014/0 (socket says 172.21.15.44:58014) 2024-08-20T03:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b7fff640 1 -- 172.21.15.44:0/4020872153 learned_addr learned my addr 172.21.15.44:0/4020872153 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b6ffd640 1 --2- 172.21.15.44:0/4020872153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f18200 0x559c39f1a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b77fe640 1 -- 172.21.15.44:0/4020872153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f18200 msgr2=0x559c39f1a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b77fe640 1 --2- 172.21.15.44:0/4020872153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f18200 0x559c39f1a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b77fe640 1 -- 172.21.15.44:0/4020872153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f0b950 msgr2=0x559c39f0bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b77fe640 1 --2- 172.21.15.44:0/4020872153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f0b950 0x559c39f0bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b77fe640 1 -- 172.21.15.44:0/4020872153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c39e17e40 con 0x559c39f0c270 2024-08-20T03:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b7fff640 1 --2- 172.21.15.44:0/4020872153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f0b950 0x559c39f0bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.844+0000 7f87b77fe640 1 --2- 172.21.15.44:0/4020872153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0c270 0x559c39f17cc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009d50 tx=0x7f87ac00e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8391e17daec79a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.845+0000 7f87b4ff9640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f87a00b3d40 0x7f87a00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.845+0000 7f879dffb640 1 -- 172.21.15.44:0/4020872153 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac017070 con 0x559c39f0c270 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.845+0000 7f879dffb640 1 -- 172.21.15.44:0/4020872153 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ac013920 con 0x559c39f0c270 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.845+0000 7f879dffb640 1 -- 172.21.15.44:0/4020872153 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac012790 con 0x559c39f0c270 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.845+0000 7f87b4ff9640 1 -- 172.21.15.44:0/1349633081 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f87a8103b80 con 0x7f87b810f8c0 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.845+0000 7f87b7fff640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f87a00b3d40 0x7f87a00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.846+0000 7f87c0c90740 1 -- 172.21.15.44:0/4020872153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0c270 msgr2=0x559c39f17cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.846+0000 7f87c0c90740 1 --2- 172.21.15.44:0/4020872153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0c270 0x559c39f17cc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009d50 tx=0x7f87ac00e5a0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.846+0000 7f87b7fff640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f87a00b3d40 0x7f87a00b6200 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f87b000a090 tx=0x7f87b000c040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.846+0000 7f87c0c90740 1 -- 172.21.15.44:0/4020872153 shutdown_connections 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.846+0000 7f87c0c90740 1 --2- 172.21.15.44:0/4020872153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f18200 0x559c39f1a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.846+0000 7f87c0c90740 1 --2- 172.21.15.44:0/4020872153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0c270 0x559c39f17cc0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.846+0000 7f87c0c90740 1 --2- 172.21.15.44:0/4020872153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f0b950 0x559c39f0bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.846+0000 7f87c0c90740 1 -- 172.21.15.44:0/4020872153 >> 172.21.15.44:0/4020872153 conn(0x559c39f09730 msgr2=0x559c39f0a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.846+0000 7f87c0c90740 1 -- 172.21.15.44:0/4020872153 shutdown_connections 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.846+0000 7f87c0c90740 1 -- 172.21.15.44:0/4020872153 wait complete. 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.846+0000 7f87c0c90740 1 -- start start 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.846+0000 7f87c0c90740 1 Processor -- start 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87c0c90740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 0x559c39f17a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87c0c90740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0c270 0x559c39f14e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87c0c90740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f18200 0x559c39f153e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87c0c90740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559c39da9ab0 con 0x559c39f0b950 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87c0c90740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559c39df7780 con 0x559c39f18200 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87c0c90740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559c39f1aee0 con 0x559c39f0c270 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87c0c90740 10 client.0 _ll_register_callbacks cb 0x559c39b7fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 0x559c39f17a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 0x559c39f17a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58024/0 (socket says 172.21.15.44:58024) 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87b77fe640 1 -- 172.21.15.44:0/250286476 learned_addr learned my addr 172.21.15.44:0/250286476 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87b7fff640 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f18200 0x559c39f153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87b6ffd640 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0c270 0x559c39f14e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87b77fe640 1 -- 172.21.15.44:0/250286476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0c270 msgr2=0x559c39f14e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87b77fe640 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0c270 0x559c39f14e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87b77fe640 1 -- 172.21.15.44:0/250286476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f18200 msgr2=0x559c39f153e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87b77fe640 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f18200 0x559c39f153e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87b77fe640 1 -- 172.21.15.44:0/250286476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c39bd3790 con 0x559c39f0b950 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87b7fff640 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f18200 0x559c39f153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f87b77fe640 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 0x559c39f17a40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009950 tx=0x7f87ac013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f651ce4c103a54c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.847+0000 7f8796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f8796ffd640 1 -- 172.21.15.44:0/250286476 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac017040 con 0x559c39f0b950 2024-08-20T03:04:03.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f8796ffd640 1 -- 172.21.15.44:0/250286476 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f87ac0128e0 con 0x559c39f0b950 2024-08-20T03:04:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f87c0c90740 1 -- 172.21.15.44:0/250286476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559c39e1fe00 con 0x559c39f0b950 2024-08-20T03:04:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f8796ffd640 1 -- 172.21.15.44:0/250286476 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87ac013e20 con 0x559c39f0b950 2024-08-20T03:04:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f87c0c90740 10 client.16077 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f87c0c90740 1 -- 172.21.15.44:0/250286476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559c39c4c620 con 0x559c39f0b950 2024-08-20T03:04:03.858 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f8796ffd640 1 -- 172.21.15.44:0/250286476 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559c39e1fe00 con 0x559c39f0b950 2024-08-20T03:04:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f8796ffd640 1 -- 172.21.15.44:0/250286476 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f87ac023070 con 0x559c39f0b950 2024-08-20T03:04:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f87c0c90740 10 client.16077 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f87c0c90740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f87c0c90740 20 client.16077 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f87c0c90740 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559c39f27e40 0x559c39f2a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.848+0000 7f87c0c90740 4 client.16077 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:04:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.849+0000 7f87c0c90740 1 -- 172.21.15.44:0/250286476 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c39e832b0 con 0x559c39f27e40 2024-08-20T03:04:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.849+0000 7f87b6ffd640 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559c39f27e40 0x559c39f2a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.849+0000 7f87b6ffd640 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559c39f27e40 0x559c39f2a2a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547fc6a74f988031 server_cookie=148b6d3c4933dfb7 in_seq=0 out_seq=0 2024-08-20T03:04:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.849+0000 7f8796ffd640 10 client.16077 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:03.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.858+0000 7f8796ffd640 1 -- 172.21.15.44:0/250286476 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c39e832b0 con 0x559c39f27e40 2024-08-20T03:04:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.858+0000 7f8796ffd640 10 client.16077 handle_command_reply: tid=0 2024-08-20T03:04:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.858+0000 7f87c0c90740 1 client.16077 shutdown 2024-08-20T03:04:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.858+0000 7f87c0c90740 1 -- 172.21.15.44:0/250286476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 msgr2=0x559c39f17a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.858+0000 7f87c0c90740 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 0x559c39f17a40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009950 tx=0x7f87ac013c00 comp rx=0 tx=0).stop 2024-08-20T03:04:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.858+0000 7f87c0c90740 1 -- 172.21.15.44:0/250286476 shutdown_connections 2024-08-20T03:04:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.858+0000 7f87c0c90740 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559c39f27e40 0x559c39f2a2a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.858+0000 7f87c0c90740 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f18200 0x559c39f153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.861 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.858+0000 7f87c0c90740 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0c270 0x559c39f14e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.858+0000 7f87c0c90740 1 --2- 172.21.15.44:0/250286476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 0x559c39f17a40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.858+0000 7f87c0c90740 1 -- 172.21.15.44:0/250286476 >> 172.21.15.44:0/250286476 conn(0x559c39f09730 msgr2=0x559c39f0f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.858+0000 7f8796ffd640 0 client.16077 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.859+0000 7f87c0c90740 1 -- 172.21.15.44:0/250286476 shutdown_connections 2024-08-20T03:04:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.859+0000 7f87c0c90740 1 -- 172.21.15.44:0/250286476 wait complete. 2024-08-20T03:04:03.862 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.859+0000 7f87c0c90740 20 client.16077 trim_cache size 0 max 16384 2024-08-20T03:04:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.875+0000 7f87c0c90740 1 Processor -- start 2024-08-20T03:04:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.875+0000 7f87c0c90740 1 -- start start 2024-08-20T03:04:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87c0c90740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0b950 0x559c39f09040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87c0c90740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f0c270 0x559c39f174c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87c0c90740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f18200 0x559c39f17a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87c0c90740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81b9240 con 0x559c39f18200 2024-08-20T03:04:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87c0c90740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81bae40 con 0x559c39f0c270 2024-08-20T03:04:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f18200 0x559c39f17a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87c0c90740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81b9910 con 0x559c39f0b950 2024-08-20T03:04:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87b7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f18200 0x559c39f17a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58036/0 (socket says 172.21.15.44:58036) 2024-08-20T03:04:03.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87b7fff640 1 -- 172.21.15.44:0/1633300778 learned_addr learned my addr 172.21.15.44:0/1633300778 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87b6ffd640 1 --2- 172.21.15.44:0/1633300778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f0c270 0x559c39f174c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87b77fe640 1 --2- 172.21.15.44:0/1633300778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0b950 0x559c39f09040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87b7fff640 1 -- 172.21.15.44:0/1633300778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0b950 msgr2=0x559c39f09040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87b7fff640 1 --2- 172.21.15.44:0/1633300778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0b950 0x559c39f09040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87b7fff640 1 -- 172.21.15.44:0/1633300778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f0c270 msgr2=0x559c39f174c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87b7fff640 1 --2- 172.21.15.44:0/1633300778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f0c270 0x559c39f174c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87b7fff640 1 -- 172.21.15.44:0/1633300778 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c39f0faa0 con 0x559c39f18200 2024-08-20T03:04:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87b77fe640 1 --2- 172.21.15.44:0/1633300778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0b950 0x559c39f09040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:04:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.876+0000 7f87b7fff640 1 --2- 172.21.15.44:0/1633300778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f18200 0x559c39f17a00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f87b0005e30 tx=0x7f87b0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445381b2fb83636d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.877+0000 7f879d7fa640 1 -- 172.21.15.44:0/1633300778 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0004b20 con 0x559c39f18200 2024-08-20T03:04:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.877+0000 7f879d7fa640 1 -- 172.21.15.44:0/1633300778 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b0004cc0 con 0x559c39f18200 2024-08-20T03:04:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.877+0000 7f879d7fa640 1 -- 172.21.15.44:0/1633300778 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0005580 con 0x559c39f18200 2024-08-20T03:04:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.877+0000 7f87c0c90740 1 -- 172.21.15.44:0/1633300778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f18200 msgr2=0x559c39f17a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.877+0000 7f87c0c90740 1 --2- 172.21.15.44:0/1633300778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f18200 0x559c39f17a00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f87b0005e30 tx=0x7f87b0016000 comp rx=0 tx=0).stop 2024-08-20T03:04:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.878+0000 7f87c0c90740 1 -- 172.21.15.44:0/1633300778 shutdown_connections 2024-08-20T03:04:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.878+0000 7f87c0c90740 1 --2- 172.21.15.44:0/1633300778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f18200 0x559c39f17a00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.878+0000 7f87c0c90740 1 --2- 172.21.15.44:0/1633300778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f0c270 0x559c39f174c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.878+0000 7f87c0c90740 1 --2- 172.21.15.44:0/1633300778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0b950 0x559c39f09040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.878+0000 7f87c0c90740 1 -- 172.21.15.44:0/1633300778 >> 172.21.15.44:0/1633300778 conn(0x559c39f09730 msgr2=0x559c39f0a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.878+0000 7f87c0c90740 1 -- 172.21.15.44:0/1633300778 shutdown_connections 2024-08-20T03:04:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.878+0000 7f87c0c90740 1 -- 172.21.15.44:0/1633300778 wait complete. 2024-08-20T03:04:03.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.878+0000 7f87c0c90740 1 -- start start 2024-08-20T03:04:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.878+0000 7f87c0c90740 1 Processor -- start 2024-08-20T03:04:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87c0c90740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 0x559c39f203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87c0c90740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0c270 0x559c39f20920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87c0c90740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f18200 0x559c39f11120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87c0c90740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81ba6b0 con 0x559c39f0b950 2024-08-20T03:04:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87c0c90740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81b9fe0 con 0x559c39f18200 2024-08-20T03:04:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87c0c90740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81b7220 con 0x559c39f0c270 2024-08-20T03:04:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87c0c90740 10 client.0 _ll_register_callbacks cb 0x7f87b00097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 0x559c39f203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 0x559c39f203c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58040/0 (socket says 172.21.15.44:58040) 2024-08-20T03:04:03.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87b77fe640 1 -- 172.21.15.44:0/2294492885 learned_addr learned my addr 172.21.15.44:0/2294492885 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87b7fff640 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f18200 0x559c39f11120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87b6ffd640 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0c270 0x559c39f20920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87b7fff640 1 -- 172.21.15.44:0/2294492885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0c270 msgr2=0x559c39f20920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87b7fff640 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0c270 0x559c39f20920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87b7fff640 1 -- 172.21.15.44:0/2294492885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 msgr2=0x559c39f203c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87b7fff640 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 0x559c39f203c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87b7fff640 1 -- 172.21.15.44:0/2294492885 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c39f11950 con 0x559c39f18200 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87b77fe640 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 0x559c39f203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f87b7fff640 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f18200 0x559c39f11120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f87b0005e30 tx=0x7f87b0005aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2e5647378b9dec3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.879+0000 7f8796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f8796ffd640 1 -- 172.21.15.44:0/2294492885 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0004a40 con 0x559c39f18200 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f87c0c90740 1 -- 172.21.15.44:0/2294492885 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87b0004cc0 con 0x559c39f18200 2024-08-20T03:04:03.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f8796ffd640 1 -- 172.21.15.44:0/2294492885 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f87b0025660 con 0x559c39f18200 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f8796ffd640 1 -- 172.21.15.44:0/2294492885 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f87b003c050 con 0x559c39f18200 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f87c0c90740 10 client.25574 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f87c0c90740 1 -- 172.21.15.44:0/2294492885 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559c39f12c60 con 0x559c39f18200 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f8796ffd640 1 -- 172.21.15.44:0/2294492885 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b0004cc0 con 0x559c39f18200 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f8796ffd640 1 -- 172.21.15.44:0/2294492885 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f87b002ea40 con 0x559c39f18200 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f87c0c90740 10 client.25574 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f87c0c90740 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f87c0c90740 20 client.25574 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f87c0c90740 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559c39f91e20 0x559c39f94240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f87c0c90740 4 client.25574 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.880+0000 7f87c0c90740 1 -- 172.21.15.44:0/2294492885 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c39e7ed50 con 0x559c39f91e20 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.882+0000 7f87b77fe640 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559c39f91e20 0x559c39f94240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.882+0000 7f8796ffd640 10 client.25574 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:03.882+0000 7f87b77fe640 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559c39f91e20 0x559c39f94240 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc8aca8bc5c7a32e server_cookie=5db5a1b488418265 in_seq=0 out_seq=0 2024-08-20T03:04:04.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:04 smithi044 ceph-mon[18790]: pgmap v367: 97 pgs: 97 active+clean; 130 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.6 MiB/s wr, 1.10k op/s 2024-08-20T03:04:04.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:04 smithi165 ceph-mon[22357]: pgmap v367: 97 pgs: 97 active+clean; 130 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.6 MiB/s wr, 1.10k op/s 2024-08-20T03:04:04.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:04 smithi099 ceph-mon[21984]: pgmap v367: 97 pgs: 97 active+clean; 130 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.6 MiB/s wr, 1.10k op/s 2024-08-20T03:04:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.744+0000 7f8796ffd640 1 -- 172.21.15.44:0/2294492885 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c39e7ed50 con 0x559c39f91e20 2024-08-20T03:04:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.744+0000 7f8796ffd640 10 client.25574 handle_command_reply: tid=0 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.744+0000 7f87c0c90740 1 client.25574 shutdown 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.744+0000 7f87c0c90740 1 -- 172.21.15.44:0/2294492885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f18200 msgr2=0x559c39f11120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.744+0000 7f87c0c90740 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f18200 0x559c39f11120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f87b0005e30 tx=0x7f87b0005aa0 comp rx=0 tx=0).stop 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.744+0000 7f87c0c90740 1 -- 172.21.15.44:0/2294492885 shutdown_connections 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.744+0000 7f87c0c90740 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559c39f91e20 0x559c39f94240 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.744+0000 7f87c0c90740 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559c39f18200 0x559c39f11120 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.744+0000 7f87c0c90740 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559c39f0c270 0x559c39f20920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.744+0000 7f87c0c90740 1 --2- 172.21.15.44:0/2294492885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c39f0b950 0x559c39f203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.744+0000 7f87c0c90740 1 -- 172.21.15.44:0/2294492885 >> 172.21.15.44:0/2294492885 conn(0x559c39f09730 msgr2=0x559c39f13590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.745+0000 7f8796ffd640 0 client.25574 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.745+0000 7f87c0c90740 1 -- 172.21.15.44:0/2294492885 shutdown_connections 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.745+0000 7f87c0c90740 1 -- 172.21.15.44:0/2294492885 wait complete. 2024-08-20T03:04:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.745+0000 7f87c0c90740 20 client.25574 trim_cache size 0 max 16384 2024-08-20T03:04:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:04:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:04:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:04:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:04:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.746+0000 7f87bddbf640 1 -- 172.21.15.44:0/1349633081 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f87a00b3d40 msgr2=0x7f87a00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.746+0000 7f87bddbf640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f87a00b3d40 0x7f87a00b6200 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f87b000a090 tx=0x7f87b000c040 comp rx=0 tx=0).stop 2024-08-20T03:04:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.746+0000 7f87bddbf640 1 -- 172.21.15.44:0/1349633081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b810f8c0 msgr2=0x7f87b81a1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.746+0000 7f87bddbf640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b810f8c0 0x7f87b81a1fb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f87a80057c0 tx=0x7f87a80042f0 comp rx=0 tx=0).stop 2024-08-20T03:04:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.746+0000 7f87bddbf640 1 -- 172.21.15.44:0/1349633081 shutdown_connections 2024-08-20T03:04:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.746+0000 7f87bddbf640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f87a00b3d40 0x7f87a00b6200 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.746+0000 7f87bddbf640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b81a7340 0x7f87b81a77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.746+0000 7f87bddbf640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87b810f8c0 0x7f87b81a1fb0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.746+0000 7f87bddbf640 1 --2- 172.21.15.44:0/1349633081 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b810a0f0 0x7f87b81a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.746+0000 7f87bddbf640 1 -- 172.21.15.44:0/1349633081 >> 172.21.15.44:0/1349633081 conn(0x7f87b806dec0 msgr2=0x7f87b8072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.746+0000 7f87bddbf640 1 -- 172.21.15.44:0/1349633081 shutdown_connections 2024-08-20T03:04:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:05.746+0000 7f87bddbf640 1 -- 172.21.15.44:0/1349633081 wait complete. 2024-08-20T03:04:06.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:06 smithi044 ceph-mon[18790]: pgmap v368: 97 pgs: 97 active+clean; 170 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 942 op/s 2024-08-20T03:04:06.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:06 smithi165 ceph-mon[22357]: pgmap v368: 97 pgs: 97 active+clean; 170 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 942 op/s 2024-08-20T03:04:06.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:06 smithi099 ceph-mon[21984]: pgmap v368: 97 pgs: 97 active+clean; 170 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 942 op/s 2024-08-20T03:04:07.476 INFO:tasks.workunit.client.0.smithi044.stdout: 47 62460 105 40289 159 31238 435 2024-08-20T03:04:08.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:08 smithi165 ceph-mon[22357]: pgmap v369: 97 pgs: 97 active+clean; 165 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 964 op/s 2024-08-20T03:04:08.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:08 smithi099 ceph-mon[21984]: pgmap v369: 97 pgs: 97 active+clean; 165 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 964 op/s 2024-08-20T03:04:08.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:08 smithi044 ceph-mon[18790]: pgmap v369: 97 pgs: 97 active+clean; 165 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 11 MiB/s wr, 964 op/s 2024-08-20T03:04:10.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:10 smithi165 ceph-mon[22357]: pgmap v370: 97 pgs: 97 active+clean; 167 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-08-20T03:04:10.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:10 smithi099 ceph-mon[21984]: pgmap v370: 97 pgs: 97 active+clean; 167 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-08-20T03:04:10.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:10 smithi044 ceph-mon[18790]: pgmap v370: 97 pgs: 97 active+clean; 167 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-08-20T03:04:12.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:12 smithi165 ceph-mon[22357]: pgmap v371: 97 pgs: 97 active+clean; 158 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-08-20T03:04:12.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:12 smithi099 ceph-mon[21984]: pgmap v371: 97 pgs: 97 active+clean; 158 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-08-20T03:04:12.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:12 smithi044 ceph-mon[18790]: pgmap v371: 97 pgs: 97 active+clean; 158 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-08-20T03:04:13.527 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:04:13.947 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:04:14.311 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:14 smithi044 ceph-mon[18790]: pgmap v372: 97 pgs: 97 active+clean; 169 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.26k op/s 2024-08-20T03:04:14.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:14 smithi165 ceph-mon[22357]: pgmap v372: 97 pgs: 97 active+clean; 169 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.26k op/s 2024-08-20T03:04:14.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:14 smithi099 ceph-mon[21984]: pgmap v372: 97 pgs: 97 active+clean; 169 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.26k op/s 2024-08-20T03:04:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.598+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3298690582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ba0107e10 msgr2=0x7f1ba010e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.598+0000 7f1ba7b20640 1 --2- 172.21.15.44:0/3298690582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ba0107e10 0x7f1ba010e540 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009be0 tx=0x7f1b9802a6c0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.598+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3298690582 shutdown_connections 2024-08-20T03:04:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.598+0000 7f1ba7b20640 1 --2- 172.21.15.44:0/3298690582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ba0107e10 0x7f1ba010e540 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.598+0000 7f1ba7b20640 1 --2- 172.21.15.44:0/3298690582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ba0107490 0x7f1ba01078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.598+0000 7f1ba7b20640 1 --2- 172.21.15.44:0/3298690582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba01026a0 0x7f1ba0102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.598+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3298690582 >> 172.21.15.44:0/3298690582 conn(0x7f1ba0076bd0 msgr2=0x7f1ba0076fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.598+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3298690582 shutdown_connections 2024-08-20T03:04:14.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.598+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3298690582 wait complete. 2024-08-20T03:04:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.599+0000 7f1ba7b20640 1 Processor -- start 2024-08-20T03:04:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.600+0000 7f1ba7b20640 1 -- start start 2024-08-20T03:04:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.600+0000 7f1ba7b20640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ba01026a0 0x7f1ba0197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.600+0000 7f1ba7b20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba0107490 0x7f1ba01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.600+0000 7f1ba7b20640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ba0107e10 0x7f1ba019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba7b20640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba0111980 con 0x7f1ba0107490 2024-08-20T03:04:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba50bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba0107490 0x7f1ba01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba50bb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba0107490 0x7f1ba01983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:56672/0 (socket says 172.21.15.44:56672) 2024-08-20T03:04:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba50bb640 1 -- 172.21.15.44:0/3173505065 learned_addr learned my addr 172.21.15.44:0/3173505065 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:14.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ba01026a0 0x7f1ba0197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3173505065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba0111800 con 0x7f1ba0107e10 2024-08-20T03:04:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3173505065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba0111b00 con 0x7f1ba01026a0 2024-08-20T03:04:14.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba60bd640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ba0107e10 0x7f1ba019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba58bc640 1 -- 172.21.15.44:0/3173505065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ba0107e10 msgr2=0x7f1ba019d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ba0107e10 0x7f1ba019d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba58bc640 1 -- 172.21.15.44:0/3173505065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba0107490 msgr2=0x7f1ba01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba0107490 0x7f1ba01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba58bc640 1 -- 172.21.15.44:0/3173505065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba019df20 con 0x7f1ba01026a0 2024-08-20T03:04:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.601+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ba01026a0 0x7f1ba0197e70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009fd0 tx=0x7f1b9c00ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.602+0000 7f1b96ffd640 1 -- 172.21.15.44:0/3173505065 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c00ccc0 con 0x7f1ba01026a0 2024-08-20T03:04:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.602+0000 7f1b96ffd640 1 -- 172.21.15.44:0/3173505065 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b9c00ee70 con 0x7f1ba01026a0 2024-08-20T03:04:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.602+0000 7f1b96ffd640 1 -- 172.21.15.44:0/3173505065 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c004a00 con 0x7f1ba01026a0 2024-08-20T03:04:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.602+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3173505065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba019e210 con 0x7f1ba01026a0 2024-08-20T03:04:14.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.602+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3173505065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba0101330 con 0x7f1ba01026a0 2024-08-20T03:04:14.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.606+0000 7f1b96ffd640 1 -- 172.21.15.44:0/3173505065 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1b9c004ba0 con 0x7f1ba01026a0 2024-08-20T03:04:14.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.606+0000 7f1ba9f19740 1 Processor -- start 2024-08-20T03:04:14.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.606+0000 7f1ba9f19740 1 -- start start 2024-08-20T03:04:14.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba9f19740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x559154046c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba9f19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x559154052ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba9f19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x5591540554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba9f19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559153fbd8e0 con 0x5591540530e0 2024-08-20T03:04:14.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba9f19740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559153facde0 con 0x559154046830 2024-08-20T03:04:14.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba9f19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559153fb0e20 con 0x559154047150 2024-08-20T03:04:14.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba58bc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x559154052ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba58bc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x559154052ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53158/0 (socket says 172.21.15.44:53158) 2024-08-20T03:04:14.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba58bc640 1 -- 172.21.15.44:0/2080587839 learned_addr learned my addr 172.21.15.44:0/2080587839 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:14.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba50bb640 1 --2- 172.21.15.44:0/2080587839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x5591540554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba60bd640 1 --2- 172.21.15.44:0/2080587839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x559154046c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba58bc640 1 -- 172.21.15.44:0/2080587839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 msgr2=0x559154046c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:14.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/2080587839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x559154046c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1ba58bc640 1 -- 172.21.15.44:0/2080587839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 msgr2=0x5591540554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:14.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.607+0000 7f1b96ffd640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1b8c0b3dd0 0x7f1b8c0b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.608+0000 7f1ba60bd640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1b8c0b3dd0 0x7f1b8c0b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.608+0000 7f1b96ffd640 1 -- 172.21.15.44:0/3173505065 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1b9c0e40d0 con 0x7f1ba01026a0 2024-08-20T03:04:14.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.608+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/2080587839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x5591540554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.608+0000 7f1ba58bc640 1 -- 172.21.15.44:0/2080587839 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559153d0c650 con 0x559154047150 2024-08-20T03:04:14.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.608+0000 7f1ba50bb640 1 --2- 172.21.15.44:0/2080587839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x5591540554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:04:14.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.609+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/2080587839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x559154052ba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0e17d0 tx=0x7f1b9c0052e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3325b94916b8a1dd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:14.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.609+0000 7f1ba60bd640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1b8c0b3dd0 0x7f1b8c0b6290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1b98006010 tx=0x7f1b98009840 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:14.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.609+0000 7f1b8bfff640 1 -- 172.21.15.44:0/2080587839 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0aab20 con 0x559154047150 2024-08-20T03:04:14.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.609+0000 7f1b8bfff640 1 -- 172.21.15.44:0/2080587839 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b9c0aa860 con 0x559154047150 2024-08-20T03:04:14.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.609+0000 7f1b8bfff640 1 -- 172.21.15.44:0/2080587839 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0aa4c0 con 0x559154047150 2024-08-20T03:04:14.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.610+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2080587839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 msgr2=0x559154052ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:14.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.610+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2080587839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x559154052ba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0e17d0 tx=0x7f1b9c0052e0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.610+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2080587839 shutdown_connections 2024-08-20T03:04:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.610+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2080587839 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x5591540554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.610+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2080587839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x559154052ba0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.610+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2080587839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x559154046c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.610+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2080587839 >> 172.21.15.44:0/2080587839 conn(0x559154044660 msgr2=0x559154045800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.610+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2080587839 shutdown_connections 2024-08-20T03:04:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.610+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2080587839 wait complete. 2024-08-20T03:04:14.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.611+0000 7f1ba9f19740 1 -- start start 2024-08-20T03:04:14.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.611+0000 7f1ba9f19740 1 Processor -- start 2024-08-20T03:04:14.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.611+0000 7f1ba9f19740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x559154050060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.611+0000 7f1ba9f19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x5591540505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.611+0000 7f1ba9f19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x559154050ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba9f19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559153e0f700 con 0x5591540530e0 2024-08-20T03:04:14.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba9f19740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559153f337c0 con 0x559154046830 2024-08-20T03:04:14.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba9f19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559154055cd0 con 0x559154047150 2024-08-20T03:04:14.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba50bb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x5591540505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba50bb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x5591540505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53172/0 (socket says 172.21.15.44:53172) 2024-08-20T03:04:14.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba50bb640 1 -- 172.21.15.44:0/2546650510 learned_addr learned my addr 172.21.15.44:0/2546650510 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:14.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x559154050060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba9f19740 10 client.0 _ll_register_callbacks cb 0x559153fa66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:14.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba60bd640 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x559154050ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba50bb640 1 -- 172.21.15.44:0/2546650510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 msgr2=0x559154050060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:14.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba50bb640 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x559154050060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba50bb640 1 -- 172.21.15.44:0/2546650510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 msgr2=0x559154050ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:14.627 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba50bb640 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x559154050ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba50bb640 1 -- 172.21.15.44:0/2546650510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55915405f100 con 0x559154047150 2024-08-20T03:04:14.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba60bd640 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x559154050ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:04:14.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2546650510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559153f5c100 con 0x559154047150 2024-08-20T03:04:14.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1ba50bb640 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x5591540505a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009cc0 tx=0x7f1b9000e780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=711fe67786298153 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:14.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.612+0000 7f1b897fa640 10 client.25555 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:04:14.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.613+0000 7f1b897fa640 1 -- 172.21.15.44:0/2546650510 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90027060 con 0x559154047150 2024-08-20T03:04:14.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.613+0000 7f1b897fa640 1 -- 172.21.15.44:0/2546650510 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1b90022600 con 0x559154047150 2024-08-20T03:04:14.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.613+0000 7f1b897fa640 1 -- 172.21.15.44:0/2546650510 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1b90022840 con 0x559154047150 2024-08-20T03:04:14.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.613+0000 7f1ba9f19740 10 client.25555 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:14.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.613+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2546650510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({fsmap=30}) -- 0x559154061a40 con 0x559154047150 2024-08-20T03:04:14.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.613+0000 7f1b897fa640 1 -- 172.21.15.44:0/2546650510 <== mon.1 v2:172.21.15.165:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1b9002b3d0 con 0x559154047150 2024-08-20T03:04:14.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.613+0000 7f1ba9f19740 10 client.25555 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:14.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.613+0000 7f1ba9f19740 10 client.25555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:14.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.613+0000 7f1ba9f19740 20 client.25555 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:14.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.613+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559154062e10 0x559154065230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.613+0000 7f1ba9f19740 4 client.25555 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:04:14.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.613+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2546650510 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559153fbf830 con 0x559154062e10 2024-08-20T03:04:14.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.614+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559154062e10 0x559154065230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.614+0000 7f1b897fa640 10 client.25555 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:14.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.614+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559154062e10 0x559154065230 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0dc4f5f1c4ff530 server_cookie=7e0e333db45511e in_seq=0 out_seq=0 2024-08-20T03:04:14.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.617+0000 7f1b897fa640 1 -- 172.21.15.44:0/2546650510 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559153fbf830 con 0x559154062e10 2024-08-20T03:04:14.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.617+0000 7f1b897fa640 10 client.25555 handle_command_reply: tid=0 2024-08-20T03:04:14.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.617+0000 7f1ba9f19740 1 client.25555 shutdown 2024-08-20T03:04:14.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.617+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2546650510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 msgr2=0x5591540505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:14.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.617+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x5591540505a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009cc0 tx=0x7f1b9000e780 comp rx=0 tx=0).stop 2024-08-20T03:04:14.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.618+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2546650510 shutdown_connections 2024-08-20T03:04:14.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.618+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559154062e10 0x559154065230 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.618+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x559154050ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.618+0000 7f1b897fa640 0 client.25555 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:14.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.618+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x5591540505a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.618+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2546650510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x559154050060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.618+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2546650510 >> 172.21.15.44:0/2546650510 conn(0x559154044660 msgr2=0x55915404a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:14.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.618+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2546650510 shutdown_connections 2024-08-20T03:04:14.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.618+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2546650510 wait complete. 2024-08-20T03:04:14.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.618+0000 7f1ba9f19740 20 client.25555 trim_cache size 0 max 16384 2024-08-20T03:04:14.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.637+0000 7f1ba9f19740 1 Processor -- start 2024-08-20T03:04:14.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.637+0000 7f1ba9f19740 1 -- start start 2024-08-20T03:04:14.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.637+0000 7f1ba9f19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154046830 0x55915404a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.637+0000 7f1ba9f19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559154047150 0x559154050560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.637+0000 7f1ba9f19740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591540530e0 0x559154050aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.637+0000 7f1ba9f19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01ac090 con 0x559154047150 2024-08-20T03:04:14.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.637+0000 7f1ba9f19740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01adc90 con 0x5591540530e0 2024-08-20T03:04:14.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.637+0000 7f1ba9f19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01ac760 con 0x559154046830 2024-08-20T03:04:14.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba58bc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154046830 0x55915404a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba58bc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154046830 0x55915404a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53182/0 (socket says 172.21.15.44:53182) 2024-08-20T03:04:14.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba58bc640 1 -- 172.21.15.44:0/2275112514 learned_addr learned my addr 172.21.15.44:0/2275112514 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:14.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba50bb640 1 --2- 172.21.15.44:0/2275112514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559154047150 0x559154050560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba60bd640 1 --2- 172.21.15.44:0/2275112514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591540530e0 0x559154050aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba58bc640 1 -- 172.21.15.44:0/2275112514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591540530e0 msgr2=0x559154050aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:14.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/2275112514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591540530e0 0x559154050aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba58bc640 1 -- 172.21.15.44:0/2275112514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559154047150 msgr2=0x559154050560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:14.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/2275112514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559154047150 0x559154050560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba58bc640 1 -- 172.21.15.44:0/2275112514 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559154046f80 con 0x559154046830 2024-08-20T03:04:14.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba60bd640 1 --2- 172.21.15.44:0/2275112514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591540530e0 0x559154050aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:04:14.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba50bb640 1 --2- 172.21.15.44:0/2275112514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559154047150 0x559154050560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:04:14.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/2275112514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154046830 0x55915404a9f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0aaa50 tx=0x7f1b9c00ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a39f08ff2d95cbbc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:14.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1b8b7fe640 1 -- 172.21.15.44:0/2275112514 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0c4020 con 0x559154046830 2024-08-20T03:04:14.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1b8b7fe640 1 -- 172.21.15.44:0/2275112514 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b9c0af350 con 0x559154046830 2024-08-20T03:04:14.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.638+0000 7f1b8b7fe640 1 -- 172.21.15.44:0/2275112514 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0aa6a0 con 0x559154046830 2024-08-20T03:04:14.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.640+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2275112514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154046830 msgr2=0x55915404a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:14.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.640+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2275112514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154046830 0x55915404a9f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0aaa50 tx=0x7f1b9c00ce60 comp rx=0 tx=0).stop 2024-08-20T03:04:14.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.640+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2275112514 shutdown_connections 2024-08-20T03:04:14.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.640+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2275112514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591540530e0 0x559154050aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.640+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2275112514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559154047150 0x559154050560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.640+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/2275112514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154046830 0x55915404a9f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.640+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2275112514 >> 172.21.15.44:0/2275112514 conn(0x559154044660 msgr2=0x559154051eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:14.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.640+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2275112514 shutdown_connections 2024-08-20T03:04:14.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.640+0000 7f1ba9f19740 1 -- 172.21.15.44:0/2275112514 wait complete. 2024-08-20T03:04:14.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.640+0000 7f1ba9f19740 1 -- start start 2024-08-20T03:04:14.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.640+0000 7f1ba9f19740 1 Processor -- start 2024-08-20T03:04:14.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.641+0000 7f1ba9f19740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x55915405b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.641+0000 7f1ba9f19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x55915405b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.641+0000 7f1ba9f19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x559154056aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.641+0000 7f1ba9f19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01ad500 con 0x5591540530e0 2024-08-20T03:04:14.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.641+0000 7f1ba9f19740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01ace30 con 0x559154046830 2024-08-20T03:04:14.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.641+0000 7f1ba9f19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba01aa070 con 0x559154047150 2024-08-20T03:04:14.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.641+0000 7f1ba9f19740 10 client.0 _ll_register_callbacks cb 0x7f1b90009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:14.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.641+0000 7f1ba58bc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x55915405b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.641+0000 7f1ba50bb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x55915405b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.641+0000 7f1ba60bd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x559154056aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.641+0000 7f1ba50bb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x55915405b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53188/0 (socket says 172.21.15.44:53188) 2024-08-20T03:04:14.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.641+0000 7f1ba50bb640 1 -- 172.21.15.44:0/1941553031 learned_addr learned my addr 172.21.15.44:0/1941553031 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:14.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.642+0000 7f1ba50bb640 1 -- 172.21.15.44:0/1941553031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 msgr2=0x55915405b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:14.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.642+0000 7f1ba50bb640 1 --2- 172.21.15.44:0/1941553031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x55915405b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.642+0000 7f1ba50bb640 1 -- 172.21.15.44:0/1941553031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 msgr2=0x559154056aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:14.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.642+0000 7f1ba50bb640 1 --2- 172.21.15.44:0/1941553031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x559154056aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.642+0000 7f1ba50bb640 1 -- 172.21.15.44:0/1941553031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591540655a0 con 0x559154047150 2024-08-20T03:04:14.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.642+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/1941553031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x55915405b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:04:14.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.643+0000 7f1ba50bb640 1 --2- 172.21.15.44:0/1941553031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x55915405b900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000e7c0 tx=0x7f1b900046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d0fc9381a92426 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:14.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.643+0000 7f1b897fa640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:04:14.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.643+0000 7f1b897fa640 1 -- 172.21.15.44:0/1941553031 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90027040 con 0x559154047150 2024-08-20T03:04:14.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.643+0000 7f1b897fa640 1 -- 172.21.15.44:0/1941553031 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1b90022600 con 0x559154047150 2024-08-20T03:04:14.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.643+0000 7f1ba9f19740 1 -- 172.21.15.44:0/1941553031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559153fbb2d0 con 0x559154047150 2024-08-20T03:04:14.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.643+0000 7f1b897fa640 1 -- 172.21.15.44:0/1941553031 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1b900229e0 con 0x559154047150 2024-08-20T03:04:14.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.643+0000 7f1ba9f19740 10 client.25561 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:14.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.643+0000 7f1ba9f19740 1 -- 172.21.15.44:0/1941553031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591540cd6d0 con 0x559154047150 2024-08-20T03:04:14.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.643+0000 7f1b897fa640 1 -- 172.21.15.44:0/1941553031 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559153fbb2d0 con 0x559154047150 2024-08-20T03:04:14.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.643+0000 7f1b897fa640 1 -- 172.21.15.44:0/1941553031 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1b9003d050 con 0x559154047150 2024-08-20T03:04:14.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.644+0000 7f1ba9f19740 10 client.25561 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:14.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.644+0000 7f1ba9f19740 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:14.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.644+0000 7f1ba9f19740 20 client.25561 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:14.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.644+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/1941553031 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559154062e10 0x5591540cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:14.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.644+0000 7f1ba9f19740 4 client.25561 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:04:14.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.644+0000 7f1ba9f19740 1 -- 172.21.15.44:0/1941553031 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559153fb8a10 con 0x559154062e10 2024-08-20T03:04:14.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.644+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/1941553031 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559154062e10 0x5591540cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:14.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.644+0000 7f1b897fa640 10 client.25561 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.644+0000 7f1ba58bc640 1 --2- 172.21.15.44:0/1941553031 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559154062e10 0x5591540cffd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a3b89b66cd5790 server_cookie=36677fa1b03b85ef in_seq=0 out_seq=0 2024-08-20T03:04:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.649+0000 7f1b897fa640 1 -- 172.21.15.44:0/1941553031 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559153fb8a10 con 0x559154062e10 2024-08-20T03:04:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.649+0000 7f1b897fa640 10 client.25561 handle_command_reply: tid=0 2024-08-20T03:04:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.649+0000 7f1ba9f19740 1 client.25561 shutdown 2024-08-20T03:04:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.650+0000 7f1ba9f19740 1 -- 172.21.15.44:0/1941553031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 msgr2=0x55915405b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:14.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.650+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/1941553031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x55915405b900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000e7c0 tx=0x7f1b900046c0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.650+0000 7f1ba9f19740 1 -- 172.21.15.44:0/1941553031 shutdown_connections 2024-08-20T03:04:14.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.650+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/1941553031 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559154062e10 0x5591540cffd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.650+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/1941553031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591540530e0 0x559154056aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.650+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/1941553031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559154047150 0x55915405b900 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.650+0000 7f1ba9f19740 1 --2- 172.21.15.44:0/1941553031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559154046830 0x55915405b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.650+0000 7f1b897fa640 0 client.25561 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:14.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.650+0000 7f1ba9f19740 1 -- 172.21.15.44:0/1941553031 >> 172.21.15.44:0/1941553031 conn(0x559154044660 msgr2=0x559154042f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:14.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.650+0000 7f1ba9f19740 1 -- 172.21.15.44:0/1941553031 shutdown_connections 2024-08-20T03:04:14.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.650+0000 7f1ba9f19740 1 -- 172.21.15.44:0/1941553031 wait complete. 2024-08-20T03:04:14.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.650+0000 7f1ba9f19740 20 client.25561 trim_cache size 0 max 16384 2024-08-20T03:04:14.657 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:04:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-08-20T03:04:14.657 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-08-20T03:04:14.657 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:04:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.651+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3173505065 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1b8c0b3dd0 msgr2=0x7f1b8c0b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.651+0000 7f1ba7b20640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1b8c0b3dd0 0x7f1b8c0b6290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1b98006010 tx=0x7f1b98009840 comp rx=0 tx=0).stop 2024-08-20T03:04:14.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.651+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3173505065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ba01026a0 msgr2=0x7f1ba0197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.651+0000 7f1ba7b20640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ba01026a0 0x7f1ba0197e70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c009fd0 tx=0x7f1b9c00ec20 comp rx=0 tx=0).stop 2024-08-20T03:04:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.651+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3173505065 shutdown_connections 2024-08-20T03:04:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.651+0000 7f1ba7b20640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1b8c0b3dd0 0x7f1b8c0b6290 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.651+0000 7f1ba7b20640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1ba0107e10 0x7f1ba019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.651+0000 7f1ba7b20640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1ba0107490 0x7f1ba01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.651+0000 7f1ba7b20640 1 --2- 172.21.15.44:0/3173505065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ba01026a0 0x7f1ba0197e70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.651+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3173505065 >> 172.21.15.44:0/3173505065 conn(0x7f1ba0076bd0 msgr2=0x7f1ba00ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.651+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3173505065 shutdown_connections 2024-08-20T03:04:14.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:14.651+0000 7f1ba7b20640 1 -- 172.21.15.44:0/3173505065 wait complete. 2024-08-20T03:04:15.555 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T03:04:15.556 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:04:15.955 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:04:16.331 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:16 smithi044 ceph-mon[18790]: pgmap v373: 97 pgs: 97 active+clean; 169 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 1.05k op/s 2024-08-20T03:04:16.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:16 smithi165 ceph-mon[22357]: pgmap v373: 97 pgs: 97 active+clean; 169 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 1.05k op/s 2024-08-20T03:04:16.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:16 smithi099 ceph-mon[21984]: pgmap v373: 97 pgs: 97 active+clean; 169 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 1.05k op/s 2024-08-20T03:04:16.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.562+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1886832492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fc810a260 msgr2=0x7f6fc810a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:16.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.563+0000 7f6fce6e4640 1 --2- 172.21.15.44:0/1886832492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fc810a260 0x7f6fc810a6a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009860 tx=0x7f6fb802a410 comp rx=0 tx=0).stop 2024-08-20T03:04:16.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.564+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1886832492 shutdown_connections 2024-08-20T03:04:16.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.564+0000 7f6fce6e4640 1 --2- 172.21.15.44:0/1886832492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fc811bdd0 0x7f6fc811e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:16.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.564+0000 7f6fce6e4640 1 --2- 172.21.15.44:0/1886832492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fc810a260 0x7f6fc810a6a0 secure :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009860 tx=0x7f6fb802a410 comp rx=0 tx=0).stop 2024-08-20T03:04:16.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.564+0000 7f6fce6e4640 1 --2- 172.21.15.44:0/1886832492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc8109e80 0x7f6fc8110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:16.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.564+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1886832492 >> 172.21.15.44:0/1886832492 conn(0x7f6fc806dcc0 msgr2=0x7f6fc806e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.567+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1886832492 shutdown_connections 2024-08-20T03:04:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.570+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1886832492 wait complete. 2024-08-20T03:04:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.572+0000 7f6fce6e4640 1 Processor -- start 2024-08-20T03:04:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.572+0000 7f6fce6e4640 1 -- start start 2024-08-20T03:04:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.573+0000 7f6fce6e4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc8109e80 0x7f6fc81a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.573+0000 7f6fce6e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fc811bdd0 0x7f6fc81a84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.573+0000 7f6fce6e4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fc81ad890 0x7f6fc81add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.573+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc8109e80 0x7f6fc81a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.573+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc8109e80 0x7f6fc81a7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53198/0 (socket says 172.21.15.44:53198) 2024-08-20T03:04:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.573+0000 7f6fc7fff640 1 -- 172.21.15.44:0/1051545941 learned_addr learned my addr 172.21.15.44:0/1051545941 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.575+0000 7f6fc77fe640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fc811bdd0 0x7f6fc81a84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:16.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.575+0000 7f6fccc81640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fc81ad890 0x7f6fc81add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:16.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.575+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc8121870 con 0x7f6fc81ad890 2024-08-20T03:04:16.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.575+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81216f0 con 0x7f6fc811bdd0 2024-08-20T03:04:16.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.575+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc81219f0 con 0x7f6fc8109e80 2024-08-20T03:04:16.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.577+0000 7f6fc7fff640 1 -- 172.21.15.44:0/1051545941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fc811bdd0 msgr2=0x7f6fc81a84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:16.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.577+0000 7f6fc7fff640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fc811bdd0 0x7f6fc81a84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:16.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.577+0000 7f6fc7fff640 1 -- 172.21.15.44:0/1051545941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fc81ad890 msgr2=0x7f6fc81add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:16.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.577+0000 7f6fc7fff640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fc81ad890 0x7f6fc81add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:16.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.577+0000 7f6fc7fff640 1 -- 172.21.15.44:0/1051545941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc81ae420 con 0x7f6fc8109e80 2024-08-20T03:04:16.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.579+0000 7f6fc7fff640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc8109e80 0x7f6fc81a7fb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000cce0 tx=0x7f6fc000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:16.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.581+0000 7f6fc57fa640 1 -- 172.21.15.44:0/1051545941 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0004b20 con 0x7f6fc8109e80 2024-08-20T03:04:16.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.581+0000 7f6fc57fa640 1 -- 172.21.15.44:0/1051545941 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fc0004cc0 con 0x7f6fc8109e80 2024-08-20T03:04:16.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.581+0000 7f6fc57fa640 1 -- 172.21.15.44:0/1051545941 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc000f6b0 con 0x7f6fc8109e80 2024-08-20T03:04:16.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.581+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc81b4a10 con 0x7f6fc8109e80 2024-08-20T03:04:16.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.581+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc81b4db0 con 0x7f6fc8109e80 2024-08-20T03:04:16.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.585+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb4005290 con 0x7f6fc8109e80 2024-08-20T03:04:16.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.587+0000 7f6fc57fa640 1 -- 172.21.15.44:0/1051545941 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6fc000f850 con 0x7f6fc8109e80 2024-08-20T03:04:16.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.588+0000 7f6fc57fa640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6fb00b3ed0 0x7f6fb00b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:16.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.589+0000 7f6fc57fa640 1 -- 172.21.15.44:0/1051545941 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6fc00e4190 con 0x7f6fc8109e80 2024-08-20T03:04:16.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.590+0000 7f6fc77fe640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6fb00b3ed0 0x7f6fb00b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:16.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.591+0000 7f6fc77fe640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6fb00b3ed0 0x7f6fb00b6390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6fb802aa00 tx=0x7f6fb8000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:16.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:16.598+0000 7f6fc57fa640 1 -- 172.21.15.44:0/1051545941 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f6fc0014030 con 0x7f6fc8109e80 2024-08-20T03:04:17.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.003+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fb4005850 con 0x7f6fc8109e80 2024-08-20T03:04:17.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.004+0000 7f6fc57fa640 1 -- 172.21.15.44:0/1051545941 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11972 (secure 0 0 0) 0x7f6fc00ab510 con 0x7f6fc8109e80 2024-08-20T03:04:17.007 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:04:17.007 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":30,"btime":"2024-08-20T03:03:44:997954+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:03:44.335961+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24569,24583,24596,24589,24604]},"id":1}]} 2024-08-20T03:04:17.008 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 30 2024-08-20T03:04:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.006+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6fb00b3ed0 msgr2=0x7f6fb00b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.006+0000 7f6fce6e4640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6fb00b3ed0 0x7f6fb00b6390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6fb802aa00 tx=0x7f6fb8000bc0 comp rx=0 tx=0).stop 2024-08-20T03:04:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.006+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc8109e80 msgr2=0x7f6fc81a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.006+0000 7f6fce6e4640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc8109e80 0x7f6fc81a7fb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000cce0 tx=0x7f6fc000a620 comp rx=0 tx=0).stop 2024-08-20T03:04:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.006+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 shutdown_connections 2024-08-20T03:04:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.006+0000 7f6fce6e4640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6fb00b3ed0 0x7f6fb00b6390 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:17.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.006+0000 7f6fce6e4640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fc81ad890 0x7f6fc81add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:17.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.006+0000 7f6fce6e4640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fc811bdd0 0x7f6fc81a84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:17.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.006+0000 7f6fce6e4640 1 --2- 172.21.15.44:0/1051545941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fc8109e80 0x7f6fc81a7fb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:17.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.006+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 >> 172.21.15.44:0/1051545941 conn(0x7f6fc806dcc0 msgr2=0x7f6fc811c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:17.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.006+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 shutdown_connections 2024-08-20T03:04:17.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:17.006+0000 7f6fce6e4640 1 -- 172.21.15.44:0/1051545941 wait complete. 2024-08-20T03:04:17.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:17 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1051545941' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:04:17.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:17 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1051545941' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:04:17.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:17 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1051545941' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:04:17.476 INFO:tasks.workunit.client.0.smithi044.stdout: 58 1469384 587 1047073 581 789732 1614 2024-08-20T03:04:17.732 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-20T03:04:18.319 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:04:18.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:18 smithi165 ceph-mon[22357]: pgmap v374: 97 pgs: 97 active+clean; 177 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 848 op/s 2024-08-20T03:04:18.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:18 smithi099 ceph-mon[21984]: pgmap v374: 97 pgs: 97 active+clean; 177 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 848 op/s 2024-08-20T03:04:18.467 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:18 smithi044 ceph-mon[18790]: pgmap v374: 97 pgs: 97 active+clean; 177 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 848 op/s 2024-08-20T03:04:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.874+0000 7ff5cab31640 1 -- 172.21.15.44:0/2356864310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5c41025d0 msgr2=0x7ff5c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.874+0000 7ff5cab31640 1 --2- 172.21.15.44:0/2356864310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5c41025d0 0x7ff5c41029b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009900 tx=0x7ff5b802a410 comp rx=0 tx=0).stop 2024-08-20T03:04:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.874+0000 7ff5cab31640 1 -- 172.21.15.44:0/2356864310 shutdown_connections 2024-08-20T03:04:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.874+0000 7ff5cab31640 1 --2- 172.21.15.44:0/2356864310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c4107d50 0x7ff5c410e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.874+0000 7ff5cab31640 1 --2- 172.21.15.44:0/2356864310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c41073b0 0x7ff5c4107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.874+0000 7ff5cab31640 1 --2- 172.21.15.44:0/2356864310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5c41025d0 0x7ff5c41029b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.874+0000 7ff5cab31640 1 -- 172.21.15.44:0/2356864310 >> 172.21.15.44:0/2356864310 conn(0x7ff5c4076ba0 msgr2=0x7ff5c4076fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.874+0000 7ff5cab31640 1 -- 172.21.15.44:0/2356864310 shutdown_connections 2024-08-20T03:04:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.874+0000 7ff5cab31640 1 -- 172.21.15.44:0/2356864310 wait complete. 2024-08-20T03:04:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.875+0000 7ff5cab31640 1 Processor -- start 2024-08-20T03:04:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.875+0000 7ff5cab31640 1 -- start start 2024-08-20T03:04:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.875+0000 7ff5cab31640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c41025d0 0x7ff5c4199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.875+0000 7ff5cab31640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5c41073b0 0x7ff5c419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.875+0000 7ff5cab31640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c4107d50 0x7ff5c419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.875+0000 7ff5cab31640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c41118d0 con 0x7ff5c41073b0 2024-08-20T03:04:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.875+0000 7ff5cab31640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c4111750 con 0x7ff5c4107d50 2024-08-20T03:04:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.875+0000 7ff5cab31640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c4111a50 con 0x7ff5c41025d0 2024-08-20T03:04:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.876+0000 7ff5c88cd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c41025d0 0x7ff5c4199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.876+0000 7ff5c88cd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c41025d0 0x7ff5c4199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59920/0 (socket says 172.21.15.44:59920) 2024-08-20T03:04:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.876+0000 7ff5c88cd640 1 -- 172.21.15.44:0/3756056631 learned_addr learned my addr 172.21.15.44:0/3756056631 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.876+0000 7ff5c3fff640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5c41073b0 0x7ff5c419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.876+0000 7ff5c90ce640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c4107d50 0x7ff5c419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.876+0000 7ff5c88cd640 1 -- 172.21.15.44:0/3756056631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c4107d50 msgr2=0x7ff5c419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.876+0000 7ff5c88cd640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c4107d50 0x7ff5c419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.876+0000 7ff5c88cd640 1 -- 172.21.15.44:0/3756056631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5c41073b0 msgr2=0x7ff5c419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.876+0000 7ff5c88cd640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5c41073b0 0x7ff5c419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.876+0000 7ff5c88cd640 1 -- 172.21.15.44:0/3756056631 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c41a5140 con 0x7ff5c41025d0 2024-08-20T03:04:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.876+0000 7ff5c88cd640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c41025d0 0x7ff5c4199d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8000c00 tx=0x7ff5b8032a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.877+0000 7ff5c1ffb640 1 -- 172.21.15.44:0/3756056631 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b803f030 con 0x7ff5c41025d0 2024-08-20T03:04:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.877+0000 7ff5c1ffb640 1 -- 172.21.15.44:0/3756056631 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5b8032e00 con 0x7ff5c41025d0 2024-08-20T03:04:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.877+0000 7ff5cab31640 1 -- 172.21.15.44:0/3756056631 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c41a53d0 con 0x7ff5c41025d0 2024-08-20T03:04:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.877+0000 7ff5c1ffb640 1 -- 172.21.15.44:0/3756056631 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8035070 con 0x7ff5c41025d0 2024-08-20T03:04:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.877+0000 7ff5cab31640 1 -- 172.21.15.44:0/3756056631 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c41a58c0 con 0x7ff5c41025d0 2024-08-20T03:04:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.880+0000 7ff5c1ffb640 1 -- 172.21.15.44:0/3756056631 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff5b8042030 con 0x7ff5c41025d0 2024-08-20T03:04:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5cd9fa740 1 Processor -- start 2024-08-20T03:04:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5cd9fa740 1 -- start start 2024-08-20T03:04:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5c1ffb640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff5b40b3d40 0x7ff5b40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5cd9fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 0x55f399fe2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5cd9fa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f399ff3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5cd9fa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399ff38b0 0x55f399ff5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5cd9fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f399f58360 con 0x55f399fe2bf0 2024-08-20T03:04:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5cd9fa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f399f47860 con 0x55f399fe3510 2024-08-20T03:04:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5cd9fa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f399f4b8a0 con 0x55f399ff38b0 2024-08-20T03:04:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5c90ce640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff5b40b3d40 0x7ff5b40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f399ff3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f399ff3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51696/0 (socket says 172.21.15.44:51696) 2024-08-20T03:04:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.881+0000 7ff5c3fff640 1 -- 172.21.15.44:0/2361941113 learned_addr learned my addr 172.21.15.44:0/2361941113 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.882+0000 7ff5c90ce640 1 --2- 172.21.15.44:0/2361941113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399ff38b0 0x55f399ff5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.882+0000 7ff5c88cd640 1 --2- 172.21.15.44:0/2361941113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 0x55f399fe2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.882+0000 7ff5c90ce640 1 -- 172.21.15.44:0/2361941113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 msgr2=0x55f399ff3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.882+0000 7ff5c90ce640 1 --2- 172.21.15.44:0/2361941113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f399ff3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.882+0000 7ff5c90ce640 1 -- 172.21.15.44:0/2361941113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 msgr2=0x55f399fe2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.882+0000 7ff5c90ce640 1 --2- 172.21.15.44:0/2361941113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 0x55f399fe2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.882+0000 7ff5c90ce640 1 -- 172.21.15.44:0/2361941113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f399eeee40 con 0x55f399ff38b0 2024-08-20T03:04:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.882+0000 7ff5c88cd640 1 --2- 172.21.15.44:0/2361941113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 0x55f399fe2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:04:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.882+0000 7ff5c1ffb640 1 -- 172.21.15.44:0/3756056631 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff5b81042e0 con 0x7ff5c41025d0 2024-08-20T03:04:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.882+0000 7ff5c90ce640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff5b40b3d40 0x7ff5b40b6200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4102330 tx=0x7ff5bc011bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.882+0000 7ff5c90ce640 1 --2- 172.21.15.44:0/2361941113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399ff38b0 0x55f399ff5ca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc005fd0 tx=0x7ff5bc01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf76c6d2a0164394 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.883+0000 7ff5aeffd640 1 -- 172.21.15.44:0/2361941113 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc01a040 con 0x55f399ff38b0 2024-08-20T03:04:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.883+0000 7ff5aeffd640 1 -- 172.21.15.44:0/2361941113 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5bc025030 con 0x55f399ff38b0 2024-08-20T03:04:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.883+0000 7ff5aeffd640 1 -- 172.21.15.44:0/2361941113 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc005490 con 0x55f399ff38b0 2024-08-20T03:04:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.883+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/2361941113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399ff38b0 msgr2=0x55f399ff5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.883+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/2361941113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399ff38b0 0x55f399ff5ca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc005fd0 tx=0x7ff5bc01f000 comp rx=0 tx=0).stop 2024-08-20T03:04:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.884+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/2361941113 shutdown_connections 2024-08-20T03:04:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.884+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/2361941113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399ff38b0 0x55f399ff5ca0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.884+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/2361941113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f399ff3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.884+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/2361941113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 0x55f399fe2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.884+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/2361941113 >> 172.21.15.44:0/2361941113 conn(0x55f399fe0a20 msgr2=0x55f399fe1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.884+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/2361941113 shutdown_connections 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.884+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/2361941113 wait complete. 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.884+0000 7ff5cd9fa740 1 -- start start 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.884+0000 7ff5cd9fa740 1 Processor -- start 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5cd9fa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe2bf0 0x55f399ff31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5cd9fa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f399ff05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5cd9fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 0x55f399ff0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5cd9fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f399e80ab0 con 0x55f399ff38b0 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5cd9fa740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f399ece780 con 0x55f399fe3510 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5cd9fa740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f399ff6530 con 0x55f399fe2bf0 2024-08-20T03:04:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5cd9fa740 10 client.0 _ll_register_callbacks cb 0x55f399c56a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5c88cd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe2bf0 0x55f399ff31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5c88cd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe2bf0 0x55f399ff31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59942/0 (socket says 172.21.15.44:59942) 2024-08-20T03:04:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5c88cd640 1 -- 172.21.15.44:0/3493518928 learned_addr learned my addr 172.21.15.44:0/3493518928 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5c90ce640 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 0x55f399ff0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5c3fff640 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f399ff05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5c88cd640 1 -- 172.21.15.44:0/3493518928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 msgr2=0x55f399ff05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5c88cd640 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f399ff05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.885+0000 7ff5c88cd640 1 -- 172.21.15.44:0/3493518928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 msgr2=0x55f399ff0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.886+0000 7ff5c88cd640 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 0x55f399ff0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.886+0000 7ff5c88cd640 1 -- 172.21.15.44:0/3493518928 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f399caa790 con 0x55f399fe2bf0 2024-08-20T03:04:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.886+0000 7ff5c90ce640 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 0x55f399ff0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.886+0000 7ff5c88cd640 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe2bf0 0x55f399ff31a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80c8280 tx=0x7ff5b8047000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30ba0b49498e9507 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.886+0000 7ff5a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:04:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.886+0000 7ff5a3fff640 1 -- 172.21.15.44:0/3493518928 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8049070 con 0x55f399fe2bf0 2024-08-20T03:04:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.886+0000 7ff5a3fff640 1 -- 172.21.15.44:0/3493518928 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff5b8101d20 con 0x55f399fe2bf0 2024-08-20T03:04:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.886+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/3493518928 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f399ef6e00 con 0x55f399fe2bf0 2024-08-20T03:04:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.886+0000 7ff5a3fff640 1 -- 172.21.15.44:0/3493518928 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff5b80e5050 con 0x55f399fe2bf0 2024-08-20T03:04:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.887+0000 7ff5cd9fa740 10 client.25576 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.887+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/3493518928 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f399d23620 con 0x55f399fe2bf0 2024-08-20T03:04:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.887+0000 7ff5a3fff640 1 -- 172.21.15.44:0/3493518928 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f399ef6e00 con 0x55f399fe2bf0 2024-08-20T03:04:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.887+0000 7ff5a3fff640 1 -- 172.21.15.44:0/3493518928 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff5b80cb270 con 0x55f399fe2bf0 2024-08-20T03:04:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.887+0000 7ff5cd9fa740 10 client.25576 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.887+0000 7ff5cd9fa740 10 client.25576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.887+0000 7ff5cd9fa740 20 client.25576 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.887+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55f39a003280 0x55f39a0056e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.887+0000 7ff5cd9fa740 4 client.25576 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:04:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.887+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/3493518928 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f399f5a2b0 con 0x55f39a003280 2024-08-20T03:04:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.887+0000 7ff5c3fff640 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55f39a003280 0x55f39a0056e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.888+0000 7ff5a3fff640 10 client.25576 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.888+0000 7ff5c3fff640 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55f39a003280 0x55f39a0056e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50755af5cf79863d server_cookie=6e1d6651dfdd42e6 in_seq=0 out_seq=0 2024-08-20T03:04:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.890+0000 7ff5a3fff640 1 -- 172.21.15.44:0/3493518928 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f399f5a2b0 con 0x55f39a003280 2024-08-20T03:04:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.890+0000 7ff5a3fff640 10 client.25576 handle_command_reply: tid=0 2024-08-20T03:04:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.890+0000 7ff5cd9fa740 1 client.25576 shutdown 2024-08-20T03:04:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.890+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/3493518928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe2bf0 msgr2=0x55f399ff31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.890+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe2bf0 0x55f399ff31a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80c8280 tx=0x7ff5b8047000 comp rx=0 tx=0).stop 2024-08-20T03:04:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.890+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/3493518928 shutdown_connections 2024-08-20T03:04:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.890+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55f39a003280 0x55f39a0056e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.890+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 0x55f399ff0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.890+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f399ff05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.890+0000 7ff5a3fff640 0 client.25576 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.890+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/3493518928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe2bf0 0x55f399ff31a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.890+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/3493518928 >> 172.21.15.44:0/3493518928 conn(0x55f399fe0a20 msgr2=0x55f399fe65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.891+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/3493518928 shutdown_connections 2024-08-20T03:04:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.891+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/3493518928 wait complete. 2024-08-20T03:04:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.891+0000 7ff5cd9fa740 20 client.25576 trim_cache size 0 max 16384 2024-08-20T03:04:18.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.898+0000 7fb46b426640 1 -- 172.21.15.44:0/1927769197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb464101330 msgr2=0x7fb464101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.898+0000 7fb46b426640 1 --2- 172.21.15.44:0/1927769197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb464101330 0x7fb464101710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb460009860 tx=0x7fb46002a340 comp rx=0 tx=0).stop 2024-08-20T03:04:18.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.898+0000 7fb46b426640 1 -- 172.21.15.44:0/1927769197 shutdown_connections 2024-08-20T03:04:18.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.898+0000 7fb46b426640 1 --2- 172.21.15.44:0/1927769197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46410efe0 0x7fb464111450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.898+0000 7fb46b426640 1 --2- 172.21.15.44:0/1927769197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb464101d30 0x7fb46410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.898+0000 7fb46b426640 1 --2- 172.21.15.44:0/1927769197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb464101330 0x7fb464101710 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.898+0000 7fb46b426640 1 -- 172.21.15.44:0/1927769197 >> 172.21.15.44:0/1927769197 conn(0x7fb464076bd0 msgr2=0x7fb464076fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:18.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.899+0000 7fb46b426640 1 -- 172.21.15.44:0/1927769197 shutdown_connections 2024-08-20T03:04:18.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.899+0000 7fb46b426640 1 -- 172.21.15.44:0/1927769197 wait complete. 2024-08-20T03:04:18.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.899+0000 7fb46b426640 1 Processor -- start 2024-08-20T03:04:18.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.899+0000 7fb46b426640 1 -- start start 2024-08-20T03:04:18.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.899+0000 7fb46b426640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb464101330 0x7fb46419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb46b426640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb464101d30 0x7fb46419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb46b426640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46410efe0 0x7fb4641a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb46b426640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb464114c60 con 0x7fb464101330 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb46b426640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb464114ae0 con 0x7fb46410efe0 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb46b426640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb464114de0 con 0x7fb464101d30 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb4689c1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb464101d30 0x7fb46419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb4689c1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb464101d30 0x7fb46419b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59970/0 (socket says 172.21.15.44:59970) 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb4689c1640 1 -- 172.21.15.44:0/2981633604 learned_addr learned my addr 172.21.15.44:0/2981633604 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb4691c2640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb464101330 0x7fb46419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb4699c3640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46410efe0 0x7fb4641a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb4689c1640 1 -- 172.21.15.44:0/2981633604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46410efe0 msgr2=0x7fb4641a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb4689c1640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46410efe0 0x7fb4641a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb4689c1640 1 -- 172.21.15.44:0/2981633604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb464101330 msgr2=0x7fb46419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb4689c1640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb464101330 0x7fb46419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb4689c1640 1 -- 172.21.15.44:0/2981633604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4641a12c0 con 0x7fb464101d30 2024-08-20T03:04:18.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb4691c2640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb464101330 0x7fb46419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:18.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.900+0000 7fb4689c1640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb464101d30 0x7fb46419b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb45400e9c0 tx=0x7fb45400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:18.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.901+0000 7fb45a7fc640 1 -- 172.21.15.44:0/2981633604 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45400cde0 con 0x7fb464101d30 2024-08-20T03:04:18.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.901+0000 7fb45a7fc640 1 -- 172.21.15.44:0/2981633604 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4540035f0 con 0x7fb464101d30 2024-08-20T03:04:18.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.901+0000 7fb46b426640 1 -- 172.21.15.44:0/2981633604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4641a15b0 con 0x7fb464101d30 2024-08-20T03:04:18.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.901+0000 7fb45a7fc640 1 -- 172.21.15.44:0/2981633604 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454010690 con 0x7fb464101d30 2024-08-20T03:04:18.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.901+0000 7fb46b426640 1 -- 172.21.15.44:0/2981633604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46406ca70 con 0x7fb464101d30 2024-08-20T03:04:18.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.904+0000 7fb45a7fc640 1 -- 172.21.15.44:0/2981633604 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb454004000 con 0x7fb464101d30 2024-08-20T03:04:18.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.905+0000 7fb46d81f740 1 Processor -- start 2024-08-20T03:04:18.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.905+0000 7fb46d81f740 1 -- start start 2024-08-20T03:04:18.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.905+0000 7fb46d81f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 0x55d99f338470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.905+0000 7fb46d81f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f3389b0 0x55d99f344400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.905+0000 7fb46d81f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f346d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.905+0000 7fb46d81f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d99f2a6930 con 0x55d99f344940 2024-08-20T03:04:18.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.905+0000 7fb46d81f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d99f295e30 con 0x55d99f3389b0 2024-08-20T03:04:18.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.905+0000 7fb46d81f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d99f299e70 con 0x55d99f338090 2024-08-20T03:04:18.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.905+0000 7fb4699c3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 0x55d99f338470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.905+0000 7fb4699c3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 0x55d99f338470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59972/0 (socket says 172.21.15.44:59972) 2024-08-20T03:04:18.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.905+0000 7fb4699c3640 1 -- 172.21.15.44:0/1779229655 learned_addr learned my addr 172.21.15.44:0/1779229655 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:18.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.906+0000 7fb4691c2640 1 --2- 172.21.15.44:0/1779229655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f3389b0 0x55d99f344400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.906+0000 7fb4689c1640 1 --2- 172.21.15.44:0/1779229655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f346d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.906+0000 7fb45a7fc640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb4500b3dc0 0x7fb4500b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.906+0000 7fb4699c3640 1 -- 172.21.15.44:0/1779229655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f3389b0 msgr2=0x55d99f344400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.906+0000 7fb4699c3640 1 --2- 172.21.15.44:0/1779229655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f3389b0 0x55d99f344400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.906+0000 7fb4699c3640 1 -- 172.21.15.44:0/1779229655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 msgr2=0x55d99f346d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.906+0000 7fb4699c3640 1 --2- 172.21.15.44:0/1779229655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f346d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.906+0000 7fb4699c3640 1 -- 172.21.15.44:0/1779229655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99eff5650 con 0x55d99f338090 2024-08-20T03:04:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.906+0000 7fb45a7fc640 1 -- 172.21.15.44:0/2981633604 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb4540e4210 con 0x7fb464101d30 2024-08-20T03:04:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.907+0000 7fb4699c3640 1 --2- 172.21.15.44:0/1779229655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 0x55d99f338470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009f50 tx=0x7fb45c0228e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9767e9b6f0a49abf server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:18.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.907+0000 7fb4699c3640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb4500b3dc0 0x7fb4500b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.907+0000 7fb44f7fe640 1 -- 172.21.15.44:0/1779229655 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c004250 con 0x55d99f338090 2024-08-20T03:04:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.908+0000 7fb44f7fe640 1 -- 172.21.15.44:0/1779229655 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c020e60 con 0x55d99f338090 2024-08-20T03:04:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.908+0000 7fb44f7fe640 1 -- 172.21.15.44:0/1779229655 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c02ab40 con 0x55d99f338090 2024-08-20T03:04:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.909+0000 7fb46d81f740 1 -- 172.21.15.44:0/1779229655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 msgr2=0x55d99f338470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.909+0000 7fb4699c3640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb4500b3dc0 0x7fb4500b6280 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb45c031000 tx=0x7fb45c032000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:18.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.909+0000 7fb46d81f740 1 --2- 172.21.15.44:0/1779229655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 0x55d99f338470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009f50 tx=0x7fb45c0228e0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.909+0000 7ff5cd9fa740 1 Processor -- start 2024-08-20T03:04:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.909+0000 7fb46d81f740 1 -- 172.21.15.44:0/1779229655 shutdown_connections 2024-08-20T03:04:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.909+0000 7fb46d81f740 1 --2- 172.21.15.44:0/1779229655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f346d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.909+0000 7fb46d81f740 1 --2- 172.21.15.44:0/1779229655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f3389b0 0x55d99f344400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.909+0000 7ff5cd9fa740 1 -- start start 2024-08-20T03:04:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.909+0000 7fb46d81f740 1 --2- 172.21.15.44:0/1779229655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 0x55d99f338470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.909+0000 7fb46d81f740 1 -- 172.21.15.44:0/1779229655 >> 172.21.15.44:0/1779229655 conn(0x55d99f335ec0 msgr2=0x55d99f337060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.909+0000 7fb46d81f740 1 -- 172.21.15.44:0/1779229655 shutdown_connections 2024-08-20T03:04:18.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.909+0000 7fb46d81f740 1 -- 172.21.15.44:0/1779229655 wait complete. 2024-08-20T03:04:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.909+0000 7ff5cd9fa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe2bf0 0x55f399fff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.909+0000 7ff5cd9fa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe3510 0x55f399fefe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5cd9fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 0x55f399ff2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.910+0000 7fb46d81f740 1 -- start start 2024-08-20T03:04:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5c3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe3510 0x55f399fefe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5c90ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 0x55f399ff2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5c88cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe2bf0 0x55f399fff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5c88cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe2bf0 0x55f399fff6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51734/0 (socket says 172.21.15.44:51734) 2024-08-20T03:04:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5c3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe3510 0x55f399fefe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59980/0 (socket says 172.21.15.44:59980) 2024-08-20T03:04:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5c90ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 0x55f399ff2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52500/0 (socket says 172.21.15.44:52500) 2024-08-20T03:04:18.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.910+0000 7fb46d81f740 1 Processor -- start 2024-08-20T03:04:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5c88cd640 1 -- 172.21.15.44:0/1352557935 learned_addr learned my addr 172.21.15.44:0/1352557935 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1352557935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c41abef0 con 0x55f399ff38b0 2024-08-20T03:04:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1352557935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c41adaf0 con 0x55f399fe2bf0 2024-08-20T03:04:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1352557935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c41ac5c0 con 0x55f399fe3510 2024-08-20T03:04:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5c3fff640 1 -- 172.21.15.44:0/1352557935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe2bf0 msgr2=0x55f399fff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.910+0000 7fb46d81f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f338090 0x55d99f3443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5c3fff640 1 --2- 172.21.15.44:0/1352557935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe2bf0 0x55f399fff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5c3fff640 1 -- 172.21.15.44:0/1352557935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 msgr2=0x55f399ff2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5c3fff640 1 --2- 172.21.15.44:0/1352557935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 0x55f399ff2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.910+0000 7ff5c3fff640 1 -- 172.21.15.44:0/1352557935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f399ff3550 con 0x55f399fe3510 2024-08-20T03:04:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.910+0000 7fb46d81f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f3389b0 0x55d99f341810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.911+0000 7ff5c90ce640 1 --2- 172.21.15.44:0/1352557935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 0x55f399ff2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb4691c2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f338090 0x55d99f3443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb4691c2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f338090 0x55d99f3443c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51736/0 (socket says 172.21.15.44:51736) 2024-08-20T03:04:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb46d81f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f341fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb4689c1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f3389b0 0x55d99f341810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.911+0000 7ff5c3fff640 1 --2- 172.21.15.44:0/1352557935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe3510 0x55f399fefe50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f39a005ef0 tx=0x7ff5b0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b19cb69088ceb65 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb46d81f740 1 -- 172.21.15.44:0/888143978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d99f0f8700 con 0x55d99f344940 2024-08-20T03:04:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb4691c2640 1 -- 172.21.15.44:0/888143978 learned_addr learned my addr 172.21.15.44:0/888143978 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb46d81f740 1 -- 172.21.15.44:0/888143978 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d99f21c810 con 0x55d99f338090 2024-08-20T03:04:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb46d81f740 1 -- 172.21.15.44:0/888143978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d99f3475c0 con 0x55d99f3389b0 2024-08-20T03:04:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb4699c3640 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f341fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb46d81f740 10 client.0 _ll_register_callbacks cb 0x55d99f28f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.911+0000 7ff5ae7fc640 1 -- 172.21.15.44:0/1352557935 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0004310 con 0x55f399fe3510 2024-08-20T03:04:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.911+0000 7ff5ae7fc640 1 -- 172.21.15.44:0/1352557935 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5b00044b0 con 0x55f399fe3510 2024-08-20T03:04:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.911+0000 7ff5ae7fc640 1 -- 172.21.15.44:0/1352557935 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b001da10 con 0x55f399fe3510 2024-08-20T03:04:18.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb4699c3640 1 -- 172.21.15.44:0/888143978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f3389b0 msgr2=0x55d99f341810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb4699c3640 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f3389b0 0x55d99f341810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb4699c3640 1 -- 172.21.15.44:0/888143978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f338090 msgr2=0x55d99f3443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb4699c3640 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f338090 0x55d99f3443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb4699c3640 1 -- 172.21.15.44:0/888143978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99f3508f0 con 0x55d99f344940 2024-08-20T03:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.911+0000 7fb4689c1640 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f3389b0 0x55d99f341810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.912+0000 7fb4699c3640 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f341fa0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb45c02a480 tx=0x7fb45c003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd417d15382eb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.912+0000 7fb44cff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:04:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.912+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1352557935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe3510 msgr2=0x55f399fefe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.912+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/1352557935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe3510 0x55f399fefe50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f39a005ef0 tx=0x7ff5b0005f90 comp rx=0 tx=0).stop 2024-08-20T03:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.912+0000 7fb44cff9640 1 -- 172.21.15.44:0/888143978 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c044020 con 0x55d99f344940 2024-08-20T03:04:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.912+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1352557935 shutdown_connections 2024-08-20T03:04:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.912+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/1352557935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399ff38b0 0x55f399ff2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.912+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/1352557935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399fe3510 0x55f399fefe50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.912+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/1352557935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe2bf0 0x55f399fff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.925 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.912+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1352557935 >> 172.21.15.44:0/1352557935 conn(0x55f399fe0a20 msgr2=0x55f399ff1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.912+0000 7fb44cff9640 1 -- 172.21.15.44:0/888143978 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb45c020930 con 0x55d99f344940 2024-08-20T03:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.912+0000 7fb46d81f740 1 -- 172.21.15.44:0/888143978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d99f245150 con 0x55d99f344940 2024-08-20T03:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.912+0000 7fb44cff9640 1 -- 172.21.15.44:0/888143978 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb45c04f050 con 0x55d99f344940 2024-08-20T03:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.912+0000 7fb46d81f740 10 client.16131 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.912+0000 7fb46d81f740 1 -- 172.21.15.44:0/888143978 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d99f353230 con 0x55d99f344940 2024-08-20T03:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.912+0000 7fb44cff9640 1 -- 172.21.15.44:0/888143978 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb45c03b040 con 0x55d99f344940 2024-08-20T03:04:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.913+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1352557935 shutdown_connections 2024-08-20T03:04:18.926 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.913+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1352557935 wait complete. 2024-08-20T03:04:18.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.913+0000 7fb44cff9640 1 -- 172.21.15.44:0/888143978 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fb45c02c700 con 0x55d99f344940 2024-08-20T03:04:18.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.913+0000 7fb46d81f740 10 client.16131 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:18.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.913+0000 7fb46d81f740 10 client.16131 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:04:18.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.913+0000 7fb46d81f740 20 client.16131 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:18.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.913+0000 7ff5cd9fa740 1 -- start start 2024-08-20T03:04:18.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.913+0000 7fb46d81f740 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55d99f354690 0x55d99f356a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.913+0000 7fb46d81f740 4 client.16131 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:04:18.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.913+0000 7fb46d81f740 1 -- 172.21.15.44:0/888143978 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d99f245150 con 0x55d99f354690 2024-08-20T03:04:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.914+0000 7ff5cd9fa740 1 Processor -- start 2024-08-20T03:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.913+0000 7fb4691c2640 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55d99f354690 0x55d99f356a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.914+0000 7fb4691c2640 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55d99f354690 0x55d99f356a80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9ddb3ce4518af504 server_cookie=5367ae982a519c16 in_seq=0 out_seq=0 2024-08-20T03:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.914+0000 7fb44cff9640 10 client.16131 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:04:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.914+0000 7ff5cd9fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 0x55f399ffbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.914+0000 7ff5cd9fa740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f39a00a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.914+0000 7ff5cd9fa740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399ff38b0 0x55f39a00ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.914+0000 7ff5c88cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 0x55f399ffbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.914+0000 7ff5c88cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 0x55f399ffbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52506/0 (socket says 172.21.15.44:52506) 2024-08-20T03:04:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.914+0000 7ff5cd9fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c41ad360 con 0x55f399fe2bf0 2024-08-20T03:04:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.914+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1518422537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c41acc90 con 0x55f399fe3510 2024-08-20T03:04:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.914+0000 7ff5c88cd640 1 -- 172.21.15.44:0/1518422537 learned_addr learned my addr 172.21.15.44:0/1518422537 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.914+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1518422537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c41a9ed0 con 0x55f399ff38b0 2024-08-20T03:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.914+0000 7fb44cff9640 1 -- 172.21.15.44:0/888143978 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d99f245150 con 0x55d99f354690 2024-08-20T03:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.914+0000 7fb44cff9640 10 client.16131 handle_command_reply: tid=0 2024-08-20T03:04:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.914+0000 7ff5c90ce640 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399ff38b0 0x55f39a00ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.915+0000 7ff5c3fff640 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f39a00a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.914+0000 7fb46d81f740 1 client.16131 shutdown 2024-08-20T03:04:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.915+0000 7ff5cd9fa740 10 client.0 _ll_register_callbacks cb 0x7ff5b803b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.915+0000 7fb46d81f740 1 -- 172.21.15.44:0/888143978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 msgr2=0x55d99f341fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.915+0000 7fb46d81f740 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f341fa0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb45c02a480 tx=0x7fb45c003f00 comp rx=0 tx=0).stop 2024-08-20T03:04:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.915+0000 7fb46d81f740 1 -- 172.21.15.44:0/888143978 shutdown_connections 2024-08-20T03:04:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.915+0000 7fb46d81f740 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f341fa0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.915+0000 7fb46d81f740 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f3389b0 0x55d99f341810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.915+0000 7ff5c88cd640 1 -- 172.21.15.44:0/1518422537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399ff38b0 msgr2=0x55f39a00ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.915+0000 7ff5c88cd640 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399ff38b0 0x55f39a00ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.915+0000 7ff5c88cd640 1 -- 172.21.15.44:0/1518422537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 msgr2=0x55f39a00a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.915+0000 7ff5c88cd640 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f39a00a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.915+0000 7ff5c88cd640 1 -- 172.21.15.44:0/1518422537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f39a00b270 con 0x55f399fe2bf0 2024-08-20T03:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.915+0000 7ff5c90ce640 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399ff38b0 0x55f39a00ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.915+0000 7ff5c3fff640 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f39a00a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:18.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.915+0000 7fb46d81f740 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55d99f354690 0x55d99f356a80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.915+0000 7fb46d81f740 1 --2- 172.21.15.44:0/888143978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f338090 0x55d99f3443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.915+0000 7fb46d81f740 1 -- 172.21.15.44:0/888143978 >> 172.21.15.44:0/888143978 conn(0x55d99f335ec0 msgr2=0x55d99f33ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:18.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.915+0000 7fb44cff9640 0 client.16131 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:04:18.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.915+0000 7fb46d81f740 1 -- 172.21.15.44:0/888143978 shutdown_connections 2024-08-20T03:04:18.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.915+0000 7fb46d81f740 1 -- 172.21.15.44:0/888143978 wait complete. 2024-08-20T03:04:18.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.915+0000 7fb46d81f740 20 client.16131 trim_cache size 0 max 16384 2024-08-20T03:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.916+0000 7ff5c88cd640 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 0x55f399ffbc60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80c7090 tx=0x7ff5b8101fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8894b4c0937d18c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.916+0000 7ff5a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.916+0000 7ff5a3fff640 1 -- 172.21.15.44:0/1518422537 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80eb020 con 0x55f399fe2bf0 2024-08-20T03:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.916+0000 7ff5a3fff640 1 -- 172.21.15.44:0/1518422537 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff5b80c52f0 con 0x55f399fe2bf0 2024-08-20T03:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.916+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1518422537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5b00044b0 con 0x55f399fe2bf0 2024-08-20T03:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.916+0000 7ff5a3fff640 1 -- 172.21.15.44:0/1518422537 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff5b80af050 con 0x55f399fe2bf0 2024-08-20T03:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.916+0000 7ff5cd9fa740 10 client.16134 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.916+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1518422537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f39a06dd80 con 0x55f399fe2bf0 2024-08-20T03:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.916+0000 7ff5a3fff640 1 -- 172.21.15.44:0/1518422537 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5b00044b0 con 0x55f399fe2bf0 2024-08-20T03:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.917+0000 7ff5a3fff640 1 -- 172.21.15.44:0/1518422537 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff5b80a7020 con 0x55f399fe2bf0 2024-08-20T03:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.917+0000 7ff5cd9fa740 10 client.16134 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.917+0000 7ff5cd9fa740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.917+0000 7ff5cd9fa740 20 client.16134 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.917+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55f39a06e430 0x55f39a070850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.917+0000 7ff5cd9fa740 4 client.16134 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:04:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.917+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1518422537 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f399f55d50 con 0x55f39a06e430 2024-08-20T03:04:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.917+0000 7ff5c3fff640 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55f39a06e430 0x55f39a070850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.917+0000 7ff5c3fff640 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55f39a06e430 0x55f39a070850 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24def116be0da3d server_cookie=b1eb41dfd0b96e71 in_seq=0 out_seq=0 2024-08-20T03:04:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:18.917+0000 7ff5a3fff640 10 client.16134 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:18.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.932+0000 7fb46d81f740 1 Processor -- start 2024-08-20T03:04:18.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.932+0000 7fb46d81f740 1 -- start start 2024-08-20T03:04:18.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.932+0000 7fb46d81f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 0x55d99f337580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.932+0000 7fb46d81f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f3389b0 0x55d99f33c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.932+0000 7fb46d81f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f340f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.932+0000 7fb46d81f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb464070d90 con 0x55d99f344940 2024-08-20T03:04:18.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.932+0000 7fb46d81f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb464072990 con 0x55d99f3389b0 2024-08-20T03:04:18.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb46d81f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb464071460 con 0x55d99f338090 2024-08-20T03:04:18.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4691c2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 0x55d99f337580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4699c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f340f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4691c2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 0x55d99f337580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59994/0 (socket says 172.21.15.44:59994) 2024-08-20T03:04:18.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4699c3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f340f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52508/0 (socket says 172.21.15.44:52508) 2024-08-20T03:04:18.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4691c2640 1 -- 172.21.15.44:0/1111923922 learned_addr learned my addr 172.21.15.44:0/1111923922 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:18.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4689c1640 1 --2- 172.21.15.44:0/1111923922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f3389b0 0x55d99f33c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4689c1640 1 -- 172.21.15.44:0/1111923922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 msgr2=0x55d99f337580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4689c1640 1 --2- 172.21.15.44:0/1111923922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 0x55d99f337580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4689c1640 1 -- 172.21.15.44:0/1111923922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 msgr2=0x55d99f340f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4689c1640 1 --2- 172.21.15.44:0/1111923922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f340f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4689c1640 1 -- 172.21.15.44:0/1111923922 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99f3387e0 con 0x55d99f3389b0 2024-08-20T03:04:18.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4691c2640 1 --2- 172.21.15.44:0/1111923922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 0x55d99f337580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:18.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb4689c1640 1 --2- 172.21.15.44:0/1111923922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f3389b0 0x55d99f33c170 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4540e19d0 tx=0x7fb4540e4510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=288ae966cd979803 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:18.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.933+0000 7fb44effd640 1 -- 172.21.15.44:0/1111923922 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4540aa9a0 con 0x55d99f3389b0 2024-08-20T03:04:18.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.934+0000 7fb44effd640 1 -- 172.21.15.44:0/1111923922 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4540ab590 con 0x55d99f3389b0 2024-08-20T03:04:18.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.934+0000 7fb44effd640 1 -- 172.21.15.44:0/1111923922 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4540a7120 con 0x55d99f3389b0 2024-08-20T03:04:18.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.934+0000 7fb46d81f740 1 -- 172.21.15.44:0/1111923922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f3389b0 msgr2=0x55d99f33c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.934+0000 7fb46d81f740 1 --2- 172.21.15.44:0/1111923922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f3389b0 0x55d99f33c170 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb4540e19d0 tx=0x7fb4540e4510 comp rx=0 tx=0).stop 2024-08-20T03:04:18.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.934+0000 7fb46d81f740 1 -- 172.21.15.44:0/1111923922 shutdown_connections 2024-08-20T03:04:18.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.934+0000 7fb46d81f740 1 --2- 172.21.15.44:0/1111923922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f344940 0x55d99f340f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.934+0000 7fb46d81f740 1 --2- 172.21.15.44:0/1111923922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f3389b0 0x55d99f33c170 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.934+0000 7fb46d81f740 1 --2- 172.21.15.44:0/1111923922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f338090 0x55d99f337580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.934+0000 7fb46d81f740 1 -- 172.21.15.44:0/1111923922 >> 172.21.15.44:0/1111923922 conn(0x55d99f335ec0 msgr2=0x55d99f32c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:18.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.934+0000 7fb46d81f740 1 -- 172.21.15.44:0/1111923922 shutdown_connections 2024-08-20T03:04:18.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.934+0000 7fb46d81f740 1 -- 172.21.15.44:0/1111923922 wait complete. 2024-08-20T03:04:18.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.935+0000 7fb46d81f740 1 -- start start 2024-08-20T03:04:18.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.935+0000 7fb46d81f740 1 Processor -- start 2024-08-20T03:04:18.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.935+0000 7fb46d81f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f338090 0x55d99f347920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.935+0000 7fb46d81f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f3389b0 0x55d99f347e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb46d81f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f344940 0x55d99f3483e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb46d81f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb464072200 con 0x55d99f3389b0 2024-08-20T03:04:18.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb46d81f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb464071b30 con 0x55d99f338090 2024-08-20T03:04:18.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb46d81f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb46406ed90 con 0x55d99f344940 2024-08-20T03:04:18.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb46d81f740 10 client.0 _ll_register_callbacks cb 0x7fb45c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:18.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb4689c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f3389b0 0x55d99f347e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb4689c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f3389b0 0x55d99f347e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52524/0 (socket says 172.21.15.44:52524) 2024-08-20T03:04:18.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb4689c1640 1 -- 172.21.15.44:0/3877374210 learned_addr learned my addr 172.21.15.44:0/3877374210 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb4699c3640 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f344940 0x55d99f3483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb4691c2640 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f338090 0x55d99f347920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb4699c3640 1 -- 172.21.15.44:0/3877374210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f338090 msgr2=0x55d99f347920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb4699c3640 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f338090 0x55d99f347920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb4699c3640 1 -- 172.21.15.44:0/3877374210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f3389b0 msgr2=0x55d99f347e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb4699c3640 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f3389b0 0x55d99f347e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb4699c3640 1 -- 172.21.15.44:0/3877374210 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99f3bd590 con 0x55d99f344940 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb4699c3640 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f344940 0x55d99f3483e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb45c02cb50 tx=0x7fb45c009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df08a7bbe66e2cd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.936+0000 7fb44cff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb44cff9640 1 -- 172.21.15.44:0/3877374210 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c058060 con 0x55d99f344940 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb44cff9640 1 -- 172.21.15.44:0/3877374210 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb45c02a4b0 con 0x55d99f344940 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb46d81f740 1 -- 172.21.15.44:0/3877374210 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4540ab590 con 0x55d99f344940 2024-08-20T03:04:18.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb44cff9640 1 -- 172.21.15.44:0/3877374210 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb45c03a030 con 0x55d99f344940 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb46d81f740 10 client.25600 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb46d81f740 1 -- 172.21.15.44:0/3877374210 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d99f3be890 con 0x55d99f344940 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb44cff9640 1 -- 172.21.15.44:0/3877374210 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4540ab590 con 0x55d99f344940 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb44cff9640 1 -- 172.21.15.44:0/3877374210 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fb45c05b030 con 0x55d99f344940 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb46d81f740 10 client.25600 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb46d81f740 10 client.25600 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb46d81f740 20 client.25600 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb46d81f740 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55d99f3bef80 0x55d99f3c13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb46d81f740 4 client.25600 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.937+0000 7fb46d81f740 1 -- 172.21.15.44:0/3877374210 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d99f2a8880 con 0x55d99f3bef80 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb4691c2640 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55d99f3bef80 0x55d99f3c13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb44cff9640 10 client.25600 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:04:18.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb4691c2640 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55d99f3bef80 0x55d99f3c13a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a34f2eceb9d9e7e5 server_cookie=ab4dea54e371f87a in_seq=0 out_seq=0 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb44cff9640 1 -- 172.21.15.44:0/3877374210 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d99f2a8880 con 0x55d99f3bef80 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb44cff9640 10 client.25600 handle_command_reply: tid=0 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb46d81f740 1 client.25600 shutdown 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb46d81f740 1 -- 172.21.15.44:0/3877374210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f344940 msgr2=0x55d99f3483e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb46d81f740 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f344940 0x55d99f3483e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb45c02cb50 tx=0x7fb45c009d20 comp rx=0 tx=0).stop 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb46d81f740 1 -- 172.21.15.44:0/3877374210 shutdown_connections 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb46d81f740 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d99f344940 0x55d99f3483e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb46d81f740 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d99f3389b0 0x55d99f347e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb46d81f740 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55d99f3bef80 0x55d99f3c13a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb46d81f740 1 --2- 172.21.15.44:0/3877374210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d99f338090 0x55d99f347920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.938+0000 7fb44cff9640 0 client.25600 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.939+0000 7fb46d81f740 1 -- 172.21.15.44:0/3877374210 >> 172.21.15.44:0/3877374210 conn(0x55d99f335ec0 msgr2=0x55d99f342e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:18.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.939+0000 7fb46d81f740 1 -- 172.21.15.44:0/3877374210 shutdown_connections 2024-08-20T03:04:18.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.939+0000 7fb46d81f740 1 -- 172.21.15.44:0/3877374210 wait complete. 2024-08-20T03:04:18.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.939+0000 7fb46d81f740 20 client.25600 trim_cache size 0 max 16384 2024-08-20T03:04:18.950 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:04:18.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.939+0000 7fb46b426640 1 -- 172.21.15.44:0/2981633604 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb4500b3dc0 msgr2=0x7fb4500b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.939+0000 7fb46b426640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb4500b3dc0 0x7fb4500b6280 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb45c031000 tx=0x7fb45c032000 comp rx=0 tx=0).stop 2024-08-20T03:04:18.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.939+0000 7fb46b426640 1 -- 172.21.15.44:0/2981633604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb464101d30 msgr2=0x7fb46419b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:18.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.939+0000 7fb46b426640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb464101d30 0x7fb46419b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb45400e9c0 tx=0x7fb45400ee80 comp rx=0 tx=0).stop 2024-08-20T03:04:18.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.940+0000 7fb46b426640 1 -- 172.21.15.44:0/2981633604 shutdown_connections 2024-08-20T03:04:18.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.940+0000 7fb46b426640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb4500b3dc0 0x7fb4500b6280 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.940+0000 7fb46b426640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46410efe0 0x7fb4641a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.940+0000 7fb46b426640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb464101d30 0x7fb46419b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.940+0000 7fb46b426640 1 --2- 172.21.15.44:0/2981633604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb464101330 0x7fb46419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.940+0000 7fb46b426640 1 -- 172.21.15.44:0/2981633604 >> 172.21.15.44:0/2981633604 conn(0x7fb464076bd0 msgr2=0x7fb46410f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.940+0000 7fb46b426640 1 -- 172.21.15.44:0/2981633604 shutdown_connections 2024-08-20T03:04:18.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:18.940+0000 7fb46b426640 1 -- 172.21.15.44:0/2981633604 wait complete. 2024-08-20T03:04:19.788 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-20T03:04:20.198 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:04:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5a3fff640 1 -- 172.21.15.44:0/1518422537 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f399f55d50 con 0x55f39a06e430 2024-08-20T03:04:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5a3fff640 10 client.16134 handle_command_reply: tid=0 2024-08-20T03:04:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5cd9fa740 1 client.16134 shutdown 2024-08-20T03:04:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1518422537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 msgr2=0x55f399ffbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 0x55f399ffbc60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80c7090 tx=0x7ff5b8101fd0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1518422537 shutdown_connections 2024-08-20T03:04:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55f39a06e430 0x55f39a070850 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f399ff38b0 0x55f39a00ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f399fe3510 0x55f39a00a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5cd9fa740 1 --2- 172.21.15.44:0/1518422537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f399fe2bf0 0x55f399ffbc60 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1518422537 >> 172.21.15.44:0/1518422537 conn(0x55f399fe0a20 msgr2=0x55f399fdfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5a3fff640 0 client.16134 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1518422537 shutdown_connections 2024-08-20T03:04:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5cd9fa740 1 -- 172.21.15.44:0/1518422537 wait complete. 2024-08-20T03:04:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.274+0000 7ff5cd9fa740 20 client.16134 trim_cache size 0 max 16384 2024-08-20T03:04:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:04:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:04:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:04:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:04:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.275+0000 7ff5cab31640 1 -- 172.21.15.44:0/3756056631 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff5b40b3d40 msgr2=0x7ff5b40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.275+0000 7ff5cab31640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff5b40b3d40 0x7ff5b40b6200 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4102330 tx=0x7ff5bc011bc0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.275+0000 7ff5cab31640 1 -- 172.21.15.44:0/3756056631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c41025d0 msgr2=0x7ff5c4199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.275+0000 7ff5cab31640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c41025d0 0x7ff5c4199d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8000c00 tx=0x7ff5b8032a00 comp rx=0 tx=0).stop 2024-08-20T03:04:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.275+0000 7ff5cab31640 1 -- 172.21.15.44:0/3756056631 shutdown_connections 2024-08-20T03:04:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.275+0000 7ff5cab31640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff5b40b3d40 0x7ff5b40b6200 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.275+0000 7ff5cab31640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c4107d50 0x7ff5c419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.275+0000 7ff5cab31640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff5c41073b0 0x7ff5c419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.275+0000 7ff5cab31640 1 --2- 172.21.15.44:0/3756056631 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5c41025d0 0x7ff5c4199d40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.275+0000 7ff5cab31640 1 -- 172.21.15.44:0/3756056631 >> 172.21.15.44:0/3756056631 conn(0x7ff5c4076ba0 msgr2=0x7ff5c4100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.275+0000 7ff5cab31640 1 -- 172.21.15.44:0/3756056631 shutdown_connections 2024-08-20T03:04:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:20.275+0000 7ff5cab31640 1 -- 172.21.15.44:0/3756056631 wait complete. 2024-08-20T03:04:20.328 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:20 smithi044 ceph-mon[18790]: pgmap v375: 97 pgs: 97 active+clean; 181 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 704 op/s 2024-08-20T03:04:20.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:20 smithi165 ceph-mon[22357]: pgmap v375: 97 pgs: 97 active+clean; 181 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 704 op/s 2024-08-20T03:04:20.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:20 smithi099 ceph-mon[21984]: pgmap v375: 97 pgs: 97 active+clean; 181 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 704 op/s 2024-08-20T03:04:20.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.705+0000 7ff7515ac640 1 -- 172.21.15.44:0/2536813672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff74c107d20 msgr2=0x7ff74c10e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:20.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.705+0000 7ff7515ac640 1 --2- 172.21.15.44:0/2536813672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff74c107d20 0x7ff74c10e520 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff744009ca0 tx=0x7ff74402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.706+0000 7ff7515ac640 1 -- 172.21.15.44:0/2536813672 shutdown_connections 2024-08-20T03:04:20.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.706+0000 7ff7515ac640 1 --2- 172.21.15.44:0/2536813672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff74c107d20 0x7ff74c10e520 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.706+0000 7ff7515ac640 1 --2- 172.21.15.44:0/2536813672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff74c1073a0 0x7ff74c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.706+0000 7ff7515ac640 1 --2- 172.21.15.44:0/2536813672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74c101990 0x7ff74c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.706+0000 7ff7515ac640 1 -- 172.21.15.44:0/2536813672 >> 172.21.15.44:0/2536813672 conn(0x7ff74c0fd400 msgr2=0x7ff74c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.706+0000 7ff7515ac640 1 -- 172.21.15.44:0/2536813672 shutdown_connections 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.706+0000 7ff7515ac640 1 -- 172.21.15.44:0/2536813672 wait complete. 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.706+0000 7ff7515ac640 1 Processor -- start 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.706+0000 7ff7515ac640 1 -- start start 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff7515ac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74c101990 0x7ff74c19b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff7515ac640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff74c1073a0 0x7ff74c19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff7515ac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff74c107d20 0x7ff74c1a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff7515ac640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1119a0 con 0x7ff74c107d20 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff7515ac640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c111820 con 0x7ff74c101990 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff7515ac640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c111b20 con 0x7ff74c1073a0 2024-08-20T03:04:20.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff74b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff74c107d20 0x7ff74c1a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff74b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff74c107d20 0x7ff74c1a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52552/0 (socket says 172.21.15.44:52552) 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff74b7fe640 1 -- 172.21.15.44:0/1960784588 learned_addr learned my addr 172.21.15.44:0/1960784588 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff74c1073a0 0x7ff74c19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff74affd640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74c101990 0x7ff74c19b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff74b7fe640 1 -- 172.21.15.44:0/1960784588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff74c1073a0 msgr2=0x7ff74c19b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff74b7fe640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff74c1073a0 0x7ff74c19b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff74b7fe640 1 -- 172.21.15.44:0/1960784588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74c101990 msgr2=0x7ff74c19b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff74b7fe640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74c101990 0x7ff74c19b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff74b7fe640 1 -- 172.21.15.44:0/1960784588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c1a11f0 con 0x7ff74c107d20 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff74c1073a0 0x7ff74c19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff74b7fe640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff74c107d20 0x7ff74c1a0a20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff744005cc0 tx=0x7ff744031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff733fff640 1 -- 172.21.15.44:0/1960784588 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744003b70 con 0x7ff74c107d20 2024-08-20T03:04:20.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff7515ac640 1 -- 172.21.15.44:0/1960784588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74c1a1480 con 0x7ff74c107d20 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.707+0000 7ff733fff640 1 -- 172.21.15.44:0/1960784588 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7440318f0 con 0x7ff74c107d20 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.708+0000 7ff733fff640 1 -- 172.21.15.44:0/1960784588 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74402f950 con 0x7ff74c107d20 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.708+0000 7ff7515ac640 1 -- 172.21.15.44:0/1960784588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74c1a7ba0 con 0x7ff74c107d20 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.710+0000 7ff733fff640 1 -- 172.21.15.44:0/1960784588 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff74402fb10 con 0x7ff74c107d20 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.710+0000 7ff7539a5740 1 Processor -- start 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff7539a5740 1 -- start start 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff7539a5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720ca3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff7539a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca41a0 0x559720cafbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff7539a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720cb0130 0x559720cb2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff7539a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559720c1a930 con 0x559720cb0130 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff7539a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559720c09e30 con 0x559720ca41a0 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff7539a5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559720c0de70 con 0x559720ca3880 2024-08-20T03:04:20.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff733fff640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff7340b3d40 0x7ff7340b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff74affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca41a0 0x559720cafbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff74a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720cb0130 0x559720cb2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff74b7fe640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff7340b3d40 0x7ff7340b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff74affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca41a0 0x559720cafbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51784/0 (socket says 172.21.15.44:51784) 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff74affd640 1 -- 172.21.15.44:0/1306832805 learned_addr learned my addr 172.21.15.44:0/1306832805 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/1306832805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720ca3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff733fff640 1 -- 172.21.15.44:0/1960784588 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff744105220 con 0x7ff74c107d20 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff74affd640 1 -- 172.21.15.44:0/1306832805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 msgr2=0x559720ca3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff74affd640 1 --2- 172.21.15.44:0/1306832805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720ca3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff74affd640 1 -- 172.21.15.44:0/1306832805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720cb0130 msgr2=0x559720cb2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff74affd640 1 --2- 172.21.15.44:0/1306832805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720cb0130 0x559720cb2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff74affd640 1 -- 172.21.15.44:0/1306832805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559720969650 con 0x559720ca41a0 2024-08-20T03:04:20.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.711+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/1306832805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720cb0130 0x559720cb2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.712+0000 7ff74b7fe640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff7340b3d40 0x7ff7340b6200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff7440cbc50 tx=0x7ff744098400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.712+0000 7ff74affd640 1 --2- 172.21.15.44:0/1306832805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca41a0 0x559720cafbf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009cf0 tx=0x7ff73c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d187c8a90cf1e344 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.712+0000 7ff7317fa640 1 -- 172.21.15.44:0/1306832805 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c004230 con 0x559720ca41a0 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.712+0000 7ff7317fa640 1 -- 172.21.15.44:0/1306832805 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff73c0043d0 con 0x559720ca41a0 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.712+0000 7ff7317fa640 1 -- 172.21.15.44:0/1306832805 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0197c0 con 0x559720ca41a0 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.712+0000 7ff7539a5740 1 -- 172.21.15.44:0/1306832805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca41a0 msgr2=0x559720cafbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.712+0000 7ff7539a5740 1 --2- 172.21.15.44:0/1306832805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca41a0 0x559720cafbf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009cf0 tx=0x7ff73c011810 comp rx=0 tx=0).stop 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 -- 172.21.15.44:0/1306832805 shutdown_connections 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 --2- 172.21.15.44:0/1306832805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720cb0130 0x559720cb2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 --2- 172.21.15.44:0/1306832805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca41a0 0x559720cafbf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 --2- 172.21.15.44:0/1306832805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720ca3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 -- 172.21.15.44:0/1306832805 >> 172.21.15.44:0/1306832805 conn(0x559720ca16b0 msgr2=0x559720ca2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 -- 172.21.15.44:0/1306832805 shutdown_connections 2024-08-20T03:04:20.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 -- 172.21.15.44:0/1306832805 wait complete. 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 -- start start 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 Processor -- start 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca3880 0x559720cad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 0x559720cad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720cb0130 0x559720cadb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559720a6c700 con 0x559720ca41a0 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559720b90810 con 0x559720ca3880 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559720cb2de0 con 0x559720cb0130 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.713+0000 7ff7539a5740 10 client.0 _ll_register_callbacks cb 0x559720c03700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff74affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 0x559720cad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff74affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 0x559720cad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52562/0 (socket says 172.21.15.44:52562) 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff74affd640 1 -- 172.21.15.44:0/3909933614 learned_addr learned my addr 172.21.15.44:0/3909933614 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:20.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720cb0130 0x559720cadb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca3880 0x559720cad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff74affd640 1 -- 172.21.15.44:0/3909933614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720cb0130 msgr2=0x559720cadb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:20.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff74affd640 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720cb0130 0x559720cadb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff74affd640 1 -- 172.21.15.44:0/3909933614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca3880 msgr2=0x559720cad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:20.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff74affd640 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca3880 0x559720cad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff74affd640 1 -- 172.21.15.44:0/3909933614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559720cbc160 con 0x559720ca41a0 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca3880 0x559720cad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff74affd640 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 0x559720cad5d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009560 tx=0x7ff73c004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cede054d68f195e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff72eff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff72eff5640 1 -- 172.21.15.44:0/3909933614 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c027070 con 0x559720ca41a0 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff7539a5740 1 -- 172.21.15.44:0/3909933614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559720bb9150 con 0x559720ca41a0 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff72eff5640 1 -- 172.21.15.44:0/3909933614 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff73c005240 con 0x559720ca41a0 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff72eff5640 1 -- 172.21.15.44:0/3909933614 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff73c033050 con 0x559720ca41a0 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff7539a5740 10 client.16152 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.714+0000 7ff7539a5740 1 -- 172.21.15.44:0/3909933614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559720cbeb60 con 0x559720ca41a0 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.715+0000 7ff72eff5640 1 -- 172.21.15.44:0/3909933614 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559720bb9150 con 0x559720ca41a0 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.715+0000 7ff72eff5640 1 -- 172.21.15.44:0/3909933614 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff73c004b70 con 0x559720ca41a0 2024-08-20T03:04:20.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.715+0000 7ff7539a5740 10 client.16152 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.715+0000 7ff7539a5740 10 client.16152 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.715+0000 7ff7539a5740 20 client.16152 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.715+0000 7ff7539a5740 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x559720cbff70 0x559720cc2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.715+0000 7ff7539a5740 4 client.16152 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.715+0000 7ff7539a5740 1 -- 172.21.15.44:0/3909933614 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559720c1c880 con 0x559720cbff70 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.715+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x559720cbff70 0x559720cc2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.715+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x559720cbff70 0x559720cc2360 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=418509d4047d9d16 server_cookie=834ff0ab5d4564aa in_seq=0 out_seq=0 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.715+0000 7ff72eff5640 10 client.16152 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.716+0000 7ff72eff5640 1 -- 172.21.15.44:0/3909933614 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559720c1c880 con 0x559720cbff70 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.716+0000 7ff72eff5640 10 client.16152 handle_command_reply: tid=0 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.716+0000 7ff7539a5740 1 client.16152 shutdown 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.717+0000 7ff7539a5740 1 -- 172.21.15.44:0/3909933614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 msgr2=0x559720cad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:20.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.717+0000 7ff7539a5740 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 0x559720cad5d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009560 tx=0x7ff73c004220 comp rx=0 tx=0).stop 2024-08-20T03:04:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.717+0000 7ff7539a5740 1 -- 172.21.15.44:0/3909933614 shutdown_connections 2024-08-20T03:04:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.717+0000 7ff7539a5740 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x559720cbff70 0x559720cc2360 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.717+0000 7ff7539a5740 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720cb0130 0x559720cadb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.717+0000 7ff7539a5740 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 0x559720cad5d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.717+0000 7ff7539a5740 1 --2- 172.21.15.44:0/3909933614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720ca3880 0x559720cad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.717+0000 7ff7539a5740 1 -- 172.21.15.44:0/3909933614 >> 172.21.15.44:0/3909933614 conn(0x559720ca16b0 msgr2=0x559720ca7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.717+0000 7ff72eff5640 0 client.16152 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:04:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.717+0000 7ff7539a5740 1 -- 172.21.15.44:0/3909933614 shutdown_connections 2024-08-20T03:04:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.717+0000 7ff7539a5740 1 -- 172.21.15.44:0/3909933614 wait complete. 2024-08-20T03:04:20.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.717+0000 7ff7539a5740 20 client.16152 trim_cache size 0 max 16384 2024-08-20T03:04:20.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff7539a5740 1 Processor -- start 2024-08-20T03:04:20.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff7539a5740 1 -- start start 2024-08-20T03:04:20.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff7539a5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720ca7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff7539a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 0x559720caedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff7539a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720cb0130 0x559720cc7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff7539a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1b0460 con 0x559720ca41a0 2024-08-20T03:04:20.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff7539a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1b2060 con 0x559720cb0130 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff7539a5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1b0b30 con 0x559720ca3880 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff74affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 0x559720caedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff74affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 0x559720caedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52570/0 (socket says 172.21.15.44:52570) 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff74affd640 1 -- 172.21.15.44:0/3557199371 learned_addr learned my addr 172.21.15.44:0/3557199371 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/3557199371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720ca7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/3557199371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720cb0130 0x559720cc7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff74affd640 1 -- 172.21.15.44:0/3557199371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 msgr2=0x559720ca7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff74affd640 1 --2- 172.21.15.44:0/3557199371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720ca7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff74affd640 1 -- 172.21.15.44:0/3557199371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720cb0130 msgr2=0x559720cc7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff74affd640 1 --2- 172.21.15.44:0/3557199371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720cb0130 0x559720cc7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff74affd640 1 -- 172.21.15.44:0/3557199371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559720ca2130 con 0x559720ca41a0 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.733+0000 7ff74affd640 1 --2- 172.21.15.44:0/3557199371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 0x559720caedd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0041f0 tx=0x7ff73c0096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23bcb887e72628f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:20.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.734+0000 7ff730ff9640 1 -- 172.21.15.44:0/3557199371 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c027040 con 0x559720ca41a0 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.734+0000 7ff730ff9640 1 -- 172.21.15.44:0/3557199371 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff73c004290 con 0x559720ca41a0 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.734+0000 7ff730ff9640 1 -- 172.21.15.44:0/3557199371 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c019950 con 0x559720ca41a0 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.735+0000 7ff7539a5740 1 -- 172.21.15.44:0/3557199371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 msgr2=0x559720caedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.735+0000 7ff7539a5740 1 --2- 172.21.15.44:0/3557199371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 0x559720caedd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0041f0 tx=0x7ff73c0096f0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.735+0000 7ff7539a5740 1 -- 172.21.15.44:0/3557199371 shutdown_connections 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.735+0000 7ff7539a5740 1 --2- 172.21.15.44:0/3557199371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720cb0130 0x559720cc7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.735+0000 7ff7539a5740 1 --2- 172.21.15.44:0/3557199371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720ca41a0 0x559720caedd0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.735+0000 7ff7539a5740 1 --2- 172.21.15.44:0/3557199371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720ca7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.735+0000 7ff7539a5740 1 -- 172.21.15.44:0/3557199371 >> 172.21.15.44:0/3557199371 conn(0x559720ca16b0 msgr2=0x559720cae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.736+0000 7ff7539a5740 1 -- 172.21.15.44:0/3557199371 shutdown_connections 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.736+0000 7ff7539a5740 1 -- 172.21.15.44:0/3557199371 wait complete. 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.736+0000 7ff7539a5740 1 -- start start 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.736+0000 7ff7539a5740 1 Processor -- start 2024-08-20T03:04:20.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.737+0000 7ff7539a5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720cb38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.737+0000 7ff7539a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720cb0130 0x559720cb3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.737+0000 7ff7539a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720cb7e60 0x559720cb82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.737+0000 7ff7539a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1b18d0 con 0x559720cb0130 2024-08-20T03:04:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.737+0000 7ff7539a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1b1200 con 0x559720cb7e60 2024-08-20T03:04:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.737+0000 7ff7539a5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1ae440 con 0x559720ca3880 2024-08-20T03:04:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.737+0000 7ff7539a5740 10 client.0 _ll_register_callbacks cb 0x7ff740009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.737+0000 7ff74a7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720cb38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.737+0000 7ff74a7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720cb38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:60074/0 (socket says 172.21.15.44:60074) 2024-08-20T03:04:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.737+0000 7ff74a7fc640 1 -- 172.21.15.44:0/2749576471 learned_addr learned my addr 172.21.15.44:0/2749576471 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.737+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720cb7e60 0x559720cb82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.737+0000 7ff74affd640 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720cb0130 0x559720cb3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.738+0000 7ff74a7fc640 1 -- 172.21.15.44:0/2749576471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720cb7e60 msgr2=0x559720cb82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.738+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720cb7e60 0x559720cb82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.738+0000 7ff74a7fc640 1 -- 172.21.15.44:0/2749576471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720cb0130 msgr2=0x559720cb3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.738+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720cb0130 0x559720cb3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.738+0000 7ff74a7fc640 1 -- 172.21.15.44:0/2749576471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559720cb8950 con 0x559720ca3880 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.738+0000 7ff74affd640 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720cb0130 0x559720cb3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.739+0000 7ff74a7fc640 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720cb38e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff740011710 tx=0x7ff73c003900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6768a204ef2916c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.739+0000 7ff72eff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.739+0000 7ff72eff5640 1 -- 172.21.15.44:0/2749576471 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74001a040 con 0x559720ca3880 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.739+0000 7ff72eff5640 1 -- 172.21.15.44:0/2749576471 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff740011270 con 0x559720ca3880 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.739+0000 7ff7539a5740 1 -- 172.21.15.44:0/2749576471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff73c004290 con 0x559720ca3880 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.740+0000 7ff72eff5640 1 -- 172.21.15.44:0/2749576471 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff7400118b0 con 0x559720ca3880 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.740+0000 7ff7539a5740 10 client.25618 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:20.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.740+0000 7ff7539a5740 1 -- 172.21.15.44:0/2749576471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559720d23910 con 0x559720ca3880 2024-08-20T03:04:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.740+0000 7ff72eff5640 1 -- 172.21.15.44:0/2749576471 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff73c004290 con 0x559720ca3880 2024-08-20T03:04:20.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.740+0000 7ff72eff5640 1 -- 172.21.15.44:0/2749576471 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff740020040 con 0x559720ca3880 2024-08-20T03:04:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.741+0000 7ff7539a5740 10 client.25618 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.741+0000 7ff7539a5740 10 client.25618 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:04:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.741+0000 7ff7539a5740 20 client.25618 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.741+0000 7ff7539a5740 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x559720d24070 0x559720d26490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.741+0000 7ff7539a5740 4 client.25618 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:04:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.741+0000 7ff7539a5740 1 -- 172.21.15.44:0/2749576471 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559720c18320 con 0x559720d24070 2024-08-20T03:04:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.741+0000 7ff74affd640 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x559720d24070 0x559720d26490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.742+0000 7ff74affd640 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x559720d24070 0x559720d26490 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1d7715527455c139 server_cookie=a46e24056447ae4c in_seq=0 out_seq=0 2024-08-20T03:04:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.742+0000 7ff72eff5640 10 client.25618 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:04:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.742+0000 7ff72eff5640 1 -- 172.21.15.44:0/2749576471 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559720c18320 con 0x559720d24070 2024-08-20T03:04:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.742+0000 7ff72eff5640 10 client.25618 handle_command_reply: tid=0 2024-08-20T03:04:20.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.742+0000 7ff7539a5740 1 client.25618 shutdown 2024-08-20T03:04:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.742+0000 7ff7539a5740 1 -- 172.21.15.44:0/2749576471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 msgr2=0x559720cb38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.742+0000 7ff7539a5740 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720cb38e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff740011710 tx=0x7ff73c003900 comp rx=0 tx=0).stop 2024-08-20T03:04:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.742+0000 7ff7539a5740 1 -- 172.21.15.44:0/2749576471 shutdown_connections 2024-08-20T03:04:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.742+0000 7ff7539a5740 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x559720d24070 0x559720d26490 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.743+0000 7ff7539a5740 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559720cb7e60 0x559720cb82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.743+0000 7ff72eff5640 0 client.25618 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:04:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.743+0000 7ff7539a5740 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559720cb0130 0x559720cb3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.743+0000 7ff7539a5740 1 --2- 172.21.15.44:0/2749576471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559720ca3880 0x559720cb38e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.743+0000 7ff7539a5740 1 -- 172.21.15.44:0/2749576471 >> 172.21.15.44:0/2749576471 conn(0x559720ca16b0 msgr2=0x559720caeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.743+0000 7ff7539a5740 1 -- 172.21.15.44:0/2749576471 shutdown_connections 2024-08-20T03:04:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.743+0000 7ff7539a5740 1 -- 172.21.15.44:0/2749576471 wait complete. 2024-08-20T03:04:20.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.743+0000 7ff7539a5740 20 client.25618 trim_cache size 0 max 16384 2024-08-20T03:04:20.752 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:04:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.744+0000 7ff7515ac640 1 -- 172.21.15.44:0/1960784588 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff7340b3d40 msgr2=0x7ff7340b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.744+0000 7ff7515ac640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff7340b3d40 0x7ff7340b6200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff7440cbc50 tx=0x7ff744098400 comp rx=0 tx=0).stop 2024-08-20T03:04:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.745+0000 7ff7515ac640 1 -- 172.21.15.44:0/1960784588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff74c107d20 msgr2=0x7ff74c1a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.745+0000 7ff7515ac640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff74c107d20 0x7ff74c1a0a20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff744005cc0 tx=0x7ff744031700 comp rx=0 tx=0).stop 2024-08-20T03:04:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.745+0000 7ff7515ac640 1 -- 172.21.15.44:0/1960784588 shutdown_connections 2024-08-20T03:04:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.745+0000 7ff7515ac640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff7340b3d40 0x7ff7340b6200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.745+0000 7ff7515ac640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff74c107d20 0x7ff74c1a0a20 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.745+0000 7ff7515ac640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff74c1073a0 0x7ff74c19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.745+0000 7ff7515ac640 1 --2- 172.21.15.44:0/1960784588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74c101990 0x7ff74c19b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.745+0000 7ff7515ac640 1 -- 172.21.15.44:0/1960784588 >> 172.21.15.44:0/1960784588 conn(0x7ff74c0fd400 msgr2=0x7ff74c06a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.745+0000 7ff7515ac640 1 -- 172.21.15.44:0/1960784588 shutdown_connections 2024-08-20T03:04:20.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:20.745+0000 7ff7515ac640 1 -- 172.21.15.44:0/1960784588 wait complete. 2024-08-20T03:04:21.728 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T03:04:22.088 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:04:22.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:22 smithi165 ceph-mon[22357]: pgmap v376: 97 pgs: 97 active+clean; 181 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 428 op/s 2024-08-20T03:04:22.387 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:22 smithi044 ceph-mon[18790]: pgmap v376: 97 pgs: 97 active+clean; 181 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 428 op/s 2024-08-20T03:04:22.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:22 smithi099 ceph-mon[21984]: pgmap v376: 97 pgs: 97 active+clean; 181 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 428 op/s 2024-08-20T03:04:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.737+0000 7f6e0a202640 1 -- 172.21.15.44:0/2438364505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6dfc09a1f0 msgr2=0x7f6dfc09a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.737+0000 7f6e0a202640 1 --2- 172.21.15.44:0/2438364505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6dfc09a1f0 0x7f6dfc09a630 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009860 tx=0x7f6df402a340 comp rx=0 tx=0).stop 2024-08-20T03:04:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.737+0000 7f6e0a202640 1 -- 172.21.15.44:0/2438364505 shutdown_connections 2024-08-20T03:04:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.737+0000 7f6e0a202640 1 --2- 172.21.15.44:0/2438364505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dfc09ab70 0x7f6dfc0a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.737+0000 7f6e0a202640 1 --2- 172.21.15.44:0/2438364505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6dfc09a1f0 0x7f6dfc09a630 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.737+0000 7f6e0a202640 1 --2- 172.21.15.44:0/2438364505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc095390 0x7f6dfc095770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.737+0000 7f6e0a202640 1 -- 172.21.15.44:0/2438364505 >> 172.21.15.44:0/2438364505 conn(0x7f6dfc0098d0 msgr2=0x7f6dfc00c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.737+0000 7f6e0a202640 1 -- 172.21.15.44:0/2438364505 shutdown_connections 2024-08-20T03:04:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.738+0000 7f6e0a202640 1 -- 172.21.15.44:0/2438364505 wait complete. 2024-08-20T03:04:22.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e0a202640 1 Processor -- start 2024-08-20T03:04:22.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e0a202640 1 -- start start 2024-08-20T03:04:22.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e0a202640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dfc095390 0x7f6dfc12bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e0a202640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc09a1f0 0x7f6dfc12c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e0a202640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6dfc09ab70 0x7f6dfc1315e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e0a202640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc0a46c0 con 0x7f6dfc09a1f0 2024-08-20T03:04:22.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e0a202640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc0a4540 con 0x7f6dfc095390 2024-08-20T03:04:22.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e0a202640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc0a4840 con 0x7f6dfc09ab70 2024-08-20T03:04:22.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e09a01640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6dfc09ab70 0x7f6dfc1315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e09a01640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6dfc09ab70 0x7f6dfc1315e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:60082/0 (socket says 172.21.15.44:60082) 2024-08-20T03:04:22.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e09200640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dfc095390 0x7f6dfc12bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e09a01640 1 -- 172.21.15.44:0/2496754135 learned_addr learned my addr 172.21.15.44:0/2496754135 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:22.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.739+0000 7f6e089ff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc09a1f0 0x7f6dfc12c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.740+0000 7f6e09200640 1 -- 172.21.15.44:0/2496754135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6dfc09ab70 msgr2=0x7f6dfc1315e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:22.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.740+0000 7f6e09200640 1 --2- 172.21.15.44:0/2496754135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6dfc09ab70 0x7f6dfc1315e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.740+0000 7f6e09200640 1 -- 172.21.15.44:0/2496754135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc09a1f0 msgr2=0x7f6dfc12c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:22.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.740+0000 7f6e09200640 1 --2- 172.21.15.44:0/2496754135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc09a1f0 0x7f6dfc12c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.740+0000 7f6e09200640 1 -- 172.21.15.44:0/2496754135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc131db0 con 0x7f6dfc095390 2024-08-20T03:04:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.740+0000 7f6e09200640 1 --2- 172.21.15.44:0/2496754135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dfc095390 0x7f6dfc12bd00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000e790 tx=0x7f6e0000ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.740+0000 7f6dfa7fc640 1 -- 172.21.15.44:0/2496754135 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0000eea0 con 0x7f6dfc095390 2024-08-20T03:04:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.740+0000 7f6dfa7fc640 1 -- 172.21.15.44:0/2496754135 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e0000ccc0 con 0x7f6dfc095390 2024-08-20T03:04:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.740+0000 7f6e0a202640 1 -- 172.21.15.44:0/2496754135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dfc1320a0 con 0x7f6dfc095390 2024-08-20T03:04:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.741+0000 7f6dfa7fc640 1 -- 172.21.15.44:0/2496754135 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00018680 con 0x7f6dfc095390 2024-08-20T03:04:22.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.741+0000 7f6e0a202640 1 -- 172.21.15.44:0/2496754135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dfc138760 con 0x7f6dfc095390 2024-08-20T03:04:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.744+0000 7f6dfa7fc640 1 -- 172.21.15.44:0/2496754135 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6e00005020 con 0x7f6dfc095390 2024-08-20T03:04:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.744+0000 7f6e0d85d740 1 Processor -- start 2024-08-20T03:04:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.745+0000 7f6e0d85d740 1 -- start start 2024-08-20T03:04:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.745+0000 7f6e0d85d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 0x555ecc7a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.745+0000 7f6e0d85d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.745+0000 7f6e0d85d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7b75d0 0x555ecc7b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.745+0000 7f6e0d85d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555ecc71d930 con 0x555ecc7a6910 2024-08-20T03:04:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.745+0000 7f6e0d85d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555ecc70ce30 con 0x555ecc7a7230 2024-08-20T03:04:22.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.745+0000 7f6e0d85d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555ecc710e70 con 0x555ecc7b75d0 2024-08-20T03:04:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.745+0000 7f6e09a01640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 0x555ecc7a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.745+0000 7f6e09a01640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 0x555ecc7a6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52612/0 (socket says 172.21.15.44:52612) 2024-08-20T03:04:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.745+0000 7f6e09a01640 1 -- 172.21.15.44:0/2781336525 learned_addr learned my addr 172.21.15.44:0/2781336525 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.745+0000 7f6e089ff640 1 --2- 172.21.15.44:0/2781336525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7b75d0 0x555ecc7b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.745+0000 7f6e09200640 1 --2- 172.21.15.44:0/2781336525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.746+0000 7f6e089ff640 1 -- 172.21.15.44:0/2781336525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 msgr2=0x555ecc7b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.746+0000 7f6e089ff640 1 --2- 172.21.15.44:0/2781336525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.746+0000 7f6e089ff640 1 -- 172.21.15.44:0/2781336525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 msgr2=0x555ecc7a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.746+0000 7f6e089ff640 1 --2- 172.21.15.44:0/2781336525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 0x555ecc7a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.746+0000 7f6e089ff640 1 -- 172.21.15.44:0/2781336525 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecc46c650 con 0x555ecc7b75d0 2024-08-20T03:04:22.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.746+0000 7f6e089ff640 1 --2- 172.21.15.44:0/2781336525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7b75d0 0x555ecc7b99c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6df4005940 tx=0x7f6df4005b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fee9a6fcc00193bf server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.746+0000 7f6def7fe640 1 -- 172.21.15.44:0/2781336525 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4041070 con 0x555ecc7b75d0 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.746+0000 7f6def7fe640 1 -- 172.21.15.44:0/2781336525 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df40044d0 con 0x555ecc7b75d0 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.746+0000 7f6def7fe640 1 -- 172.21.15.44:0/2781336525 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4037040 con 0x555ecc7b75d0 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.747+0000 7f6e0d85d740 1 -- 172.21.15.44:0/2781336525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7b75d0 msgr2=0x555ecc7b99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.747+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/2781336525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7b75d0 0x555ecc7b99c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6df4005940 tx=0x7f6df4005b10 comp rx=0 tx=0).stop 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.747+0000 7f6dfa7fc640 1 --2- 172.21.15.44:0/2496754135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6df00b3dc0 0x7f6df00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.747+0000 7f6e0d85d740 1 -- 172.21.15.44:0/2781336525 shutdown_connections 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.747+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/2781336525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7b75d0 0x555ecc7b99c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.747+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/2781336525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.747+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/2781336525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 0x555ecc7a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.747+0000 7f6e0d85d740 1 -- 172.21.15.44:0/2781336525 >> 172.21.15.44:0/2781336525 conn(0x555ecc7a4740 msgr2=0x555ecc7a58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.747+0000 7f6e0d85d740 1 -- 172.21.15.44:0/2781336525 shutdown_connections 2024-08-20T03:04:22.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.747+0000 7f6e09a01640 1 --2- 172.21.15.44:0/2496754135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6df00b3dc0 0x7f6df00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.747+0000 7f6e0d85d740 1 -- 172.21.15.44:0/2781336525 wait complete. 2024-08-20T03:04:22.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.748+0000 7f6dfa7fc640 1 -- 172.21.15.44:0/2496754135 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6e000e4970 con 0x7f6dfc095390 2024-08-20T03:04:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.748+0000 7f6e0d85d740 1 -- start start 2024-08-20T03:04:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.748+0000 7f6e09a01640 1 --2- 172.21.15.44:0/2496754135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6df00b3dc0 0x7f6df00b6280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6e04068b50 tx=0x7f6e04080040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.748+0000 7f6e0d85d740 1 Processor -- start 2024-08-20T03:04:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e0d85d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 0x555ecc7b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e0d85d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e0d85d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7b75d0 0x555ecc7b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e0d85d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555ecc56f700 con 0x555ecc7a6910 2024-08-20T03:04:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e0d85d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555ecc693810 con 0x555ecc7a7230 2024-08-20T03:04:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e0d85d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555ecc7ba220 con 0x555ecc7b75d0 2024-08-20T03:04:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e0d85d740 10 client.0 _ll_register_callbacks cb 0x555ecc706700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:22.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e09200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 0x555ecc7b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e09200640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 0x555ecc7b7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52616/0 (socket says 172.21.15.44:52616) 2024-08-20T03:04:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e09200640 1 -- 172.21.15.44:0/178279381 learned_addr learned my addr 172.21.15.44:0/178279381 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e09a01640 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7b75d0 0x555ecc7b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e089ff640 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e09200640 1 -- 172.21.15.44:0/178279381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7b75d0 msgr2=0x555ecc7b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e09200640 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7b75d0 0x555ecc7b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e09200640 1 -- 172.21.15.44:0/178279381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 msgr2=0x555ecc7b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e09200640 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e09200640 1 -- 172.21.15.44:0/178279381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecc7c35b0 con 0x555ecc7a6910 2024-08-20T03:04:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.749+0000 7f6e089ff640 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:04:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.750+0000 7f6e09200640 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 0x555ecc7b7060 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6e000a8800 tx=0x7f6e000aafa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3ade9e52f13e04 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:22.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.750+0000 7f6decff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.750+0000 7f6decff9640 1 -- 172.21.15.44:0/178279381 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e000e2560 con 0x555ecc7a6910 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.750+0000 7f6decff9640 1 -- 172.21.15.44:0/178279381 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6e000b41b0 con 0x555ecc7a6910 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.750+0000 7f6e0d85d740 1 -- 172.21.15.44:0/178279381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555ecc6bc150 con 0x555ecc7a6910 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.750+0000 7f6decff9640 1 -- 172.21.15.44:0/178279381 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6e000ab590 con 0x555ecc7a6910 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.750+0000 7f6e0d85d740 10 client.16170 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.750+0000 7f6e0d85d740 1 -- 172.21.15.44:0/178279381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ecc7c5f30 con 0x555ecc7a6910 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.751+0000 7f6decff9640 1 -- 172.21.15.44:0/178279381 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555ecc6bc150 con 0x555ecc7a6910 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.751+0000 7f6decff9640 1 -- 172.21.15.44:0/178279381 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6e000aa7f0 con 0x555ecc7a6910 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.751+0000 7f6e0d85d740 10 client.16170 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.751+0000 7f6e0d85d740 10 client.16170 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.751+0000 7f6e0d85d740 20 client.16170 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.751+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x555ecc7c73e0 0x555ecc7c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.751+0000 7f6e0d85d740 4 client.16170 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:04:22.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.751+0000 7f6e0d85d740 1 -- 172.21.15.44:0/178279381 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ecc71f880 con 0x555ecc7c73e0 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.751+0000 7f6e089ff640 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x555ecc7c73e0 0x555ecc7c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.752+0000 7f6decff9640 10 client.16170 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.752+0000 7f6e089ff640 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x555ecc7c73e0 0x555ecc7c97d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63aac6a5b305cad1 server_cookie=a35b5ec879ecb436 in_seq=0 out_seq=0 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6decff9640 1 -- 172.21.15.44:0/178279381 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ecc71f880 con 0x555ecc7c73e0 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6decff9640 10 client.16170 handle_command_reply: tid=0 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6e0d85d740 1 client.16170 shutdown 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6e0d85d740 1 -- 172.21.15.44:0/178279381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 msgr2=0x555ecc7b7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 0x555ecc7b7060 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6e000a8800 tx=0x7f6e000aafa0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6e0d85d740 1 -- 172.21.15.44:0/178279381 shutdown_connections 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x555ecc7c73e0 0x555ecc7c97d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6decff9640 0 client.16170 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7b75d0 0x555ecc7b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/178279381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7a6910 0x555ecc7b7060 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6e0d85d740 1 -- 172.21.15.44:0/178279381 >> 172.21.15.44:0/178279381 conn(0x555ecc7a4740 msgr2=0x555ecc7aa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:22.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6e0d85d740 1 -- 172.21.15.44:0/178279381 shutdown_connections 2024-08-20T03:04:22.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6e0d85d740 1 -- 172.21.15.44:0/178279381 wait complete. 2024-08-20T03:04:22.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.753+0000 7f6e0d85d740 20 client.16170 trim_cache size 0 max 16384 2024-08-20T03:04:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.780+0000 7f6e0d85d740 1 Processor -- start 2024-08-20T03:04:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.780+0000 7f6e0d85d740 1 -- start start 2024-08-20T03:04:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.780+0000 7f6e0d85d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7a6910 0x555ecc7b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.780+0000 7f6e0d85d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7a5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.780+0000 7f6e0d85d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7b75d0 0x555ecc7a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.780+0000 7f6e0d85d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc13ed50 con 0x555ecc7b75d0 2024-08-20T03:04:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.780+0000 7f6e0d85d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc140950 con 0x555ecc7a7230 2024-08-20T03:04:22.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.780+0000 7f6e0d85d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc13f420 con 0x555ecc7a6910 2024-08-20T03:04:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.780+0000 7f6e09200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7a6910 0x555ecc7b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.780+0000 7f6e09200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7a6910 0x555ecc7b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:60116/0 (socket says 172.21.15.44:60116) 2024-08-20T03:04:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.780+0000 7f6e09200640 1 -- 172.21.15.44:0/3960346932 learned_addr learned my addr 172.21.15.44:0/3960346932 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.781+0000 7f6e089ff640 1 --2- 172.21.15.44:0/3960346932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.781+0000 7f6e09a01640 1 --2- 172.21.15.44:0/3960346932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7b75d0 0x555ecc7a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.782+0000 7f6e089ff640 1 -- 172.21.15.44:0/3960346932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7a6910 msgr2=0x555ecc7b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.782+0000 7f6e089ff640 1 --2- 172.21.15.44:0/3960346932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7a6910 0x555ecc7b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.782+0000 7f6e089ff640 1 -- 172.21.15.44:0/3960346932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7b75d0 msgr2=0x555ecc7a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.782+0000 7f6e089ff640 1 --2- 172.21.15.44:0/3960346932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7b75d0 0x555ecc7a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.782+0000 7f6e089ff640 1 -- 172.21.15.44:0/3960346932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecc7b4840 con 0x555ecc7a7230 2024-08-20T03:04:22.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.782+0000 7f6e089ff640 1 --2- 172.21.15.44:0/3960346932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7a5800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x555ecc7b75a0 tx=0x7f6df402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2858f50faf1e71aa server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:22.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.783+0000 7f6deeffd640 1 -- 172.21.15.44:0/3960346932 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4037030 con 0x555ecc7a7230 2024-08-20T03:04:22.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.783+0000 7f6deeffd640 1 -- 172.21.15.44:0/3960346932 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df4030900 con 0x555ecc7a7230 2024-08-20T03:04:22.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.783+0000 7f6deeffd640 1 -- 172.21.15.44:0/3960346932 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4041040 con 0x555ecc7a7230 2024-08-20T03:04:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.787+0000 7f6e0d85d740 1 -- 172.21.15.44:0/3960346932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 msgr2=0x555ecc7a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.787+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/3960346932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7a5800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x555ecc7b75a0 tx=0x7f6df402a340 comp rx=0 tx=0).stop 2024-08-20T03:04:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.789+0000 7f6e0d85d740 1 -- 172.21.15.44:0/3960346932 shutdown_connections 2024-08-20T03:04:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.789+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/3960346932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7b75d0 0x555ecc7a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.789+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/3960346932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a7230 0x555ecc7a5800 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.789+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/3960346932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7a6910 0x555ecc7b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.789+0000 7f6e0d85d740 1 -- 172.21.15.44:0/3960346932 >> 172.21.15.44:0/3960346932 conn(0x555ecc7a4740 msgr2=0x555ecc7b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:22.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.792+0000 7f6e0d85d740 1 -- 172.21.15.44:0/3960346932 shutdown_connections 2024-08-20T03:04:22.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.792+0000 7f6e0d85d740 1 -- 172.21.15.44:0/3960346932 wait complete. 2024-08-20T03:04:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.792+0000 7f6e0d85d740 1 -- start start 2024-08-20T03:04:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.793+0000 7f6e0d85d740 1 Processor -- start 2024-08-20T03:04:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.793+0000 7f6e0d85d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a6910 0x555ecc7bfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.794+0000 7f6e0d85d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7a7230 0x555ecc7bd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.794+0000 7f6e0d85d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7b75d0 0x555ecc7bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.794+0000 7f6e0d85d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc1401c0 con 0x555ecc7b75d0 2024-08-20T03:04:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.794+0000 7f6e0d85d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc13faf0 con 0x555ecc7a6910 2024-08-20T03:04:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.794+0000 7f6e0d85d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc13cd50 con 0x555ecc7a7230 2024-08-20T03:04:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.794+0000 7f6e09a01640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7b75d0 0x555ecc7bd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.794+0000 7f6e09a01640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7b75d0 0x555ecc7bd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52634/0 (socket says 172.21.15.44:52634) 2024-08-20T03:04:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.794+0000 7f6e09a01640 1 -- 172.21.15.44:0/1777531539 learned_addr learned my addr 172.21.15.44:0/1777531539 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:22.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.794+0000 7f6e0d85d740 10 client.0 _ll_register_callbacks cb 0x7f6e000e7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:22.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.795+0000 7f6e089ff640 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7a7230 0x555ecc7bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.795+0000 7f6e09200640 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a6910 0x555ecc7bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.795+0000 7f6e09a01640 1 -- 172.21.15.44:0/1777531539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7a7230 msgr2=0x555ecc7bd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:22.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.795+0000 7f6e09a01640 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7a7230 0x555ecc7bd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.795+0000 7f6e09a01640 1 -- 172.21.15.44:0/1777531539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a6910 msgr2=0x555ecc7bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:22.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.795+0000 7f6e09a01640 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a6910 0x555ecc7bfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.795+0000 7f6e09a01640 1 -- 172.21.15.44:0/1777531539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecc7bddd0 con 0x555ecc7b75d0 2024-08-20T03:04:22.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.796+0000 7f6e09a01640 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7b75d0 0x555ecc7bd5c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6e04065ff0 tx=0x7f6e04068db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2e34e4a86ee805 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.797+0000 7f6decff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:04:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.797+0000 7f6decff9640 1 -- 172.21.15.44:0/1777531539 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0409a070 con 0x555ecc7b75d0 2024-08-20T03:04:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.797+0000 7f6decff9640 1 -- 172.21.15.44:0/1777531539 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6e0406c590 con 0x555ecc7b75d0 2024-08-20T03:04:22.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.799+0000 7f6e0d85d740 1 -- 172.21.15.44:0/1777531539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555ecc79fc40 con 0x555ecc7b75d0 2024-08-20T03:04:22.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.801+0000 7f6decff9640 1 -- 172.21.15.44:0/1777531539 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6e040a4050 con 0x555ecc7b75d0 2024-08-20T03:04:22.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.807+0000 7f6e0d85d740 10 client.16176 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:22.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.807+0000 7f6e0d85d740 1 -- 172.21.15.44:0/1777531539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555ecc7b1b40 con 0x555ecc7b75d0 2024-08-20T03:04:22.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.811+0000 7f6decff9640 1 -- 172.21.15.44:0/1777531539 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555ecc79fc40 con 0x555ecc7b75d0 2024-08-20T03:04:22.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.811+0000 7f6decff9640 1 -- 172.21.15.44:0/1777531539 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6e04096480 con 0x555ecc7b75d0 2024-08-20T03:04:22.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.817+0000 7f6e0d85d740 10 client.16176 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:22.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.817+0000 7f6e0d85d740 10 client.16176 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:04:22.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.817+0000 7f6e0d85d740 20 client.16176 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:22.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.817+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x555ecc7c73e0 0x555ecc838890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:22.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.817+0000 7f6e0d85d740 4 client.16176 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:04:22.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.817+0000 7f6e0d85d740 1 -- 172.21.15.44:0/1777531539 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ecc7b80f0 con 0x555ecc7c73e0 2024-08-20T03:04:22.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.818+0000 7f6e09200640 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x555ecc7c73e0 0x555ecc838890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:22.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.820+0000 7f6e09200640 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x555ecc7c73e0 0x555ecc838890 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4160ad07de7e2453 server_cookie=4bc331a9c8d76bce in_seq=0 out_seq=0 2024-08-20T03:04:22.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.820+0000 7f6decff9640 10 client.16176 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:04:22.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.824+0000 7f6decff9640 1 -- 172.21.15.44:0/1777531539 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ecc7b80f0 con 0x555ecc7c73e0 2024-08-20T03:04:22.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.824+0000 7f6decff9640 10 client.16176 handle_command_reply: tid=0 2024-08-20T03:04:22.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.825+0000 7f6e0d85d740 1 client.16176 shutdown 2024-08-20T03:04:22.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.826+0000 7f6e0d85d740 1 -- 172.21.15.44:0/1777531539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7b75d0 msgr2=0x555ecc7bd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.826+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7b75d0 0x555ecc7bd5c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6e04065ff0 tx=0x7f6e04068db0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.826+0000 7f6e0d85d740 1 -- 172.21.15.44:0/1777531539 shutdown_connections 2024-08-20T03:04:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.826+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecc7b75d0 0x555ecc7bd5c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.826+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x555ecc7c73e0 0x555ecc838890 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.826+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555ecc7a7230 0x555ecc7bd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.826+0000 7f6e0d85d740 1 --2- 172.21.15.44:0/1777531539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ecc7a6910 0x555ecc7bfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.826+0000 7f6decff9640 0 client.16176 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:04:22.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.826+0000 7f6e0d85d740 1 -- 172.21.15.44:0/1777531539 >> 172.21.15.44:0/1777531539 conn(0x555ecc7a4740 msgr2=0x555ecc7b85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:22.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.827+0000 7f6e0d85d740 1 -- 172.21.15.44:0/1777531539 shutdown_connections 2024-08-20T03:04:22.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.828+0000 7f6e0d85d740 1 -- 172.21.15.44:0/1777531539 wait complete. 2024-08-20T03:04:22.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.828+0000 7f6e0d85d740 20 client.16176 trim_cache size 0 max 16384 2024-08-20T03:04:22.831 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:04:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.829+0000 7f6e0a202640 1 -- 172.21.15.44:0/2496754135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6df00b3dc0 msgr2=0x7f6df00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.829+0000 7f6e0a202640 1 --2- 172.21.15.44:0/2496754135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6df00b3dc0 0x7f6df00b6280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6e04068b50 tx=0x7f6e04080040 comp rx=0 tx=0).stop 2024-08-20T03:04:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.829+0000 7f6e0a202640 1 -- 172.21.15.44:0/2496754135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dfc095390 msgr2=0x7f6dfc12bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:22.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.829+0000 7f6e0a202640 1 --2- 172.21.15.44:0/2496754135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dfc095390 0x7f6dfc12bd00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000e790 tx=0x7f6e0000ec50 comp rx=0 tx=0).stop 2024-08-20T03:04:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.830+0000 7f6e0a202640 1 -- 172.21.15.44:0/2496754135 shutdown_connections 2024-08-20T03:04:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.830+0000 7f6e0a202640 1 --2- 172.21.15.44:0/2496754135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6df00b3dc0 0x7f6df00b6280 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.830+0000 7f6e0a202640 1 --2- 172.21.15.44:0/2496754135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6dfc09ab70 0x7f6dfc1315e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.830+0000 7f6e0a202640 1 --2- 172.21.15.44:0/2496754135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dfc09a1f0 0x7f6dfc12c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.830+0000 7f6e0a202640 1 --2- 172.21.15.44:0/2496754135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dfc095390 0x7f6dfc12bd00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.830+0000 7f6e0a202640 1 -- 172.21.15.44:0/2496754135 >> 172.21.15.44:0/2496754135 conn(0x7f6dfc0098d0 msgr2=0x7f6dfc0999e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.830+0000 7f6e0a202640 1 -- 172.21.15.44:0/2496754135 shutdown_connections 2024-08-20T03:04:22.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:22.830+0000 7f6e0a202640 1 -- 172.21.15.44:0/2496754135 wait complete. 2024-08-20T03:04:23.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T03:04:24.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:24 smithi044 ceph-mon[18790]: pgmap v377: 97 pgs: 97 active+clean; 157 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 509 op/s 2024-08-20T03:04:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:24 smithi165 ceph-mon[22357]: pgmap v377: 97 pgs: 97 active+clean; 157 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 509 op/s 2024-08-20T03:04:24.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:24 smithi099 ceph-mon[21984]: pgmap v377: 97 pgs: 97 active+clean; 157 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 509 op/s 2024-08-20T03:04:24.664 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:04:25.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.174+0000 7f3a77cae640 1 -- 172.21.15.44:0/3686124054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a70101990 msgr2=0x7f3a70101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:25.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.174+0000 7f3a77cae640 1 --2- 172.21.15.44:0/3686124054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a70101990 0x7f3a70101d70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009860 tx=0x7f3a6c02a340 comp rx=0 tx=0).stop 2024-08-20T03:04:25.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.175+0000 7f3a77cae640 1 -- 172.21.15.44:0/3686124054 shutdown_connections 2024-08-20T03:04:25.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.175+0000 7f3a77cae640 1 --2- 172.21.15.44:0/3686124054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a70107d20 0x7f3a7010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.175+0000 7f3a77cae640 1 --2- 172.21.15.44:0/3686124054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a701073a0 0x7f3a701077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.175+0000 7f3a77cae640 1 --2- 172.21.15.44:0/3686124054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a70101990 0x7f3a70101d70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.175+0000 7f3a77cae640 1 -- 172.21.15.44:0/3686124054 >> 172.21.15.44:0/3686124054 conn(0x7f3a700fd400 msgr2=0x7f3a700ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:25.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.175+0000 7f3a77cae640 1 -- 172.21.15.44:0/3686124054 shutdown_connections 2024-08-20T03:04:25.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.175+0000 7f3a77cae640 1 -- 172.21.15.44:0/3686124054 wait complete. 2024-08-20T03:04:25.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.175+0000 7f3a77cae640 1 Processor -- start 2024-08-20T03:04:25.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.175+0000 7f3a77cae640 1 -- start start 2024-08-20T03:04:25.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a77cae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a70101990 0x7f3a7019b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a77cae640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a701073a0 0x7f3a7019b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a77cae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a70107d20 0x7f3a701a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a77cae640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701119e0 con 0x7f3a70107d20 2024-08-20T03:04:25.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a77cae640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70111860 con 0x7f3a70101990 2024-08-20T03:04:25.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a77cae640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70111b60 con 0x7f3a701073a0 2024-08-20T03:04:25.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a75249640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a701073a0 0x7f3a7019b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a75a4a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a70101990 0x7f3a7019b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a75a4a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a70101990 0x7f3a7019b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51854/0 (socket says 172.21.15.44:51854) 2024-08-20T03:04:25.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a75249640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a701073a0 0x7f3a7019b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:60150/0 (socket says 172.21.15.44:60150) 2024-08-20T03:04:25.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a75a4a640 1 -- 172.21.15.44:0/3019937952 learned_addr learned my addr 172.21.15.44:0/3019937952 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:25.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a7624b640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a70107d20 0x7f3a701a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a75a4a640 1 -- 172.21.15.44:0/3019937952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a701073a0 msgr2=0x7f3a7019b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:25.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a75a4a640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a701073a0 0x7f3a7019b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a75a4a640 1 -- 172.21.15.44:0/3019937952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a70107d20 msgr2=0x7f3a701a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a75a4a640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a70107d20 0x7f3a701a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a75a4a640 1 -- 172.21.15.44:0/3019937952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a701a1290 con 0x7f3a70101990 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.176+0000 7f3a75249640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a701073a0 0x7f3a7019b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.177+0000 7f3a75a4a640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a70101990 0x7f3a7019b1e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c035040 tx=0x7f3a6c032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.177+0000 7f3a66ffd640 1 -- 172.21.15.44:0/3019937952 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c037040 con 0x7f3a70101990 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.177+0000 7f3a66ffd640 1 -- 172.21.15.44:0/3019937952 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a6c041070 con 0x7f3a70101990 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.177+0000 7f3a77cae640 1 -- 172.21.15.44:0/3019937952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a701a1520 con 0x7f3a70101990 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.177+0000 7f3a66ffd640 1 -- 172.21.15.44:0/3019937952 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c003850 con 0x7f3a70101990 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.177+0000 7f3a77cae640 1 -- 172.21.15.44:0/3019937952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a701083f0 con 0x7f3a70101990 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.180+0000 7f3a7a0a7740 1 Processor -- start 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.180+0000 7f3a7a0a7740 1 -- start start 2024-08-20T03:04:25.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.180+0000 7f3a7a0a7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b47a880 0x558f1b47ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.180+0000 7f3a7a0a7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b486bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.180+0000 7f3a7a0a7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b487130 0x558f1b489520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.180+0000 7f3a7a0a7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558f1b3f1930 con 0x558f1b47b1a0 2024-08-20T03:04:25.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.180+0000 7f3a7a0a7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558f1b3e0e30 con 0x558f1b47a880 2024-08-20T03:04:25.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.180+0000 7f3a7a0a7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f1b3e4e70 con 0x558f1b487130 2024-08-20T03:04:25.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.180+0000 7f3a7624b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b47a880 0x558f1b47ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a75249640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b487130 0x558f1b489520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a75249640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b487130 0x558f1b489520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:60158/0 (socket says 172.21.15.44:60158) 2024-08-20T03:04:25.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a7624b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b47a880 0x558f1b47ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51860/0 (socket says 172.21.15.44:51860) 2024-08-20T03:04:25.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a75249640 1 -- 172.21.15.44:0/941182270 learned_addr learned my addr 172.21.15.44:0/941182270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:25.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a66ffd640 1 -- 172.21.15.44:0/3019937952 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3a6c004050 con 0x7f3a70101990 2024-08-20T03:04:25.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a75249640 1 -- 172.21.15.44:0/941182270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b47a880 msgr2=0x558f1b47ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:25.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a75249640 1 --2- 172.21.15.44:0/941182270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b47a880 0x558f1b47ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a75249640 1 -- 172.21.15.44:0/941182270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 msgr2=0x558f1b486bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:04:25.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a75a4a640 1 --2- 172.21.15.44:0/941182270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b486bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a75249640 1 --2- 172.21.15.44:0/941182270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b486bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a75249640 1 -- 172.21.15.44:0/941182270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f1b140650 con 0x558f1b487130 2024-08-20T03:04:25.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a75a4a640 1 --2- 172.21.15.44:0/941182270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b486bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:04:25.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.181+0000 7f3a75249640 1 --2- 172.21.15.44:0/941182270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b487130 0x558f1b489520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a600114f0 tx=0x7f3a600119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db15ca037aafa0ba server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:25.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.182+0000 7f3a66ffd640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3a5c0b3dc0 0x7f3a5c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.182+0000 7f3a5bfff640 1 -- 172.21.15.44:0/941182270 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60005020 con 0x558f1b487130 2024-08-20T03:04:25.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.182+0000 7f3a5bfff640 1 -- 172.21.15.44:0/941182270 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a600047a0 con 0x558f1b487130 2024-08-20T03:04:25.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.182+0000 7f3a5bfff640 1 -- 172.21.15.44:0/941182270 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6000f920 con 0x558f1b487130 2024-08-20T03:04:25.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.182+0000 7f3a66ffd640 1 -- 172.21.15.44:0/3019937952 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3a6c1038c0 con 0x7f3a70101990 2024-08-20T03:04:25.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.182+0000 7f3a7624b640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3a5c0b3dc0 0x7f3a5c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.183+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/941182270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b487130 msgr2=0x558f1b489520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:25.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.183+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/941182270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b487130 0x558f1b489520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a600114f0 tx=0x7f3a600119b0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.183+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/941182270 shutdown_connections 2024-08-20T03:04:25.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.183+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/941182270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b487130 0x558f1b489520 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.183+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/941182270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b486bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.183+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/941182270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b47a880 0x558f1b47ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.183+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/941182270 >> 172.21.15.44:0/941182270 conn(0x558f1b4786b0 msgr2=0x558f1b479850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:25.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.183+0000 7f3a7624b640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3a5c0b3dc0 0x7f3a5c0b6280 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3a68005fd0 tx=0x7f3a68016040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:25.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.183+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/941182270 shutdown_connections 2024-08-20T03:04:25.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.183+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/941182270 wait complete. 2024-08-20T03:04:25.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.183+0000 7f3a7a0a7740 1 -- start start 2024-08-20T03:04:25.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a7a0a7740 1 Processor -- start 2024-08-20T03:04:25.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a7a0a7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47a880 0x558f1b4840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a7a0a7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47b1a0 0x558f1b484610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a7a0a7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b484b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a7a0a7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558f1b243700 con 0x558f1b47a880 2024-08-20T03:04:25.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a7a0a7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558f1b367810 con 0x558f1b487130 2024-08-20T03:04:25.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a7a0a7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558f1b489de0 con 0x558f1b47b1a0 2024-08-20T03:04:25.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a7a0a7740 10 client.0 _ll_register_callbacks cb 0x558f1b3da700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:25.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a75a4a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47a880 0x558f1b4840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a75249640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47b1a0 0x558f1b484610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a75249640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47b1a0 0x558f1b484610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:60162/0 (socket says 172.21.15.44:60162) 2024-08-20T03:04:25.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a75249640 1 -- 172.21.15.44:0/168288636 learned_addr learned my addr 172.21.15.44:0/168288636 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:25.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a75a4a640 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47a880 0x558f1b4840d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52664/0 (socket says 172.21.15.44:52664) 2024-08-20T03:04:25.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.184+0000 7f3a7624b640 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b484b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a75249640 1 -- 172.21.15.44:0/168288636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 msgr2=0x558f1b484b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:25.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a75249640 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b484b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a75249640 1 -- 172.21.15.44:0/168288636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47a880 msgr2=0x558f1b4840d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:25.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a75249640 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47a880 0x558f1b4840d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a75249640 1 -- 172.21.15.44:0/168288636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f1b4931c0 con 0x558f1b47b1a0 2024-08-20T03:04:25.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a75249640 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47b1a0 0x558f1b484610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a60000c00 tx=0x7f3a600097f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d0deabee527f471 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:25.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a597fa640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:04:25.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a597fa640 1 -- 172.21.15.44:0/168288636 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60026020 con 0x558f1b47b1a0 2024-08-20T03:04:25.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a597fa640 1 -- 172.21.15.44:0/168288636 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3a600048d0 con 0x558f1b47b1a0 2024-08-20T03:04:25.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/168288636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558f1b390150 con 0x558f1b47b1a0 2024-08-20T03:04:25.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a597fa640 1 -- 172.21.15.44:0/168288636 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3a60031050 con 0x558f1b47b1a0 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a7a0a7740 10 client.25645 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.185+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/168288636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558f1b495ae0 con 0x558f1b47b1a0 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.186+0000 7f3a597fa640 1 -- 172.21.15.44:0/168288636 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558f1b390150 con 0x558f1b47b1a0 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.186+0000 7f3a597fa640 1 -- 172.21.15.44:0/168288636 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3a60022460 con 0x558f1b47b1a0 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.186+0000 7f3a7a0a7740 10 client.25645 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.186+0000 7f3a7a0a7740 10 client.25645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.186+0000 7f3a7a0a7740 20 client.25645 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.186+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x558f1b496f40 0x558f1b499330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.186+0000 7f3a7a0a7740 4 client.25645 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.186+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/168288636 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f1b3f3880 con 0x558f1b496f40 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.186+0000 7f3a75a4a640 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x558f1b496f40 0x558f1b499330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.187+0000 7f3a75a4a640 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x558f1b496f40 0x558f1b499330 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=740e3e29d5ef356e server_cookie=4ba2f78441c56c56 in_seq=0 out_seq=0 2024-08-20T03:04:25.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.187+0000 7f3a597fa640 10 client.25645 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.187+0000 7f3a597fa640 1 -- 172.21.15.44:0/168288636 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f1b3f3880 con 0x558f1b496f40 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.187+0000 7f3a597fa640 10 client.25645 handle_command_reply: tid=0 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.187+0000 7f3a7a0a7740 1 client.25645 shutdown 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.188+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/168288636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47b1a0 msgr2=0x558f1b484610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.188+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47b1a0 0x558f1b484610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a60000c00 tx=0x7f3a600097f0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.188+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/168288636 shutdown_connections 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.188+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x558f1b496f40 0x558f1b499330 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.188+0000 7f3a597fa640 0 client.25645 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.188+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b484b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.188+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47b1a0 0x558f1b484610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.188+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/168288636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47a880 0x558f1b4840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.188+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/168288636 >> 172.21.15.44:0/168288636 conn(0x558f1b4786b0 msgr2=0x558f1b47e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:25.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.188+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/168288636 shutdown_connections 2024-08-20T03:04:25.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.188+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/168288636 wait complete. 2024-08-20T03:04:25.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.188+0000 7f3a7a0a7740 20 client.25645 trim_cache size 0 max 16384 2024-08-20T03:04:25.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a7a0a7740 1 Processor -- start 2024-08-20T03:04:25.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a7a0a7740 1 -- start start 2024-08-20T03:04:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a7a0a7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47a880 0x558f1b49ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a7a0a7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b49ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a7a0a7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b484040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a7a0a7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701ac110 con 0x558f1b47b1a0 2024-08-20T03:04:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a7a0a7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701add10 con 0x558f1b487130 2024-08-20T03:04:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a7a0a7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701ac7e0 con 0x558f1b47a880 2024-08-20T03:04:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a7624b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b484040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a75a4a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47a880 0x558f1b49ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a7624b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b484040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:51876/0 (socket says 172.21.15.44:51876) 2024-08-20T03:04:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a7624b640 1 -- 172.21.15.44:0/3206001404 learned_addr learned my addr 172.21.15.44:0/3206001404 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.207+0000 7f3a75249640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b49ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.208+0000 7f3a75249640 1 -- 172.21.15.44:0/3206001404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47a880 msgr2=0x558f1b49ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.208+0000 7f3a75249640 1 --2- 172.21.15.44:0/3206001404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47a880 0x558f1b49ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.208+0000 7f3a75249640 1 -- 172.21.15.44:0/3206001404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 msgr2=0x558f1b484040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:25.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.208+0000 7f3a75249640 1 --2- 172.21.15.44:0/3206001404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b484040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.208+0000 7f3a75249640 1 -- 172.21.15.44:0/3206001404 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f1b479130 con 0x558f1b47b1a0 2024-08-20T03:04:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.208+0000 7f3a7624b640 1 --2- 172.21.15.44:0/3206001404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b484040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:04:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.208+0000 7f3a75249640 1 --2- 172.21.15.44:0/3206001404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b49ef90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3a60022a20 tx=0x7f3a600225f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ce0317ed25ecd9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.208+0000 7f3a5b7fe640 1 -- 172.21.15.44:0/3206001404 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60015070 con 0x558f1b47b1a0 2024-08-20T03:04:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.208+0000 7f3a5b7fe640 1 -- 172.21.15.44:0/3206001404 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a60022ca0 con 0x558f1b47b1a0 2024-08-20T03:04:25.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.208+0000 7f3a5b7fe640 1 -- 172.21.15.44:0/3206001404 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60026040 con 0x558f1b47b1a0 2024-08-20T03:04:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.209+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206001404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 msgr2=0x558f1b49ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.209+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/3206001404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b49ef90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3a60022a20 tx=0x7f3a600225f0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.209+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206001404 shutdown_connections 2024-08-20T03:04:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.209+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/3206001404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b484040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.209+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/3206001404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b49ef90 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.209+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/3206001404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47a880 0x558f1b49ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.209+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206001404 >> 172.21.15.44:0/3206001404 conn(0x558f1b4786b0 msgr2=0x558f1b485f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:25.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.210+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206001404 shutdown_connections 2024-08-20T03:04:25.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.210+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206001404 wait complete. 2024-08-20T03:04:25.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.210+0000 7f3a7a0a7740 1 -- start start 2024-08-20T03:04:25.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.211+0000 7f3a7a0a7740 1 Processor -- start 2024-08-20T03:04:25.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.211+0000 7f3a7a0a7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47a880 0x558f1b48a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.211+0000 7f3a7a0a7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b48a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.211+0000 7f3a75249640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b48a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a75249640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b48a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52682/0 (socket says 172.21.15.44:52682) 2024-08-20T03:04:25.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a75249640 1 -- 172.21.15.44:0/3206341394 learned_addr learned my addr 172.21.15.44:0/3206341394 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:25.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b48aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a75a4a640 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47a880 0x558f1b48a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206341394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701ad580 con 0x558f1b47b1a0 2024-08-20T03:04:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206341394 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701aceb0 con 0x558f1b487130 2024-08-20T03:04:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206341394 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701aa110 con 0x558f1b47a880 2024-08-20T03:04:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a7624b640 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b48aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a75249640 1 -- 172.21.15.44:0/3206341394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47a880 msgr2=0x558f1b48a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a75249640 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47a880 0x558f1b48a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a75249640 1 -- 172.21.15.44:0/3206341394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 msgr2=0x558f1b48aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a75249640 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b48aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a75249640 1 -- 172.21.15.44:0/3206341394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f1b4ffac0 con 0x558f1b47b1a0 2024-08-20T03:04:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a7a0a7740 10 client.0 _ll_register_callbacks cb 0x7f3a68014d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:25.217 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.212+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206341394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a60022ca0 con 0x558f1b47b1a0 2024-08-20T03:04:25.218 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.213+0000 7f3a75249640 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b48a5a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3a600042b0 tx=0x7f3a600095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefe90a4d99cff63 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:25.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.214+0000 7f3a597fa640 10 client.16191 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:04:25.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.214+0000 7f3a597fa640 1 -- 172.21.15.44:0/3206341394 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6003b070 con 0x558f1b47b1a0 2024-08-20T03:04:25.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.214+0000 7f3a597fa640 1 -- 172.21.15.44:0/3206341394 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3a60004ae0 con 0x558f1b47b1a0 2024-08-20T03:04:25.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.214+0000 7f3a597fa640 1 -- 172.21.15.44:0/3206341394 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3a60015050 con 0x558f1b47b1a0 2024-08-20T03:04:25.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.214+0000 7f3a7a0a7740 10 client.16191 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:25.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.214+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206341394 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=30}) -- 0x558f1b500be0 con 0x558f1b47b1a0 2024-08-20T03:04:25.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.215+0000 7f3a597fa640 1 -- 172.21.15.44:0/3206341394 <== mon.0 v2:172.21.15.44:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3a600035f0 con 0x558f1b47b1a0 2024-08-20T03:04:25.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.215+0000 7f3a7a0a7740 10 client.16191 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:25.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.215+0000 7f3a7a0a7740 10 client.16191 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.215+0000 7f3a7a0a7740 20 client.16191 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.215+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x558f1b496f40 0x558f1b503180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.215+0000 7f3a7a0a7740 4 client.16191 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.215+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206341394 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f1b3ef320 con 0x558f1b496f40 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.215+0000 7f3a75a4a640 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x558f1b496f40 0x558f1b503180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.216+0000 7f3a597fa640 10 client.16191 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.216+0000 7f3a75a4a640 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x558f1b496f40 0x558f1b503180 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7886cc62d18f817e server_cookie=8a4705da5089e0b in_seq=0 out_seq=0 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.216+0000 7f3a597fa640 1 -- 172.21.15.44:0/3206341394 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f1b3ef320 con 0x558f1b496f40 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.216+0000 7f3a597fa640 10 client.16191 handle_command_reply: tid=0 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.216+0000 7f3a7a0a7740 1 client.16191 shutdown 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.216+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206341394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 msgr2=0x558f1b48a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.216+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b48a5a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3a600042b0 tx=0x7f3a600095a0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.217+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206341394 shutdown_connections 2024-08-20T03:04:25.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.217+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x558f1b496f40 0x558f1b503180 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.217+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558f1b487130 0x558f1b48aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.217+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558f1b47b1a0 0x558f1b48a5a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.217+0000 7f3a597fa640 0 client.16191 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:04:25.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.217+0000 7f3a7a0a7740 1 --2- 172.21.15.44:0/3206341394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558f1b47a880 0x558f1b48a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.223 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.217+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206341394 >> 172.21.15.44:0/3206341394 conn(0x558f1b4786b0 msgr2=0x558f1b484040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.217+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206341394 shutdown_connections 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.217+0000 7f3a7a0a7740 1 -- 172.21.15.44:0/3206341394 wait complete. 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.217+0000 7f3a7a0a7740 20 client.16191 trim_cache size 0 max 16384 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.218+0000 7f3a77cae640 1 -- 172.21.15.44:0/3019937952 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3a5c0b3dc0 msgr2=0x7f3a5c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.218+0000 7f3a77cae640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3a5c0b3dc0 0x7f3a5c0b6280 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3a68005fd0 tx=0x7f3a68016040 comp rx=0 tx=0).stop 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.218+0000 7f3a77cae640 1 -- 172.21.15.44:0/3019937952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a70101990 msgr2=0x7f3a7019b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.218+0000 7f3a77cae640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a70101990 0x7f3a7019b1e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c035040 tx=0x7f3a6c032d50 comp rx=0 tx=0).stop 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.219+0000 7f3a77cae640 1 -- 172.21.15.44:0/3019937952 shutdown_connections 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.219+0000 7f3a77cae640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3a5c0b3dc0 0x7f3a5c0b6280 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.219+0000 7f3a77cae640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3a70107d20 0x7f3a701a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.219+0000 7f3a77cae640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a701073a0 0x7f3a7019b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.219+0000 7f3a77cae640 1 --2- 172.21.15.44:0/3019937952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a70101990 0x7f3a7019b1e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:25.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.219+0000 7f3a77cae640 1 -- 172.21.15.44:0/3019937952 >> 172.21.15.44:0/3019937952 conn(0x7f3a700fd400 msgr2=0x7f3a7006afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:25.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.219+0000 7f3a77cae640 1 -- 172.21.15.44:0/3019937952 shutdown_connections 2024-08-20T03:04:25.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:25.219+0000 7f3a77cae640 1 -- 172.21.15.44:0/3019937952 wait complete. 2024-08-20T03:04:26.183 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:26 smithi044 ceph-mon[18790]: pgmap v378: 97 pgs: 97 active+clean; 156 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 793 op/s 2024-08-20T03:04:26.186 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T03:04:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:26 smithi165 ceph-mon[22357]: pgmap v378: 97 pgs: 97 active+clean; 156 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 793 op/s 2024-08-20T03:04:26.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:26 smithi099 ceph-mon[21984]: pgmap v378: 97 pgs: 97 active+clean; 156 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 793 op/s 2024-08-20T03:04:26.608 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:04:27.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.109+0000 7f61642f4640 1 -- 172.21.15.44:0/463197155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f615c10f120 msgr2=0x7f615c111510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:27.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.109+0000 7f61642f4640 1 --2- 172.21.15.44:0/463197155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f615c10f120 0x7f615c111510 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6154009ca0 tx=0x7f615402a950 comp rx=0 tx=0).stop 2024-08-20T03:04:27.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.109+0000 7f61642f4640 1 -- 172.21.15.44:0/463197155 shutdown_connections 2024-08-20T03:04:27.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.109+0000 7f61642f4640 1 --2- 172.21.15.44:0/463197155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f615c10f120 0x7f615c111510 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.109+0000 7f61642f4640 1 --2- 172.21.15.44:0/463197155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f615c101e90 0x7f615c10ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.109+0000 7f61642f4640 1 --2- 172.21.15.44:0/463197155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f615c101570 0x7f615c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.109+0000 7f61642f4640 1 -- 172.21.15.44:0/463197155 >> 172.21.15.44:0/463197155 conn(0x7f615c0fd460 msgr2=0x7f615c0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:27.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.109+0000 7f61642f4640 1 -- 172.21.15.44:0/463197155 shutdown_connections 2024-08-20T03:04:27.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.110+0000 7f61642f4640 1 -- 172.21.15.44:0/463197155 wait complete. 2024-08-20T03:04:27.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.110+0000 7f61642f4640 1 Processor -- start 2024-08-20T03:04:27.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.110+0000 7f61642f4640 1 -- start start 2024-08-20T03:04:27.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f61642f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f615c101570 0x7f615c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f61642f4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f615c101e90 0x7f615c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f61642f4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f615c10f120 0x7f615c1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f61642f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f615c114ca0 con 0x7f615c101570 2024-08-20T03:04:27.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f61642f4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f615c114b20 con 0x7f615c10f120 2024-08-20T03:04:27.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f61642f4640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f615c114e20 con 0x7f615c101e90 2024-08-20T03:04:27.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f616188f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f615c101e90 0x7f615c19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f6162090640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f615c101570 0x7f615c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f616188f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f615c101e90 0x7f615c19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:60188/0 (socket says 172.21.15.44:60188) 2024-08-20T03:04:27.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f616188f640 1 -- 172.21.15.44:0/3695236454 learned_addr learned my addr 172.21.15.44:0/3695236454 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:27.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f6162891640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f615c10f120 0x7f615c1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f616188f640 1 -- 172.21.15.44:0/3695236454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f615c10f120 msgr2=0x7f615c1a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:27.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f616188f640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f615c10f120 0x7f615c1a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f616188f640 1 -- 172.21.15.44:0/3695236454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f615c101570 msgr2=0x7f615c19b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:27.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f616188f640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f615c101570 0x7f615c19b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.111+0000 7f616188f640 1 -- 172.21.15.44:0/3695236454 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615c1a1280 con 0x7f615c101e90 2024-08-20T03:04:27.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.112+0000 7f616188f640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f615c101e90 0x7f615c19b710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f614c00e9f0 tx=0x7f614c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:27.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.112+0000 7f61537fe640 1 -- 172.21.15.44:0/3695236454 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c00cd80 con 0x7f615c101e90 2024-08-20T03:04:27.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.112+0000 7f61537fe640 1 -- 172.21.15.44:0/3695236454 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f614c0054d0 con 0x7f615c101e90 2024-08-20T03:04:27.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.112+0000 7f61642f4640 1 -- 172.21.15.44:0/3695236454 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615c1a1570 con 0x7f615c101e90 2024-08-20T03:04:27.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.112+0000 7f61537fe640 1 -- 172.21.15.44:0/3695236454 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c010690 con 0x7f615c101e90 2024-08-20T03:04:27.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.112+0000 7f61642f4640 1 -- 172.21.15.44:0/3695236454 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615c1a7c30 con 0x7f615c101e90 2024-08-20T03:04:27.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.115+0000 7f61666ed740 1 Processor -- start 2024-08-20T03:04:27.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.115+0000 7f61666ed740 1 -- start start 2024-08-20T03:04:27.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.115+0000 7f61666ed740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5c5880 0x556c4a5c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f61666ed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f61666ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5d2130 0x556c4a5d4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f61666ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556c4a53c930 con 0x556c4a5d2130 2024-08-20T03:04:27.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f61666ed740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556c4a52be30 con 0x556c4a5c5880 2024-08-20T03:04:27.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f61666ed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556c4a52fe70 con 0x556c4a5c61a0 2024-08-20T03:04:27.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f6162090640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f6162090640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:60194/0 (socket says 172.21.15.44:60194) 2024-08-20T03:04:27.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f6162090640 1 -- 172.21.15.44:0/3415862960 learned_addr learned my addr 172.21.15.44:0/3415862960 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:27.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f61537fe640 1 -- 172.21.15.44:0/3695236454 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f614c004e90 con 0x7f615c101e90 2024-08-20T03:04:27.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f6162891640 1 --2- 172.21.15.44:0/3415862960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5c5880 0x556c4a5c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f6162090640 1 -- 172.21.15.44:0/3415862960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5c5880 msgr2=0x556c4a5c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:27.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f6162090640 1 --2- 172.21.15.44:0/3415862960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5c5880 0x556c4a5c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f6162090640 1 -- 172.21.15.44:0/3415862960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5d2130 msgr2=0x556c4a5d4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:27.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f616188f640 1 --2- 172.21.15.44:0/3415862960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5d2130 0x556c4a5d4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.116+0000 7f6162090640 1 --2- 172.21.15.44:0/3415862960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5d2130 0x556c4a5d4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.117+0000 7f6162090640 1 -- 172.21.15.44:0/3415862960 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4a28b650 con 0x556c4a5c61a0 2024-08-20T03:04:27.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.117+0000 7f616188f640 1 --2- 172.21.15.44:0/3415862960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5d2130 0x556c4a5d4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:04:27.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.117+0000 7f6162090640 1 --2- 172.21.15.44:0/3415862960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5d1bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6158009d20 tx=0x7f6158011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f014dbbc12924ce server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:27.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.117+0000 7f6150de6640 1 -- 172.21.15.44:0/3415862960 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158004230 con 0x556c4a5c61a0 2024-08-20T03:04:27.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.117+0000 7f6150de6640 1 -- 172.21.15.44:0/3415862960 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61580043d0 con 0x556c4a5c61a0 2024-08-20T03:04:27.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.117+0000 7f6150de6640 1 -- 172.21.15.44:0/3415862960 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61580198c0 con 0x556c4a5c61a0 2024-08-20T03:04:27.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.117+0000 7f61537fe640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f61480b3dc0 0x7f61480b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.118+0000 7f6162891640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f61480b3dc0 0x7f61480b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.118+0000 7f61537fe640 1 -- 172.21.15.44:0/3695236454 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f614c0e4370 con 0x7f615c101e90 2024-08-20T03:04:27.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.118+0000 7f61666ed740 1 -- 172.21.15.44:0/3415862960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 msgr2=0x556c4a5d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:27.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.118+0000 7f61666ed740 1 --2- 172.21.15.44:0/3415862960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5d1bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6158009d20 tx=0x7f6158011810 comp rx=0 tx=0).stop 2024-08-20T03:04:27.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.118+0000 7f6162891640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f61480b3dc0 0x7f61480b6280 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6154005e30 tx=0x7f6154005c50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:27.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.118+0000 7f61666ed740 1 -- 172.21.15.44:0/3415862960 shutdown_connections 2024-08-20T03:04:27.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.118+0000 7f61666ed740 1 --2- 172.21.15.44:0/3415862960 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5d2130 0x556c4a5d4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.118+0000 7f61666ed740 1 --2- 172.21.15.44:0/3415862960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5d1bf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.118+0000 7f61666ed740 1 --2- 172.21.15.44:0/3415862960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5c5880 0x556c4a5c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.118+0000 7f61666ed740 1 -- 172.21.15.44:0/3415862960 >> 172.21.15.44:0/3415862960 conn(0x556c4a5c36b0 msgr2=0x556c4a5c4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:27.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.119+0000 7f61666ed740 1 -- 172.21.15.44:0/3415862960 shutdown_connections 2024-08-20T03:04:27.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.119+0000 7f61666ed740 1 -- 172.21.15.44:0/3415862960 wait complete. 2024-08-20T03:04:27.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.119+0000 7f61666ed740 1 -- start start 2024-08-20T03:04:27.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.119+0000 7f61666ed740 1 Processor -- start 2024-08-20T03:04:27.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.119+0000 7f61666ed740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5c5880 0x556c4a5cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.119+0000 7f61666ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c61a0 0x556c4a5cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.119+0000 7f61666ed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5d2130 0x556c4a5cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.119+0000 7f61666ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556c4a38e700 con 0x556c4a5c61a0 2024-08-20T03:04:27.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.119+0000 7f61666ed740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556c4a4b2810 con 0x556c4a5c5880 2024-08-20T03:04:27.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.119+0000 7f61666ed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556c4a5d4de0 con 0x556c4a5d2130 2024-08-20T03:04:27.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.119+0000 7f61666ed740 10 client.0 _ll_register_callbacks cb 0x556c4a525700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:27.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f616188f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c61a0 0x556c4a5cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f6162090640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5c5880 0x556c4a5cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f616188f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c61a0 0x556c4a5cf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52714/0 (socket says 172.21.15.44:52714) 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f616188f640 1 -- 172.21.15.44:0/1999839779 learned_addr learned my addr 172.21.15.44:0/1999839779 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f6162891640 1 --2- 172.21.15.44:0/1999839779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5d2130 0x556c4a5cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f6162090640 1 -- 172.21.15.44:0/1999839779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5d2130 msgr2=0x556c4a5cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f6162090640 1 --2- 172.21.15.44:0/1999839779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5d2130 0x556c4a5cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f6162090640 1 -- 172.21.15.44:0/1999839779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c61a0 msgr2=0x556c4a5cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f6162090640 1 --2- 172.21.15.44:0/1999839779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c61a0 0x556c4a5cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f6162090640 1 -- 172.21.15.44:0/1999839779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c4a5de160 con 0x556c4a5c5880 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f6162090640 1 --2- 172.21.15.44:0/1999839779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5c5880 0x556c4a5cf090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x556c4a5c8a00 tx=0x7f6158003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8757c1bd87844fa6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f6145ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.120+0000 7f6145ffb640 1 -- 172.21.15.44:0/1999839779 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158027070 con 0x556c4a5c5880 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f6145ffb640 1 -- 172.21.15.44:0/1999839779 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6158005240 con 0x556c4a5c5880 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f61666ed740 1 -- 172.21.15.44:0/1999839779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556c4a4db150 con 0x556c4a5c5880 2024-08-20T03:04:27.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f6145ffb640 1 -- 172.21.15.44:0/1999839779 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6158033050 con 0x556c4a5c5880 2024-08-20T03:04:27.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f61666ed740 10 client.25688 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:27.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f61666ed740 1 -- 172.21.15.44:0/1999839779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c4a5e0a80 con 0x556c4a5c5880 2024-08-20T03:04:27.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f6145ffb640 1 -- 172.21.15.44:0/1999839779 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556c4a4db150 con 0x556c4a5c5880 2024-08-20T03:04:27.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f6145ffb640 1 -- 172.21.15.44:0/1999839779 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6158005460 con 0x556c4a5c5880 2024-08-20T03:04:27.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f61666ed740 10 client.25688 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:27.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f61666ed740 10 client.25688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:27.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f61666ed740 20 client.25688 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:27.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f61666ed740 1 --2- 172.21.15.44:0/1999839779 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556c4a5e1e90 0x556c4a5e4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f61666ed740 4 client.25688 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:04:27.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f61666ed740 1 -- 172.21.15.44:0/1999839779 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c4a53e880 con 0x556c4a5e1e90 2024-08-20T03:04:27.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.121+0000 7f616188f640 1 --2- 172.21.15.44:0/1999839779 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556c4a5e1e90 0x556c4a5e4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.122+0000 7f6145ffb640 10 client.25688 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:27.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.122+0000 7f616188f640 1 --2- 172.21.15.44:0/1999839779 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556c4a5e1e90 0x556c4a5e4280 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fef2a4a1dc8c8a2 server_cookie=10ff87ebac6b1fa9 in_seq=0 out_seq=0 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f6145ffb640 1 -- 172.21.15.44:0/1999839779 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c4a53e880 con 0x556c4a5e1e90 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f6145ffb640 10 client.25688 handle_command_reply: tid=0 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f61666ed740 1 client.25688 shutdown 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f61666ed740 1 -- 172.21.15.44:0/1999839779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5c5880 msgr2=0x556c4a5cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f61666ed740 1 --2- 172.21.15.44:0/1999839779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5c5880 0x556c4a5cf090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x556c4a5c8a00 tx=0x7f6158003820 comp rx=0 tx=0).stop 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f61666ed740 1 -- 172.21.15.44:0/1999839779 shutdown_connections 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f61666ed740 1 --2- 172.21.15.44:0/1999839779 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556c4a5e1e90 0x556c4a5e4280 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f61666ed740 1 --2- 172.21.15.44:0/1999839779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5d2130 0x556c4a5cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f61666ed740 1 --2- 172.21.15.44:0/1999839779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c61a0 0x556c4a5cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f6145ffb640 0 client.25688 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f61666ed740 1 --2- 172.21.15.44:0/1999839779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5c5880 0x556c4a5cf090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f61666ed740 1 -- 172.21.15.44:0/1999839779 >> 172.21.15.44:0/1999839779 conn(0x556c4a5c36b0 msgr2=0x556c4a5c9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:27.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.128+0000 7f61666ed740 1 -- 172.21.15.44:0/1999839779 shutdown_connections 2024-08-20T03:04:27.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.129+0000 7f61666ed740 1 -- 172.21.15.44:0/1999839779 wait complete. 2024-08-20T03:04:27.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.129+0000 7f61666ed740 20 client.25688 trim_cache size 0 max 16384 2024-08-20T03:04:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.157+0000 7f61666ed740 1 Processor -- start 2024-08-20T03:04:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.157+0000 7f61666ed740 1 -- start start 2024-08-20T03:04:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.158+0000 7f61666ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 0x556c4a5cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.158+0000 7f61666ed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.158+0000 7f61666ed740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5d2130 0x556c4a5e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.158+0000 7f61666ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1b0520 con 0x556c4a5c5880 2024-08-20T03:04:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.158+0000 7f61666ed740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1b2120 con 0x556c4a5d2130 2024-08-20T03:04:27.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.158+0000 7f61666ed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1b0bf0 con 0x556c4a5c61a0 2024-08-20T03:04:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.159+0000 7f6162090640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 0x556c4a5cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.159+0000 7f6162090640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 0x556c4a5cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52724/0 (socket says 172.21.15.44:52724) 2024-08-20T03:04:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.159+0000 7f6162090640 1 -- 172.21.15.44:0/2182073414 learned_addr learned my addr 172.21.15.44:0/2182073414 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.159+0000 7f6162090640 1 -- 172.21.15.44:0/2182073414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 msgr2=0x556c4a5cfad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:04:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.159+0000 7f6162090640 1 --2- 172.21.15.44:0/2182073414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5cfad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.159+0000 7f6162090640 1 -- 172.21.15.44:0/2182073414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5d2130 msgr2=0x556c4a5e9b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:04:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.159+0000 7f6162090640 1 --2- 172.21.15.44:0/2182073414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5d2130 0x556c4a5e9b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.159+0000 7f6162090640 1 -- 172.21.15.44:0/2182073414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4a5c5fd0 con 0x556c4a5c5880 2024-08-20T03:04:27.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.160+0000 7f6162090640 1 --2- 172.21.15.44:0/2182073414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 0x556c4a5cf590 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6158004ed0 tx=0x7f6158009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb62a98fa76625e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.161+0000 7f6147fff640 1 -- 172.21.15.44:0/2182073414 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158030020 con 0x556c4a5c5880 2024-08-20T03:04:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.161+0000 7f6147fff640 1 -- 172.21.15.44:0/2182073414 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6158019490 con 0x556c4a5c5880 2024-08-20T03:04:27.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.161+0000 7f6147fff640 1 -- 172.21.15.44:0/2182073414 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6158027070 con 0x556c4a5c5880 2024-08-20T03:04:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.165+0000 7f61666ed740 1 -- 172.21.15.44:0/2182073414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 msgr2=0x556c4a5cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:27.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.165+0000 7f61666ed740 1 --2- 172.21.15.44:0/2182073414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 0x556c4a5cf590 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6158004ed0 tx=0x7f6158009640 comp rx=0 tx=0).stop 2024-08-20T03:04:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.170+0000 7f61666ed740 1 -- 172.21.15.44:0/2182073414 shutdown_connections 2024-08-20T03:04:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.170+0000 7f61666ed740 1 --2- 172.21.15.44:0/2182073414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5d2130 0x556c4a5e9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.170+0000 7f61666ed740 1 --2- 172.21.15.44:0/2182073414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.170+0000 7f61666ed740 1 --2- 172.21.15.44:0/2182073414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 0x556c4a5cf590 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.170+0000 7f61666ed740 1 -- 172.21.15.44:0/2182073414 >> 172.21.15.44:0/2182073414 conn(0x556c4a5c36b0 msgr2=0x556c4a5d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.176+0000 7f61666ed740 1 -- 172.21.15.44:0/2182073414 shutdown_connections 2024-08-20T03:04:27.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.176+0000 7f61666ed740 1 -- 172.21.15.44:0/2182073414 wait complete. 2024-08-20T03:04:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.176+0000 7f61666ed740 1 -- start start 2024-08-20T03:04:27.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.177+0000 7f61666ed740 1 Processor -- start 2024-08-20T03:04:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.177+0000 7f61666ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 0x556c4a5d6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.178+0000 7f61666ed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5d6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.178+0000 7f6162090640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 0x556c4a5d6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.178+0000 7f6162090640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 0x556c4a5d6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:52738/0 (socket says 172.21.15.44:52738) 2024-08-20T03:04:27.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.178+0000 7f6162090640 1 -- 172.21.15.44:0/1036446280 learned_addr learned my addr 172.21.15.44:0/1036446280 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:27.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.178+0000 7f61666ed740 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5d2130 0x556c4a5d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.178+0000 7f61666ed740 1 -- 172.21.15.44:0/1036446280 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1b1990 con 0x556c4a5c5880 2024-08-20T03:04:27.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.178+0000 7f616188f640 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5d6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.178+0000 7f61666ed740 1 -- 172.21.15.44:0/1036446280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1b12c0 con 0x556c4a5d2130 2024-08-20T03:04:27.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.178+0000 7f61666ed740 1 -- 172.21.15.44:0/1036446280 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f615c1ae500 con 0x556c4a5c61a0 2024-08-20T03:04:27.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.179+0000 7f61666ed740 10 client.0 _ll_register_callbacks cb 0x7f6158009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.179+0000 7f6162891640 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5d2130 0x556c4a5d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.180+0000 7f6162891640 1 -- 172.21.15.44:0/1036446280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 msgr2=0x556c4a5d6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.180+0000 7f6162891640 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5d6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.180+0000 7f6162891640 1 -- 172.21.15.44:0/1036446280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 msgr2=0x556c4a5d6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.180+0000 7f6162891640 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 0x556c4a5d6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.180+0000 7f6162891640 1 -- 172.21.15.44:0/1036446280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c4a6534f0 con 0x556c4a5d2130 2024-08-20T03:04:27.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.181+0000 7f6162090640 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 0x556c4a5d6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:04:27.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.181+0000 7f616188f640 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5d6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:04:27.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.181+0000 7f6162891640 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5d2130 0x556c4a5d7090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6154005cc0 tx=0x7f61540095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ae57d0bd554ff5d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:27.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.181+0000 7f6145ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:04:27.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.183+0000 7f6145ffb640 1 -- 172.21.15.44:0/1036446280 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154034040 con 0x556c4a5d2130 2024-08-20T03:04:27.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.183+0000 7f6145ffb640 1 -- 172.21.15.44:0/1036446280 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6154030440 con 0x556c4a5d2130 2024-08-20T03:04:27.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.184+0000 7f61666ed740 1 -- 172.21.15.44:0/1036446280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6158019490 con 0x556c4a5d2130 2024-08-20T03:04:27.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.187+0000 7f6145ffb640 1 -- 172.21.15.44:0/1036446280 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f615402f680 con 0x556c4a5d2130 2024-08-20T03:04:27.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.188+0000 7f61666ed740 10 client.25691 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:27.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.188+0000 7f61666ed740 1 -- 172.21.15.44:0/1036446280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c4a6547f0 con 0x556c4a5d2130 2024-08-20T03:04:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.190+0000 7f6145ffb640 1 -- 172.21.15.44:0/1036446280 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6154030af0 con 0x556c4a5d2130 2024-08-20T03:04:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.190+0000 7f6145ffb640 1 -- 172.21.15.44:0/1036446280 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f615405da50 con 0x556c4a5d2130 2024-08-20T03:04:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.193+0000 7f61666ed740 10 client.25691 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.193+0000 7f61666ed740 10 client.25691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.193+0000 7f61666ed740 20 client.25691 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.193+0000 7f61666ed740 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556c4a655010 0x556c4a657430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.193+0000 7f61666ed740 4 client.25691 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:04:27.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.193+0000 7f61666ed740 1 -- 172.21.15.44:0/1036446280 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6158019490 con 0x556c4a655010 2024-08-20T03:04:27.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.193+0000 7f6162090640 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556c4a655010 0x556c4a657430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:27.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.198+0000 7f6162090640 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556c4a655010 0x556c4a657430 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ad3d3f406da751 server_cookie=e78d14502031d71 in_seq=0 out_seq=0 2024-08-20T03:04:27.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.198+0000 7f6145ffb640 10 client.25691 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:27.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.203+0000 7f6145ffb640 1 -- 172.21.15.44:0/1036446280 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6158019490 con 0x556c4a655010 2024-08-20T03:04:27.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.203+0000 7f6145ffb640 10 client.25691 handle_command_reply: tid=0 2024-08-20T03:04:27.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.203+0000 7f61666ed740 1 client.25691 shutdown 2024-08-20T03:04:27.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.204+0000 7f61666ed740 1 -- 172.21.15.44:0/1036446280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5d2130 msgr2=0x556c4a5d7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:27.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.204+0000 7f61666ed740 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5d2130 0x556c4a5d7090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6154005cc0 tx=0x7f61540095c0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.205+0000 7f61666ed740 1 -- 172.21.15.44:0/1036446280 shutdown_connections 2024-08-20T03:04:27.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.205+0000 7f61666ed740 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556c4a655010 0x556c4a657430 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.205+0000 7f61666ed740 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556c4a5d2130 0x556c4a5d7090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.205+0000 7f61666ed740 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556c4a5c61a0 0x556c4a5d6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.205+0000 7f61666ed740 1 --2- 172.21.15.44:0/1036446280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c4a5c5880 0x556c4a5d6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.205+0000 7f61666ed740 1 -- 172.21.15.44:0/1036446280 >> 172.21.15.44:0/1036446280 conn(0x556c4a5c36b0 msgr2=0x556c4a5c1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:27.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.205+0000 7f6145ffb640 0 client.25691 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:27.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.205+0000 7f61666ed740 1 -- 172.21.15.44:0/1036446280 shutdown_connections 2024-08-20T03:04:27.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.206+0000 7f61666ed740 1 -- 172.21.15.44:0/1036446280 wait complete. 2024-08-20T03:04:27.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.206+0000 7f61666ed740 20 client.25691 trim_cache size 0 max 16384 2024-08-20T03:04:27.210 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:04:27.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.208+0000 7f61642f4640 1 -- 172.21.15.44:0/3695236454 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f61480b3dc0 msgr2=0x7f61480b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:27.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.208+0000 7f61642f4640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f61480b3dc0 0x7f61480b6280 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6154005e30 tx=0x7f6154005c50 comp rx=0 tx=0).stop 2024-08-20T03:04:27.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.208+0000 7f61642f4640 1 -- 172.21.15.44:0/3695236454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f615c101e90 msgr2=0x7f615c19b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:27.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.208+0000 7f61642f4640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f615c101e90 0x7f615c19b710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f614c00e9f0 tx=0x7f614c00edf0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.208+0000 7f61642f4640 1 -- 172.21.15.44:0/3695236454 shutdown_connections 2024-08-20T03:04:27.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.208+0000 7f61642f4640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f61480b3dc0 0x7f61480b6280 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.208+0000 7f61642f4640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f615c10f120 0x7f615c1a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.208+0000 7f61642f4640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f615c101e90 0x7f615c19b710 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.208+0000 7f61642f4640 1 --2- 172.21.15.44:0/3695236454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f615c101570 0x7f615c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:27.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.208+0000 7f61642f4640 1 -- 172.21.15.44:0/3695236454 >> 172.21.15.44:0/3695236454 conn(0x7f615c0fd460 msgr2=0x7f615c111320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:27.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.209+0000 7f61642f4640 1 -- 172.21.15.44:0/3695236454 shutdown_connections 2024-08-20T03:04:27.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:27.209+0000 7f61642f4640 1 -- 172.21.15.44:0/3695236454 wait complete. 2024-08-20T03:04:27.476 INFO:tasks.workunit.client.0.smithi044.stdout: 67 1259877 550 871274 513 669602 1314 2024-08-20T03:04:28.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:28 smithi165 ceph-mon[22357]: pgmap v379: 97 pgs: 97 active+clean; 156 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 782 op/s 2024-08-20T03:04:28.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:28 smithi099 ceph-mon[21984]: pgmap v379: 97 pgs: 97 active+clean; 156 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 782 op/s 2024-08-20T03:04:28.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:28 smithi044 ceph-mon[18790]: pgmap v379: 97 pgs: 97 active+clean; 156 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 782 op/s 2024-08-20T03:04:29.220 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T03:04:29.221 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T03:04:29.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:29 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:04:29.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:29 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:04:29.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:29 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:04:29.919 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:04:30.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:30 smithi044 ceph-mon[18790]: pgmap v380: 97 pgs: 97 active+clean; 161 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 802 op/s 2024-08-20T03:04:30.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:30 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:04:30.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:30 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:04:30.376 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:30 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:04:30.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:30 smithi165 ceph-mon[22357]: pgmap v380: 97 pgs: 97 active+clean; 161 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 802 op/s 2024-08-20T03:04:30.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:30 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:04:30.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:30 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:04:30.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:30 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:04:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:30 smithi099 ceph-mon[21984]: pgmap v380: 97 pgs: 97 active+clean; 161 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 802 op/s 2024-08-20T03:04:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:30 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:04:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:30 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:04:30.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:30 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:04:30.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.530+0000 7f529aabd640 1 -- 172.21.15.44:0/187192704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5294102920 msgr2=0x7f5294102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:30.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.531+0000 7f529aabd640 1 --2- 172.21.15.44:0/187192704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5294102920 0x7f5294102d00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5290009900 tx=0x7f529002a400 comp rx=0 tx=0).stop 2024-08-20T03:04:30.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.531+0000 7f529aabd640 1 -- 172.21.15.44:0/187192704 shutdown_connections 2024-08-20T03:04:30.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.531+0000 7f529aabd640 1 --2- 172.21.15.44:0/187192704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5294108cb0 0x7f529410f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.532+0000 7f529aabd640 1 --2- 172.21.15.44:0/187192704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5294108330 0x7f5294108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.532+0000 7f529aabd640 1 --2- 172.21.15.44:0/187192704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5294102920 0x7f5294102d00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.532+0000 7f529aabd640 1 -- 172.21.15.44:0/187192704 >> 172.21.15.44:0/187192704 conn(0x7f52940fe350 msgr2=0x7f5294100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:30.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.532+0000 7f529aabd640 1 -- 172.21.15.44:0/187192704 shutdown_connections 2024-08-20T03:04:30.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.532+0000 7f529aabd640 1 -- 172.21.15.44:0/187192704 wait complete. 2024-08-20T03:04:30.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.533+0000 7f529aabd640 1 Processor -- start 2024-08-20T03:04:30.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.534+0000 7f529aabd640 1 -- start start 2024-08-20T03:04:30.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.534+0000 7f529aabd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5294102920 0x7f5294198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.534+0000 7f529aabd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5294108330 0x7f52941990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.534+0000 7f529aabd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5294108cb0 0x7f529419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.535+0000 7f529aabd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5294112980 con 0x7f5294108cb0 2024-08-20T03:04:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.535+0000 7f529aabd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5294112800 con 0x7f5294108330 2024-08-20T03:04:30.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.535+0000 7f529aabd640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5294112b00 con 0x7f5294102920 2024-08-20T03:04:30.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.535+0000 7f529a2bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5294108cb0 0x7f529419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.535+0000 7f529a2bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5294108cb0 0x7f529419e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60248/0 (socket says 172.21.15.44:60248) 2024-08-20T03:04:30.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.535+0000 7f529a2bc640 1 -- 172.21.15.44:0/3570995075 learned_addr learned my addr 172.21.15.44:0/3570995075 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:30.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.536+0000 7f52992ba640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5294108330 0x7f52941990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.536+0000 7f529a2bc640 1 -- 172.21.15.44:0/3570995075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5294102920 msgr2=0x7f5294198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:30.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.536+0000 7f529a2bc640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5294102920 0x7f5294198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.536+0000 7f529a2bc640 1 -- 172.21.15.44:0/3570995075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5294108330 msgr2=0x7f52941990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:30.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.536+0000 7f529a2bc640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5294108330 0x7f52941990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.536+0000 7f529a2bc640 1 -- 172.21.15.44:0/3570995075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529419ec10 con 0x7f5294108cb0 2024-08-20T03:04:30.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.537+0000 7f529a2bc640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5294108cb0 0x7f529419e440 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f528c00eaa0 tx=0x7f528c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:30.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.537+0000 7f528affd640 1 -- 172.21.15.44:0/3570995075 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c00cdb0 con 0x7f5294108cb0 2024-08-20T03:04:30.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.537+0000 7f529aabd640 1 -- 172.21.15.44:0/3570995075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529419ef00 con 0x7f5294108cb0 2024-08-20T03:04:30.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.537+0000 7f529aabd640 1 -- 172.21.15.44:0/3570995075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52941a55c0 con 0x7f5294108cb0 2024-08-20T03:04:30.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.540+0000 7f528affd640 1 -- 172.21.15.44:0/3570995075 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f528c005020 con 0x7f5294108cb0 2024-08-20T03:04:30.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.540+0000 7f528affd640 1 -- 172.21.15.44:0/3570995075 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c0116d0 con 0x7f5294108cb0 2024-08-20T03:04:30.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.541+0000 7f529e118740 1 Processor -- start 2024-08-20T03:04:30.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.541+0000 7f529e118740 1 -- start start 2024-08-20T03:04:30.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.542+0000 7f529e118740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3550 0x555668ee3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.542+0000 7f529e118740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee3e70 0x555668eef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.542+0000 7f529e118740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668eefe00 0x555668ef21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.542+0000 7f529e118740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555668e59940 con 0x555668ee3e70 2024-08-20T03:04:30.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.542+0000 7f529e118740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555668e48e40 con 0x555668eefe00 2024-08-20T03:04:30.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.542+0000 7f529e118740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555668e4ce80 con 0x555668ee3550 2024-08-20T03:04:30.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.542+0000 7f5299abb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee3e70 0x555668eef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.542+0000 7f52992ba640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3550 0x555668ee3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.542+0000 7f52992ba640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3550 0x555668ee3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:50402/0 (socket says 172.21.15.44:50402) 2024-08-20T03:04:30.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.542+0000 7f52992ba640 1 -- 172.21.15.44:0/1811433297 learned_addr learned my addr 172.21.15.44:0/1811433297 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:30.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.542+0000 7f52992ba640 1 --2- 172.21.15.44:0/1811433297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668eefe00 0x555668ef21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.543+0000 7f52992ba640 1 -- 172.21.15.44:0/1811433297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668eefe00 msgr2=0x555668ef21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:30.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.543+0000 7f52992ba640 1 --2- 172.21.15.44:0/1811433297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668eefe00 0x555668ef21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.543+0000 7f52992ba640 1 -- 172.21.15.44:0/1811433297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee3e70 msgr2=0x555668eef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:30.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.543+0000 7f52992ba640 1 --2- 172.21.15.44:0/1811433297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee3e70 0x555668eef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.543+0000 7f52992ba640 1 -- 172.21.15.44:0/1811433297 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555668de5520 con 0x555668ee3550 2024-08-20T03:04:30.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.546+0000 7f528affd640 1 -- 172.21.15.44:0/3570995075 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f528c004a40 con 0x7f5294108cb0 2024-08-20T03:04:30.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.547+0000 7f52992ba640 1 --2- 172.21.15.44:0/1811433297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3550 0x555668ee3930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5284009530 tx=0x7f52840097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5a207f1aaf031e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:30.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.548+0000 7f528affd640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f52800b3f30 0x7f52800b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.548+0000 7f528affd640 1 -- 172.21.15.44:0/3570995075 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f528c0e4140 con 0x7f5294108cb0 2024-08-20T03:04:30.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.549+0000 7f527ffff640 1 -- 172.21.15.44:0/1811433297 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284028070 con 0x555668ee3550 2024-08-20T03:04:30.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.549+0000 7f527ffff640 1 -- 172.21.15.44:0/1811433297 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f528400f760 con 0x555668ee3550 2024-08-20T03:04:30.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.549+0000 7f527ffff640 1 -- 172.21.15.44:0/1811433297 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52840116d0 con 0x555668ee3550 2024-08-20T03:04:30.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.551+0000 7f5299abb640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f52800b3f30 0x7f52800b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.554+0000 7f529e118740 1 -- 172.21.15.44:0/1811433297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3550 msgr2=0x555668ee3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:30.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.554+0000 7f529e118740 1 --2- 172.21.15.44:0/1811433297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3550 0x555668ee3930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5284009530 tx=0x7f52840097b0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.554+0000 7f5299abb640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f52800b3f30 0x7f52800b63f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f52900096c0 tx=0x7f529000b040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:30.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.555+0000 7f529e118740 1 -- 172.21.15.44:0/1811433297 shutdown_connections 2024-08-20T03:04:30.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.555+0000 7f529e118740 1 --2- 172.21.15.44:0/1811433297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668eefe00 0x555668ef21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.555+0000 7f529e118740 1 --2- 172.21.15.44:0/1811433297 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee3e70 0x555668eef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.555+0000 7f529e118740 1 --2- 172.21.15.44:0/1811433297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3550 0x555668ee3930 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5284009530 tx=0x7f52840097b0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.555+0000 7f529e118740 1 -- 172.21.15.44:0/1811433297 >> 172.21.15.44:0/1811433297 conn(0x555668ee1380 msgr2=0x555668ee2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.557+0000 7f529e118740 1 -- 172.21.15.44:0/1811433297 shutdown_connections 2024-08-20T03:04:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.557+0000 7f529e118740 1 -- 172.21.15.44:0/1811433297 wait complete. 2024-08-20T03:04:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.558+0000 7f529e118740 1 -- start start 2024-08-20T03:04:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.558+0000 7f529e118740 1 Processor -- start 2024-08-20T03:04:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.558+0000 7f529e118740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3e70 0x555668ee9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.558+0000 7f529e118740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668eefe00 0x555668ee96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.559+0000 7f529e118740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee9d90 0x555668eee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.559+0000 7f5299abb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3e70 0x555668ee9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.559+0000 7f5299abb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3e70 0x555668ee9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:50418/0 (socket says 172.21.15.44:50418) 2024-08-20T03:04:30.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.559+0000 7f5299abb640 1 -- 172.21.15.44:0/1032648412 learned_addr learned my addr 172.21.15.44:0/1032648412 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:30.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.559+0000 7f52992ba640 1 --2- 172.21.15.44:0/1032648412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668eefe00 0x555668ee96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.559+0000 7f529e118740 1 -- 172.21.15.44:0/1032648412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555668cab890 con 0x555668ee9d90 2024-08-20T03:04:30.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.559+0000 7f529e118740 1 -- 172.21.15.44:0/1032648412 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555668dcf700 con 0x555668eefe00 2024-08-20T03:04:30.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.559+0000 7f529e118740 1 -- 172.21.15.44:0/1032648412 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555668ef29b0 con 0x555668ee3e70 2024-08-20T03:04:30.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.559+0000 7f529e118740 10 client.0 _ll_register_callbacks cb 0x555668e446f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:30.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.559+0000 7f5299abb640 1 -- 172.21.15.44:0/1032648412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668eefe00 msgr2=0x555668ee96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:30.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.559+0000 7f5299abb640 1 --2- 172.21.15.44:0/1032648412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668eefe00 0x555668ee96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.559+0000 7f5299abb640 1 -- 172.21.15.44:0/1032648412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee9d90 msgr2=0x555668eee9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:30.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.560+0000 7f5299abb640 1 --2- 172.21.15.44:0/1032648412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee9d90 0x555668eee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.560+0000 7f5299abb640 1 -- 172.21.15.44:0/1032648412 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555668ba8930 con 0x555668ee3e70 2024-08-20T03:04:30.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.561+0000 7f5299abb640 1 --2- 172.21.15.44:0/1032648412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3e70 0x555668ee9160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5290033000 tx=0x7f5290031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59a79b212ed65ebf server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:30.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.561+0000 7f527d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:04:30.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.562+0000 7f527d7fa640 1 -- 172.21.15.44:0/1032648412 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529004c020 con 0x555668ee3e70 2024-08-20T03:04:30.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.562+0000 7f529e118740 1 -- 172.21.15.44:0/1032648412 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f528400f760 con 0x555668ee3e70 2024-08-20T03:04:30.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.562+0000 7f527d7fa640 1 -- 172.21.15.44:0/1032648412 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f52900328c0 con 0x555668ee3e70 2024-08-20T03:04:30.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.562+0000 7f527d7fa640 1 -- 172.21.15.44:0/1032648412 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5290040070 con 0x555668ee3e70 2024-08-20T03:04:30.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.562+0000 7f529e118740 10 client.25672 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:30.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.562+0000 7f529e118740 1 -- 172.21.15.44:0/1032648412 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555668ef8d10 con 0x555668ee3e70 2024-08-20T03:04:30.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.563+0000 7f527d7fa640 1 -- 172.21.15.44:0/1032648412 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5290041040 con 0x555668ee3e70 2024-08-20T03:04:30.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.563+0000 7f527d7fa640 1 -- 172.21.15.44:0/1032648412 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f529003a040 con 0x555668ee3e70 2024-08-20T03:04:30.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.566+0000 7f529e118740 10 client.25672 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:30.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.566+0000 7f529e118740 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:30.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.566+0000 7f529e118740 20 client.25672 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:30.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.566+0000 7f529e118740 1 --2- 172.21.15.44:0/1032648412 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x555668efa260 0x555668efc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.566+0000 7f529e118740 4 client.25672 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:04:30.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.566+0000 7f529e118740 1 -- 172.21.15.44:0/1032648412 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f528400f760 con 0x555668efa260 2024-08-20T03:04:30.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.568+0000 7f52992ba640 1 --2- 172.21.15.44:0/1032648412 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x555668efa260 0x555668efc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.573+0000 7f52992ba640 1 --2- 172.21.15.44:0/1032648412 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x555668efa260 0x555668efc720 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0fdd2598c055d1c server_cookie=24a1f7227848c059 in_seq=0 out_seq=0 2024-08-20T03:04:30.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.574+0000 7f527d7fa640 10 client.25672 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.576+0000 7f527d7fa640 1 -- 172.21.15.44:0/1032648412 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f528400f760 con 0x555668efa260 2024-08-20T03:04:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.576+0000 7f527d7fa640 10 client.25672 handle_command_reply: tid=0 2024-08-20T03:04:30.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.576+0000 7f529e118740 1 client.25672 shutdown 2024-08-20T03:04:30.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.577+0000 7f529e118740 1 -- 172.21.15.44:0/1032648412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3e70 msgr2=0x555668ee9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:30.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.577+0000 7f529e118740 1 --2- 172.21.15.44:0/1032648412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3e70 0x555668ee9160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5290033000 tx=0x7f5290031400 comp rx=0 tx=0).stop 2024-08-20T03:04:30.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.577+0000 7f529e118740 1 -- 172.21.15.44:0/1032648412 shutdown_connections 2024-08-20T03:04:30.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.577+0000 7f529e118740 1 --2- 172.21.15.44:0/1032648412 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x555668efa260 0x555668efc720 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.577+0000 7f529e118740 1 --2- 172.21.15.44:0/1032648412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee9d90 0x555668eee9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.577+0000 7f529e118740 1 --2- 172.21.15.44:0/1032648412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668eefe00 0x555668ee96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.577+0000 7f529e118740 1 --2- 172.21.15.44:0/1032648412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668ee3e70 0x555668ee9160 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.577+0000 7f529e118740 1 -- 172.21.15.44:0/1032648412 >> 172.21.15.44:0/1032648412 conn(0x555668ee1380 msgr2=0x555668ee6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:30.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.577+0000 7f527d7fa640 0 client.25672 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:30.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.579+0000 7f529e118740 1 -- 172.21.15.44:0/1032648412 shutdown_connections 2024-08-20T03:04:30.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.579+0000 7f529e118740 1 -- 172.21.15.44:0/1032648412 wait complete. 2024-08-20T03:04:30.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.580+0000 7f529e118740 20 client.25672 trim_cache size 0 max 16384 2024-08-20T03:04:30.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.620+0000 7f529e118740 1 Processor -- start 2024-08-20T03:04:30.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.621+0000 7f529e118740 1 -- start start 2024-08-20T03:04:30.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.622+0000 7f529e118740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee3e70 0x555668f0d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.622+0000 7f5299abb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee3e70 0x555668f0d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.622+0000 7f5299abb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee3e70 0x555668f0d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60276/0 (socket says 172.21.15.44:60276) 2024-08-20T03:04:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.623+0000 7f529e118740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668ee9d90 0x555668f02000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.623+0000 7f5299abb640 1 -- 172.21.15.44:0/3188959341 learned_addr learned my addr 172.21.15.44:0/3188959341 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.623+0000 7f529e118740 1 --2- 172.21.15.44:0/3188959341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668eefe00 0x555668f02540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.626 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.623+0000 7f52992ba640 1 --2- 172.21.15.44:0/3188959341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668ee9d90 0x555668f02000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.623+0000 7f529e118740 1 -- 172.21.15.44:0/3188959341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f527800e5f0 con 0x555668ee3e70 2024-08-20T03:04:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.623+0000 7f529e118740 1 -- 172.21.15.44:0/3188959341 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f52780101f0 con 0x555668ee9d90 2024-08-20T03:04:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.623+0000 7f529e118740 1 -- 172.21.15.44:0/3188959341 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f527800ecc0 con 0x555668eefe00 2024-08-20T03:04:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.624+0000 7f529a2bc640 1 --2- 172.21.15.44:0/3188959341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668eefe00 0x555668f02540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.624+0000 7f529a2bc640 1 -- 172.21.15.44:0/3188959341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668ee9d90 msgr2=0x555668f02000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.624+0000 7f529a2bc640 1 --2- 172.21.15.44:0/3188959341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668ee9d90 0x555668f02000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.624+0000 7f529a2bc640 1 -- 172.21.15.44:0/3188959341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee3e70 msgr2=0x555668f0d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.624+0000 7f529a2bc640 1 --2- 172.21.15.44:0/3188959341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee3e70 0x555668f0d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.624+0000 7f529a2bc640 1 -- 172.21.15.44:0/3188959341 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555668ee3ca0 con 0x555668eefe00 2024-08-20T03:04:30.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.625+0000 7f529a2bc640 1 --2- 172.21.15.44:0/3188959341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668eefe00 0x555668f02540 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f528c0e18a0 tx=0x7f528c0ab460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b764daaf9f258d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:30.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.626+0000 7f527f7fe640 1 -- 172.21.15.44:0/3188959341 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c01f040 con 0x555668eefe00 2024-08-20T03:04:30.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.626+0000 7f527f7fe640 1 -- 172.21.15.44:0/3188959341 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f528c0b3e10 con 0x555668eefe00 2024-08-20T03:04:30.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.626+0000 7f527f7fe640 1 -- 172.21.15.44:0/3188959341 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c0a66d0 con 0x555668eefe00 2024-08-20T03:04:30.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.627+0000 7f529e118740 1 -- 172.21.15.44:0/3188959341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668eefe00 msgr2=0x555668f02540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:30.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.627+0000 7f529e118740 1 --2- 172.21.15.44:0/3188959341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668eefe00 0x555668f02540 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f528c0e18a0 tx=0x7f528c0ab460 comp rx=0 tx=0).stop 2024-08-20T03:04:30.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.628+0000 7f529e118740 1 -- 172.21.15.44:0/3188959341 shutdown_connections 2024-08-20T03:04:30.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.628+0000 7f529e118740 1 --2- 172.21.15.44:0/3188959341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668eefe00 0x555668f02540 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.628+0000 7f529e118740 1 --2- 172.21.15.44:0/3188959341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668ee9d90 0x555668f02000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.628+0000 7f529e118740 1 --2- 172.21.15.44:0/3188959341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee3e70 0x555668f0d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.628+0000 7f529e118740 1 -- 172.21.15.44:0/3188959341 >> 172.21.15.44:0/3188959341 conn(0x555668ee1380 msgr2=0x555668f0cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:30.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.630+0000 7f529e118740 1 -- 172.21.15.44:0/3188959341 shutdown_connections 2024-08-20T03:04:30.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.630+0000 7f529e118740 1 -- 172.21.15.44:0/3188959341 wait complete. 2024-08-20T03:04:30.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.631+0000 7f529e118740 1 -- start start 2024-08-20T03:04:30.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.631+0000 7f529e118740 1 Processor -- start 2024-08-20T03:04:30.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.632+0000 7f529e118740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668ee3e70 0x555668f01270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.632+0000 7f529e118740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee9d90 0x555668f017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.632+0000 7f5299abb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668ee3e70 0x555668f01270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.632+0000 7f5299abb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668ee3e70 0x555668f01270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47374/0 (socket says 172.21.15.44:47374) 2024-08-20T03:04:30.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.632+0000 7f52992ba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee9d90 0x555668f017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.632+0000 7f52992ba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee9d90 0x555668f017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60280/0 (socket says 172.21.15.44:60280) 2024-08-20T03:04:30.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.633+0000 7f5299abb640 1 -- 172.21.15.44:0/4271529663 learned_addr learned my addr 172.21.15.44:0/4271529663 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:30.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.633+0000 7f529e118740 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668eefe00 0x555668f8be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.633+0000 7f529e118740 1 -- 172.21.15.44:0/4271529663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f527800fa60 con 0x555668ee9d90 2024-08-20T03:04:30.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.633+0000 7f529a2bc640 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668eefe00 0x555668f8be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.633+0000 7f529e118740 1 -- 172.21.15.44:0/4271529663 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f527800f390 con 0x555668ee3e70 2024-08-20T03:04:30.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.633+0000 7f529e118740 1 -- 172.21.15.44:0/4271529663 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f527800c5d0 con 0x555668eefe00 2024-08-20T03:04:30.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.633+0000 7f529e118740 10 client.0 _ll_register_callbacks cb 0x7f5284005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:30.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.636+0000 7f529a2bc640 1 -- 172.21.15.44:0/4271529663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668ee3e70 msgr2=0x555668f01270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:30.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.636+0000 7f529a2bc640 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668ee3e70 0x555668f01270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.636+0000 7f529a2bc640 1 -- 172.21.15.44:0/4271529663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee9d90 msgr2=0x555668f017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:30.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.636+0000 7f529a2bc640 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee9d90 0x555668f017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.636+0000 7f529a2bc640 1 -- 172.21.15.44:0/4271529663 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555668f8c380 con 0x555668eefe00 2024-08-20T03:04:30.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.638+0000 7f529a2bc640 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668eefe00 0x555668f8be40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f528c005f00 tx=0x7f528c0aa1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6c805b090e883 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:30.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.638+0000 7f527d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:04:30.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.638+0000 7f527d7fa640 1 -- 172.21.15.44:0/4271529663 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c01f070 con 0x555668eefe00 2024-08-20T03:04:30.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.638+0000 7f527d7fa640 1 -- 172.21.15.44:0/4271529663 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f528c0a7350 con 0x555668eefe00 2024-08-20T03:04:30.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.639+0000 7f529e118740 1 -- 172.21.15.44:0/4271529663 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f528c0b3e10 con 0x555668eefe00 2024-08-20T03:04:30.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.640+0000 7f527d7fa640 1 -- 172.21.15.44:0/4271529663 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f528c0e1870 con 0x555668eefe00 2024-08-20T03:04:30.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.641+0000 7f529e118740 10 client.25678 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:30.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.641+0000 7f529e118740 1 -- 172.21.15.44:0/4271529663 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555668f8d5c0 con 0x555668eefe00 2024-08-20T03:04:30.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.642+0000 7f527d7fa640 1 -- 172.21.15.44:0/4271529663 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f528c0a68e0 con 0x555668eefe00 2024-08-20T03:04:30.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.642+0000 7f527d7fa640 1 -- 172.21.15.44:0/4271529663 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f528c00f040 con 0x555668eefe00 2024-08-20T03:04:30.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.644+0000 7f529e118740 10 client.25678 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:30.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.644+0000 7f529e118740 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:30.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.644+0000 7f529e118740 20 client.25678 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:30.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.644+0000 7f529e118740 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x555668f8dc60 0x555668f90080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:30.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.644+0000 7f529e118740 4 client.25678 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T03:04:30.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.644+0000 7f529e118740 1 -- 172.21.15.44:0/4271529663 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f528c0b3e10 con 0x555668f8dc60 2024-08-20T03:04:30.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.646+0000 7f5299abb640 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x555668f8dc60 0x555668f90080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:30.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.646+0000 7f5299abb640 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x555668f8dc60 0x555668f90080 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28f2905075f79612 server_cookie=3564e1780e8a4ec1 in_seq=0 out_seq=0 2024-08-20T03:04:30.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.648+0000 7f527d7fa640 10 client.25678 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:30.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.651+0000 7f527d7fa640 1 -- 172.21.15.44:0/4271529663 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f528c0b3e10 con 0x555668f8dc60 2024-08-20T03:04:30.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.651+0000 7f527d7fa640 10 client.25678 handle_command_reply: tid=0 2024-08-20T03:04:30.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.651+0000 7f529e118740 1 client.25678 shutdown 2024-08-20T03:04:30.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.656+0000 7f529e118740 1 -- 172.21.15.44:0/4271529663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668eefe00 msgr2=0x555668f8be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:30.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.656+0000 7f529e118740 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668eefe00 0x555668f8be40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f528c005f00 tx=0x7f528c0aa1f0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.656+0000 7f529e118740 1 -- 172.21.15.44:0/4271529663 shutdown_connections 2024-08-20T03:04:30.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.656+0000 7f529e118740 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x555668f8dc60 0x555668f90080 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.656+0000 7f529e118740 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555668eefe00 0x555668f8be40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.656+0000 7f529e118740 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555668ee9d90 0x555668f017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.656+0000 7f529e118740 1 --2- 172.21.15.44:0/4271529663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555668ee3e70 0x555668f01270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.656+0000 7f529e118740 1 -- 172.21.15.44:0/4271529663 >> 172.21.15.44:0/4271529663 conn(0x555668ee1380 msgr2=0x555668ee65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:30.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.656+0000 7f527d7fa640 0 client.25678 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:30.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.657+0000 7f529e118740 1 -- 172.21.15.44:0/4271529663 shutdown_connections 2024-08-20T03:04:30.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.658+0000 7f529e118740 1 -- 172.21.15.44:0/4271529663 wait complete. 2024-08-20T03:04:30.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.658+0000 7f529e118740 20 client.25678 trim_cache size 0 max 16384 2024-08-20T03:04:30.661 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:04:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-08-20T03:04:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "ce02f953-98f5-4617-85e4-88059e468dec", 2024-08-20T03:04:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-08-20T03:04:30.662 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:04:30.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.660+0000 7f529aabd640 1 -- 172.21.15.44:0/3570995075 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f52800b3f30 msgr2=0x7f52800b63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:30.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.660+0000 7f529aabd640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f52800b3f30 0x7f52800b63f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f52900096c0 tx=0x7f529000b040 comp rx=0 tx=0).stop 2024-08-20T03:04:30.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.660+0000 7f529aabd640 1 -- 172.21.15.44:0/3570995075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5294108cb0 msgr2=0x7f529419e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:30.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.660+0000 7f529aabd640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5294108cb0 0x7f529419e440 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f528c00eaa0 tx=0x7f528c00ef60 comp rx=0 tx=0).stop 2024-08-20T03:04:30.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.662+0000 7f529aabd640 1 -- 172.21.15.44:0/3570995075 shutdown_connections 2024-08-20T03:04:30.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.662+0000 7f529aabd640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f52800b3f30 0x7f52800b63f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.662+0000 7f529aabd640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5294108cb0 0x7f529419e440 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.662+0000 7f529aabd640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5294108330 0x7f52941990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.662+0000 7f529aabd640 1 --2- 172.21.15.44:0/3570995075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5294102920 0x7f5294198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:30.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.662+0000 7f529aabd640 1 -- 172.21.15.44:0/3570995075 >> 172.21.15.44:0/3570995075 conn(0x7f52940fe350 msgr2=0x7f5294100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:30.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.662+0000 7f529aabd640 1 -- 172.21.15.44:0/3570995075 shutdown_connections 2024-08-20T03:04:30.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:30.662+0000 7f529aabd640 1 -- 172.21.15.44:0/3570995075 wait complete. 2024-08-20T03:04:31.442 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:04:31.713 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:04:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.206+0000 7f6f0357a640 1 -- 172.21.15.44:0/1059910653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc1084e0 msgr2=0x7f6efc108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.206+0000 7f6f0357a640 1 --2- 172.21.15.44:0/1059910653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc1084e0 0x7f6efc108920 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009900 tx=0x7f6eec02a410 comp rx=0 tx=0).stop 2024-08-20T03:04:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.206+0000 7f6f0357a640 1 -- 172.21.15.44:0/1059910653 shutdown_connections 2024-08-20T03:04:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.206+0000 7f6f0357a640 1 --2- 172.21.15.44:0/1059910653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6efc108e60 0x7f6efc10f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.206+0000 7f6f0357a640 1 --2- 172.21.15.44:0/1059910653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc1084e0 0x7f6efc108920 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.206+0000 7f6f0357a640 1 --2- 172.21.15.44:0/1059910653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6efc102ad0 0x7f6efc102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.206+0000 7f6f0357a640 1 -- 172.21.15.44:0/1059910653 >> 172.21.15.44:0/1059910653 conn(0x7f6efc0fe540 msgr2=0x7f6efc100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.206+0000 7f6f0357a640 1 -- 172.21.15.44:0/1059910653 shutdown_connections 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.206+0000 7f6f0357a640 1 -- 172.21.15.44:0/1059910653 wait complete. 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.206+0000 7f6f0357a640 1 Processor -- start 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f0357a640 1 -- start start 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f0357a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6efc102ad0 0x7f6efc198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f0357a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6efc1084e0 0x7f6efc1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f0357a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc108e60 0x7f6efc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f0357a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc112ab0 con 0x7f6efc108e60 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f0357a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc112930 con 0x7f6efc102ad0 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f0357a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc112c30 con 0x7f6efc1084e0 2024-08-20T03:04:32.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f01b17640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc108e60 0x7f6efc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f01b17640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc108e60 0x7f6efc19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60300/0 (socket says 172.21.15.44:60300) 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f01b17640 1 -- 172.21.15.44:0/501235047 learned_addr learned my addr 172.21.15.44:0/501235047 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f00b15640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6efc1084e0 0x7f6efc1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f01316640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6efc102ad0 0x7f6efc198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f01b17640 1 -- 172.21.15.44:0/501235047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6efc1084e0 msgr2=0x7f6efc1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f01b17640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6efc1084e0 0x7f6efc1994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f01b17640 1 -- 172.21.15.44:0/501235047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6efc102ad0 msgr2=0x7f6efc198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f01b17640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6efc102ad0 0x7f6efc198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f01b17640 1 -- 172.21.15.44:0/501235047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6efc19f010 con 0x7f6efc108e60 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.207+0000 7f6f01b17640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc108e60 0x7f6efc19e840 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400eaa0 tx=0x7f6ef400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.208+0000 7f6ef27fc640 1 -- 172.21.15.44:0/501235047 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400cdb0 con 0x7f6efc108e60 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.208+0000 7f6ef27fc640 1 -- 172.21.15.44:0/501235047 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef4005020 con 0x7f6efc108e60 2024-08-20T03:04:32.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.208+0000 7f6f0357a640 1 -- 172.21.15.44:0/501235047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6efc19f300 con 0x7f6efc108e60 2024-08-20T03:04:32.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.208+0000 7f6ef27fc640 1 -- 172.21.15.44:0/501235047 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40116d0 con 0x7f6efc108e60 2024-08-20T03:04:32.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.208+0000 7f6f0357a640 1 -- 172.21.15.44:0/501235047 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6efc1a59c0 con 0x7f6efc108e60 2024-08-20T03:04:32.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.210+0000 7f6ef27fc640 1 -- 172.21.15.44:0/501235047 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6ef40051d0 con 0x7f6efc108e60 2024-08-20T03:04:32.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f05973740 1 Processor -- start 2024-08-20T03:04:32.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f05973740 1 -- start start 2024-08-20T03:04:32.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f05973740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 0x55e98f6aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f05973740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6af150 0x55e98f6baba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6ef27fc640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6ee80b3d40 0x7f6ee80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f05973740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6bd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f05973740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e98f6258e0 con 0x55e98f6ae830 2024-08-20T03:04:32.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f05973740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e98f614de0 con 0x55e98f6af150 2024-08-20T03:04:32.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f05973740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e98f618e20 con 0x55e98f6bb0e0 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f00b15640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 0x55e98f6aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f00b15640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 0x55e98f6aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60308/0 (socket says 172.21.15.44:60308) 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f00b15640 1 -- 172.21.15.44:0/2640304395 learned_addr learned my addr 172.21.15.44:0/2640304395 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f01b17640 1 --2- 172.21.15.44:0/2640304395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6bd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f00b15640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6ee80b3d40 0x7f6ee80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6f01316640 1 --2- 172.21.15.44:0/2640304395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6af150 0x55e98f6baba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.211+0000 7f6ef27fc640 1 -- 172.21.15.44:0/501235047 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6ef40e40a0 con 0x7f6efc108e60 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.212+0000 7f6f00b15640 1 -- 172.21.15.44:0/2640304395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 msgr2=0x55e98f6bd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.212+0000 7f6f00b15640 1 --2- 172.21.15.44:0/2640304395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6bd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.212+0000 7f6f00b15640 1 -- 172.21.15.44:0/2640304395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6af150 msgr2=0x55e98f6baba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.212+0000 7f6f00b15640 1 --2- 172.21.15.44:0/2640304395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6af150 0x55e98f6baba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.212+0000 7f6f00b15640 1 -- 172.21.15.44:0/2640304395 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98f374650 con 0x55e98f6ae830 2024-08-20T03:04:32.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.212+0000 7f6f01b17640 1 --2- 172.21.15.44:0/2640304395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6bd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.212+0000 7f6f01316640 1 --2- 172.21.15.44:0/2640304395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6af150 0x55e98f6baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.212+0000 7f6f00b15640 1 --2- 172.21.15.44:0/2640304395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 0x55e98f6aec10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6eec02a930 tx=0x7f6eec004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdcbe07230f4080 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.212+0000 7f6f00b15640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6ee80b3d40 0x7f6ee80b6200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009900 tx=0x7f6eec035000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.212+0000 7f6ee77fe640 1 -- 172.21.15.44:0/2640304395 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec035070 con 0x55e98f6ae830 2024-08-20T03:04:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.212+0000 7f6ee77fe640 1 -- 172.21.15.44:0/2640304395 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eec0044d0 con 0x55e98f6ae830 2024-08-20T03:04:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.212+0000 7f6ee77fe640 1 -- 172.21.15.44:0/2640304395 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec032020 con 0x55e98f6ae830 2024-08-20T03:04:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.213+0000 7f6f05973740 1 -- 172.21.15.44:0/2640304395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 msgr2=0x55e98f6aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.213+0000 7f6f05973740 1 --2- 172.21.15.44:0/2640304395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 0x55e98f6aec10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6eec02a930 tx=0x7f6eec004040 comp rx=0 tx=0).stop 2024-08-20T03:04:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.213+0000 7f6f05973740 1 -- 172.21.15.44:0/2640304395 shutdown_connections 2024-08-20T03:04:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.213+0000 7f6f05973740 1 --2- 172.21.15.44:0/2640304395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6bd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.213+0000 7f6f05973740 1 --2- 172.21.15.44:0/2640304395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6af150 0x55e98f6baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.213+0000 7f6f05973740 1 --2- 172.21.15.44:0/2640304395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 0x55e98f6aec10 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.213+0000 7f6f05973740 1 -- 172.21.15.44:0/2640304395 >> 172.21.15.44:0/2640304395 conn(0x55e98f6ac660 msgr2=0x55e98f6ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.213+0000 7f6f05973740 1 -- 172.21.15.44:0/2640304395 shutdown_connections 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.213+0000 7f6f05973740 1 -- 172.21.15.44:0/2640304395 wait complete. 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.213+0000 7f6f05973740 1 -- start start 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.213+0000 7f6f05973740 1 Processor -- start 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f05973740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 0x55e98f6b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f05973740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6af150 0x55e98f6b8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f05973740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f05973740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e98f477700 con 0x55e98f6ae830 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f05973740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e98f59b7c0 con 0x55e98f6af150 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f05973740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e98f6bdd30 con 0x55e98f6bb0e0 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f05973740 10 client.0 _ll_register_callbacks cb 0x55e98f60e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:32.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f01316640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 0x55e98f6b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f00b15640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6af150 0x55e98f6b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f01316640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 0x55e98f6b8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60316/0 (socket says 172.21.15.44:60316) 2024-08-20T03:04:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f01316640 1 -- 172.21.15.44:0/3887205928 learned_addr learned my addr 172.21.15.44:0/3887205928 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f01b17640 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f00b15640 1 -- 172.21.15.44:0/3887205928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 msgr2=0x55e98f6b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f00b15640 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f00b15640 1 -- 172.21.15.44:0/3887205928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 msgr2=0x55e98f6b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f00b15640 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 0x55e98f6b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f00b15640 1 -- 172.21.15.44:0/3887205928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98f6c7160 con 0x55e98f6af150 2024-08-20T03:04:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f01316640 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 0x55e98f6b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6f00b15640 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6af150 0x55e98f6b8640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6eec032fd0 tx=0x7f6eec005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=610cc695175151dd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:32.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6ee4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:04:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.214+0000 7f6ee4ff9640 1 -- 172.21.15.44:0/3887205928 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec031b10 con 0x55e98f6af150 2024-08-20T03:04:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6ee4ff9640 1 -- 172.21.15.44:0/3887205928 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6eec031cb0 con 0x55e98f6af150 2024-08-20T03:04:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6f05973740 1 -- 172.21.15.44:0/3887205928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e98f5c4100 con 0x55e98f6af150 2024-08-20T03:04:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6ee4ff9640 1 -- 172.21.15.44:0/3887205928 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6eec061050 con 0x55e98f6af150 2024-08-20T03:04:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6f05973740 10 client.25721 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6f05973740 1 -- 172.21.15.44:0/3887205928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e98f6c9b10 con 0x55e98f6af150 2024-08-20T03:04:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6ee4ff9640 1 -- 172.21.15.44:0/3887205928 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e98f5c4100 con 0x55e98f6af150 2024-08-20T03:04:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6ee4ff9640 1 -- 172.21.15.44:0/3887205928 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6eec048070 con 0x55e98f6af150 2024-08-20T03:04:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6f05973740 10 client.25721 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6f05973740 10 client.25721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6f05973740 20 client.25721 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:32.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6f05973740 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e98f6cae60 0x55e98f6cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6f05973740 4 client.25721 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:04:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6f05973740 1 -- 172.21.15.44:0/3887205928 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e98f627830 con 0x55e98f6cae60 2024-08-20T03:04:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.215+0000 7f6f01316640 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e98f6cae60 0x55e98f6cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.216+0000 7f6ee4ff9640 10 client.25721 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.216+0000 7f6f01316640 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e98f6cae60 0x55e98f6cd250 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf042e49dae81084 server_cookie=ffd7b2f8d4f2ff8b in_seq=0 out_seq=0 2024-08-20T03:04:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6ee4ff9640 1 -- 172.21.15.44:0/3887205928 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e98f627830 con 0x55e98f6cae60 2024-08-20T03:04:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6ee4ff9640 10 client.25721 handle_command_reply: tid=0 2024-08-20T03:04:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6f05973740 1 client.25721 shutdown 2024-08-20T03:04:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6f05973740 1 -- 172.21.15.44:0/3887205928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6af150 msgr2=0x55e98f6b8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6f05973740 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6af150 0x55e98f6b8640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6eec032fd0 tx=0x7f6eec005b50 comp rx=0 tx=0).stop 2024-08-20T03:04:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6f05973740 1 -- 172.21.15.44:0/3887205928 shutdown_connections 2024-08-20T03:04:32.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6f05973740 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e98f6cae60 0x55e98f6cd250 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6f05973740 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6f05973740 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6af150 0x55e98f6b8640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6ee4ff9640 0 client.25721 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:32.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6f05973740 1 --2- 172.21.15.44:0/3887205928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6ae830 0x55e98f6b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6f05973740 1 -- 172.21.15.44:0/3887205928 >> 172.21.15.44:0/3887205928 conn(0x55e98f6ac660 msgr2=0x55e98f6b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:32.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6f05973740 1 -- 172.21.15.44:0/3887205928 shutdown_connections 2024-08-20T03:04:32.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6f05973740 1 -- 172.21.15.44:0/3887205928 wait complete. 2024-08-20T03:04:32.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.218+0000 7f6f05973740 20 client.25721 trim_cache size 0 max 16384 2024-08-20T03:04:32.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.236+0000 7f6f05973740 1 Processor -- start 2024-08-20T03:04:32.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.236+0000 7f6f05973740 1 -- start start 2024-08-20T03:04:32.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.236+0000 7f6f05973740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6ae830 0x55e98f6b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.236+0000 7f6f05973740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6af150 0x55e98f6b9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.236+0000 7f6f05973740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6bb0e0 0x55e98f6d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.236+0000 7f6f05973740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1ae230 con 0x55e98f6bb0e0 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.236+0000 7f6f05973740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1afe30 con 0x55e98f6ae830 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.236+0000 7f6f05973740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1ae900 con 0x55e98f6af150 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6f01b17640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6bb0e0 0x55e98f6d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6f01b17640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6bb0e0 0x55e98f6d2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60318/0 (socket says 172.21.15.44:60318) 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6f01b17640 1 -- 172.21.15.44:0/695282512 learned_addr learned my addr 172.21.15.44:0/695282512 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6f00b15640 1 --2- 172.21.15.44:0/695282512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6af150 0x55e98f6b9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6f01316640 1 --2- 172.21.15.44:0/695282512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6ae830 0x55e98f6b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6f01b17640 1 -- 172.21.15.44:0/695282512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6af150 msgr2=0x55e98f6b9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6f01b17640 1 --2- 172.21.15.44:0/695282512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6af150 0x55e98f6b9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6f01b17640 1 -- 172.21.15.44:0/695282512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6ae830 msgr2=0x55e98f6b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6f01b17640 1 --2- 172.21.15.44:0/695282512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6ae830 0x55e98f6b75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6f01b17640 1 -- 172.21.15.44:0/695282512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98f6b7060 con 0x55e98f6bb0e0 2024-08-20T03:04:32.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6f01b17640 1 --2- 172.21.15.44:0/695282512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6bb0e0 0x55e98f6d2850 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400cfd0 tx=0x7f6ef40aa8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1c0ca50496e5dd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:32.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6ee6ffd640 1 -- 172.21.15.44:0/695282512 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef401f040 con 0x55e98f6bb0e0 2024-08-20T03:04:32.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6ee6ffd640 1 -- 172.21.15.44:0/695282512 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef40af320 con 0x55e98f6bb0e0 2024-08-20T03:04:32.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.237+0000 7f6ee6ffd640 1 -- 172.21.15.44:0/695282512 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40cdb90 con 0x55e98f6bb0e0 2024-08-20T03:04:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.238+0000 7f6f05973740 1 -- 172.21.15.44:0/695282512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6bb0e0 msgr2=0x55e98f6d2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.238+0000 7f6f05973740 1 --2- 172.21.15.44:0/695282512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6bb0e0 0x55e98f6d2850 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400cfd0 tx=0x7f6ef40aa8e0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.238+0000 7f6f05973740 1 -- 172.21.15.44:0/695282512 shutdown_connections 2024-08-20T03:04:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.238+0000 7f6f05973740 1 --2- 172.21.15.44:0/695282512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6bb0e0 0x55e98f6d2850 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.238+0000 7f6f05973740 1 --2- 172.21.15.44:0/695282512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6af150 0x55e98f6b9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.238+0000 7f6f05973740 1 --2- 172.21.15.44:0/695282512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6ae830 0x55e98f6b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.238+0000 7f6f05973740 1 -- 172.21.15.44:0/695282512 >> 172.21.15.44:0/695282512 conn(0x55e98f6ac660 msgr2=0x55e98f6b9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.238+0000 7f6f05973740 1 -- 172.21.15.44:0/695282512 shutdown_connections 2024-08-20T03:04:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.238+0000 7f6f05973740 1 -- 172.21.15.44:0/695282512 wait complete. 2024-08-20T03:04:32.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.238+0000 7f6f05973740 1 -- start start 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f05973740 1 Processor -- start 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f05973740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6ae830 0x55e98f6c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f05973740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6af150 0x55e98f6c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f05973740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f05973740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1af6a0 con 0x55e98f6af150 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f05973740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1aefd0 con 0x55e98f6ae830 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f05973740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1ac210 con 0x55e98f6bb0e0 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f05973740 10 client.0 _ll_register_callbacks cb 0x7f6ef800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f00b15640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6af150 0x55e98f6c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f01316640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6ae830 0x55e98f6c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f00b15640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6af150 0x55e98f6c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60324/0 (socket says 172.21.15.44:60324) 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f00b15640 1 -- 172.21.15.44:0/2019233435 learned_addr learned my addr 172.21.15.44:0/2019233435 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f01b17640 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f00b15640 1 -- 172.21.15.44:0/2019233435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 msgr2=0x55e98f6be900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f00b15640 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6be900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f00b15640 1 -- 172.21.15.44:0/2019233435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6ae830 msgr2=0x55e98f6c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f00b15640 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6ae830 0x55e98f6c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f00b15640 1 -- 172.21.15.44:0/2019233435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98f6bf130 con 0x55e98f6af150 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.239+0000 7f6f01316640 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6ae830 0x55e98f6c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6f00b15640 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6af150 0x55e98f6c3820 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55e98f6ba9f0 tx=0x7f6eec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278941aa2a47a4fe server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6ee4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6ee4ff9640 1 -- 172.21.15.44:0/2019233435 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec032d00 con 0x55e98f6af150 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6ee4ff9640 1 -- 172.21.15.44:0/2019233435 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6eec031ca0 con 0x55e98f6af150 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6f05973740 1 -- 172.21.15.44:0/2019233435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ef40af320 con 0x55e98f6af150 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6ee4ff9640 1 -- 172.21.15.44:0/2019233435 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6eec06c050 con 0x55e98f6af150 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6f05973740 10 client.16236 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6f05973740 1 -- 172.21.15.44:0/2019233435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e98f7354e0 con 0x55e98f6af150 2024-08-20T03:04:32.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6ee4ff9640 1 -- 172.21.15.44:0/2019233435 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef40af320 con 0x55e98f6af150 2024-08-20T03:04:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6ee4ff9640 1 -- 172.21.15.44:0/2019233435 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6eec048040 con 0x55e98f6af150 2024-08-20T03:04:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6f05973740 10 client.16236 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6f05973740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6f05973740 20 client.16236 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6f05973740 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e98f735c60 0x55e98f738080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6f05973740 4 client.16236 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:04:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.240+0000 7f6f05973740 1 -- 172.21.15.44:0/2019233435 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e98f6a7830 con 0x55e98f735c60 2024-08-20T03:04:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.241+0000 7f6f01316640 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e98f735c60 0x55e98f738080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.241+0000 7f6ee4ff9640 10 client.16236 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:32.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.241+0000 7f6f01316640 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e98f735c60 0x55e98f738080 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ef207548bde9db server_cookie=af581dd2c6be590b in_seq=0 out_seq=0 2024-08-20T03:04:32.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6ee4ff9640 1 -- 172.21.15.44:0/2019233435 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55e98f6a7830 con 0x55e98f735c60 2024-08-20T03:04:32.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6ee4ff9640 10 client.16236 handle_command_reply: tid=0 2024-08-20T03:04:32.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6f05973740 1 client.16236 shutdown 2024-08-20T03:04:32.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6f05973740 1 -- 172.21.15.44:0/2019233435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6af150 msgr2=0x55e98f6c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:32.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6f05973740 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6af150 0x55e98f6c3820 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55e98f6ba9f0 tx=0x7f6eec000bc0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6f05973740 1 -- 172.21.15.44:0/2019233435 shutdown_connections 2024-08-20T03:04:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6f05973740 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e98f735c60 0x55e98f738080 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6f05973740 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98f6bb0e0 0x55e98f6be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6ee4ff9640 0 client.16236 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6f05973740 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98f6af150 0x55e98f6c3820 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6f05973740 1 --2- 172.21.15.44:0/2019233435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98f6ae830 0x55e98f6c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6f05973740 1 -- 172.21.15.44:0/2019233435 >> 172.21.15.44:0/2019233435 conn(0x55e98f6ac660 msgr2=0x55e98f6b9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6f05973740 1 -- 172.21.15.44:0/2019233435 shutdown_connections 2024-08-20T03:04:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6f05973740 1 -- 172.21.15.44:0/2019233435 wait complete. 2024-08-20T03:04:32.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.244+0000 7f6f05973740 20 client.16236 trim_cache size 0 max 16384 2024-08-20T03:04:32.251 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:04:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "scrub active (22 inodes in the stack)", 2024-08-20T03:04:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-08-20T03:04:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "ce02f953-98f5-4617-85e4-88059e468dec": { 2024-08-20T03:04:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-08-20T03:04:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "ce02f953-98f5-4617-85e4-88059e468dec", 2024-08-20T03:04:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-08-20T03:04:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:04:32.251 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:04:32.251 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:04:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.245+0000 7f6f0357a640 1 -- 172.21.15.44:0/501235047 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6ee80b3d40 msgr2=0x7f6ee80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.245+0000 7f6f0357a640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6ee80b3d40 0x7f6ee80b6200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6eec009900 tx=0x7f6eec035000 comp rx=0 tx=0).stop 2024-08-20T03:04:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.245+0000 7f6f0357a640 1 -- 172.21.15.44:0/501235047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc108e60 msgr2=0x7f6efc19e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.245+0000 7f6f0357a640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc108e60 0x7f6efc19e840 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400eaa0 tx=0x7f6ef400ef60 comp rx=0 tx=0).stop 2024-08-20T03:04:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.245+0000 7f6f0357a640 1 -- 172.21.15.44:0/501235047 shutdown_connections 2024-08-20T03:04:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.245+0000 7f6f0357a640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6ee80b3d40 0x7f6ee80b6200 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.245+0000 7f6f0357a640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6efc108e60 0x7f6efc19e840 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.245+0000 7f6f0357a640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6efc1084e0 0x7f6efc1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.245+0000 7f6f0357a640 1 --2- 172.21.15.44:0/501235047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6efc102ad0 0x7f6efc198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.245+0000 7f6f0357a640 1 -- 172.21.15.44:0/501235047 >> 172.21.15.44:0/501235047 conn(0x7f6efc0fe540 msgr2=0x7f6efc100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:32.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.245+0000 7f6f0357a640 1 -- 172.21.15.44:0/501235047 shutdown_connections 2024-08-20T03:04:32.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:04:32.245+0000 7f6f0357a640 1 -- 172.21.15.44:0/501235047 wait complete. 2024-08-20T03:04:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:32 smithi165 ceph-mon[22357]: pgmap v381: 97 pgs: 97 active+clean; 167 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 828 op/s 2024-08-20T03:04:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:32 smithi165 ceph-mon[22357]: scrub queued for path: / 2024-08-20T03:04:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:32 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:32 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:04:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:32 smithi099 ceph-mon[21984]: pgmap v381: 97 pgs: 97 active+clean; 167 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 828 op/s 2024-08-20T03:04:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:32 smithi099 ceph-mon[21984]: scrub queued for path: / 2024-08-20T03:04:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:32 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:32.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:32 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:04:32.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:32 smithi044 ceph-mon[18790]: pgmap v381: 97 pgs: 97 active+clean; 167 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 828 op/s 2024-08-20T03:04:32.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:32 smithi044 ceph-mon[18790]: scrub queued for path: / 2024-08-20T03:04:32.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:32 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:32.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:32 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:04:33.078 INFO:tasks.cephfs.filesystem:scrub status for tag:ce02f953-98f5-4617-85e4-88059e468dec - {'path': '/', 'tag': 'ce02f953-98f5-4617-85e4-88059e468dec', 'options': 'recursive,force'} 2024-08-20T03:04:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.370+0000 7f72e6be7640 1 -- 172.21.15.44:0/3958801014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e01029f0 msgr2=0x7f72e0102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.370+0000 7f72e6be7640 1 --2- 172.21.15.44:0/3958801014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e01029f0 0x7f72e0102dd0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009900 tx=0x7f72d402a410 comp rx=0 tx=0).stop 2024-08-20T03:04:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.370+0000 7f72e6be7640 1 -- 172.21.15.44:0/3958801014 shutdown_connections 2024-08-20T03:04:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.370+0000 7f72e6be7640 1 --2- 172.21.15.44:0/3958801014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e0108d70 0x7f72e010f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.370+0000 7f72e6be7640 1 --2- 172.21.15.44:0/3958801014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e01083f0 0x7f72e0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.370+0000 7f72e6be7640 1 --2- 172.21.15.44:0/3958801014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e01029f0 0x7f72e0102dd0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.370+0000 7f72e6be7640 1 -- 172.21.15.44:0/3958801014 >> 172.21.15.44:0/3958801014 conn(0x7f72e00fe430 msgr2=0x7f72e0100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.370+0000 7f72e6be7640 1 -- 172.21.15.44:0/3958801014 shutdown_connections 2024-08-20T03:04:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.370+0000 7f72e6be7640 1 -- 172.21.15.44:0/3958801014 wait complete. 2024-08-20T03:04:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72e6be7640 1 Processor -- start 2024-08-20T03:04:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72e6be7640 1 -- start start 2024-08-20T03:04:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72e6be7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e01029f0 0x7f72e00751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72e6be7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e01083f0 0x7f72e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72e6be7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e0108d70 0x7f72e0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72e6be7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01129f0 con 0x7f72e0108d70 2024-08-20T03:04:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72e6be7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f72e0112870 con 0x7f72e01029f0 2024-08-20T03:04:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72e6be7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72e0112b70 con 0x7f72e01083f0 2024-08-20T03:04:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72e5184640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e0108d70 0x7f72e0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72e5184640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e0108d70 0x7f72e0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60342/0 (socket says 172.21.15.44:60342) 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72e5184640 1 -- 172.21.15.44:0/811395995 learned_addr learned my addr 172.21.15.44:0/811395995 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72dffff640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e01083f0 0x7f72e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.371+0000 7f72e4983640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e01029f0 0x7f72e00751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.372+0000 7f72e5184640 1 -- 172.21.15.44:0/811395995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e01083f0 msgr2=0x7f72e0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.372+0000 7f72e5184640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e01083f0 0x7f72e0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.372+0000 7f72e5184640 1 -- 172.21.15.44:0/811395995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e01029f0 msgr2=0x7f72e00751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.372+0000 7f72e5184640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e01029f0 0x7f72e00751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.372+0000 7f72e5184640 1 -- 172.21.15.44:0/811395995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e01a7320 con 0x7f72e0108d70 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.372+0000 7f72e5184640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e0108d70 0x7f72e0075c50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f72d800eb10 tx=0x7f72d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.372+0000 7f72ddffb640 1 -- 172.21.15.44:0/811395995 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8004b20 con 0x7f72e0108d70 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.372+0000 7f72ddffb640 1 -- 172.21.15.44:0/811395995 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d8004cc0 con 0x7f72e0108d70 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.372+0000 7f72e6be7640 1 -- 172.21.15.44:0/811395995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e01a7610 con 0x7f72e0108d70 2024-08-20T03:04:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.372+0000 7f72ddffb640 1 -- 172.21.15.44:0/811395995 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8010670 con 0x7f72e0108d70 2024-08-20T03:04:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.372+0000 7f72e6be7640 1 -- 172.21.15.44:0/811395995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e01a7b10 con 0x7f72e0108d70 2024-08-20T03:04:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.374+0000 7f72ddffb640 1 -- 172.21.15.44:0/811395995 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f72d8005020 con 0x7f72e0108d70 2024-08-20T03:04:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72e99d5740 1 Processor -- start 2024-08-20T03:04:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72e99d5740 1 -- start start 2024-08-20T03:04:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72e99d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 0x55acc5ee3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72e99d5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee4270 0x55acc5eefcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72e99d5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5ef25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72e99d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55acc5e59360 con 0x55acc5ee3950 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72e99d5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55acc5e48860 con 0x55acc5ee4270 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72e99d5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55acc5e4c8a0 con 0x55acc5ef0200 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72dffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 0x55acc5ee3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72dffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 0x55acc5ee3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60350/0 (socket says 172.21.15.44:60350) 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72dffff640 1 -- 172.21.15.44:0/2613209644 learned_addr learned my addr 172.21.15.44:0/2613209644 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72ddffb640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f72d00b3d40 0x7f72d00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72dffff640 1 --2- 172.21.15.44:0/2613209644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5ef25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72e4983640 1 --2- 172.21.15.44:0/2613209644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee4270 0x55acc5eefcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.375+0000 7f72e5184640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f72d00b3d40 0x7f72d00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.376+0000 7f72dffff640 1 -- 172.21.15.44:0/2613209644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 msgr2=0x55acc5ef25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.376+0000 7f72dffff640 1 --2- 172.21.15.44:0/2613209644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5ef25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.376+0000 7f72dffff640 1 -- 172.21.15.44:0/2613209644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee4270 msgr2=0x55acc5eefcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.376+0000 7f72dffff640 1 --2- 172.21.15.44:0/2613209644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee4270 0x55acc5eefcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.376+0000 7f72dffff640 1 -- 172.21.15.44:0/2613209644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc5defe40 con 0x55acc5ee3950 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.376+0000 7f72ddffb640 1 -- 172.21.15.44:0/811395995 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f72d80e4560 con 0x7f72e0108d70 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.376+0000 7f72dffff640 1 --2- 172.21.15.44:0/2613209644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 0x55acc5ee3d30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00bfd0 tx=0x7f72cc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ea6cba9368924c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.376+0000 7f72e5184640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f72d00b3d40 0x7f72d00b6200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f72d80ab170 tx=0x7f72d8078930 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.376+0000 7f72caffd640 1 -- 172.21.15.44:0/2613209644 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc012ce0 con 0x55acc5ee3950 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.376+0000 7f72caffd640 1 -- 172.21.15.44:0/2613209644 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72cc010c20 con 0x55acc5ee3950 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.376+0000 7f72caffd640 1 -- 172.21.15.44:0/2613209644 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc01ea20 con 0x55acc5ee3950 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.377+0000 7f72e99d5740 1 -- 172.21.15.44:0/2613209644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 msgr2=0x55acc5ee3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.377+0000 7f72e99d5740 1 --2- 172.21.15.44:0/2613209644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 0x55acc5ee3d30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00bfd0 tx=0x7f72cc00b790 comp rx=0 tx=0).stop 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.377+0000 7f72e99d5740 1 -- 172.21.15.44:0/2613209644 shutdown_connections 2024-08-20T03:04:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.377+0000 7f72e99d5740 1 --2- 172.21.15.44:0/2613209644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5ef25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.377+0000 7f72e99d5740 1 --2- 172.21.15.44:0/2613209644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee4270 0x55acc5eefcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.377+0000 7f72e99d5740 1 --2- 172.21.15.44:0/2613209644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 0x55acc5ee3d30 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.377+0000 7f72e99d5740 1 -- 172.21.15.44:0/2613209644 >> 172.21.15.44:0/2613209644 conn(0x55acc5ee1730 msgr2=0x55acc5ee2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.377+0000 7f72e99d5740 1 -- 172.21.15.44:0/2613209644 shutdown_connections 2024-08-20T03:04:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.377+0000 7f72e99d5740 1 -- 172.21.15.44:0/2613209644 wait complete. 2024-08-20T03:04:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.377+0000 7f72e99d5740 1 -- start start 2024-08-20T03:04:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.377+0000 7f72e99d5740 1 Processor -- start 2024-08-20T03:04:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.378+0000 7f72e99d5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 0x55acc5eefb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.378+0000 7f72e99d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee4270 0x55acc5eecf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.378+0000 7f72e99d5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5eed600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.378+0000 7f72e99d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55acc5d81ab0 con 0x55acc5ee4270 2024-08-20T03:04:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.378+0000 7f72e99d5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55acc5dcf780 con 0x55acc5ee3950 2024-08-20T03:04:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.378+0000 7f72e99d5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55acc5ef2ee0 con 0x55acc5ef0200 2024-08-20T03:04:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.378+0000 7f72e99d5740 10 client.0 _ll_register_callbacks cb 0x55acc5b57a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.378+0000 7f72dffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 0x55acc5eefb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.378+0000 7f72dffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 0x55acc5eefb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47458/0 (socket says 172.21.15.44:47458) 2024-08-20T03:04:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.378+0000 7f72dffff640 1 -- 172.21.15.44:0/1039717414 learned_addr learned my addr 172.21.15.44:0/1039717414 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.378+0000 7f72e4983640 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee4270 0x55acc5eecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.378+0000 7f72dffff640 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5eed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72dffff640 1 -- 172.21.15.44:0/1039717414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 msgr2=0x55acc5eed600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72dffff640 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5eed600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72dffff640 1 -- 172.21.15.44:0/1039717414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee4270 msgr2=0x55acc5eecf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72dffff640 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee4270 0x55acc5eecf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72dffff640 1 -- 172.21.15.44:0/1039717414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acc5bab790 con 0x55acc5ee3950 2024-08-20T03:04:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72dffff640 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 0x55acc5eefb40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b440 tx=0x7f72cc01e6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f5cd974341ef201 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72bffff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:04:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72bffff640 1 -- 172.21.15.44:0/1039717414 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc024040 con 0x55acc5ee3950 2024-08-20T03:04:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72bffff640 1 -- 172.21.15.44:0/1039717414 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f72cc004450 con 0x55acc5ee3950 2024-08-20T03:04:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72e99d5740 1 -- 172.21.15.44:0/1039717414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55acc5df7e00 con 0x55acc5ee3950 2024-08-20T03:04:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72bffff640 1 -- 172.21.15.44:0/1039717414 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72cc01e7a0 con 0x55acc5ee3950 2024-08-20T03:04:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72e99d5740 10 client.25739 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.379+0000 7f72e99d5740 1 -- 172.21.15.44:0/1039717414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55acc5c24620 con 0x55acc5ee3950 2024-08-20T03:04:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.380+0000 7f72bffff640 1 -- 172.21.15.44:0/1039717414 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55acc5df7e00 con 0x55acc5ee3950 2024-08-20T03:04:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.380+0000 7f72bffff640 1 -- 172.21.15.44:0/1039717414 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f72cc010ca0 con 0x55acc5ee3950 2024-08-20T03:04:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.380+0000 7f72e99d5740 10 client.25739 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.380+0000 7f72e99d5740 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.380+0000 7f72e99d5740 20 client.25739 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.380+0000 7f72e99d5740 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55acc5eff9e0 0x55acc5f01dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.380+0000 7f72e99d5740 4 client.25739 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:04:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.380+0000 7f72e99d5740 1 -- 172.21.15.44:0/1039717414 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acc5e5b2b0 con 0x55acc5eff9e0 2024-08-20T03:04:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.380+0000 7f72e4983640 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55acc5eff9e0 0x55acc5f01dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.381+0000 7f72e4983640 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55acc5eff9e0 0x55acc5f01dd0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2550bd251a214545 server_cookie=eec7262cee20a94 in_seq=0 out_seq=0 2024-08-20T03:04:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.381+0000 7f72bffff640 10 client.25739 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72bffff640 1 -- 172.21.15.44:0/1039717414 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55acc5e5b2b0 con 0x55acc5eff9e0 2024-08-20T03:04:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72bffff640 10 client.25739 handle_command_reply: tid=0 2024-08-20T03:04:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72e99d5740 1 client.25739 shutdown 2024-08-20T03:04:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72e99d5740 1 -- 172.21.15.44:0/1039717414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 msgr2=0x55acc5eefb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72e99d5740 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 0x55acc5eefb40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b440 tx=0x7f72cc01e6a0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72e99d5740 1 -- 172.21.15.44:0/1039717414 shutdown_connections 2024-08-20T03:04:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72e99d5740 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55acc5eff9e0 0x55acc5f01dd0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72e99d5740 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5eed600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72bffff640 0 client.25739 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72e99d5740 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee4270 0x55acc5eecf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72e99d5740 1 --2- 172.21.15.44:0/1039717414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 0x55acc5eefb40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72e99d5740 1 -- 172.21.15.44:0/1039717414 >> 172.21.15.44:0/1039717414 conn(0x55acc5ee1730 msgr2=0x55acc5ee72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72e99d5740 1 -- 172.21.15.44:0/1039717414 shutdown_connections 2024-08-20T03:04:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72e99d5740 1 -- 172.21.15.44:0/1039717414 wait complete. 2024-08-20T03:04:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.382+0000 7f72e99d5740 20 client.25739 trim_cache size 0 max 16384 2024-08-20T03:04:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.397+0000 7f72e99d5740 1 Processor -- start 2024-08-20T03:04:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.397+0000 7f72e99d5740 1 -- start start 2024-08-20T03:04:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.397+0000 7f72e99d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 0x55acc5efc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.397+0000 7f72e99d5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee4270 0x55acc5ee27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.397+0000 7f72e99d5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5ee2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.397+0000 7f72e99d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01b03b0 con 0x55acc5ee3950 2024-08-20T03:04:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.397+0000 7f72e99d5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01b1fb0 con 0x55acc5ee4270 2024-08-20T03:04:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.397+0000 7f72e99d5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01b0a80 con 0x55acc5ef0200 2024-08-20T03:04:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.397+0000 7f72dffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 0x55acc5efc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72dffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 0x55acc5efc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60376/0 (socket says 172.21.15.44:60376) 2024-08-20T03:04:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72dffff640 1 -- 172.21.15.44:0/88483914 learned_addr learned my addr 172.21.15.44:0/88483914 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72dffff640 1 --2- 172.21.15.44:0/88483914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5ee2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72e4983640 1 --2- 172.21.15.44:0/88483914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee4270 0x55acc5ee27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72dffff640 1 -- 172.21.15.44:0/88483914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 msgr2=0x55acc5ee2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72dffff640 1 --2- 172.21.15.44:0/88483914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5ee2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72dffff640 1 -- 172.21.15.44:0/88483914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee4270 msgr2=0x55acc5ee27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72dffff640 1 --2- 172.21.15.44:0/88483914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee4270 0x55acc5ee27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72dffff640 1 -- 172.21.15.44:0/88483914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc5f02140 con 0x55acc5ee3950 2024-08-20T03:04:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72e4983640 1 --2- 172.21.15.44:0/88483914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee4270 0x55acc5ee27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:04:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72dffff640 1 --2- 172.21.15.44:0/88483914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 0x55acc5efc000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0057c0 tx=0x7f72cc0123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e213d6a1817ecc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72ca7fc640 1 -- 172.21.15.44:0/88483914 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc003960 con 0x55acc5ee3950 2024-08-20T03:04:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72ca7fc640 1 -- 172.21.15.44:0/88483914 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72cc01edf0 con 0x55acc5ee3950 2024-08-20T03:04:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.398+0000 7f72ca7fc640 1 -- 172.21.15.44:0/88483914 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc010740 con 0x55acc5ee3950 2024-08-20T03:04:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 -- 172.21.15.44:0/88483914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 msgr2=0x55acc5efc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 --2- 172.21.15.44:0/88483914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 0x55acc5efc000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0057c0 tx=0x7f72cc0123f0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 -- 172.21.15.44:0/88483914 shutdown_connections 2024-08-20T03:04:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 --2- 172.21.15.44:0/88483914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5ee2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 --2- 172.21.15.44:0/88483914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee4270 0x55acc5ee27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 --2- 172.21.15.44:0/88483914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee3950 0x55acc5efc000 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 -- 172.21.15.44:0/88483914 >> 172.21.15.44:0/88483914 conn(0x55acc5ee1730 msgr2=0x55acc5ee8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 -- 172.21.15.44:0/88483914 shutdown_connections 2024-08-20T03:04:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 -- 172.21.15.44:0/88483914 wait complete. 2024-08-20T03:04:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 -- start start 2024-08-20T03:04:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 Processor -- start 2024-08-20T03:04:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 0x55acc5ef0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee4270 0x55acc5ef13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.399+0000 7f72e99d5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5ef7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e99d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01b1820 con 0x55acc5ee4270 2024-08-20T03:04:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e99d5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01b1150 con 0x55acc5ee3950 2024-08-20T03:04:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e99d5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01ae370 con 0x55acc5ef0200 2024-08-20T03:04:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e99d5740 10 client.0 _ll_register_callbacks cb 0x7f72d4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e4983640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee4270 0x55acc5ef13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e4983640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee4270 0x55acc5ef13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60392/0 (socket says 172.21.15.44:60392) 2024-08-20T03:04:33.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e4983640 1 -- 172.21.15.44:0/3846131166 learned_addr learned my addr 172.21.15.44:0/3846131166 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72dffff640 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5ef7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72dffff640 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 0x55acc5ef0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e4983640 1 -- 172.21.15.44:0/3846131166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 msgr2=0x55acc5ef7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e4983640 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5ef7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e4983640 1 -- 172.21.15.44:0/3846131166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 msgr2=0x55acc5ef0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e4983640 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 0x55acc5ef0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e4983640 1 -- 172.21.15.44:0/3846131166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acc5ef78b0 con 0x55acc5ee4270 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72dffff640 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 0x55acc5ef0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e4983640 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee4270 0x55acc5ef13a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f72d402a450 tx=0x7f72d4005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735caa09c3cfcf1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72bffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72bffff640 1 -- 172.21.15.44:0/3846131166 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4035040 con 0x55acc5ee4270 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72bffff640 1 -- 172.21.15.44:0/3846131166 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f72d4005040 con 0x55acc5ee4270 2024-08-20T03:04:33.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e99d5740 1 -- 172.21.15.44:0/3846131166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72cc01edf0 con 0x55acc5ee4270 2024-08-20T03:04:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72bffff640 1 -- 172.21.15.44:0/3846131166 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72d4005240 con 0x55acc5ee4270 2024-08-20T03:04:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e99d5740 10 client.16254 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.400+0000 7f72e99d5740 1 -- 172.21.15.44:0/3846131166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55acc5ef8b60 con 0x55acc5ee4270 2024-08-20T03:04:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.401+0000 7f72bffff640 1 -- 172.21.15.44:0/3846131166 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72cc01edf0 con 0x55acc5ee4270 2024-08-20T03:04:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.401+0000 7f72bffff640 1 -- 172.21.15.44:0/3846131166 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f72d4003710 con 0x55acc5ee4270 2024-08-20T03:04:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.401+0000 7f72e99d5740 10 client.16254 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.401+0000 7f72e99d5740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.401+0000 7f72e99d5740 20 client.16254 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.401+0000 7f72e99d5740 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55acc5f6aa30 0x55acc5f6ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.401+0000 7f72e99d5740 4 client.16254 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:04:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.401+0000 7f72e99d5740 1 -- 172.21.15.44:0/3846131166 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55acc5e56d50 con 0x55acc5f6aa30 2024-08-20T03:04:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.401+0000 7f72dffff640 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55acc5f6aa30 0x55acc5f6ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.401+0000 7f72bffff640 10 client.16254 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:33.401+0000 7f72dffff640 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55acc5f6aa30 0x55acc5f6ce50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1909f0e953277b55 server_cookie=5098f0320f205db9 in_seq=0 out_seq=0 2024-08-20T03:04:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:34 smithi165 ceph-mon[22357]: pgmap v382: 97 pgs: 97 active+clean; 221 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.9 MiB/s wr, 1.01k op/s 2024-08-20T03:04:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:34 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:04:34.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:34 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:34.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:34 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:04:34.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:34 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:34.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:34 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:04:34.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:34 smithi099 ceph-mon[21984]: pgmap v382: 97 pgs: 97 active+clean; 221 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.9 MiB/s wr, 1.01k op/s 2024-08-20T03:04:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:34 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:04:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:34 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:34 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:04:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:34 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:34 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:04:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:34 smithi044 ceph-mon[18790]: pgmap v382: 97 pgs: 97 active+clean; 221 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.9 MiB/s wr, 1.01k op/s 2024-08-20T03:04:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:34 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:04:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:34 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:34 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:04:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:34 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:34 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.840+0000 7f72bffff640 1 -- 172.21.15.44:0/3846131166 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55acc5e56d50 con 0x55acc5f6aa30 2024-08-20T03:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.841+0000 7f72bffff640 10 client.16254 handle_command_reply: tid=0 2024-08-20T03:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.841+0000 7f72e99d5740 1 client.16254 shutdown 2024-08-20T03:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.841+0000 7f72e99d5740 1 -- 172.21.15.44:0/3846131166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee4270 msgr2=0x55acc5ef13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.841+0000 7f72e99d5740 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee4270 0x55acc5ef13a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f72d402a450 tx=0x7f72d4005be0 comp rx=0 tx=0).stop 2024-08-20T03:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.841+0000 7f72e99d5740 1 -- 172.21.15.44:0/3846131166 shutdown_connections 2024-08-20T03:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.841+0000 7f72e99d5740 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55acc5f6aa30 0x55acc5f6ce50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.841+0000 7f72e99d5740 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acc5ef0200 0x55acc5ef7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.841+0000 7f72bffff640 0 client.16254 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.841+0000 7f72e99d5740 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acc5ee4270 0x55acc5ef13a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.842+0000 7f72e99d5740 1 --2- 172.21.15.44:0/3846131166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55acc5ee3950 0x55acc5ef0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.842+0000 7f72e99d5740 1 -- 172.21.15.44:0/3846131166 >> 172.21.15.44:0/3846131166 conn(0x55acc5ee1730 msgr2=0x55acc5ee27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.842+0000 7f72e99d5740 1 -- 172.21.15.44:0/3846131166 shutdown_connections 2024-08-20T03:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.842+0000 7f72e99d5740 1 -- 172.21.15.44:0/3846131166 wait complete. 2024-08-20T03:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.842+0000 7f72e99d5740 20 client.16254 trim_cache size 0 max 16384 2024-08-20T03:04:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:04:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:04:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:04:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:04:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.844+0000 7f72e6be7640 1 -- 172.21.15.44:0/811395995 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f72d00b3d40 msgr2=0x7f72d00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.844+0000 7f72e6be7640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f72d00b3d40 0x7f72d00b6200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f72d80ab170 tx=0x7f72d8078930 comp rx=0 tx=0).stop 2024-08-20T03:04:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.844+0000 7f72e6be7640 1 -- 172.21.15.44:0/811395995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e0108d70 msgr2=0x7f72e0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.844+0000 7f72e6be7640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e0108d70 0x7f72e0075c50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f72d800eb10 tx=0x7f72d800c2d0 comp rx=0 tx=0).stop 2024-08-20T03:04:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.844+0000 7f72e6be7640 1 -- 172.21.15.44:0/811395995 shutdown_connections 2024-08-20T03:04:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.844+0000 7f72e6be7640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f72d00b3d40 0x7f72d00b6200 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.844+0000 7f72e6be7640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f72e0108d70 0x7f72e0075c50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.844+0000 7f72e6be7640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e01083f0 0x7f72e0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.844+0000 7f72e6be7640 1 --2- 172.21.15.44:0/811395995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72e01029f0 0x7f72e00751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.844+0000 7f72e6be7640 1 -- 172.21.15.44:0/811395995 >> 172.21.15.44:0/811395995 conn(0x7f72e00fe430 msgr2=0x7f72e0100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.844+0000 7f72e6be7640 1 -- 172.21.15.44:0/811395995 shutdown_connections 2024-08-20T03:04:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:34.844+0000 7f72e6be7640 1 -- 172.21.15.44:0/811395995 wait complete. 2024-08-20T03:04:35.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:35 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:04:35.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:35 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:35.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:35 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:35.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:35 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:04:35.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:35 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:35.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:35 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:35.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:35 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:04:35.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:35 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:35.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:35 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:36 smithi165 ceph-mon[22357]: pgmap v383: 97 pgs: 97 active+clean; 242 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 847 op/s 2024-08-20T03:04:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:36 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:04:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:36 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:36 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:04:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:36 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:36 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:04:36.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:36 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:36 smithi099 ceph-mon[21984]: pgmap v383: 97 pgs: 97 active+clean; 242 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 847 op/s 2024-08-20T03:04:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:36 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:04:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:36 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:36 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:04:36.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:36 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:36.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:36 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:04:36.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:36 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:36 smithi044 ceph-mon[18790]: pgmap v383: 97 pgs: 97 active+clean; 242 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 847 op/s 2024-08-20T03:04:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:36 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:04:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:36 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:36 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:04:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:36 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:36 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:04:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:36 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:04:37.476 INFO:tasks.workunit.client.0.smithi044.stdout: 69 621284 290 430959 256 324701 688 2024-08-20T03:04:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:38 smithi165 ceph-mon[22357]: pgmap v384: 97 pgs: 97 active+clean; 251 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.8 MiB/s wr, 974 op/s 2024-08-20T03:04:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:38 smithi099 ceph-mon[21984]: pgmap v384: 97 pgs: 97 active+clean; 251 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.8 MiB/s wr, 974 op/s 2024-08-20T03:04:38.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:38 smithi044 ceph-mon[18790]: pgmap v384: 97 pgs: 97 active+clean; 251 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.8 MiB/s wr, 974 op/s 2024-08-20T03:04:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:40 smithi165 ceph-mon[22357]: pgmap v385: 97 pgs: 97 active+clean; 236 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.32k op/s 2024-08-20T03:04:40.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:40 smithi099 ceph-mon[21984]: pgmap v385: 97 pgs: 97 active+clean; 236 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.32k op/s 2024-08-20T03:04:40.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:40 smithi044 ceph-mon[18790]: pgmap v385: 97 pgs: 97 active+clean; 236 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 10 MiB/s wr, 1.32k op/s 2024-08-20T03:04:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:42 smithi165 ceph-mon[22357]: pgmap v386: 97 pgs: 97 active+clean; 238 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.40k op/s 2024-08-20T03:04:42.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:42 smithi099 ceph-mon[21984]: pgmap v386: 97 pgs: 97 active+clean; 238 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.40k op/s 2024-08-20T03:04:42.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:42 smithi044 ceph-mon[18790]: pgmap v386: 97 pgs: 97 active+clean; 238 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.40k op/s 2024-08-20T03:04:44.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:44 smithi165 ceph-mon[22357]: pgmap v387: 97 pgs: 97 active+clean; 247 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 10 MiB/s wr, 1.42k op/s 2024-08-20T03:04:44.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:44 smithi099 ceph-mon[21984]: pgmap v387: 97 pgs: 97 active+clean; 247 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 10 MiB/s wr, 1.42k op/s 2024-08-20T03:04:44.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:44 smithi044 ceph-mon[18790]: pgmap v387: 97 pgs: 97 active+clean; 247 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 10 MiB/s wr, 1.42k op/s 2024-08-20T03:04:46.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:46 smithi165 ceph-mon[22357]: pgmap v388: 97 pgs: 97 active+clean; 249 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 1.24k op/s 2024-08-20T03:04:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:46 smithi099 ceph-mon[21984]: pgmap v388: 97 pgs: 97 active+clean; 249 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 1.24k op/s 2024-08-20T03:04:46.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:46 smithi044 ceph-mon[18790]: pgmap v388: 97 pgs: 97 active+clean; 249 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 1.24k op/s 2024-08-20T03:04:47.478 INFO:tasks.workunit.client.0.smithi044.stdout: 78 1409230 514 989209 543 745982 1675 2024-08-20T03:04:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.098+0000 7f3f282f6640 1 -- 172.21.15.44:0/4227512209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f18098060 msgr2=0x7f3f180984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.098+0000 7f3f282f6640 1 --2- 172.21.15.44:0/4227512209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f18098060 0x7f3f180984c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009860 tx=0x7f3f1002a340 comp rx=0 tx=0).stop 2024-08-20T03:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.098+0000 7f3f282f6640 1 -- 172.21.15.44:0/4227512209 shutdown_connections 2024-08-20T03:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.098+0000 7f3f282f6640 1 --2- 172.21.15.44:0/4227512209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f18098a00 0x7f3f180a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.098+0000 7f3f282f6640 1 --2- 172.21.15.44:0/4227512209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f18098060 0x7f3f180984c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.098+0000 7f3f282f6640 1 --2- 172.21.15.44:0/4227512209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f180049b0 0x7f3f18004d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.098+0000 7f3f282f6640 1 -- 172.21.15.44:0/4227512209 >> 172.21.15.44:0/4227512209 conn(0x7f3f1800a180 msgr2=0x7f3f1800c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.098+0000 7f3f282f6640 1 -- 172.21.15.44:0/4227512209 shutdown_connections 2024-08-20T03:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.098+0000 7f3f282f6640 1 -- 172.21.15.44:0/4227512209 wait complete. 2024-08-20T03:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.099+0000 7f3f282f6640 1 Processor -- start 2024-08-20T03:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.099+0000 7f3f282f6640 1 -- start start 2024-08-20T03:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.100+0000 7f3f282f6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f180049b0 0x7f3f1812be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.100+0000 7f3f282f6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f18098060 0x7f3f1812c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.100+0000 7f3f26092640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f180049b0 0x7f3f1812be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.100+0000 7f3f25891640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f18098060 0x7f3f1812c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.100+0000 7f3f25891640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f18098060 0x7f3f1812c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60736/0 (socket says 172.21.15.44:60736) 2024-08-20T03:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.100+0000 7f3f26092640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f180049b0 0x7f3f1812be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47214/0 (socket says 172.21.15.44:47214) 2024-08-20T03:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.100+0000 7f3f25891640 1 -- 172.21.15.44:0/3072623508 learned_addr learned my addr 172.21.15.44:0/3072623508 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.100+0000 7f3f282f6640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18098a00 0x7f3f18131700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.100+0000 7f3f282f6640 1 -- 172.21.15.44:0/3072623508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f180a5890 con 0x7f3f18098060 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.100+0000 7f3f282f6640 1 -- 172.21.15.44:0/3072623508 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f180a5710 con 0x7f3f180049b0 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.100+0000 7f3f282f6640 1 -- 172.21.15.44:0/3072623508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3f180a5a10 con 0x7f3f18098a00 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.100+0000 7f3f26893640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18098a00 0x7f3f18131700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.101+0000 7f3f25891640 1 -- 172.21.15.44:0/3072623508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18098a00 msgr2=0x7f3f18131700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.101+0000 7f3f25891640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18098a00 0x7f3f18131700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.101+0000 7f3f25891640 1 -- 172.21.15.44:0/3072623508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f180049b0 msgr2=0x7f3f1812be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.101+0000 7f3f25891640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f180049b0 0x7f3f1812be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.101+0000 7f3f25891640 1 -- 172.21.15.44:0/3072623508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f18131ed0 con 0x7f3f18098060 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.101+0000 7f3f26893640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18098a00 0x7f3f18131700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.101+0000 7f3f26092640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f180049b0 0x7f3f1812be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.101+0000 7f3f25891640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f18098060 0x7f3f1812c370 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3f10035040 tx=0x7f3f10032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.102+0000 7f3f177fe640 1 -- 172.21.15.44:0/3072623508 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037040 con 0x7f3f18098060 2024-08-20T03:04:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.102+0000 7f3f177fe640 1 -- 172.21.15.44:0/3072623508 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f10041070 con 0x7f3f18098060 2024-08-20T03:04:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.102+0000 7f3f282f6640 1 -- 172.21.15.44:0/3072623508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f18132160 con 0x7f3f18098060 2024-08-20T03:04:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.102+0000 7f3f282f6640 1 -- 172.21.15.44:0/3072623508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f18138880 con 0x7f3f18098060 2024-08-20T03:04:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.103+0000 7f3f177fe640 1 -- 172.21.15.44:0/3072623508 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f100035f0 con 0x7f3f18098060 2024-08-20T03:04:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f2b1c7740 1 Processor -- start 2024-08-20T03:04:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f2b1c7740 1 -- start start 2024-08-20T03:04:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f2b1c7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 0x56119bc7abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f177fe640 1 -- 172.21.15.44:0/3072623508 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3f10004050 con 0x7f3f18098060 2024-08-20T03:04:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f2b1c7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bc8a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f2b1c7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc8aab0 0x56119bc944a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f2b1c7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56119bbf2360 con 0x56119bc8a150 2024-08-20T03:04:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f2b1c7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56119bbe1860 con 0x56119bc7a7d0 2024-08-20T03:04:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f2b1c7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56119bbe58a0 con 0x56119bc8aab0 2024-08-20T03:04:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f26893640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 0x56119bc7abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f25891640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc8aab0 0x56119bc944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f26893640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 0x56119bc7abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47218/0 (socket says 172.21.15.44:47218) 2024-08-20T03:04:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.107+0000 7f3f26893640 1 -- 172.21.15.44:0/993937257 learned_addr learned my addr 172.21.15.44:0/993937257 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.108+0000 7f3f26092640 1 --2- 172.21.15.44:0/993937257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bc8a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.108+0000 7f3f26893640 1 -- 172.21.15.44:0/993937257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc8aab0 msgr2=0x56119bc944a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.108+0000 7f3f26893640 1 --2- 172.21.15.44:0/993937257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc8aab0 0x56119bc944a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.108+0000 7f3f26893640 1 -- 172.21.15.44:0/993937257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 msgr2=0x56119bc8a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.108+0000 7f3f26893640 1 --2- 172.21.15.44:0/993937257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bc8a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.108+0000 7f3f26893640 1 -- 172.21.15.44:0/993937257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119bb88e40 con 0x56119bc7a7d0 2024-08-20T03:04:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.108+0000 7f3f26092640 1 --2- 172.21.15.44:0/993937257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bc8a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:04:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.108+0000 7f3f26893640 1 --2- 172.21.15.44:0/993937257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 0x56119bc7abb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f200666d0 tx=0x7f3f200512b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2695caadce3819 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.108+0000 7f3f14bfe640 1 -- 172.21.15.44:0/993937257 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2006c5b0 con 0x56119bc7a7d0 2024-08-20T03:04:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.108+0000 7f3f177fe640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3f0c0b3c20 0x7f3f0c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.108+0000 7f3f14bfe640 1 -- 172.21.15.44:0/993937257 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f2007c8c0 con 0x56119bc7a7d0 2024-08-20T03:04:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.108+0000 7f3f14bfe640 1 -- 172.21.15.44:0/993937257 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20086740 con 0x56119bc7a7d0 2024-08-20T03:04:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.109+0000 7f3f26092640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3f0c0b3c20 0x7f3f0c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.109+0000 7f3f177fe640 1 -- 172.21.15.44:0/3072623508 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3f10103b20 con 0x7f3f18098060 2024-08-20T03:04:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.109+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/993937257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 msgr2=0x56119bc7abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.109+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/993937257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 0x56119bc7abb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f200666d0 tx=0x7f3f200512b0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.109+0000 7f3f26092640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3f0c0b3c20 0x7f3f0c0b60e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c005fd0 tx=0x7f3f1c009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.109+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/993937257 shutdown_connections 2024-08-20T03:04:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.109+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/993937257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc8aab0 0x56119bc944a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.109+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/993937257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bc8a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.109+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/993937257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 0x56119bc7abb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.109+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/993937257 >> 172.21.15.44:0/993937257 conn(0x56119bc7cce0 msgr2=0x56119bc7d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:48.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.110+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/993937257 shutdown_connections 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.110+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/993937257 wait complete. 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.110+0000 7f3f2b1c7740 1 -- start start 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.110+0000 7f3f2b1c7740 1 Processor -- start 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.110+0000 7f3f2b1c7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc7a7d0 0x56119bc94220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.110+0000 7f3f2b1c7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc8a150 0x56119bc916b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.110+0000 7f3f2b1c7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8aab0 0x56119bc91c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.110+0000 7f3f2b1c7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56119bb1aab0 con 0x56119bc8aab0 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.110+0000 7f3f2b1c7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56119bb68780 con 0x56119bc8a150 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.110+0000 7f3f2b1c7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56119bc94ad0 con 0x56119bc7a7d0 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.110+0000 7f3f2b1c7740 10 client.0 _ll_register_callbacks cb 0x56119b8f0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f26893640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8aab0 0x56119bc91c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f25891640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc8a150 0x56119bc916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f26893640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8aab0 0x56119bc91c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60756/0 (socket says 172.21.15.44:60756) 2024-08-20T03:04:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f26893640 1 -- 172.21.15.44:0/1586256059 learned_addr learned my addr 172.21.15.44:0/1586256059 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f26092640 1 --2- 172.21.15.44:0/1586256059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc7a7d0 0x56119bc94220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f25891640 1 -- 172.21.15.44:0/1586256059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc7a7d0 msgr2=0x56119bc94220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f25891640 1 --2- 172.21.15.44:0/1586256059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc7a7d0 0x56119bc94220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f25891640 1 -- 172.21.15.44:0/1586256059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8aab0 msgr2=0x56119bc91c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f25891640 1 --2- 172.21.15.44:0/1586256059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8aab0 0x56119bc91c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f25891640 1 -- 172.21.15.44:0/1586256059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56119b944790 con 0x56119bc8a150 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f25891640 1 --2- 172.21.15.44:0/1586256059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc8a150 0x56119bc916b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f101013f0 tx=0x7f3f10030450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5fd20f48321ab1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f09ffb640 1 -- 172.21.15.44:0/1586256059 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f100c6770 con 0x56119bc8a150 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f09ffb640 1 -- 172.21.15.44:0/1586256059 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3f100d34d0 con 0x56119bc8a150 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.111+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1586256059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56119bb90e00 con 0x56119bc8a150 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.112+0000 7f3f09ffb640 1 -- 172.21.15.44:0/1586256059 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3f100ca020 con 0x56119bc8a150 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.112+0000 7f3f2b1c7740 10 client.25751 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.112+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1586256059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56119b9bd620 con 0x56119bc8a150 2024-08-20T03:04:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.112+0000 7f3f09ffb640 1 -- 172.21.15.44:0/1586256059 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56119bb90e00 con 0x56119bc8a150 2024-08-20T03:04:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.112+0000 7f3f09ffb640 1 -- 172.21.15.44:0/1586256059 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3f100ed0f0 con 0x56119bc8a150 2024-08-20T03:04:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.112+0000 7f3f2b1c7740 10 client.25751 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.112+0000 7f3f2b1c7740 10 client.25751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.112+0000 7f3f2b1c7740 20 client.25751 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.112+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1586256059 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56119bca1670 0x56119bca3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.112+0000 7f3f2b1c7740 4 client.25751 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:04:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.112+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1586256059 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56119bbf42b0 con 0x56119bca1670 2024-08-20T03:04:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.113+0000 7f3f26092640 1 --2- 172.21.15.44:0/1586256059 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56119bca1670 0x56119bca3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.114+0000 7f3f26092640 1 --2- 172.21.15.44:0/1586256059 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56119bca1670 0x56119bca3ad0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797dce8bff111d55 server_cookie=8518e7b4b7c6abbe in_seq=0 out_seq=0 2024-08-20T03:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.114+0000 7f3f09ffb640 10 client.25751 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.117+0000 7f3f09ffb640 1 -- 172.21.15.44:0/1586256059 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56119bbf42b0 con 0x56119bca1670 2024-08-20T03:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.117+0000 7f3f09ffb640 10 client.25751 handle_command_reply: tid=0 2024-08-20T03:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.117+0000 7f3f2b1c7740 1 client.25751 shutdown 2024-08-20T03:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.117+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1586256059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc8a150 msgr2=0x56119bc916b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.117+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1586256059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc8a150 0x56119bc916b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f101013f0 tx=0x7f3f10030450 comp rx=0 tx=0).stop 2024-08-20T03:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.117+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1586256059 shutdown_connections 2024-08-20T03:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.117+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1586256059 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56119bca1670 0x56119bca3ad0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.117+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1586256059 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8aab0 0x56119bc91c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.117+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1586256059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc8a150 0x56119bc916b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.117+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1586256059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc7a7d0 0x56119bc94220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.117+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1586256059 >> 172.21.15.44:0/1586256059 conn(0x56119bc7cce0 msgr2=0x56119bc8b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.117+0000 7f3f09ffb640 0 client.25751 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.118+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1586256059 shutdown_connections 2024-08-20T03:04:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.118+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1586256059 wait complete. 2024-08-20T03:04:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.118+0000 7f3f2b1c7740 20 client.25751 trim_cache size 0 max 16384 2024-08-20T03:04:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.150+0000 7f3f2b1c7740 1 Processor -- start 2024-08-20T03:04:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.150+0000 7f3f2b1c7740 1 -- start start 2024-08-20T03:04:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.151+0000 7f3f2b1c7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc7a7d0 0x56119bc8d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.151+0000 7f3f2b1c7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bc9dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.151+0000 7f3f2b1c7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc8aab0 0x56119bc90f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.151+0000 7f3f26092640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc7a7d0 0x56119bc8d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.151+0000 7f3f26092640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc7a7d0 0x56119bc8d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:42764/0 (socket says 172.21.15.44:42764) 2024-08-20T03:04:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.152+0000 7f3f25891640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bc9dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.152+0000 7f3f25891640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bc9dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60772/0 (socket says 172.21.15.44:60772) 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.152+0000 7f3f26092640 1 -- 172.21.15.44:0/2122713327 learned_addr learned my addr 172.21.15.44:0/2122713327 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.152+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/2122713327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0400e5d0 con 0x56119bc8a150 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.152+0000 7f3f26893640 1 --2- 172.21.15.44:0/2122713327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc8aab0 0x56119bc90f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.152+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/2122713327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f040101d0 con 0x56119bc8aab0 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.152+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/2122713327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0400eca0 con 0x56119bc7a7d0 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.152+0000 7f3f26092640 1 -- 172.21.15.44:0/2122713327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc8aab0 msgr2=0x56119bc90f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.152+0000 7f3f26092640 1 --2- 172.21.15.44:0/2122713327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc8aab0 0x56119bc90f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.152+0000 7f3f26092640 1 -- 172.21.15.44:0/2122713327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 msgr2=0x56119bc9dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.153+0000 7f3f26092640 1 --2- 172.21.15.44:0/2122713327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bc9dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.153+0000 7f3f26092640 1 -- 172.21.15.44:0/2122713327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56119bc7af20 con 0x56119bc7a7d0 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.153+0000 7f3f26893640 1 --2- 172.21.15.44:0/2122713327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc8aab0 0x56119bc90f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.153+0000 7f3f25891640 1 --2- 172.21.15.44:0/2122713327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bc9dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:04:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.153+0000 7f3f26092640 1 --2- 172.21.15.44:0/2122713327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc7a7d0 0x56119bc8d930 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c005dc0 tx=0x7f3f1c019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=141fbf108ff30290 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.153+0000 7f3f0bfff640 1 -- 172.21.15.44:0/2122713327 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c01fdb0 con 0x56119bc7a7d0 2024-08-20T03:04:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.153+0000 7f3f0bfff640 1 -- 172.21.15.44:0/2122713327 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56119bbf42b0 con 0x56119bc7a7d0 2024-08-20T03:04:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.153+0000 7f3f0bfff640 1 -- 172.21.15.44:0/2122713327 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c0117c0 con 0x56119bc7a7d0 2024-08-20T03:04:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.154+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/2122713327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc7a7d0 msgr2=0x56119bc8d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.154+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/2122713327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc7a7d0 0x56119bc8d930 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c005dc0 tx=0x7f3f1c019000 comp rx=0 tx=0).stop 2024-08-20T03:04:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.155+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/2122713327 shutdown_connections 2024-08-20T03:04:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.155+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/2122713327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc8aab0 0x56119bc90f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.155+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/2122713327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bc9dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.155+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/2122713327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc7a7d0 0x56119bc8d930 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.155+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/2122713327 >> 172.21.15.44:0/2122713327 conn(0x56119bc7cce0 msgr2=0x56119bc8d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.155+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/2122713327 shutdown_connections 2024-08-20T03:04:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.155+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/2122713327 wait complete. 2024-08-20T03:04:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.156+0000 7f3f2b1c7740 1 -- start start 2024-08-20T03:04:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.156+0000 7f3f2b1c7740 1 Processor -- start 2024-08-20T03:04:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.156+0000 7f3f2b1c7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 0x56119bc9a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.157+0000 7f3f2b1c7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bca8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.157+0000 7f3f26092640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 0x56119bc9a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.157+0000 7f3f26092640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 0x56119bc9a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47238/0 (socket says 172.21.15.44:47238) 2024-08-20T03:04:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.157+0000 7f3f25891640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bca8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.157+0000 7f3f25891640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bca8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60786/0 (socket says 172.21.15.44:60786) 2024-08-20T03:04:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.157+0000 7f3f26092640 1 -- 172.21.15.44:0/1711654891 learned_addr learned my addr 172.21.15.44:0/1711654891 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:04:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.157+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc8aab0 0x56119bca9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.157+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1711654891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0400fa40 con 0x56119bc8a150 2024-08-20T03:04:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.157+0000 7f3f26893640 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc8aab0 0x56119bca9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.157+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1711654891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0400f370 con 0x56119bc7a7d0 2024-08-20T03:04:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.158+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1711654891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0400c5b0 con 0x56119bc8aab0 2024-08-20T03:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.158+0000 7f3f25891640 1 -- 172.21.15.44:0/1711654891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc8aab0 msgr2=0x56119bca9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.158+0000 7f3f25891640 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc8aab0 0x56119bca9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.158+0000 7f3f25891640 1 -- 172.21.15.44:0/1711654891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 msgr2=0x56119bc9a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.158+0000 7f3f25891640 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 0x56119bc9a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.158+0000 7f3f25891640 1 -- 172.21.15.44:0/1711654891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56119bc95fe0 con 0x56119bc8a150 2024-08-20T03:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.159+0000 7f3f2b1c7740 10 client.0 _ll_register_callbacks cb 0x7f3f100097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.159+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1711654891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56119bbf42b0 con 0x56119bc8a150 2024-08-20T03:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.159+0000 7f3f25891640 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bca8ee0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3f1002a930 tx=0x7f3f10005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6b94140de0b1da server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.159+0000 7f3f09ffb640 10 client.16272 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.159+0000 7f3f09ffb640 1 -- 172.21.15.44:0/1711654891 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f100ca200 con 0x56119bc8a150 2024-08-20T03:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.160+0000 7f3f09ffb640 1 -- 172.21.15.44:0/1711654891 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3f10107240 con 0x56119bc8a150 2024-08-20T03:04:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.160+0000 7f3f09ffb640 1 -- 172.21.15.44:0/1711654891 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3f100ca680 con 0x56119bc8a150 2024-08-20T03:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.160+0000 7f3f2b1c7740 10 client.16272 fetch_fsmap learned FSMap version 30 2024-08-20T03:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.160+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1711654891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=30}) -- 0x56119bd0c460 con 0x56119bc8a150 2024-08-20T03:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.160+0000 7f3f09ffb640 1 -- 172.21.15.44:0/1711654891 <== mon.0 v2:172.21.15.44:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3f100cbbe0 con 0x56119bc8a150 2024-08-20T03:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.160+0000 7f3f2b1c7740 10 client.16272 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.160+0000 7f3f2b1c7740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.161+0000 7f3f2b1c7740 20 client.16272 populate_metadata read hostname 'smithi044' 2024-08-20T03:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.161+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56119bd0ca10 0x56119bd0ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.161+0000 7f3f2b1c7740 4 client.16272 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.161+0000 7f3f26092640 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56119bd0ca10 0x56119bd0ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.161+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1711654891 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56119bbefd50 con 0x56119bd0ca10 2024-08-20T03:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.162+0000 7f3f09ffb640 10 client.16272 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:48.162+0000 7f3f26092640 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56119bd0ca10 0x56119bd0ee30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c4fc4f580d8ee28 server_cookie=256b20e2a2cc12a7 in_seq=0 out_seq=0 2024-08-20T03:04:48.385 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:48 smithi165 ceph-mon[22357]: pgmap v389: 97 pgs: 97 active+clean; 249 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 1.28k op/s 2024-08-20T03:04:48.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:48 smithi099 ceph-mon[21984]: pgmap v389: 97 pgs: 97 active+clean; 249 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 1.28k op/s 2024-08-20T03:04:48.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:48 smithi044 ceph-mon[18790]: pgmap v389: 97 pgs: 97 active+clean; 249 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 1.28k op/s 2024-08-20T03:04:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:50 smithi165 ceph-mon[22357]: pgmap v390: 97 pgs: 97 active+clean; 253 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.4 MiB/s wr, 1.01k op/s 2024-08-20T03:04:50.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:50 smithi099 ceph-mon[21984]: pgmap v390: 97 pgs: 97 active+clean; 253 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.4 MiB/s wr, 1.01k op/s 2024-08-20T03:04:50.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:50 smithi044 ceph-mon[18790]: pgmap v390: 97 pgs: 97 active+clean; 253 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.4 MiB/s wr, 1.01k op/s 2024-08-20T03:04:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.033+0000 7f3f09ffb640 1 -- 172.21.15.44:0/1711654891 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56119bbefd50 con 0x56119bd0ca10 2024-08-20T03:04:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.033+0000 7f3f09ffb640 10 client.16272 handle_command_reply: tid=0 2024-08-20T03:04:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.033+0000 7f3f2b1c7740 1 client.16272 shutdown 2024-08-20T03:04:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.034+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1711654891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 msgr2=0x56119bca8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.034+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bca8ee0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3f1002a930 tx=0x7f3f10005bd0 comp rx=0 tx=0).stop 2024-08-20T03:04:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.034+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1711654891 shutdown_connections 2024-08-20T03:04:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.034+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56119bd0ca10 0x56119bd0ee30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.034+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56119bc8aab0 0x56119bca9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.034+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56119bc8a150 0x56119bca8ee0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.034+0000 7f3f2b1c7740 1 --2- 172.21.15.44:0/1711654891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56119bc7a7d0 0x56119bc9a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.034+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1711654891 >> 172.21.15.44:0/1711654891 conn(0x56119bc7cce0 msgr2=0x56119bc8b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.034+0000 7f3f09ffb640 0 client.16272 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:04:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.034+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1711654891 shutdown_connections 2024-08-20T03:04:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.034+0000 7f3f2b1c7740 1 -- 172.21.15.44:0/1711654891 wait complete. 2024-08-20T03:04:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.034+0000 7f3f2b1c7740 20 client.16272 trim_cache size 0 max 16384 2024-08-20T03:04:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:04:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:04:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:04:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:04:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.035+0000 7f3f282f6640 1 -- 172.21.15.44:0/3072623508 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3f0c0b3c20 msgr2=0x7f3f0c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.035+0000 7f3f282f6640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3f0c0b3c20 0x7f3f0c0b60e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c005fd0 tx=0x7f3f1c009260 comp rx=0 tx=0).stop 2024-08-20T03:04:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.035+0000 7f3f282f6640 1 -- 172.21.15.44:0/3072623508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f18098060 msgr2=0x7f3f1812c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:04:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.035+0000 7f3f282f6640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f18098060 0x7f3f1812c370 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3f10035040 tx=0x7f3f10032d50 comp rx=0 tx=0).stop 2024-08-20T03:04:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.035+0000 7f3f282f6640 1 -- 172.21.15.44:0/3072623508 shutdown_connections 2024-08-20T03:04:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.035+0000 7f3f282f6640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3f0c0b3c20 0x7f3f0c0b60e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.035+0000 7f3f282f6640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f18098a00 0x7f3f18131700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.035+0000 7f3f282f6640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f18098060 0x7f3f1812c370 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.035+0000 7f3f282f6640 1 --2- 172.21.15.44:0/3072623508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3f180049b0 0x7f3f1812be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:04:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.035+0000 7f3f282f6640 1 -- 172.21.15.44:0/3072623508 >> 172.21.15.44:0/3072623508 conn(0x7f3f1800a180 msgr2=0x7f3f1809b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:04:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.035+0000 7f3f282f6640 1 -- 172.21.15.44:0/3072623508 shutdown_connections 2024-08-20T03:04:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:04:52.035+0000 7f3f282f6640 1 -- 172.21.15.44:0/3072623508 wait complete. 2024-08-20T03:04:52.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:52 smithi165 ceph-mon[22357]: pgmap v391: 97 pgs: 97 active+clean; 255 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 717 op/s 2024-08-20T03:04:52.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:52 smithi099 ceph-mon[21984]: pgmap v391: 97 pgs: 97 active+clean; 255 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 717 op/s 2024-08-20T03:04:52.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:52 smithi044 ceph-mon[18790]: pgmap v391: 97 pgs: 97 active+clean; 255 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 717 op/s 2024-08-20T03:04:54.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:54 smithi165 ceph-mon[22357]: pgmap v392: 97 pgs: 97 active+clean; 253 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.3 MiB/s wr, 1.22k op/s 2024-08-20T03:04:54.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:54 smithi099 ceph-mon[21984]: pgmap v392: 97 pgs: 97 active+clean; 253 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.3 MiB/s wr, 1.22k op/s 2024-08-20T03:04:54.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:54 smithi044 ceph-mon[18790]: pgmap v392: 97 pgs: 97 active+clean; 253 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.3 MiB/s wr, 1.22k op/s 2024-08-20T03:04:56.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:56 smithi099 ceph-mon[21984]: pgmap v393: 97 pgs: 97 active+clean; 245 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 1.36k op/s 2024-08-20T03:04:56.539 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:56 smithi044 ceph-mon[18790]: pgmap v393: 97 pgs: 97 active+clean; 245 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 1.36k op/s 2024-08-20T03:04:56.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:56 smithi165 ceph-mon[22357]: pgmap v393: 97 pgs: 97 active+clean; 245 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 1.36k op/s 2024-08-20T03:04:57.476 INFO:tasks.workunit.client.0.smithi044.stdout: 83 1004254 513 720144 387 568728 1118 2024-08-20T03:04:58.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:04:58 smithi099 ceph-mon[21984]: pgmap v394: 97 pgs: 97 active+clean; 262 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.2 MiB/s wr, 1.46k op/s 2024-08-20T03:04:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:04:58 smithi044 ceph-mon[18790]: pgmap v394: 97 pgs: 97 active+clean; 262 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.2 MiB/s wr, 1.46k op/s 2024-08-20T03:04:58.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:04:58 smithi165 ceph-mon[22357]: pgmap v394: 97 pgs: 97 active+clean; 262 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.2 MiB/s wr, 1.46k op/s 2024-08-20T03:05:00.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:00 smithi099 ceph-mon[21984]: pgmap v395: 97 pgs: 97 active+clean; 294 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.0 MiB/s wr, 1.55k op/s 2024-08-20T03:05:00.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:00 smithi044 ceph-mon[18790]: pgmap v395: 97 pgs: 97 active+clean; 294 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.0 MiB/s wr, 1.55k op/s 2024-08-20T03:05:00.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:00 smithi165 ceph-mon[22357]: pgmap v395: 97 pgs: 97 active+clean; 294 MiB data, 989 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.0 MiB/s wr, 1.55k op/s 2024-08-20T03:05:02.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:02 smithi099 ceph-mon[21984]: pgmap v396: 97 pgs: 97 active+clean; 302 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.3 MiB/s wr, 1.45k op/s 2024-08-20T03:05:02.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:02 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:05:02.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:02 smithi044 ceph-mon[18790]: pgmap v396: 97 pgs: 97 active+clean; 302 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.3 MiB/s wr, 1.45k op/s 2024-08-20T03:05:02.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:02 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:05:02.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:02 smithi165 ceph-mon[22357]: pgmap v396: 97 pgs: 97 active+clean; 302 MiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.3 MiB/s wr, 1.45k op/s 2024-08-20T03:05:02.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:02 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:05:03.079 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:05:03.368 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:05:03.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.908+0000 7f3e72b43640 1 -- 172.21.15.44:0/2134305186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e6c1073a0 msgr2=0x7f3e6c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:03.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.908+0000 7f3e72b43640 1 --2- 172.21.15.44:0/2134305186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e6c1073a0 0x7f3e6c1077e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009900 tx=0x7f3e5802a410 comp rx=0 tx=0).stop 2024-08-20T03:05:03.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.908+0000 7f3e72b43640 1 -- 172.21.15.44:0/2134305186 shutdown_connections 2024-08-20T03:05:03.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.908+0000 7f3e72b43640 1 --2- 172.21.15.44:0/2134305186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e6c107d20 0x7f3e6c10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.908+0000 7f3e72b43640 1 --2- 172.21.15.44:0/2134305186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e6c1073a0 0x7f3e6c1077e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.908+0000 7f3e72b43640 1 --2- 172.21.15.44:0/2134305186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e6c101990 0x7f3e6c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.908+0000 7f3e72b43640 1 -- 172.21.15.44:0/2134305186 >> 172.21.15.44:0/2134305186 conn(0x7f3e6c0fd400 msgr2=0x7f3e6c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:03.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.908+0000 7f3e72b43640 1 -- 172.21.15.44:0/2134305186 shutdown_connections 2024-08-20T03:05:03.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.908+0000 7f3e72b43640 1 -- 172.21.15.44:0/2134305186 wait complete. 2024-08-20T03:05:03.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e72b43640 1 Processor -- start 2024-08-20T03:05:03.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e72b43640 1 -- start start 2024-08-20T03:05:03.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e72b43640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e6c101990 0x7f3e6c198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e72b43640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e6c1073a0 0x7f3e6c199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e72b43640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e6c107d20 0x7f3e6c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e72b43640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c111970 con 0x7f3e6c1073a0 2024-08-20T03:05:03.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e72b43640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1117f0 con 0x7f3e6c107d20 2024-08-20T03:05:03.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e72b43640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c111af0 con 0x7f3e6c101990 2024-08-20T03:05:03.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e6bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e6c1073a0 0x7f3e6c199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e708df640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e6c101990 0x7f3e6c198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e6bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e6c1073a0 0x7f3e6c199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44084/0 (socket says 172.21.15.44:44084) 2024-08-20T03:05:03.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e6bfff640 1 -- 172.21.15.44:0/898510983 learned_addr learned my addr 172.21.15.44:0/898510983 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:03.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e710e0640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e6c107d20 0x7f3e6c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e6bfff640 1 -- 172.21.15.44:0/898510983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e6c101990 msgr2=0x7f3e6c198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:03.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e6bfff640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e6c101990 0x7f3e6c198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e6bfff640 1 -- 172.21.15.44:0/898510983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e6c107d20 msgr2=0x7f3e6c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:03.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.909+0000 7f3e6bfff640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e6c107d20 0x7f3e6c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.910+0000 7f3e6bfff640 1 -- 172.21.15.44:0/898510983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6c19f080 con 0x7f3e6c1073a0 2024-08-20T03:05:03.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.910+0000 7f3e708df640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e6c101990 0x7f3e6c198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:03.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.910+0000 7f3e710e0640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e6c107d20 0x7f3e6c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:03.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.910+0000 7f3e6bfff640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e6c1073a0 0x7f3e6c199510 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3e58005940 tx=0x7f3e58004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:03.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.910+0000 7f3e69ffb640 1 -- 172.21.15.44:0/898510983 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5803f030 con 0x7f3e6c1073a0 2024-08-20T03:05:03.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.910+0000 7f3e69ffb640 1 -- 172.21.15.44:0/898510983 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e58003aa0 con 0x7f3e6c1073a0 2024-08-20T03:05:03.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.910+0000 7f3e72b43640 1 -- 172.21.15.44:0/898510983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6c19f310 con 0x7f3e6c1073a0 2024-08-20T03:05:03.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.910+0000 7f3e69ffb640 1 -- 172.21.15.44:0/898510983 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58035070 con 0x7f3e6c1073a0 2024-08-20T03:05:03.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.910+0000 7f3e72b43640 1 -- 172.21.15.44:0/898510983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e6c1a5a30 con 0x7f3e6c1073a0 2024-08-20T03:05:03.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e74f3c740 1 Processor -- start 2024-08-20T03:05:03.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e69ffb640 1 -- 172.21.15.44:0/898510983 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3e58042030 con 0x7f3e6c1073a0 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e74f3c740 1 -- start start 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e74f3c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bfe0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e74f3c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfeca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e74f3c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfecf90 0x55889bfef380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e74f3c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55889bf578e0 con 0x55889bfe1000 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e74f3c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55889bf46de0 con 0x55889bfecf90 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e74f3c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55889bf4ae20 con 0x55889bfe06e0 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e708df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfeca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e708df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfeca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44090/0 (socket says 172.21.15.44:44090) 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e708df640 1 -- 172.21.15.44:0/2674865775 learned_addr learned my addr 172.21.15.44:0/2674865775 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e710e0640 1 --2- 172.21.15.44:0/2674865775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bfe0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e6bfff640 1 --2- 172.21.15.44:0/2674865775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfecf90 0x55889bfef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e708df640 1 -- 172.21.15.44:0/2674865775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 msgr2=0x55889bfe0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e708df640 1 --2- 172.21.15.44:0/2674865775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bfe0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e708df640 1 -- 172.21.15.44:0/2674865775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfecf90 msgr2=0x55889bfef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e708df640 1 --2- 172.21.15.44:0/2674865775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfecf90 0x55889bfef380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e708df640 1 -- 172.21.15.44:0/2674865775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889bca6650 con 0x55889bfe1000 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.913+0000 7f3e710e0640 1 --2- 172.21.15.44:0/2674865775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bfe0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.914+0000 7f3e69ffb640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3e5c0b3dc0 0x7f3e5c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.914+0000 7f3e708df640 1 --2- 172.21.15.44:0/2674865775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfeca50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3e600105b0 tx=0x7f3e60010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdad057eddf0cd4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.914+0000 7f3e710e0640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3e5c0b3dc0 0x7f3e5c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.914+0000 7f3e69ffb640 1 -- 172.21.15.44:0/898510983 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3e58104300 con 0x7f3e6c1073a0 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.914+0000 7f3e56ffd640 1 -- 172.21.15.44:0/2674865775 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6000e770 con 0x55889bfe1000 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.914+0000 7f3e56ffd640 1 -- 172.21.15.44:0/2674865775 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e6000edc0 con 0x55889bfe1000 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.914+0000 7f3e56ffd640 1 -- 172.21.15.44:0/2674865775 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6000e770 con 0x55889bfe1000 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.915+0000 7f3e74f3c740 1 -- 172.21.15.44:0/2674865775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 msgr2=0x55889bfeca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:03.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.915+0000 7f3e710e0640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3e5c0b3dc0 0x7f3e5c0b6280 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009fd0 tx=0x7f3e64009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.915+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/2674865775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfeca50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3e600105b0 tx=0x7f3e60010a70 comp rx=0 tx=0).stop 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.915+0000 7f3e74f3c740 1 -- 172.21.15.44:0/2674865775 shutdown_connections 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.915+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/2674865775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfecf90 0x55889bfef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.915+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/2674865775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfeca50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.915+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/2674865775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bfe0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.915+0000 7f3e74f3c740 1 -- 172.21.15.44:0/2674865775 >> 172.21.15.44:0/2674865775 conn(0x55889bfde510 msgr2=0x55889bfdf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.915+0000 7f3e74f3c740 1 -- 172.21.15.44:0/2674865775 shutdown_connections 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.915+0000 7f3e74f3c740 1 -- 172.21.15.44:0/2674865775 wait complete. 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.915+0000 7f3e74f3c740 1 -- start start 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.915+0000 7f3e74f3c740 1 Processor -- start 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.916+0000 7f3e74f3c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfe06e0 0x55889bfe9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.916+0000 7f3e74f3c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfea4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.916+0000 7f3e74f3c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfecf90 0x55889bfea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.916+0000 7f3e74f3c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55889bda9700 con 0x55889bfe1000 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.916+0000 7f3e74f3c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55889becd7c0 con 0x55889bfe06e0 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.916+0000 7f3e74f3c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55889bfefc40 con 0x55889bfecf90 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.916+0000 7f3e6bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfea4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.916+0000 7f3e708df640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfe06e0 0x55889bfe9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.916+0000 7f3e6bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfea4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44092/0 (socket says 172.21.15.44:44092) 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.916+0000 7f3e6bfff640 1 -- 172.21.15.44:0/1508606581 learned_addr learned my addr 172.21.15.44:0/1508606581 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.916+0000 7f3e710e0640 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfecf90 0x55889bfea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e74f3c740 10 client.0 _ll_register_callbacks cb 0x55889bf406b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e708df640 1 -- 172.21.15.44:0/1508606581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfecf90 msgr2=0x55889bfea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e708df640 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfecf90 0x55889bfea9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e708df640 1 -- 172.21.15.44:0/1508606581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 msgr2=0x55889bfea4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e708df640 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfea4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e708df640 1 -- 172.21.15.44:0/1508606581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889bff9050 con 0x55889bfe06e0 2024-08-20T03:05:03.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e6bfff640 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfea4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e708df640 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfe06e0 0x55889bfe9f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3e60000c00 tx=0x7f3e60005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7d8e117bfe646f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e4bfff640 1 -- 172.21.15.44:0/1508606581 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6000e770 con 0x55889bfe06e0 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e4bfff640 1 -- 172.21.15.44:0/1508606581 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3e6000e910 con 0x55889bfe06e0 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1508606581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55889bef6100 con 0x55889bfe06e0 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.917+0000 7f3e4bfff640 1 -- 172.21.15.44:0/1508606581 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3e600116d0 con 0x55889bfe06e0 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.918+0000 7f3e74f3c740 10 client.25769 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.918+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1508606581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55889bffb9e0 con 0x55889bfe06e0 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.918+0000 7f3e4bfff640 1 -- 172.21.15.44:0/1508606581 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55889bef6100 con 0x55889bfe06e0 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.918+0000 7f3e4bfff640 1 -- 172.21.15.44:0/1508606581 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3e6001cb50 con 0x55889bfe06e0 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.918+0000 7f3e74f3c740 10 client.25769 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.918+0000 7f3e74f3c740 10 client.25769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.918+0000 7f3e74f3c740 20 client.25769 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:03.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.918+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55889bffce40 0x55889bfff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.918+0000 7f3e74f3c740 4 client.25769 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:05:03.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.918+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1508606581 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55889bf59830 con 0x55889bffce40 2024-08-20T03:05:03.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.918+0000 7f3e6bfff640 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55889bffce40 0x55889bfff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.919+0000 7f3e4bfff640 10 client.25769 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:03.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.919+0000 7f3e6bfff640 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55889bffce40 0x55889bfff230 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805e1fb3457011f5 server_cookie=22644626c0e2a134 in_seq=0 out_seq=0 2024-08-20T03:05:03.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e4bfff640 1 -- 172.21.15.44:0/1508606581 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55889bf59830 con 0x55889bffce40 2024-08-20T03:05:03.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e4bfff640 10 client.25769 handle_command_reply: tid=0 2024-08-20T03:05:03.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e74f3c740 1 client.25769 shutdown 2024-08-20T03:05:03.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1508606581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfe06e0 msgr2=0x55889bfe9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:03.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfe06e0 0x55889bfe9f60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3e60000c00 tx=0x7f3e60005750 comp rx=0 tx=0).stop 2024-08-20T03:05:03.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1508606581 shutdown_connections 2024-08-20T03:05:03.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55889bffce40 0x55889bfff230 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfecf90 0x55889bfea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfea4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e4bfff640 0 client.25769 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:03.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1508606581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfe06e0 0x55889bfe9f60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1508606581 >> 172.21.15.44:0/1508606581 conn(0x55889bfde510 msgr2=0x55889bfe40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:03.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1508606581 shutdown_connections 2024-08-20T03:05:03.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1508606581 wait complete. 2024-08-20T03:05:03.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.922+0000 7f3e74f3c740 20 client.25769 trim_cache size 0 max 16384 2024-08-20T03:05:03.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.941+0000 7f3e74f3c740 1 Processor -- start 2024-08-20T03:05:03.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.941+0000 7f3e74f3c740 1 -- start start 2024-08-20T03:05:03.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.941+0000 7f3e74f3c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bfe47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.941+0000 7f3e74f3c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfdd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.941+0000 7f3e74f3c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfecf90 0x55889bfea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.941+0000 7f3e74f3c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1ae2d0 con 0x55889bfe1000 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.941+0000 7f3e74f3c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1afed0 con 0x55889bfecf90 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.941+0000 7f3e74f3c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1ae9a0 con 0x55889bfe06e0 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e6bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfdd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e6bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfdd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44104/0 (socket says 172.21.15.44:44104) 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e6bfff640 1 -- 172.21.15.44:0/1888031774 learned_addr learned my addr 172.21.15.44:0/1888031774 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e708df640 1 --2- 172.21.15.44:0/1888031774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bfe47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e710e0640 1 --2- 172.21.15.44:0/1888031774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfecf90 0x55889bfea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e6bfff640 1 -- 172.21.15.44:0/1888031774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 msgr2=0x55889bfe47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e6bfff640 1 --2- 172.21.15.44:0/1888031774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bfe47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e6bfff640 1 -- 172.21.15.44:0/1888031774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfecf90 msgr2=0x55889bfea460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:03.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e6bfff640 1 --2- 172.21.15.44:0/1888031774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfecf90 0x55889bfea460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e6bfff640 1 -- 172.21.15.44:0/1888031774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889bfe8ec0 con 0x55889bfe1000 2024-08-20T03:05:03.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e6bfff640 1 --2- 172.21.15.44:0/1888031774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfdd390 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3e58101b70 tx=0x7f3e58109040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca6565d865975 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:03.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e567fc640 1 -- 172.21.15.44:0/1888031774 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e580ae070 con 0x55889bfe1000 2024-08-20T03:05:03.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e567fc640 1 -- 172.21.15.44:0/1888031774 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e580cab40 con 0x55889bfe1000 2024-08-20T03:05:03.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.942+0000 7f3e567fc640 1 -- 172.21.15.44:0/1888031774 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e58049040 con 0x55889bfe1000 2024-08-20T03:05:03.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.943+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1888031774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 msgr2=0x55889bfdd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:03.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.943+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1888031774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfdd390 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3e58101b70 tx=0x7f3e58109040 comp rx=0 tx=0).stop 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.943+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1888031774 shutdown_connections 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.943+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1888031774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfecf90 0x55889bfea460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.943+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1888031774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfe1000 0x55889bfdd390 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.943+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1888031774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bfe47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.943+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1888031774 >> 172.21.15.44:0/1888031774 conn(0x55889bfde510 msgr2=0x55889bfdcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.943+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1888031774 shutdown_connections 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.943+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1888031774 wait complete. 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.943+0000 7f3e74f3c740 1 -- start start 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.943+0000 7f3e74f3c740 1 Processor -- start 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e74f3c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bff5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e74f3c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfe1000 0x55889bfefec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e74f3c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfecf90 0x55889bff0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e74f3c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1af740 con 0x55889bfecf90 2024-08-20T03:05:03.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e74f3c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1af070 con 0x55889bfe1000 2024-08-20T03:05:03.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e74f3c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c1ac2b0 con 0x55889bfe06e0 2024-08-20T03:05:03.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e74f3c740 10 client.0 _ll_register_callbacks cb 0x7f3e58107f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:03.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e710e0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfecf90 0x55889bff0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e6bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfe1000 0x55889bfefec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e708df640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bff5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e710e0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfecf90 0x55889bff0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44110/0 (socket says 172.21.15.44:44110) 2024-08-20T03:05:03.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e710e0640 1 -- 172.21.15.44:0/1328203407 learned_addr learned my addr 172.21.15.44:0/1328203407 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:03.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e710e0640 1 -- 172.21.15.44:0/1328203407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 msgr2=0x55889bff5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:03.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e710e0640 1 --2- 172.21.15.44:0/1328203407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bff5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e710e0640 1 -- 172.21.15.44:0/1328203407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfe1000 msgr2=0x55889bfefec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:03.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e710e0640 1 --2- 172.21.15.44:0/1328203407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfe1000 0x55889bfefec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e710e0640 1 -- 172.21.15.44:0/1328203407 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889bff0b90 con 0x55889bfecf90 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e6bfff640 1 --2- 172.21.15.44:0/1328203407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfe1000 0x55889bfefec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e710e0640 1 --2- 172.21.15.44:0/1328203407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfecf90 0x55889bff0400 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3e640121c0 tx=0x7f3e64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c923171118fbe45 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.944+0000 7f3e4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e4bfff640 1 -- 172.21.15.44:0/1328203407 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6402f060 con 0x55889bfecf90 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e4bfff640 1 -- 172.21.15.44:0/1328203407 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f3e640119c0 con 0x55889bfecf90 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1328203407 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e580cab40 con 0x55889bfecf90 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e4bfff640 1 -- 172.21.15.44:0/1328203407 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3e64011bc0 con 0x55889bfecf90 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e74f3c740 10 client.16290 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1328203407 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55889c066ad0 con 0x55889bfecf90 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e4bfff640 1 -- 172.21.15.44:0/1328203407 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e580cab40 con 0x55889bfecf90 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e4bfff640 1 -- 172.21.15.44:0/1328203407 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f3e64032020 con 0x55889bfecf90 2024-08-20T03:05:03.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e74f3c740 10 client.16290 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e74f3c740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e74f3c740 20 client.16290 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1328203407 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55889c0672e0 0x55889c069700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e74f3c740 4 client.16290 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.945+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1328203407 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55889bf552d0 con 0x55889c0672e0 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.946+0000 7f3e708df640 1 --2- 172.21.15.44:0/1328203407 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55889c0672e0 0x55889c069700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.946+0000 7f3e708df640 1 --2- 172.21.15.44:0/1328203407 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55889c0672e0 0x55889c069700 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc61f90516c29a84 server_cookie=30e8cbe843a3a02d in_seq=0 out_seq=0 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.946+0000 7f3e4bfff640 10 client.16290 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e4bfff640 1 -- 172.21.15.44:0/1328203407 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55889bf552d0 con 0x55889c0672e0 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e4bfff640 10 client.16290 handle_command_reply: tid=0 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e74f3c740 1 client.16290 shutdown 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1328203407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfecf90 msgr2=0x55889bff0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:03.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1328203407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfecf90 0x55889bff0400 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3e640121c0 tx=0x7f3e64004a40 comp rx=0 tx=0).stop 2024-08-20T03:05:03.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1328203407 shutdown_connections 2024-08-20T03:05:03.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1328203407 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55889c0672e0 0x55889c069700 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1328203407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55889bfecf90 0x55889bff0400 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1328203407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55889bfe1000 0x55889bfefec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e4bfff640 0 client.16290 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:03.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e74f3c740 1 --2- 172.21.15.44:0/1328203407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55889bfe06e0 0x55889bff5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1328203407 >> 172.21.15.44:0/1328203407 conn(0x55889bfde510 msgr2=0x55889bfdd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:03.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1328203407 shutdown_connections 2024-08-20T03:05:03.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e74f3c740 1 -- 172.21.15.44:0/1328203407 wait complete. 2024-08-20T03:05:03.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.947+0000 7f3e74f3c740 20 client.16290 trim_cache size 0 max 16384 2024-08-20T03:05:03.955 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:05:03.955 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-20T03:05:03.955 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-08-20T03:05:03.955 INFO:teuthology.orchestra.run.smithi044.stdout: "ce02f953-98f5-4617-85e4-88059e468dec": { 2024-08-20T03:05:03.955 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-08-20T03:05:03.955 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "ce02f953-98f5-4617-85e4-88059e468dec", 2024-08-20T03:05:03.955 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-08-20T03:05:03.955 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:05:03.955 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:05:03.955 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:05:03.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.948+0000 7f3e72b43640 1 -- 172.21.15.44:0/898510983 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3e5c0b3dc0 msgr2=0x7f3e5c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:03.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.948+0000 7f3e72b43640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3e5c0b3dc0 0x7f3e5c0b6280 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3e64009fd0 tx=0x7f3e64009500 comp rx=0 tx=0).stop 2024-08-20T03:05:03.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.948+0000 7f3e72b43640 1 -- 172.21.15.44:0/898510983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e6c1073a0 msgr2=0x7f3e6c199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:03.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.948+0000 7f3e72b43640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e6c1073a0 0x7f3e6c199510 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3e58005940 tx=0x7f3e58004270 comp rx=0 tx=0).stop 2024-08-20T03:05:03.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.948+0000 7f3e72b43640 1 -- 172.21.15.44:0/898510983 shutdown_connections 2024-08-20T03:05:03.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.948+0000 7f3e72b43640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3e5c0b3dc0 0x7f3e5c0b6280 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.948+0000 7f3e72b43640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e6c107d20 0x7f3e6c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.948+0000 7f3e72b43640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e6c1073a0 0x7f3e6c199510 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.948+0000 7f3e72b43640 1 --2- 172.21.15.44:0/898510983 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e6c101990 0x7f3e6c198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:03.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.948+0000 7f3e72b43640 1 -- 172.21.15.44:0/898510983 >> 172.21.15.44:0/898510983 conn(0x7f3e6c0fd400 msgr2=0x7f3e6c1044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:03.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.948+0000 7f3e72b43640 1 -- 172.21.15.44:0/898510983 shutdown_connections 2024-08-20T03:05:03.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:03.948+0000 7f3e72b43640 1 -- 172.21.15.44:0/898510983 wait complete. 2024-08-20T03:05:04.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:04 smithi099 ceph-mon[21984]: pgmap v397: 97 pgs: 97 active+clean; 325 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.4 MiB/s wr, 1.69k op/s 2024-08-20T03:05:04.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:04 smithi044 ceph-mon[18790]: pgmap v397: 97 pgs: 97 active+clean; 325 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.4 MiB/s wr, 1.69k op/s 2024-08-20T03:05:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:04 smithi165 ceph-mon[22357]: pgmap v397: 97 pgs: 97 active+clean; 325 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 9.4 MiB/s wr, 1.69k op/s 2024-08-20T03:05:05.156 INFO:tasks.cephfs.filesystem:scrub status for tag:ce02f953-98f5-4617-85e4-88059e468dec - {'path': '/', 'tag': 'ce02f953-98f5-4617-85e4-88059e468dec', 'options': 'recursive,force'} 2024-08-20T03:05:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.393+0000 7ff442143640 1 -- 172.21.15.44:0/687707630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff43c104230 msgr2=0x7ff43c10c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.393+0000 7ff442143640 1 --2- 172.21.15.44:0/687707630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff43c104230 0x7ff43c10c710 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff430009900 tx=0x7ff43002a410 comp rx=0 tx=0).stop 2024-08-20T03:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.393+0000 7ff442143640 1 -- 172.21.15.44:0/687707630 shutdown_connections 2024-08-20T03:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.393+0000 7ff442143640 1 --2- 172.21.15.44:0/687707630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c10cd30 0x7ff43c10f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.393+0000 7ff442143640 1 --2- 172.21.15.44:0/687707630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff43c104230 0x7ff43c10c710 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.393+0000 7ff442143640 1 --2- 172.21.15.44:0/687707630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff43c103910 0x7ff43c103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.393+0000 7ff442143640 1 -- 172.21.15.44:0/687707630 >> 172.21.15.44:0/687707630 conn(0x7ff43c078620 msgr2=0x7ff43c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.393+0000 7ff442143640 1 -- 172.21.15.44:0/687707630 shutdown_connections 2024-08-20T03:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.393+0000 7ff442143640 1 -- 172.21.15.44:0/687707630 wait complete. 2024-08-20T03:05:05.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff442143640 1 Processor -- start 2024-08-20T03:05:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff442143640 1 -- start start 2024-08-20T03:05:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff442143640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff43c103910 0x7ff43c199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff442143640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff43c104230 0x7ff43c199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff442143640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c10cd30 0x7ff43c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff442143640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1129c0 con 0x7ff43c104230 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff442143640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c112840 con 0x7ff43c103910 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff442143640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c112b40 con 0x7ff43c10cd30 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff43affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff43c104230 0x7ff43c199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff43affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff43c104230 0x7ff43c199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44130/0 (socket says 172.21.15.44:44130) 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff43affd640 1 -- 172.21.15.44:0/3835576471 learned_addr learned my addr 172.21.15.44:0/3835576471 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff43bfff640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c10cd30 0x7ff43c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.394+0000 7ff43b7fe640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff43c103910 0x7ff43c199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.395+0000 7ff43affd640 1 -- 172.21.15.44:0/3835576471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c10cd30 msgr2=0x7ff43c19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.395+0000 7ff43affd640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c10cd30 0x7ff43c19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.395+0000 7ff43affd640 1 -- 172.21.15.44:0/3835576471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff43c103910 msgr2=0x7ff43c199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.395+0000 7ff43affd640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff43c103910 0x7ff43c199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.395+0000 7ff43affd640 1 -- 172.21.15.44:0/3835576471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c19f0a0 con 0x7ff43c104230 2024-08-20T03:05:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.395+0000 7ff43bfff640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c10cd30 0x7ff43c19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.395+0000 7ff43affd640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff43c104230 0x7ff43c199540 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff430004270 tx=0x7ff4300042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.395+0000 7ff438ff9640 1 -- 172.21.15.44:0/3835576471 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43003f030 con 0x7ff43c104230 2024-08-20T03:05:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.395+0000 7ff438ff9640 1 -- 172.21.15.44:0/3835576471 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff430031880 con 0x7ff43c104230 2024-08-20T03:05:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.395+0000 7ff442143640 1 -- 172.21.15.44:0/3835576471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c19f330 con 0x7ff43c104230 2024-08-20T03:05:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.395+0000 7ff438ff9640 1 -- 172.21.15.44:0/3835576471 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430035070 con 0x7ff43c104230 2024-08-20T03:05:05.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.395+0000 7ff442143640 1 -- 172.21.15.44:0/3835576471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c1a5a50 con 0x7ff43c104230 2024-08-20T03:05:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff438ff9640 1 -- 172.21.15.44:0/3835576471 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff430042030 con 0x7ff43c104230 2024-08-20T03:05:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff44500b740 1 Processor -- start 2024-08-20T03:05:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff44500b740 1 -- start start 2024-08-20T03:05:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff44500b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c51950 0x5591c5c51d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff44500b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff44500b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c62610 0x5591c5c64a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff44500b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5591c5bc7360 con 0x5591c5c52270 2024-08-20T03:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff44500b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5591c5bb6860 con 0x5591c5c51950 2024-08-20T03:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff44500b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5591c5bba8a0 con 0x5591c5c62610 2024-08-20T03:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff43b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff43bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c51950 0x5591c5c51d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff43affd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c62610 0x5591c5c64a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff43bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c51950 0x5591c5c51d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:35146/0 (socket says 172.21.15.44:35146) 2024-08-20T03:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff43bfff640 1 -- 172.21.15.44:0/2530403759 learned_addr learned my addr 172.21.15.44:0/2530403759 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.398+0000 7ff43affd640 1 --2- 172.21.15.44:0/2530403759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c62610 0x5591c5c64a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:37468/0 (socket says 172.21.15.44:37468) 2024-08-20T03:05:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.399+0000 7ff43b7fe640 1 -- 172.21.15.44:0/2530403759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c62610 msgr2=0x5591c5c64a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.399+0000 7ff43b7fe640 1 --2- 172.21.15.44:0/2530403759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c62610 0x5591c5c64a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.399+0000 7ff43b7fe640 1 -- 172.21.15.44:0/2530403759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c51950 msgr2=0x5591c5c51d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.399+0000 7ff43b7fe640 1 --2- 172.21.15.44:0/2530403759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c51950 0x5591c5c51d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.399+0000 7ff43b7fe640 1 -- 172.21.15.44:0/2530403759 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c5b5de40 con 0x5591c5c52270 2024-08-20T03:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.399+0000 7ff438ff9640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff4240b3dc0 0x7ff4240b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.399+0000 7ff43b7fe640 1 --2- 172.21.15.44:0/2530403759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c620d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff42c005fd0 tx=0x7ff42c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab789e23c9cc9dd6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.399+0000 7ff43bfff640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff4240b3dc0 0x7ff4240b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.399+0000 7ff421ffb640 1 -- 172.21.15.44:0/2530403759 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c01c480 con 0x5591c5c52270 2024-08-20T03:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.399+0000 7ff438ff9640 1 -- 172.21.15.44:0/3835576471 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff430104330 con 0x7ff43c104230 2024-08-20T03:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.399+0000 7ff421ffb640 1 -- 172.21.15.44:0/2530403759 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff42c01cad0 con 0x5591c5c52270 2024-08-20T03:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.399+0000 7ff421ffb640 1 -- 172.21.15.44:0/2530403759 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c011bc0 con 0x5591c5c52270 2024-08-20T03:05:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.400+0000 7ff43bfff640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff4240b3dc0 0x7ff4240b6280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff434009fd0 tx=0x7ff434009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.400+0000 7ff44500b740 1 -- 172.21.15.44:0/2530403759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 msgr2=0x5591c5c620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.400+0000 7ff44500b740 1 --2- 172.21.15.44:0/2530403759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c620d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff42c005fd0 tx=0x7ff42c0107a0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.400+0000 7ff44500b740 1 -- 172.21.15.44:0/2530403759 shutdown_connections 2024-08-20T03:05:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.400+0000 7ff44500b740 1 --2- 172.21.15.44:0/2530403759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c62610 0x5591c5c64a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.400+0000 7ff44500b740 1 --2- 172.21.15.44:0/2530403759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c620d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.400+0000 7ff44500b740 1 --2- 172.21.15.44:0/2530403759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c51950 0x5591c5c51d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.400+0000 7ff44500b740 1 -- 172.21.15.44:0/2530403759 >> 172.21.15.44:0/2530403759 conn(0x5591c5c4f730 msgr2=0x5591c5c50920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.400+0000 7ff44500b740 1 -- 172.21.15.44:0/2530403759 shutdown_connections 2024-08-20T03:05:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.400+0000 7ff44500b740 1 -- 172.21.15.44:0/2530403759 wait complete. 2024-08-20T03:05:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff44500b740 1 -- start start 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff44500b740 1 Processor -- start 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff44500b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c51950 0x5591c5c61f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff44500b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c52270 0x5591c5c5f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff44500b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c62610 0x5591c5c5fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff44500b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5591c5aefab0 con 0x5591c5c62610 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff44500b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5591c5b3d780 con 0x5591c5c52270 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff44500b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5591c5c652f0 con 0x5591c5c51950 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff44500b740 10 client.0 _ll_register_callbacks cb 0x5591c58c5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff43bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c62610 0x5591c5c5fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff43bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c62610 0x5591c5c5fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44142/0 (socket says 172.21.15.44:44142) 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff43b7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c51950 0x5591c5c61f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff43bfff640 1 -- 172.21.15.44:0/2343994661 learned_addr learned my addr 172.21.15.44:0/2343994661 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.401+0000 7ff43affd640 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c52270 0x5591c5c5f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff43bfff640 1 -- 172.21.15.44:0/2343994661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c51950 msgr2=0x5591c5c61f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff43bfff640 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c51950 0x5591c5c61f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff43bfff640 1 -- 172.21.15.44:0/2343994661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c52270 msgr2=0x5591c5c5f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff43bfff640 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c52270 0x5591c5c5f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff43bfff640 1 -- 172.21.15.44:0/2343994661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c5919790 con 0x5591c5c62610 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff43b7fe640 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c51950 0x5591c5c61f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff43bfff640 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c62610 0x5591c5c5fa00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff434012800 tx=0x7ff434004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b465a86a2a93b2df server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff41affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff41affd640 1 -- 172.21.15.44:0/2343994661 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434011d20 con 0x5591c5c62610 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff41affd640 1 -- 172.21.15.44:0/2343994661 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff434003970 con 0x5591c5c62610 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff44500b740 1 -- 172.21.15.44:0/2343994661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5591c5b65e00 con 0x5591c5c62610 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff41affd640 1 -- 172.21.15.44:0/2343994661 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff434003b70 con 0x5591c5c62610 2024-08-20T03:05:05.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff44500b740 10 client.16302 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff44500b740 1 -- 172.21.15.44:0/2343994661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591c5992620 con 0x5591c5c62610 2024-08-20T03:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.402+0000 7ff41affd640 1 -- 172.21.15.44:0/2343994661 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5591c5b65e00 con 0x5591c5c62610 2024-08-20T03:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.403+0000 7ff41affd640 1 -- 172.21.15.44:0/2343994661 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff434031020 con 0x5591c5c62610 2024-08-20T03:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.403+0000 7ff44500b740 10 client.16302 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.403+0000 7ff44500b740 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.403+0000 7ff44500b740 20 client.16302 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.403+0000 7ff44500b740 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5591c5c72120 0x5591c5c74560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.403+0000 7ff44500b740 4 client.16302 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.403+0000 7ff44500b740 1 -- 172.21.15.44:0/2343994661 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591c5bc92b0 con 0x5591c5c72120 2024-08-20T03:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.403+0000 7ff43b7fe640 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5591c5c72120 0x5591c5c74560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.403+0000 7ff43b7fe640 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5591c5c72120 0x5591c5c74560 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d961c32add24dae6 server_cookie=973502b755b91630 in_seq=0 out_seq=0 2024-08-20T03:05:05.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.403+0000 7ff41affd640 10 client.16302 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.406+0000 7ff41affd640 1 -- 172.21.15.44:0/2343994661 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591c5bc92b0 con 0x5591c5c72120 2024-08-20T03:05:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.406+0000 7ff41affd640 10 client.16302 handle_command_reply: tid=0 2024-08-20T03:05:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.406+0000 7ff44500b740 1 client.16302 shutdown 2024-08-20T03:05:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.407+0000 7ff44500b740 1 -- 172.21.15.44:0/2343994661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c62610 msgr2=0x5591c5c5fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.407+0000 7ff44500b740 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c62610 0x5591c5c5fa00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff434012800 tx=0x7ff434004ab0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.407+0000 7ff44500b740 1 -- 172.21.15.44:0/2343994661 shutdown_connections 2024-08-20T03:05:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.407+0000 7ff44500b740 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5591c5c72120 0x5591c5c74560 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.407+0000 7ff44500b740 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c62610 0x5591c5c5fa00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.407+0000 7ff44500b740 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c52270 0x5591c5c5f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.407+0000 7ff44500b740 1 --2- 172.21.15.44:0/2343994661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c51950 0x5591c5c61f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.407+0000 7ff44500b740 1 -- 172.21.15.44:0/2343994661 >> 172.21.15.44:0/2343994661 conn(0x5591c5c4f730 msgr2=0x5591c5c55320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:05.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.407+0000 7ff41affd640 0 client.16302 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.407+0000 7ff44500b740 1 -- 172.21.15.44:0/2343994661 shutdown_connections 2024-08-20T03:05:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.407+0000 7ff44500b740 1 -- 172.21.15.44:0/2343994661 wait complete. 2024-08-20T03:05:05.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.407+0000 7ff44500b740 20 client.16302 trim_cache size 0 max 16384 2024-08-20T03:05:05.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:05 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:05:05.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:05 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:05:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.422+0000 7ff44500b740 1 Processor -- start 2024-08-20T03:05:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.422+0000 7ff44500b740 1 -- start start 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.422+0000 7ff44500b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c51950 0x5591c5c6e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.422+0000 7ff44500b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c5dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.422+0000 7ff44500b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c62610 0x5591c5c5e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.422+0000 7ff44500b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1ac070 con 0x5591c5c52270 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.422+0000 7ff44500b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1adc70 con 0x5591c5c51950 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.422+0000 7ff44500b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1ac740 con 0x5591c5c62610 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.422+0000 7ff43affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c5dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff43affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c5dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44152/0 (socket says 172.21.15.44:44152) 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff43affd640 1 -- 172.21.15.44:0/1533867802 learned_addr learned my addr 172.21.15.44:0/1533867802 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff43bfff640 1 --2- 172.21.15.44:0/1533867802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c62610 0x5591c5c5e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff43b7fe640 1 --2- 172.21.15.44:0/1533867802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c51950 0x5591c5c6e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff43affd640 1 -- 172.21.15.44:0/1533867802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c62610 msgr2=0x5591c5c5e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff43affd640 1 --2- 172.21.15.44:0/1533867802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c62610 0x5591c5c5e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff43affd640 1 -- 172.21.15.44:0/1533867802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c51950 msgr2=0x5591c5c6e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff43affd640 1 --2- 172.21.15.44:0/1533867802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c51950 0x5591c5c6e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff43affd640 1 -- 172.21.15.44:0/1533867802 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c5c57ab0 con 0x5591c5c52270 2024-08-20T03:05:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff43affd640 1 --2- 172.21.15.44:0/1533867802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c5dbe0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff4300cad00 tx=0x7ff43003b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d62205374e9db7e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff4217fa640 1 -- 172.21.15.44:0/1533867802 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430049040 con 0x5591c5c52270 2024-08-20T03:05:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff4217fa640 1 -- 172.21.15.44:0/1533867802 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff430003e40 con 0x5591c5c52270 2024-08-20T03:05:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.423+0000 7ff4217fa640 1 -- 172.21.15.44:0/1533867802 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4300d4000 con 0x5591c5c52270 2024-08-20T03:05:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.424+0000 7ff44500b740 1 -- 172.21.15.44:0/1533867802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 msgr2=0x5591c5c5dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.424+0000 7ff44500b740 1 --2- 172.21.15.44:0/1533867802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c5dbe0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff4300cad00 tx=0x7ff43003b640 comp rx=0 tx=0).stop 2024-08-20T03:05:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.424+0000 7ff44500b740 1 -- 172.21.15.44:0/1533867802 shutdown_connections 2024-08-20T03:05:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.424+0000 7ff44500b740 1 --2- 172.21.15.44:0/1533867802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c62610 0x5591c5c5e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.424+0000 7ff44500b740 1 --2- 172.21.15.44:0/1533867802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c5dbe0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.424+0000 7ff44500b740 1 --2- 172.21.15.44:0/1533867802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c51950 0x5591c5c6e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.424+0000 7ff44500b740 1 -- 172.21.15.44:0/1533867802 >> 172.21.15.44:0/1533867802 conn(0x5591c5c4f730 msgr2=0x5591c5c508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.424+0000 7ff44500b740 1 -- 172.21.15.44:0/1533867802 shutdown_connections 2024-08-20T03:05:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.424+0000 7ff44500b740 1 -- 172.21.15.44:0/1533867802 wait complete. 2024-08-20T03:05:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.424+0000 7ff44500b740 1 -- start start 2024-08-20T03:05:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff44500b740 1 Processor -- start 2024-08-20T03:05:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff44500b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c51950 0x5591c5c6a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff44500b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c6ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff44500b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c62610 0x5591c5c79260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff44500b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1ad4e0 con 0x5591c5c52270 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff44500b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1ace10 con 0x5591c5c62610 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff44500b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1aa050 con 0x5591c5c51950 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff44500b740 10 client.0 _ll_register_callbacks cb 0x7ff434012730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff43affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c6ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff43affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c6ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44158/0 (socket says 172.21.15.44:44158) 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff43affd640 1 -- 172.21.15.44:0/3104316854 learned_addr learned my addr 172.21.15.44:0/3104316854 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff43b7fe640 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c51950 0x5591c5c6a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff43bfff640 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c62610 0x5591c5c79260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff43affd640 1 -- 172.21.15.44:0/3104316854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c51950 msgr2=0x5591c5c6a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff43affd640 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c51950 0x5591c5c6a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff43affd640 1 -- 172.21.15.44:0/3104316854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c62610 msgr2=0x5591c5c79260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff43affd640 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c62610 0x5591c5c79260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff43affd640 1 -- 172.21.15.44:0/3104316854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c5c799d0 con 0x5591c5c52270 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff43bfff640 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c62610 0x5591c5c79260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff43affd640 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c6ad10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff430005c40 tx=0x7ff4300c74a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f12f5e16f7950a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff41affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.425+0000 7ff41affd640 1 -- 172.21.15.44:0/3104316854 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4300e5060 con 0x5591c5c52270 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff41affd640 1 -- 172.21.15.44:0/3104316854 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7ff4300cb260 con 0x5591c5c52270 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff44500b740 1 -- 172.21.15.44:0/3104316854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5591c5bc4d50 con 0x5591c5c52270 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff41affd640 1 -- 172.21.15.44:0/3104316854 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff430049070 con 0x5591c5c52270 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff44500b740 10 client.16308 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff44500b740 1 -- 172.21.15.44:0/3104316854 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5591c5cdc870 con 0x5591c5c52270 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff41affd640 1 -- 172.21.15.44:0/3104316854 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5591c5bc4d50 con 0x5591c5c52270 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff41affd640 1 -- 172.21.15.44:0/3104316854 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff4300d3b70 con 0x5591c5c52270 2024-08-20T03:05:05.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff44500b740 10 client.16308 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff44500b740 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff44500b740 20 client.16308 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff44500b740 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5591c5cdce20 0x5591c5cdf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff44500b740 4 client.16308 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff44500b740 1 -- 172.21.15.44:0/3104316854 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591c5bc2490 con 0x5591c5cdce20 2024-08-20T03:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.426+0000 7ff43b7fe640 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5591c5cdce20 0x5591c5cdf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.427+0000 7ff43b7fe640 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5591c5cdce20 0x5591c5cdf240 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5750645360951acc server_cookie=cfb2b082f9fd7467 in_seq=0 out_seq=0 2024-08-20T03:05:05.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:05.427+0000 7ff41affd640 10 client.16308 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:05.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:05 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:05:05.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:05 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:05:05.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:05 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:05:05.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:05 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:05:06.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:06 smithi099 ceph-mon[21984]: pgmap v398: 97 pgs: 97 active+clean; 328 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.2 MiB/s wr, 1.12k op/s 2024-08-20T03:05:06.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:06 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:05:06.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:06 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:05:06.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:06 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:05:06.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:06 smithi044 ceph-mon[18790]: pgmap v398: 97 pgs: 97 active+clean; 328 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.2 MiB/s wr, 1.12k op/s 2024-08-20T03:05:06.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:06 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:05:06.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:06 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:05:06.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:06 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:05:06.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:06 smithi165 ceph-mon[22357]: pgmap v398: 97 pgs: 97 active+clean; 328 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.2 MiB/s wr, 1.12k op/s 2024-08-20T03:05:06.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:06 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:05:06.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:06 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:05:06.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:06 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:05:07.476 INFO:tasks.workunit.client.0.smithi044.stdout: 89 790175 394 573743 456 453246 1306 2024-08-20T03:05:08.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:08 smithi099 ceph-mon[21984]: pgmap v399: 97 pgs: 97 active+clean; 329 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.4 MiB/s wr, 956 op/s 2024-08-20T03:05:08.545 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:08 smithi044 ceph-mon[18790]: pgmap v399: 97 pgs: 97 active+clean; 329 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.4 MiB/s wr, 956 op/s 2024-08-20T03:05:08.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:08 smithi165 ceph-mon[22357]: pgmap v399: 97 pgs: 97 active+clean; 329 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.4 MiB/s wr, 956 op/s 2024-08-20T03:05:10.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:10 smithi044 ceph-mon[18790]: pgmap v400: 97 pgs: 97 active+clean; 339 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.3 MiB/s wr, 790 op/s 2024-08-20T03:05:10.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:10 smithi165 ceph-mon[22357]: pgmap v400: 97 pgs: 97 active+clean; 339 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.3 MiB/s wr, 790 op/s 2024-08-20T03:05:10.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:10 smithi099 ceph-mon[21984]: pgmap v400: 97 pgs: 97 active+clean; 339 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.3 MiB/s wr, 790 op/s 2024-08-20T03:05:12.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:12 smithi044 ceph-mon[18790]: pgmap v401: 97 pgs: 97 active+clean; 342 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 595 op/s 2024-08-20T03:05:12.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:12 smithi165 ceph-mon[22357]: pgmap v401: 97 pgs: 97 active+clean; 342 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 595 op/s 2024-08-20T03:05:12.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:12 smithi099 ceph-mon[21984]: pgmap v401: 97 pgs: 97 active+clean; 342 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 595 op/s 2024-08-20T03:05:14.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:14 smithi044 ceph-mon[18790]: pgmap v402: 97 pgs: 97 active+clean; 346 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 943 op/s 2024-08-20T03:05:14.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:14 smithi165 ceph-mon[22357]: pgmap v402: 97 pgs: 97 active+clean; 346 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 943 op/s 2024-08-20T03:05:14.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:14 smithi099 ceph-mon[21984]: pgmap v402: 97 pgs: 97 active+clean; 346 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 943 op/s 2024-08-20T03:05:16.466 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:16 smithi099 ceph-mon[21984]: pgmap v403: 97 pgs: 97 active+clean; 348 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 768 op/s 2024-08-20T03:05:16.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:16 smithi044 ceph-mon[18790]: pgmap v403: 97 pgs: 97 active+clean; 348 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 768 op/s 2024-08-20T03:05:16.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:16 smithi165 ceph-mon[22357]: pgmap v403: 97 pgs: 97 active+clean; 348 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 768 op/s 2024-08-20T03:05:17.476 INFO:tasks.workunit.client.0.smithi044.stdout: 95 1411961 304 1020827 395 816575 1655 2024-08-20T03:05:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.995+0000 7ff41affd640 1 -- 172.21.15.44:0/3104316854 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591c5bc2490 con 0x5591c5cdce20 2024-08-20T03:05:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.995+0000 7ff41affd640 10 client.16308 handle_command_reply: tid=0 2024-08-20T03:05:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.995+0000 7ff44500b740 1 client.16308 shutdown 2024-08-20T03:05:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.996+0000 7ff44500b740 1 -- 172.21.15.44:0/3104316854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 msgr2=0x5591c5c6ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.996+0000 7ff44500b740 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c6ad10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff430005c40 tx=0x7ff4300c74a0 comp rx=0 tx=0).stop 2024-08-20T03:05:17.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.998+0000 7ff44500b740 1 -- 172.21.15.44:0/3104316854 shutdown_connections 2024-08-20T03:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.998+0000 7ff44500b740 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5591c5cdce20 0x5591c5cdf240 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.998+0000 7ff44500b740 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591c5c62610 0x5591c5c79260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.998+0000 7ff44500b740 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591c5c52270 0x5591c5c6ad10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.998+0000 7ff44500b740 1 --2- 172.21.15.44:0/3104316854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5591c5c51950 0x5591c5c6a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.998+0000 7ff44500b740 1 -- 172.21.15.44:0/3104316854 >> 172.21.15.44:0/3104316854 conn(0x5591c5c4f730 msgr2=0x5591c5c5dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.998+0000 7ff41affd640 0 client.16308 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:18.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.999+0000 7ff44500b740 1 -- 172.21.15.44:0/3104316854 shutdown_connections 2024-08-20T03:05:18.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.999+0000 7ff44500b740 1 -- 172.21.15.44:0/3104316854 wait complete. 2024-08-20T03:05:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:17.999+0000 7ff44500b740 20 client.16308 trim_cache size 0 max 16384 2024-08-20T03:05:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:05:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:05:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:05:18.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:05:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:18.001+0000 7ff442143640 1 -- 172.21.15.44:0/3835576471 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff4240b3dc0 msgr2=0x7ff4240b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:18.001+0000 7ff442143640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff4240b3dc0 0x7ff4240b6280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff434009fd0 tx=0x7ff434009500 comp rx=0 tx=0).stop 2024-08-20T03:05:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:18.001+0000 7ff442143640 1 -- 172.21.15.44:0/3835576471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff43c104230 msgr2=0x7ff43c199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:18.001+0000 7ff442143640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff43c104230 0x7ff43c199540 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff430004270 tx=0x7ff4300042a0 comp rx=0 tx=0).stop 2024-08-20T03:05:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:18.001+0000 7ff442143640 1 -- 172.21.15.44:0/3835576471 shutdown_connections 2024-08-20T03:05:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:18.001+0000 7ff442143640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff4240b3dc0 0x7ff4240b6280 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:18.001+0000 7ff442143640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff43c10cd30 0x7ff43c19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:18.001+0000 7ff442143640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff43c104230 0x7ff43c199540 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:18.001+0000 7ff442143640 1 --2- 172.21.15.44:0/3835576471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff43c103910 0x7ff43c199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:18.001+0000 7ff442143640 1 -- 172.21.15.44:0/3835576471 >> 172.21.15.44:0/3835576471 conn(0x7ff43c078620 msgr2=0x7ff43c0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:18.002+0000 7ff442143640 1 -- 172.21.15.44:0/3835576471 shutdown_connections 2024-08-20T03:05:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:18.002+0000 7ff442143640 1 -- 172.21.15.44:0/3835576471 wait complete. 2024-08-20T03:05:18.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:18 smithi044 ceph-mon[18790]: pgmap v404: 97 pgs: 97 active+clean; 359 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 778 op/s 2024-08-20T03:05:18.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:18 smithi165 ceph-mon[22357]: pgmap v404: 97 pgs: 97 active+clean; 359 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 778 op/s 2024-08-20T03:05:18.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:18 smithi099 ceph-mon[21984]: pgmap v404: 97 pgs: 97 active+clean; 359 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 778 op/s 2024-08-20T03:05:20.437 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:20 smithi165 ceph-mon[22357]: pgmap v405: 97 pgs: 97 active+clean; 355 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 797 op/s 2024-08-20T03:05:20.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:20 smithi044 ceph-mon[18790]: pgmap v405: 97 pgs: 97 active+clean; 355 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 797 op/s 2024-08-20T03:05:20.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:20 smithi099 ceph-mon[21984]: pgmap v405: 97 pgs: 97 active+clean; 355 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 797 op/s 2024-08-20T03:05:22.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:22 smithi044 ceph-mon[18790]: pgmap v406: 97 pgs: 97 active+clean; 350 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 800 op/s 2024-08-20T03:05:22.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:22 smithi165 ceph-mon[22357]: pgmap v406: 97 pgs: 97 active+clean; 350 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 800 op/s 2024-08-20T03:05:22.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:22 smithi099 ceph-mon[21984]: pgmap v406: 97 pgs: 97 active+clean; 350 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 800 op/s 2024-08-20T03:05:24.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:24 smithi044 ceph-mon[18790]: pgmap v407: 97 pgs: 97 active+clean; 334 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 834 op/s 2024-08-20T03:05:24.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:24 smithi165 ceph-mon[22357]: pgmap v407: 97 pgs: 97 active+clean; 334 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 834 op/s 2024-08-20T03:05:24.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:24 smithi099 ceph-mon[21984]: pgmap v407: 97 pgs: 97 active+clean; 334 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 834 op/s 2024-08-20T03:05:26.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:26 smithi044 ceph-mon[18790]: pgmap v408: 97 pgs: 97 active+clean; 334 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.6 MiB/s wr, 434 op/s 2024-08-20T03:05:26.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:26 smithi165 ceph-mon[22357]: pgmap v408: 97 pgs: 97 active+clean; 334 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.6 MiB/s wr, 434 op/s 2024-08-20T03:05:26.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:26 smithi099 ceph-mon[21984]: pgmap v408: 97 pgs: 97 active+clean; 334 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.6 MiB/s wr, 434 op/s 2024-08-20T03:05:27.476 INFO:tasks.workunit.client.0.smithi044.stdout: 104 1011668 621 713525 396 574987 1366 2024-08-20T03:05:28.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:28 smithi044 ceph-mon[18790]: pgmap v409: 97 pgs: 97 active+clean; 338 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.7 MiB/s wr, 356 op/s 2024-08-20T03:05:28.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:28 smithi165 ceph-mon[22357]: pgmap v409: 97 pgs: 97 active+clean; 338 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.7 MiB/s wr, 356 op/s 2024-08-20T03:05:28.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:28 smithi099 ceph-mon[21984]: pgmap v409: 97 pgs: 97 active+clean; 338 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.7 MiB/s wr, 356 op/s 2024-08-20T03:05:30.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:30 smithi044 ceph-mon[18790]: pgmap v410: 97 pgs: 97 active+clean; 375 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-08-20T03:05:30.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:30 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:05:30.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:30 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:05:30.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:30 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:05:30.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:30 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:05:30.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:30 smithi165 ceph-mon[22357]: pgmap v410: 97 pgs: 97 active+clean; 375 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-08-20T03:05:30.659 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:30 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:05:30.659 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:30 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:05:30.659 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:30 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:05:30.659 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:30 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:05:30.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:30 smithi099 ceph-mon[21984]: pgmap v410: 97 pgs: 97 active+clean; 375 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.9 MiB/s wr, 556 op/s 2024-08-20T03:05:30.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:30 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:05:30.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:30 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:05:30.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:30 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:05:30.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:30 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:05:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.401+0000 7f9065030640 1 -- 172.21.15.44:0/3964680658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906010a3c0 msgr2=0x7f906010a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.401+0000 7f9065030640 1 --2- 172.21.15.44:0/3964680658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906010a3c0 0x7f906010a800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9058009ca0 tx=0x7f905802a8d0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.402+0000 7f9065030640 1 -- 172.21.15.44:0/3964680658 shutdown_connections 2024-08-20T03:05:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.402+0000 7f9065030640 1 --2- 172.21.15.44:0/3964680658 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90601175f0 0x7f9060119a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.402+0000 7f9065030640 1 --2- 172.21.15.44:0/3964680658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906010a3c0 0x7f906010a800 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.402+0000 7f9065030640 1 --2- 172.21.15.44:0/3964680658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9060074ae0 0x7f9060109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.402+0000 7f9065030640 1 -- 172.21.15.44:0/3964680658 >> 172.21.15.44:0/3964680658 conn(0x7f906006d8e0 msgr2=0x7f906006dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.402+0000 7f9065030640 1 -- 172.21.15.44:0/3964680658 shutdown_connections 2024-08-20T03:05:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.402+0000 7f9065030640 1 -- 172.21.15.44:0/3964680658 wait complete. 2024-08-20T03:05:32.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.404+0000 7f9065030640 1 Processor -- start 2024-08-20T03:05:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.404+0000 7f9065030640 1 -- start start 2024-08-20T03:05:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.405+0000 7f9065030640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9060074ae0 0x7f90601a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.405+0000 7f9065030640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90601175f0 0x7f90601a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.405+0000 7f9065030640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90601a8e40 0x7f90601a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.405+0000 7f9065030640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f906011d250 con 0x7f90601a8e40 2024-08-20T03:05:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.405+0000 7f9065030640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f906011d0d0 con 0x7f9060074ae0 2024-08-20T03:05:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.405+0000 7f9065030640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f906011d3d0 con 0x7f90601175f0 2024-08-20T03:05:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.406+0000 7f905ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9060074ae0 0x7f90601a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.406+0000 7f905ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9060074ae0 0x7f90601a3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53314/0 (socket says 172.21.15.44:53314) 2024-08-20T03:05:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.406+0000 7f905ed9d640 1 -- 172.21.15.44:0/3552853884 learned_addr learned my addr 172.21.15.44:0/3552853884 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.406+0000 7f905e59c640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90601175f0 0x7f90601a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.406+0000 7f905ed9d640 1 -- 172.21.15.44:0/3552853884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90601175f0 msgr2=0x7f90601a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:32.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.407+0000 7f905ed9d640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90601175f0 0x7f90601a3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.410 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.407+0000 7f905ed9d640 1 -- 172.21.15.44:0/3552853884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90601a8e40 msgr2=0x7f90601a92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.407+0000 7f905ed9d640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90601a8e40 0x7f90601a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.407+0000 7f905ed9d640 1 -- 172.21.15.44:0/3552853884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90601a9a70 con 0x7f9060074ae0 2024-08-20T03:05:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.407+0000 7f905ed9d640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9060074ae0 0x7f90601a3570 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f905000d950 tx=0x7f905000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.411+0000 7f9047fff640 1 -- 172.21.15.44:0/3552853884 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050017070 con 0x7f9060074ae0 2024-08-20T03:05:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.411+0000 7f9065030640 1 -- 172.21.15.44:0/3552853884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90601affc0 con 0x7f9060074ae0 2024-08-20T03:05:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.411+0000 7f9065030640 1 -- 172.21.15.44:0/3552853884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90601b03c0 con 0x7f9060074ae0 2024-08-20T03:05:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.411+0000 7f9047fff640 1 -- 172.21.15.44:0/3552853884 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9050009c30 con 0x7f9060074ae0 2024-08-20T03:05:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.411+0000 7f9047fff640 1 -- 172.21.15.44:0/3552853884 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905000f620 con 0x7f9060074ae0 2024-08-20T03:05:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.416+0000 7f9067e1f740 1 Processor -- start 2024-08-20T03:05:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.416+0000 7f9067e1f740 1 -- start start 2024-08-20T03:05:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.416+0000 7f9067e1f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd0fd40 0x557dffd10120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.416+0000 7f9067e1f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd10660 0x557dffd248c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.416+0000 7f9067e1f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd271f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.416+0000 7f9067e1f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557dffc85360 con 0x557dffd24e00 2024-08-20T03:05:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.416+0000 7f9067e1f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557dffc74860 con 0x557dffd10660 2024-08-20T03:05:32.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.417+0000 7f9067e1f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557dffc788a0 con 0x557dffd0fd40 2024-08-20T03:05:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.417+0000 7f905f59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd0fd40 0x557dffd10120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.417+0000 7f905f59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd0fd40 0x557dffd10120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47378/0 (socket says 172.21.15.44:47378) 2024-08-20T03:05:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.417+0000 7f905f59e640 1 -- 172.21.15.44:0/3788765476 learned_addr learned my addr 172.21.15.44:0/3788765476 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:32.421 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.418+0000 7f905f59e640 1 -- 172.21.15.44:0/3788765476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd10660 msgr2=0x557dffd248c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:05:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.418+0000 7f905f59e640 1 --2- 172.21.15.44:0/3788765476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd10660 0x557dffd248c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.418+0000 7f905f59e640 1 -- 172.21.15.44:0/3788765476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 msgr2=0x557dffd271f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:05:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.419+0000 7f905f59e640 1 --2- 172.21.15.44:0/3788765476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd271f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.419+0000 7f905f59e640 1 -- 172.21.15.44:0/3788765476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dffc1be40 con 0x557dffd0fd40 2024-08-20T03:05:32.422 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.419+0000 7f905f59e640 1 --2- 172.21.15.44:0/3788765476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd0fd40 0x557dffd10120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f905400a890 tx=0x7f905400ad50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9cc3c2b37ec3f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:32.425 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.421+0000 7f9047fff640 1 -- 172.21.15.44:0/3552853884 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9050010460 con 0x7f9060074ae0 2024-08-20T03:05:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.423+0000 7f9047fff640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f90480b3ed0 0x7f90480b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.424+0000 7f9047fff640 1 -- 172.21.15.44:0/3552853884 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f90500a6c00 con 0x7f9060074ae0 2024-08-20T03:05:32.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.425+0000 7f905ed9d640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f90480b3ed0 0x7f90480b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.437+0000 7f905ed9d640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f90480b3ed0 0x7f90480b6390 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f90500a4c90 tx=0x7f90500b1f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.440+0000 7f90457fa640 1 -- 172.21.15.44:0/3788765476 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905400ccc0 con 0x557dffd0fd40 2024-08-20T03:05:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.440+0000 7f90457fa640 1 -- 172.21.15.44:0/3788765476 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905400f900 con 0x557dffd0fd40 2024-08-20T03:05:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.441+0000 7f9067e1f740 1 -- 172.21.15.44:0/3788765476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd0fd40 msgr2=0x557dffd10120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.441+0000 7f9067e1f740 1 --2- 172.21.15.44:0/3788765476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd0fd40 0x557dffd10120 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f905400a890 tx=0x7f905400ad50 comp rx=0 tx=0).stop 2024-08-20T03:05:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.441+0000 7f9067e1f740 1 -- 172.21.15.44:0/3788765476 shutdown_connections 2024-08-20T03:05:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.441+0000 7f9067e1f740 1 --2- 172.21.15.44:0/3788765476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd271f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.441+0000 7f9067e1f740 1 --2- 172.21.15.44:0/3788765476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd10660 0x557dffd248c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.441+0000 7f9067e1f740 1 --2- 172.21.15.44:0/3788765476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd0fd40 0x557dffd10120 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.441+0000 7f9067e1f740 1 -- 172.21.15.44:0/3788765476 >> 172.21.15.44:0/3788765476 conn(0x557dffd0db70 msgr2=0x557dffd0ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.442+0000 7f9067e1f740 1 -- 172.21.15.44:0/3788765476 shutdown_connections 2024-08-20T03:05:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.442+0000 7f9067e1f740 1 -- 172.21.15.44:0/3788765476 wait complete. 2024-08-20T03:05:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.442+0000 7f9067e1f740 1 -- start start 2024-08-20T03:05:32.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.442+0000 7f9067e1f740 1 Processor -- start 2024-08-20T03:05:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.443+0000 7f9067e1f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd10660 0x557dffd19a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.443+0000 7f9067e1f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd24e00 0x557dffd19ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.443+0000 7f9067e1f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 0x557dffd24730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.443+0000 7f9067e1f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557dffbadab0 con 0x557dffd10660 2024-08-20T03:05:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.443+0000 7f9067e1f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557dffbfb780 con 0x557dffd24e00 2024-08-20T03:05:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.443+0000 7f9067e1f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x557dffd16470 con 0x557dffd242b0 2024-08-20T03:05:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.443+0000 7f9067e1f740 10 client.0 _ll_register_callbacks cb 0x557dff983a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.443+0000 7f905e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd24e00 0x557dffd19ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.443+0000 7f905e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd24e00 0x557dffd19ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53338/0 (socket says 172.21.15.44:53338) 2024-08-20T03:05:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.443+0000 7f905e59c640 1 -- 172.21.15.44:0/3513119871 learned_addr learned my addr 172.21.15.44:0/3513119871 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.444+0000 7f905f59e640 1 --2- 172.21.15.44:0/3513119871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 0x557dffd24730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.444+0000 7f905f59e640 1 -- 172.21.15.44:0/3513119871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd24e00 msgr2=0x557dffd19ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.444+0000 7f905f59e640 1 --2- 172.21.15.44:0/3513119871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd24e00 0x557dffd19ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.444+0000 7f905f59e640 1 -- 172.21.15.44:0/3513119871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd10660 msgr2=0x557dffd19a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.444+0000 7f905f59e640 1 --2- 172.21.15.44:0/3513119871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd10660 0x557dffd19a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.444+0000 7f905f59e640 1 -- 172.21.15.44:0/3513119871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dff9d7790 con 0x557dffd242b0 2024-08-20T03:05:32.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.445+0000 7f905f59e640 1 --2- 172.21.15.44:0/3513119871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 0x557dffd24730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f905400c6a0 tx=0x7f905400a3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47dbe6a84c99f4de server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.448+0000 7f9042ff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:05:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.448+0000 7f9042ff5640 1 -- 172.21.15.44:0/3513119871 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905400ccd0 con 0x557dffd242b0 2024-08-20T03:05:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.448+0000 7f9067e1f740 1 -- 172.21.15.44:0/3513119871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557dffc23e00 con 0x557dffd242b0 2024-08-20T03:05:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.448+0000 7f9042ff5640 1 -- 172.21.15.44:0/3513119871 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f9054018b00 con 0x557dffd242b0 2024-08-20T03:05:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.454+0000 7f9042ff5640 1 -- 172.21.15.44:0/3513119871 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f905400ce70 con 0x557dffd242b0 2024-08-20T03:05:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.454+0000 7f9067e1f740 10 client.25777 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.454+0000 7f9067e1f740 1 -- 172.21.15.44:0/3513119871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557dffa50620 con 0x557dffd242b0 2024-08-20T03:05:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.454+0000 7f9042ff5640 1 -- 172.21.15.44:0/3513119871 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557dffc23e00 con 0x557dffd242b0 2024-08-20T03:05:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.455+0000 7f9042ff5640 1 -- 172.21.15.44:0/3513119871 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f905402c050 con 0x557dffd242b0 2024-08-20T03:05:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.455+0000 7f9067e1f740 10 client.25777 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.455+0000 7f9067e1f740 10 client.25777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.455+0000 7f9067e1f740 20 client.25777 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.455+0000 7f9067e1f740 1 --2- 172.21.15.44:0/3513119871 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x557dffd3d6a0 0x557dffd3fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.455+0000 7f9067e1f740 4 client.25777 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:05:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.455+0000 7f9067e1f740 1 -- 172.21.15.44:0/3513119871 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dffc872b0 con 0x557dffd3d6a0 2024-08-20T03:05:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.455+0000 7f905e59c640 1 --2- 172.21.15.44:0/3513119871 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x557dffd3d6a0 0x557dffd3fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.456+0000 7f9042ff5640 10 client.25777 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:32.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.456+0000 7f905e59c640 1 --2- 172.21.15.44:0/3513119871 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x557dffd3d6a0 0x557dffd3fac0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd351fcb80dff9b server_cookie=eb3b1198b917e301 in_seq=0 out_seq=0 2024-08-20T03:05:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.458+0000 7f9042ff5640 1 -- 172.21.15.44:0/3513119871 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dffc872b0 con 0x557dffd3d6a0 2024-08-20T03:05:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.458+0000 7f9042ff5640 10 client.25777 handle_command_reply: tid=0 2024-08-20T03:05:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.458+0000 7f9067e1f740 1 client.25777 shutdown 2024-08-20T03:05:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.459+0000 7f9067e1f740 1 -- 172.21.15.44:0/3513119871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 msgr2=0x557dffd24730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.459+0000 7f9067e1f740 1 --2- 172.21.15.44:0/3513119871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 0x557dffd24730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f905400c6a0 tx=0x7f905400a3e0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.459+0000 7f9067e1f740 1 -- 172.21.15.44:0/3513119871 shutdown_connections 2024-08-20T03:05:32.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.459+0000 7f9067e1f740 1 --2- 172.21.15.44:0/3513119871 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x557dffd3d6a0 0x557dffd3fac0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.459+0000 7f9067e1f740 1 --2- 172.21.15.44:0/3513119871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 0x557dffd24730 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.459+0000 7f9067e1f740 1 --2- 172.21.15.44:0/3513119871 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd24e00 0x557dffd19ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.459+0000 7f9067e1f740 1 --2- 172.21.15.44:0/3513119871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd10660 0x557dffd19a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.459+0000 7f9067e1f740 1 -- 172.21.15.44:0/3513119871 >> 172.21.15.44:0/3513119871 conn(0x557dffd0db70 msgr2=0x557dffd12ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.459+0000 7f9042ff5640 0 client.25777 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.459+0000 7f9067e1f740 1 -- 172.21.15.44:0/3513119871 shutdown_connections 2024-08-20T03:05:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.459+0000 7f9067e1f740 1 -- 172.21.15.44:0/3513119871 wait complete. 2024-08-20T03:05:32.463 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.459+0000 7f9067e1f740 20 client.25777 trim_cache size 0 max 16384 2024-08-20T03:05:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.505+0000 7f9067e1f740 1 Processor -- start 2024-08-20T03:05:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.505+0000 7f9067e1f740 1 -- start start 2024-08-20T03:05:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.505+0000 7f9067e1f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd10660 0x557dffd0fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.505+0000 7f9067e1f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd242b0 0x557dffd0d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.505+0000 7f9067e1f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd18100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.505+0000 7f9067e1f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f904c00e730 con 0x557dffd24e00 2024-08-20T03:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.505+0000 7f9067e1f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f904c010330 con 0x557dffd242b0 2024-08-20T03:05:32.509 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.505+0000 7f9067e1f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f904c00ee00 con 0x557dffd10660 2024-08-20T03:05:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.506+0000 7f905f59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd18100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.506+0000 7f905e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd242b0 0x557dffd0d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.506+0000 7f905f59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd18100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50470/0 (socket says 172.21.15.44:50470) 2024-08-20T03:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.506+0000 7f905f59e640 1 -- 172.21.15.44:0/1605225372 learned_addr learned my addr 172.21.15.44:0/1605225372 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.506+0000 7f905e59c640 1 -- 172.21.15.44:0/1605225372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd10660 msgr2=0x557dffd0fc90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.506+0000 7f905e59c640 1 --2- 172.21.15.44:0/1605225372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd10660 0x557dffd0fc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.506+0000 7f905e59c640 1 -- 172.21.15.44:0/1605225372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 msgr2=0x557dffd18100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.506+0000 7f905e59c640 1 --2- 172.21.15.44:0/1605225372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd18100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.506+0000 7f905e59c640 1 -- 172.21.15.44:0/1605225372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dffd10a40 con 0x557dffd242b0 2024-08-20T03:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.506+0000 7f905f59e640 1 --2- 172.21.15.44:0/1605225372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd18100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.507+0000 7f905e59c640 1 --2- 172.21.15.44:0/1605225372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd242b0 0x557dffd0d420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x557dffd40320 tx=0x7f9058009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7842223d1aa4e03b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.507+0000 7f9044ff9640 1 -- 172.21.15.44:0/1605225372 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058003c30 con 0x557dffd242b0 2024-08-20T03:05:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.507+0000 7f9044ff9640 1 -- 172.21.15.44:0/1605225372 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9058036040 con 0x557dffd242b0 2024-08-20T03:05:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.507+0000 7f9044ff9640 1 -- 172.21.15.44:0/1605225372 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058030890 con 0x557dffd242b0 2024-08-20T03:05:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.508+0000 7f9067e1f740 1 -- 172.21.15.44:0/1605225372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd242b0 msgr2=0x557dffd0d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:32.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.508+0000 7f9067e1f740 1 --2- 172.21.15.44:0/1605225372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd242b0 0x557dffd0d420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x557dffd40320 tx=0x7f9058009870 comp rx=0 tx=0).stop 2024-08-20T03:05:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.511+0000 7f9067e1f740 1 -- 172.21.15.44:0/1605225372 shutdown_connections 2024-08-20T03:05:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.511+0000 7f9067e1f740 1 --2- 172.21.15.44:0/1605225372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd18100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.511+0000 7f9067e1f740 1 --2- 172.21.15.44:0/1605225372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd242b0 0x557dffd0d420 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.511+0000 7f9067e1f740 1 --2- 172.21.15.44:0/1605225372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd10660 0x557dffd0fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.511+0000 7f9067e1f740 1 -- 172.21.15.44:0/1605225372 >> 172.21.15.44:0/1605225372 conn(0x557dffd0db70 msgr2=0x557dffd0e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.518+0000 7f9067e1f740 1 -- 172.21.15.44:0/1605225372 shutdown_connections 2024-08-20T03:05:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.518+0000 7f9067e1f740 1 -- 172.21.15.44:0/1605225372 wait complete. 2024-08-20T03:05:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.518+0000 7f9067e1f740 1 -- start start 2024-08-20T03:05:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.518+0000 7f9067e1f740 1 Processor -- start 2024-08-20T03:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f9067e1f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd10660 0x557dffd51980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f9067e1f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 0x557dffd4c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f9067e1f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd4c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f9067e1f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f904c00fba0 con 0x557dffd24e00 2024-08-20T03:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f9067e1f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f904c00f4d0 con 0x557dffd10660 2024-08-20T03:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f9067e1f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f904c00c710 con 0x557dffd242b0 2024-08-20T03:05:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f9067e1f740 10 client.0 _ll_register_callbacks cb 0x7f905400a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f905e59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 0x557dffd4c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f905e59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 0x557dffd4c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47404/0 (socket says 172.21.15.44:47404) 2024-08-20T03:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f905e59c640 1 -- 172.21.15.44:0/1920905720 learned_addr learned my addr 172.21.15.44:0/1920905720 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f905f59e640 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd4c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f905f59e640 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd10660 0x557dffd51980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.519+0000 7f905e59c640 1 -- 172.21.15.44:0/1920905720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd10660 msgr2=0x557dffd51980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.520+0000 7f905e59c640 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd10660 0x557dffd51980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.520+0000 7f905e59c640 1 -- 172.21.15.44:0/1920905720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 msgr2=0x557dffd4c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.520+0000 7f905e59c640 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd4c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.520+0000 7f905e59c640 1 -- 172.21.15.44:0/1920905720 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dffda61f0 con 0x557dffd242b0 2024-08-20T03:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.520+0000 7f905f59e640 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd10660 0x557dffd51980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:05:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.520+0000 7f905e59c640 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 0x557dffd4c0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f905802a910 tx=0x7f9058003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea6bcbf2ec8eef3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.528+0000 7f9042ff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:05:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.528+0000 7f9042ff5640 1 -- 172.21.15.44:0/1920905720 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058054030 con 0x557dffd242b0 2024-08-20T03:05:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.528+0000 7f9042ff5640 1 -- 172.21.15.44:0/1920905720 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f9058004250 con 0x557dffd242b0 2024-08-20T03:05:32.531 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.528+0000 7f9067e1f740 1 -- 172.21.15.44:0/1920905720 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9058036040 con 0x557dffd242b0 2024-08-20T03:05:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.530+0000 7f9042ff5640 1 -- 172.21.15.44:0/1920905720 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9058061050 con 0x557dffd242b0 2024-08-20T03:05:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.530+0000 7f9067e1f740 10 client.25780 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.530+0000 7f9067e1f740 1 -- 172.21.15.44:0/1920905720 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557dffda7460 con 0x557dffd242b0 2024-08-20T03:05:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.530+0000 7f9042ff5640 1 -- 172.21.15.44:0/1920905720 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9058036040 con 0x557dffd242b0 2024-08-20T03:05:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.531+0000 7f9042ff5640 1 -- 172.21.15.44:0/1920905720 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f9058030b60 con 0x557dffd242b0 2024-08-20T03:05:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.531+0000 7f9067e1f740 10 client.25780 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.531+0000 7f9067e1f740 10 client.25780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.531+0000 7f9067e1f740 20 client.25780 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.531+0000 7f9067e1f740 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x557dffda7b70 0x557dffda9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.531+0000 7f9067e1f740 4 client.25780 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:05:32.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.531+0000 7f9067e1f740 1 -- 172.21.15.44:0/1920905720 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557dffc82d50 con 0x557dffda7b70 2024-08-20T03:05:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.531+0000 7f905f59e640 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x557dffda7b70 0x557dffda9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.532+0000 7f905f59e640 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x557dffda7b70 0x557dffda9f90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa252aa833950847 server_cookie=5e97f03c17dbce9e in_seq=0 out_seq=0 2024-08-20T03:05:32.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:32.532+0000 7f9042ff5640 10 client.25780 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:32.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:32 smithi044 ceph-mon[18790]: pgmap v411: 97 pgs: 97 active+clean; 391 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 558 op/s 2024-08-20T03:05:32.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:32 smithi165 ceph-mon[22357]: pgmap v411: 97 pgs: 97 active+clean; 391 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 558 op/s 2024-08-20T03:05:32.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:32 smithi099 ceph-mon[21984]: pgmap v411: 97 pgs: 97 active+clean; 391 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.3 MiB/s wr, 558 op/s 2024-08-20T03:05:34.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:34 smithi044 ceph-mon[18790]: pgmap v412: 97 pgs: 97 active+clean; 430 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.6 MiB/s wr, 796 op/s 2024-08-20T03:05:34.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:34 smithi165 ceph-mon[22357]: pgmap v412: 97 pgs: 97 active+clean; 430 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.6 MiB/s wr, 796 op/s 2024-08-20T03:05:34.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:34 smithi099 ceph-mon[21984]: pgmap v412: 97 pgs: 97 active+clean; 430 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 9.6 MiB/s wr, 796 op/s 2024-08-20T03:05:35.158 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:05:36.234 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:05:36.530 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:36 smithi044 ceph-mon[18790]: pgmap v413: 97 pgs: 97 active+clean; 434 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.0 MiB/s wr, 771 op/s 2024-08-20T03:05:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:36 smithi165 ceph-mon[22357]: pgmap v413: 97 pgs: 97 active+clean; 434 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.0 MiB/s wr, 771 op/s 2024-08-20T03:05:36.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:36 smithi099 ceph-mon[21984]: pgmap v413: 97 pgs: 97 active+clean; 434 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 9.0 MiB/s wr, 771 op/s 2024-08-20T03:05:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.738+0000 7f5e03924640 1 -- 172.21.15.44:0/2118229987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dfc107d20 msgr2=0x7f5dfc10e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.738+0000 7f5e03924640 1 --2- 172.21.15.44:0/2118229987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dfc107d20 0x7f5dfc10e520 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009ca0 tx=0x7f5df402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.738+0000 7f5e03924640 1 -- 172.21.15.44:0/2118229987 shutdown_connections 2024-08-20T03:05:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.738+0000 7f5e03924640 1 --2- 172.21.15.44:0/2118229987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dfc107d20 0x7f5dfc10e520 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.738+0000 7f5e03924640 1 --2- 172.21.15.44:0/2118229987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dfc1073a0 0x7f5dfc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.738+0000 7f5e03924640 1 --2- 172.21.15.44:0/2118229987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dfc101990 0x7f5dfc101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.738+0000 7f5e03924640 1 -- 172.21.15.44:0/2118229987 >> 172.21.15.44:0/2118229987 conn(0x7f5dfc0fd400 msgr2=0x7f5dfc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e03924640 1 -- 172.21.15.44:0/2118229987 shutdown_connections 2024-08-20T03:05:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e03924640 1 -- 172.21.15.44:0/2118229987 wait complete. 2024-08-20T03:05:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e03924640 1 Processor -- start 2024-08-20T03:05:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e03924640 1 -- start start 2024-08-20T03:05:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e03924640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dfc101990 0x7f5dfc198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e03924640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dfc1073a0 0x7f5dfc199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e03924640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dfc107d20 0x7f5dfc19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e03924640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc112b00 con 0x7f5dfc101990 2024-08-20T03:05:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e03924640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc112980 con 0x7f5dfc107d20 2024-08-20T03:05:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e03924640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc112c80 con 0x7f5dfc1073a0 2024-08-20T03:05:36.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e016c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dfc101990 0x7f5dfc198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e016c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dfc101990 0x7f5dfc198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50510/0 (socket says 172.21.15.44:50510) 2024-08-20T03:05:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e016c0640 1 -- 172.21.15.44:0/1110604479 learned_addr learned my addr 172.21.15.44:0/1110604479 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.739+0000 7f5e00ebf640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dfc1073a0 0x7f5dfc199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5e01ec1640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dfc107d20 0x7f5dfc19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5e016c0640 1 -- 172.21.15.44:0/1110604479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dfc1073a0 msgr2=0x7f5dfc199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5e016c0640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dfc1073a0 0x7f5dfc199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5e016c0640 1 -- 172.21.15.44:0/1110604479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dfc107d20 msgr2=0x7f5dfc19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5e016c0640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dfc107d20 0x7f5dfc19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5e016c0640 1 -- 172.21.15.44:0/1110604479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc19efe0 con 0x7f5dfc101990 2024-08-20T03:05:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5e01ec1640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dfc107d20 0x7f5dfc19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:05:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5e016c0640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dfc101990 0x7f5dfc198f30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5df800e9f0 tx=0x7f5df800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:36.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5df27fc640 1 -- 172.21.15.44:0/1110604479 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df800cd80 con 0x7f5dfc101990 2024-08-20T03:05:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5df27fc640 1 -- 172.21.15.44:0/1110604479 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df80054d0 con 0x7f5dfc101990 2024-08-20T03:05:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5e03924640 1 -- 172.21.15.44:0/1110604479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dfc19f2d0 con 0x7f5dfc101990 2024-08-20T03:05:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5df27fc640 1 -- 172.21.15.44:0/1110604479 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8010690 con 0x7f5dfc101990 2024-08-20T03:05:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.740+0000 7f5e03924640 1 -- 172.21.15.44:0/1110604479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dfc1a5990 con 0x7f5dfc101990 2024-08-20T03:05:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5df27fc640 1 -- 172.21.15.44:0/1110604479 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5df8004e90 con 0x7f5dfc101990 2024-08-20T03:05:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5e05d1d740 1 Processor -- start 2024-08-20T03:05:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5e05d1d740 1 -- start start 2024-08-20T03:05:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5e05d1d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4b830 0x562747e4bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5e05d1d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4c150 0x562747e57ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5e05d1d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e5a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5e05d1d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562747dc28e0 con 0x562747e4c150 2024-08-20T03:05:36.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5e05d1d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562747db1de0 con 0x562747e4b830 2024-08-20T03:05:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5e05d1d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562747db5e20 con 0x562747e580e0 2024-08-20T03:05:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5e016c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4c150 0x562747e57ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5e016c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4c150 0x562747e57ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50520/0 (socket says 172.21.15.44:50520) 2024-08-20T03:05:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5e00ebf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e5a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.743+0000 7f5e016c0640 1 -- 172.21.15.44:0/3885362262 learned_addr learned my addr 172.21.15.44:0/3885362262 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5e01ec1640 1 --2- 172.21.15.44:0/3885362262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4b830 0x562747e4bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5e00ebf640 1 -- 172.21.15.44:0/3885362262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4b830 msgr2=0x562747e4bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5e00ebf640 1 --2- 172.21.15.44:0/3885362262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4b830 0x562747e4bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5e00ebf640 1 -- 172.21.15.44:0/3885362262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4c150 msgr2=0x562747e57ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5e00ebf640 1 --2- 172.21.15.44:0/3885362262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4c150 0x562747e57ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5e00ebf640 1 -- 172.21.15.44:0/3885362262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562747b11650 con 0x562747e580e0 2024-08-20T03:05:36.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5df27fc640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5de80b3d40 0x7f5de80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5e01ec1640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5de80b3d40 0x7f5de80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5e00ebf640 1 --2- 172.21.15.44:0/3885362262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e5a4d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5dec005fd0 tx=0x7f5dec0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b4e8f1d782ab24 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5df27fc640 1 -- 172.21.15.44:0/1110604479 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5df80e4420 con 0x7f5dfc101990 2024-08-20T03:05:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5de77fe640 1 -- 172.21.15.44:0/3885362262 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec01c480 con 0x562747e580e0 2024-08-20T03:05:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5de77fe640 1 -- 172.21.15.44:0/3885362262 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dec01cad0 con 0x562747e580e0 2024-08-20T03:05:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.744+0000 7f5de77fe640 1 -- 172.21.15.44:0/3885362262 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec011b60 con 0x562747e580e0 2024-08-20T03:05:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.745+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3885362262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 msgr2=0x562747e5a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.745+0000 7f5e01ec1640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5de80b3d40 0x7f5de80b6200 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5df4005e30 tx=0x7f5df40098c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:36.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.745+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/3885362262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e5a4d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5dec005fd0 tx=0x7f5dec0107a0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.745+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3885362262 shutdown_connections 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.745+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/3885362262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e5a4d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.745+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/3885362262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4c150 0x562747e57ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.745+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/3885362262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4b830 0x562747e4bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.745+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3885362262 >> 172.21.15.44:0/3885362262 conn(0x562747e49660 msgr2=0x562747e4a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.745+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3885362262 shutdown_connections 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.745+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3885362262 wait complete. 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.745+0000 7f5e05d1d740 1 -- start start 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.745+0000 7f5e05d1d740 1 Processor -- start 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e05d1d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 0x562747e55080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e05d1d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e05d1d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e55b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e05d1d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562747c14700 con 0x562747e4b830 2024-08-20T03:05:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e05d1d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562747d387c0 con 0x562747e4c150 2024-08-20T03:05:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e05d1d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562747e5ad90 con 0x562747e580e0 2024-08-20T03:05:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e05d1d740 10 client.0 _ll_register_callbacks cb 0x562747dab6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e016c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 0x562747e55080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e016c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 0x562747e55080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50528/0 (socket says 172.21.15.44:50528) 2024-08-20T03:05:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e016c0640 1 -- 172.21.15.44:0/1097953768 learned_addr learned my addr 172.21.15.44:0/1097953768 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e01ec1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e55b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e00ebf640 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e016c0640 1 -- 172.21.15.44:0/1097953768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 msgr2=0x562747e55b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e016c0640 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e55b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e016c0640 1 -- 172.21.15.44:0/1097953768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 msgr2=0x562747e555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e016c0640 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e016c0640 1 -- 172.21.15.44:0/1097953768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562747e64170 con 0x562747e4b830 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.746+0000 7f5e00ebf640 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5e016c0640 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 0x562747e55080 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5df80aada0 tx=0x7f5df80048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3575e83d49ed546a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5de4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5de4ff9640 1 -- 172.21.15.44:0/1097953768 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80aa9f0 con 0x562747e4b830 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5de4ff9640 1 -- 172.21.15.44:0/1097953768 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5df80b3ed0 con 0x562747e4b830 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5e05d1d740 1 -- 172.21.15.44:0/1097953768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562747d61100 con 0x562747e4b830 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5de4ff9640 1 -- 172.21.15.44:0/1097953768 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5df8005180 con 0x562747e4b830 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5e05d1d740 10 client.16329 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5e05d1d740 1 -- 172.21.15.44:0/1097953768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562747e66b80 con 0x562747e4b830 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5de4ff9640 1 -- 172.21.15.44:0/1097953768 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562747d61100 con 0x562747e4b830 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5de4ff9640 1 -- 172.21.15.44:0/1097953768 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5df80af120 con 0x562747e4b830 2024-08-20T03:05:36.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5e05d1d740 10 client.16329 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5e05d1d740 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5e05d1d740 20 client.16329 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562747e68060 0x562747e6a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5e05d1d740 4 client.16329 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:05:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.747+0000 7f5e05d1d740 1 -- 172.21.15.44:0/1097953768 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562747dc4830 con 0x562747e68060 2024-08-20T03:05:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.748+0000 7f5e00ebf640 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562747e68060 0x562747e6a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.748+0000 7f5de4ff9640 10 client.16329 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:36.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.748+0000 7f5e00ebf640 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562747e68060 0x562747e6a520 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a64608f766d7b424 server_cookie=f462ffa4cd732183 in_seq=0 out_seq=0 2024-08-20T03:05:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.751+0000 7f5de4ff9640 1 -- 172.21.15.44:0/1097953768 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562747dc4830 con 0x562747e68060 2024-08-20T03:05:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.751+0000 7f5de4ff9640 10 client.16329 handle_command_reply: tid=0 2024-08-20T03:05:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.751+0000 7f5e05d1d740 1 client.16329 shutdown 2024-08-20T03:05:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.751+0000 7f5e05d1d740 1 -- 172.21.15.44:0/1097953768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 msgr2=0x562747e55080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.751+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 0x562747e55080 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5df80aada0 tx=0x7f5df80048d0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.751+0000 7f5e05d1d740 1 -- 172.21.15.44:0/1097953768 shutdown_connections 2024-08-20T03:05:36.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.751+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562747e68060 0x562747e6a520 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.751+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e55b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.751+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.751+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/1097953768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 0x562747e55080 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.751+0000 7f5e05d1d740 1 -- 172.21.15.44:0/1097953768 >> 172.21.15.44:0/1097953768 conn(0x562747e49660 msgr2=0x562747e4f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.751+0000 7f5de4ff9640 0 client.16329 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.752+0000 7f5e05d1d740 1 -- 172.21.15.44:0/1097953768 shutdown_connections 2024-08-20T03:05:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.752+0000 7f5e05d1d740 1 -- 172.21.15.44:0/1097953768 wait complete. 2024-08-20T03:05:36.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.752+0000 7f5e05d1d740 20 client.16329 trim_cache size 0 max 16384 2024-08-20T03:05:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.770+0000 7f5e05d1d740 1 Processor -- start 2024-08-20T03:05:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.770+0000 7f5e05d1d740 1 -- start start 2024-08-20T03:05:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.770+0000 7f5e05d1d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 0x562747e4f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.770+0000 7f5e05d1d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e6fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.770+0000 7f5e05d1d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e70370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.770+0000 7f5e05d1d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1ae1a0 con 0x562747e4b830 2024-08-20T03:05:36.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.770+0000 7f5e05d1d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1afda0 con 0x562747e4c150 2024-08-20T03:05:36.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.770+0000 7f5e05d1d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1ae870 con 0x562747e580e0 2024-08-20T03:05:36.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.770+0000 7f5e016c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 0x562747e4f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5e016c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 0x562747e4f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50530/0 (socket says 172.21.15.44:50530) 2024-08-20T03:05:36.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5e016c0640 1 -- 172.21.15.44:0/2295311147 learned_addr learned my addr 172.21.15.44:0/2295311147 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:36.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5e01ec1640 1 --2- 172.21.15.44:0/2295311147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e70370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5e00ebf640 1 --2- 172.21.15.44:0/2295311147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e6fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5e01ec1640 1 -- 172.21.15.44:0/2295311147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 msgr2=0x562747e6fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5e01ec1640 1 --2- 172.21.15.44:0/2295311147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e6fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5e01ec1640 1 -- 172.21.15.44:0/2295311147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 msgr2=0x562747e4f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5e01ec1640 1 --2- 172.21.15.44:0/2295311147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 0x562747e4f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5e01ec1640 1 -- 172.21.15.44:0/2295311147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562747e4a0e0 con 0x562747e580e0 2024-08-20T03:05:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5e016c0640 1 --2- 172.21.15.44:0/2295311147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 0x562747e4f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5e01ec1640 1 --2- 172.21.15.44:0/2295311147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e70370 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009ca0 tx=0x7f5df403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e289d0770186e1e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5de6ffd640 1 -- 172.21.15.44:0/2295311147 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4032e20 con 0x562747e580e0 2024-08-20T03:05:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.771+0000 7f5de6ffd640 1 -- 172.21.15.44:0/2295311147 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df4036040 con 0x562747e580e0 2024-08-20T03:05:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.772+0000 7f5de6ffd640 1 -- 172.21.15.44:0/2295311147 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df402f8c0 con 0x562747e580e0 2024-08-20T03:05:36.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.772+0000 7f5e05d1d740 1 -- 172.21.15.44:0/2295311147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 msgr2=0x562747e70370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.772+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/2295311147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e70370 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009ca0 tx=0x7f5df403f000 comp rx=0 tx=0).stop 2024-08-20T03:05:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.772+0000 7f5e05d1d740 1 -- 172.21.15.44:0/2295311147 shutdown_connections 2024-08-20T03:05:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.772+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/2295311147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e580e0 0x562747e70370 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.772+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/2295311147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e6fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.772+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/2295311147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e4b830 0x562747e4f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.772+0000 7f5e05d1d740 1 -- 172.21.15.44:0/2295311147 >> 172.21.15.44:0/2295311147 conn(0x562747e49660 msgr2=0x562747e4a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.772+0000 7f5e05d1d740 1 -- 172.21.15.44:0/2295311147 shutdown_connections 2024-08-20T03:05:36.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.772+0000 7f5e05d1d740 1 -- 172.21.15.44:0/2295311147 wait complete. 2024-08-20T03:05:36.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.773+0000 7f5e05d1d740 1 -- start start 2024-08-20T03:05:36.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.773+0000 7f5e05d1d740 1 Processor -- start 2024-08-20T03:05:36.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.773+0000 7f5e05d1d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e4b830 0x562747e60050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.773+0000 7f5e05d1d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e60590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.773+0000 7f5e05d1d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e580e0 0x562747e516b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.773+0000 7f5e05d1d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1af610 con 0x562747e580e0 2024-08-20T03:05:36.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.773+0000 7f5e05d1d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1aef40 con 0x562747e4c150 2024-08-20T03:05:36.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.773+0000 7f5e05d1d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1ac180 con 0x562747e4b830 2024-08-20T03:05:36.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.773+0000 7f5e05d1d740 10 client.0 _ll_register_callbacks cb 0x7f5df8009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:36.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e016c0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e4b830 0x562747e60050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e00ebf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e60590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e016c0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e4b830 0x562747e60050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47462/0 (socket says 172.21.15.44:47462) 2024-08-20T03:05:36.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e00ebf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e60590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53428/0 (socket says 172.21.15.44:53428) 2024-08-20T03:05:36.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e016c0640 1 -- 172.21.15.44:0/3173012367 learned_addr learned my addr 172.21.15.44:0/3173012367 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:36.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e01ec1640 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e580e0 0x562747e516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e00ebf640 1 -- 172.21.15.44:0/3173012367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e4b830 msgr2=0x562747e60050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e00ebf640 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e4b830 0x562747e60050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e00ebf640 1 -- 172.21.15.44:0/3173012367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e580e0 msgr2=0x562747e516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e00ebf640 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e580e0 0x562747e516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e00ebf640 1 -- 172.21.15.44:0/3173012367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562747e51bf0 con 0x562747e4c150 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e01ec1640 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e580e0 0x562747e516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e00ebf640 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e60590 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5dec000c00 tx=0x7f5dec00b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=164875ae852aa873 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5de4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5de4ff9640 1 -- 172.21.15.44:0/3173012367 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec004220 con 0x562747e4c150 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5de4ff9640 1 -- 172.21.15.44:0/3173012367 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5dec0043c0 con 0x562747e4c150 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.774+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3173012367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df4036040 con 0x562747e4c150 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.775+0000 7f5de4ff9640 1 -- 172.21.15.44:0/3173012367 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5dec005040 con 0x562747e4c150 2024-08-20T03:05:36.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.775+0000 7f5e05d1d740 10 client.25814 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:36.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.775+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3173012367 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562747ed2490 con 0x562747e4c150 2024-08-20T03:05:36.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.775+0000 7f5de4ff9640 1 -- 172.21.15.44:0/3173012367 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df4036040 con 0x562747e4c150 2024-08-20T03:05:36.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.775+0000 7f5de4ff9640 1 -- 172.21.15.44:0/3173012367 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5dec004660 con 0x562747e4c150 2024-08-20T03:05:36.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.775+0000 7f5e05d1d740 10 client.25814 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:36.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.775+0000 7f5e05d1d740 10 client.25814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:36.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.775+0000 7f5e05d1d740 20 client.25814 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:36.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.775+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562747ed2980 0x562747ed4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:36.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.775+0000 7f5e05d1d740 4 client.25814 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:05:36.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.775+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3173012367 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562747dc02d0 con 0x562747ed2980 2024-08-20T03:05:36.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.775+0000 7f5e016c0640 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562747ed2980 0x562747ed4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:36.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.776+0000 7f5de4ff9640 10 client.25814 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:36.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.776+0000 7f5e016c0640 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562747ed2980 0x562747ed4da0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a238b767ea5b948 server_cookie=5adeba293da6704e in_seq=0 out_seq=0 2024-08-20T03:05:36.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.779+0000 7f5de4ff9640 1 -- 172.21.15.44:0/3173012367 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562747dc02d0 con 0x562747ed2980 2024-08-20T03:05:36.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.779+0000 7f5de4ff9640 10 client.25814 handle_command_reply: tid=0 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.779+0000 7f5e05d1d740 1 client.25814 shutdown 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.780+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3173012367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 msgr2=0x562747e60590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.780+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e60590 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5dec000c00 tx=0x7f5dec00b410 comp rx=0 tx=0).stop 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.780+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3173012367 shutdown_connections 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.780+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x562747ed2980 0x562747ed4da0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.780+0000 7f5de4ff9640 0 client.25814 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.780+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562747e580e0 0x562747e516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.780+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562747e4c150 0x562747e60590 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.780+0000 7f5e05d1d740 1 --2- 172.21.15.44:0/3173012367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562747e4b830 0x562747e60050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.780+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3173012367 >> 172.21.15.44:0/3173012367 conn(0x562747e49660 msgr2=0x562747e566b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.780+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3173012367 shutdown_connections 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.780+0000 7f5e05d1d740 1 -- 172.21.15.44:0/3173012367 wait complete. 2024-08-20T03:05:36.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.780+0000 7f5e05d1d740 20 client.25814 trim_cache size 0 max 16384 2024-08-20T03:05:36.785 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:05:36.785 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-08-20T03:05:36.785 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-08-20T03:05:36.785 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:05:36.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.781+0000 7f5e03924640 1 -- 172.21.15.44:0/1110604479 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5de80b3d40 msgr2=0x7f5de80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:36.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.781+0000 7f5e03924640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5de80b3d40 0x7f5de80b6200 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5df4005e30 tx=0x7f5df40098c0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.781+0000 7f5e03924640 1 -- 172.21.15.44:0/1110604479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dfc101990 msgr2=0x7f5dfc198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:36.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.781+0000 7f5e03924640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dfc101990 0x7f5dfc198f30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5df800e9f0 tx=0x7f5df800edf0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.781+0000 7f5e03924640 1 -- 172.21.15.44:0/1110604479 shutdown_connections 2024-08-20T03:05:36.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.781+0000 7f5e03924640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5de80b3d40 0x7f5de80b6200 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.781+0000 7f5e03924640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5dfc107d20 0x7f5dfc19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.781+0000 7f5e03924640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5dfc1073a0 0x7f5dfc199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.781+0000 7f5e03924640 1 --2- 172.21.15.44:0/1110604479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5dfc101990 0x7f5dfc198f30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:36.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.781+0000 7f5e03924640 1 -- 172.21.15.44:0/1110604479 >> 172.21.15.44:0/1110604479 conn(0x7f5dfc0fd400 msgr2=0x7f5dfc0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:36.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.781+0000 7f5e03924640 1 -- 172.21.15.44:0/1110604479 shutdown_connections 2024-08-20T03:05:36.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:36.781+0000 7f5e03924640 1 -- 172.21.15.44:0/1110604479 wait complete. 2024-08-20T03:05:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.229+0000 7f9042ff5640 1 -- 172.21.15.44:0/1920905720 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557dffc82d50 con 0x557dffda7b70 2024-08-20T03:05:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.229+0000 7f9042ff5640 10 client.25780 handle_command_reply: tid=0 2024-08-20T03:05:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.229+0000 7f9067e1f740 1 client.25780 shutdown 2024-08-20T03:05:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.230+0000 7f9067e1f740 1 -- 172.21.15.44:0/1920905720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 msgr2=0x557dffd4c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.230+0000 7f9067e1f740 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 0x557dffd4c0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f905802a910 tx=0x7f9058003b90 comp rx=0 tx=0).stop 2024-08-20T03:05:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.230+0000 7f9067e1f740 1 -- 172.21.15.44:0/1920905720 shutdown_connections 2024-08-20T03:05:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.230+0000 7f9067e1f740 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x557dffda7b70 0x557dffda9f90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.230+0000 7f9067e1f740 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dffd24e00 0x557dffd4c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.230+0000 7f9067e1f740 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557dffd242b0 0x557dffd4c0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.230+0000 7f9042ff5640 0 client.25780 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.230+0000 7f9067e1f740 1 --2- 172.21.15.44:0/1920905720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557dffd10660 0x557dffd51980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.230+0000 7f9067e1f740 1 -- 172.21.15.44:0/1920905720 >> 172.21.15.44:0/1920905720 conn(0x557dffd0db70 msgr2=0x557dffd180c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.230+0000 7f9067e1f740 1 -- 172.21.15.44:0/1920905720 shutdown_connections 2024-08-20T03:05:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.230+0000 7f9067e1f740 1 -- 172.21.15.44:0/1920905720 wait complete. 2024-08-20T03:05:37.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.231+0000 7f9067e1f740 20 client.25780 trim_cache size 0 max 16384 2024-08-20T03:05:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:05:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:05:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:05:37.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:05:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.231+0000 7f9065030640 1 -- 172.21.15.44:0/3552853884 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f90480b3ed0 msgr2=0x7f90480b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.231+0000 7f9065030640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f90480b3ed0 0x7f90480b6390 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f90500a4c90 tx=0x7f90500b1f90 comp rx=0 tx=0).stop 2024-08-20T03:05:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.231+0000 7f9065030640 1 -- 172.21.15.44:0/3552853884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9060074ae0 msgr2=0x7f90601a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.231+0000 7f9065030640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9060074ae0 0x7f90601a3570 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f905000d950 tx=0x7f905000de10 comp rx=0 tx=0).stop 2024-08-20T03:05:37.236 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.232+0000 7f9065030640 1 -- 172.21.15.44:0/3552853884 shutdown_connections 2024-08-20T03:05:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.232+0000 7f9065030640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f90480b3ed0 0x7f90480b6390 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.232+0000 7f9065030640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90601a8e40 0x7f90601a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.232+0000 7f9065030640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90601175f0 0x7f90601a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.232+0000 7f9065030640 1 --2- 172.21.15.44:0/3552853884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9060074ae0 0x7f90601a3570 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.232+0000 7f9065030640 1 -- 172.21.15.44:0/3552853884 >> 172.21.15.44:0/3552853884 conn(0x7f906006d8e0 msgr2=0x7f90600731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.232+0000 7f9065030640 1 -- 172.21.15.44:0/3552853884 shutdown_connections 2024-08-20T03:05:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:37.232+0000 7f9065030640 1 -- 172.21.15.44:0/3552853884 wait complete. 2024-08-20T03:05:37.477 INFO:tasks.workunit.client.0.smithi044.stdout: 107 1246834 360 878378 167 708552 810 2024-08-20T03:05:37.722 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T03:05:37.722 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:05:37.997 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:05:38.393 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:38 smithi044 ceph-mon[18790]: pgmap v414: 97 pgs: 97 active+clean; 446 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 1.26k op/s 2024-08-20T03:05:38.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.476+0000 7f95b8531640 1 -- 172.21.15.44:0/2402827865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95b01039a0 msgr2=0x7f95b0103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:38.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.476+0000 7f95b8531640 1 --2- 172.21.15.44:0/2402827865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95b01039a0 0x7f95b0103d80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009900 tx=0x7f95ac02a410 comp rx=0 tx=0).stop 2024-08-20T03:05:38.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.476+0000 7f95b8531640 1 -- 172.21.15.44:0/2402827865 shutdown_connections 2024-08-20T03:05:38.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.476+0000 7f95b8531640 1 --2- 172.21.15.44:0/2402827865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95b010ce00 0x7f95b010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:38.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.476+0000 7f95b8531640 1 --2- 172.21.15.44:0/2402827865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95b01042c0 0x7f95b010c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:38.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.476+0000 7f95b8531640 1 --2- 172.21.15.44:0/2402827865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95b01039a0 0x7f95b0103d80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:38.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.476+0000 7f95b8531640 1 -- 172.21.15.44:0/2402827865 >> 172.21.15.44:0/2402827865 conn(0x7f95b00fd400 msgr2=0x7f95b00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:38.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.476+0000 7f95b8531640 1 -- 172.21.15.44:0/2402827865 shutdown_connections 2024-08-20T03:05:38.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.476+0000 7f95b8531640 1 -- 172.21.15.44:0/2402827865 wait complete. 2024-08-20T03:05:38.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.477+0000 7f95b8531640 1 Processor -- start 2024-08-20T03:05:38.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.477+0000 7f95b8531640 1 -- start start 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.477+0000 7f95b8531640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95b01039a0 0x7f95b0198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.477+0000 7f95b8531640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95b01042c0 0x7f95b0199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.477+0000 7f95b8531640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95b010ce00 0x7f95b019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.477+0000 7f95b8531640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95b0112a70 con 0x7f95b010ce00 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.477+0000 7f95b8531640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95b01128f0 con 0x7f95b01039a0 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.477+0000 7f95b8531640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f95b0112bf0 con 0x7f95b01042c0 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.477+0000 7f95b6ace640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95b010ce00 0x7f95b019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.477+0000 7f95b6ace640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95b010ce00 0x7f95b019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37974/0 (socket says 172.21.15.44:37974) 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95b6ace640 1 -- 172.21.15.44:0/1768005141 learned_addr learned my addr 172.21.15.44:0/1768005141 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95b5acc640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95b01042c0 0x7f95b0199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95b62cd640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95b01039a0 0x7f95b0198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95b6ace640 1 -- 172.21.15.44:0/1768005141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95b01042c0 msgr2=0x7f95b0199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:38.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95b6ace640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95b01042c0 0x7f95b0199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95b6ace640 1 -- 172.21.15.44:0/1768005141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95b01039a0 msgr2=0x7f95b0198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95b6ace640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95b01039a0 0x7f95b0198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95b6ace640 1 -- 172.21.15.44:0/1768005141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b019f070 con 0x7f95b010ce00 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95b6ace640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95b010ce00 0x7f95b019e8a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f95a800eb10 tx=0x7f95a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95a77fe640 1 -- 172.21.15.44:0/1768005141 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8004b20 con 0x7f95b010ce00 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95a77fe640 1 -- 172.21.15.44:0/1768005141 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95a8004cc0 con 0x7f95b010ce00 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95b8531640 1 -- 172.21.15.44:0/1768005141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b019f360 con 0x7f95b010ce00 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95a77fe640 1 -- 172.21.15.44:0/1768005141 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8010670 con 0x7f95b010ce00 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.478+0000 7f95b8531640 1 -- 172.21.15.44:0/1768005141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b0101530 con 0x7f95b010ce00 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.479+0000 7f95b8531640 1 -- 172.21.15.44:0/1768005141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b0103d80 con 0x7f95b010ce00 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.481+0000 7f95a77fe640 1 -- 172.21.15.44:0/1768005141 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f95a8005000 con 0x7f95b010ce00 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.483+0000 7f95a77fe640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f959c0b3d40 0x7f959c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:38.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.483+0000 7f95b62cd640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f959c0b3d40 0x7f959c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:38.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.483+0000 7f95a77fe640 1 -- 172.21.15.44:0/1768005141 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f95a80e4890 con 0x7f95b010ce00 2024-08-20T03:05:38.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.483+0000 7f95b62cd640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f959c0b3d40 0x7f959c0b6200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f95ac031440 tx=0x7f95ac0095f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:38.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.486+0000 7f95a77fe640 1 -- 172.21.15.44:0/1768005141 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f95a8055e20 con 0x7f95b010ce00 2024-08-20T03:05:38.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:38 smithi165 ceph-mon[22357]: pgmap v414: 97 pgs: 97 active+clean; 446 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 1.26k op/s 2024-08-20T03:05:38.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:38 smithi099 ceph-mon[21984]: pgmap v414: 97 pgs: 97 active+clean; 446 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 1.26k op/s 2024-08-20T03:05:38.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.744+0000 7f95b8531640 1 -- 172.21.15.44:0/1768005141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95b019a0f0 con 0x7f95b010ce00 2024-08-20T03:05:38.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.745+0000 7f95a77fe640 1 -- 172.21.15.44:0/1768005141 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11972 (secure 0 0 0) 0x7f95a80abc10 con 0x7f95b010ce00 2024-08-20T03:05:38.748 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:05:38.748 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":30,"btime":"2024-08-20T03:03:44:997954+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:03:44.335961+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24569,24583,24596,24589,24604]},"id":1}]} 2024-08-20T03:05:38.748 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 30 2024-08-20T03:05:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.747+0000 7f95b8531640 1 -- 172.21.15.44:0/1768005141 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f959c0b3d40 msgr2=0x7f959c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.747+0000 7f95b8531640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f959c0b3d40 0x7f959c0b6200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f95ac031440 tx=0x7f95ac0095f0 comp rx=0 tx=0).stop 2024-08-20T03:05:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.747+0000 7f95b8531640 1 -- 172.21.15.44:0/1768005141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95b010ce00 msgr2=0x7f95b019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.747+0000 7f95b8531640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95b010ce00 0x7f95b019e8a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f95a800eb10 tx=0x7f95a800c2d0 comp rx=0 tx=0).stop 2024-08-20T03:05:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.747+0000 7f95b8531640 1 -- 172.21.15.44:0/1768005141 shutdown_connections 2024-08-20T03:05:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.747+0000 7f95b8531640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f959c0b3d40 0x7f959c0b6200 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:38.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.747+0000 7f95b8531640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95b010ce00 0x7f95b019e8a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:38.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.747+0000 7f95b8531640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f95b01042c0 0x7f95b0199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:38.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.747+0000 7f95b8531640 1 --2- 172.21.15.44:0/1768005141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95b01039a0 0x7f95b0198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:38.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.747+0000 7f95b8531640 1 -- 172.21.15.44:0/1768005141 >> 172.21.15.44:0/1768005141 conn(0x7f95b00fd400 msgr2=0x7f95b00fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:38.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.747+0000 7f95b8531640 1 -- 172.21.15.44:0/1768005141 shutdown_connections 2024-08-20T03:05:38.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:38.747+0000 7f95b8531640 1 -- 172.21.15.44:0/1768005141 wait complete. 2024-08-20T03:05:39.500 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:39 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1768005141' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:05:39.534 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-20T03:05:39.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:39 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1768005141' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:05:39.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:39 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1768005141' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:05:39.866 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:05:40.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.417+0000 7f7f4dd42640 1 -- 172.21.15.44:0/3519941457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f481039b0 msgr2=0x7f7f48103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:40.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.417+0000 7f7f4dd42640 1 --2- 172.21.15.44:0/3519941457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f481039b0 0x7f7f48103d90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009900 tx=0x7f7f3802a410 comp rx=0 tx=0).stop 2024-08-20T03:05:40.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.417+0000 7f7f4dd42640 1 -- 172.21.15.44:0/3519941457 shutdown_connections 2024-08-20T03:05:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.417+0000 7f7f4dd42640 1 --2- 172.21.15.44:0/3519941457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f4810ce10 0x7f7f4810f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.417+0000 7f7f4dd42640 1 --2- 172.21.15.44:0/3519941457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f481042d0 0x7f7f4810c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.417+0000 7f7f4dd42640 1 --2- 172.21.15.44:0/3519941457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f481039b0 0x7f7f48103d90 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.417+0000 7f7f4dd42640 1 -- 172.21.15.44:0/3519941457 >> 172.21.15.44:0/3519941457 conn(0x7f7f480fd430 msgr2=0x7f7f480ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.417+0000 7f7f4dd42640 1 -- 172.21.15.44:0/3519941457 shutdown_connections 2024-08-20T03:05:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.417+0000 7f7f4dd42640 1 -- 172.21.15.44:0/3519941457 wait complete. 2024-08-20T03:05:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.418+0000 7f7f4dd42640 1 Processor -- start 2024-08-20T03:05:40.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.418+0000 7f7f4dd42640 1 -- start start 2024-08-20T03:05:40.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f4dd42640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f481039b0 0x7f7f48101660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f4dd42640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f481042d0 0x7f7f48101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f4dd42640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f4810ce10 0x7f7f481020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f4dd42640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f48112a80 con 0x7f7f481039b0 2024-08-20T03:05:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f4dd42640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f48112900 con 0x7f7f481042d0 2024-08-20T03:05:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f4dd42640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f48112c00 con 0x7f7f4810ce10 2024-08-20T03:05:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f46ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f481042d0 0x7f7f48101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f481039b0 0x7f7f48101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f47fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f4810ce10 0x7f7f481020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f46ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f481042d0 0x7f7f48101ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:42972/0 (socket says 172.21.15.44:42972) 2024-08-20T03:05:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f46ffd640 1 -- 172.21.15.44:0/1366482369 learned_addr learned my addr 172.21.15.44:0/1366482369 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:40.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f477fe640 1 -- 172.21.15.44:0/1366482369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f4810ce10 msgr2=0x7f7f481020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f477fe640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f4810ce10 0x7f7f481020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f477fe640 1 -- 172.21.15.44:0/1366482369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f481042d0 msgr2=0x7f7f48101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f477fe640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f481042d0 0x7f7f48101ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f477fe640 1 -- 172.21.15.44:0/1366482369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f481a73d0 con 0x7f7f481039b0 2024-08-20T03:05:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f46ffd640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f481042d0 0x7f7f48101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f47fff640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f4810ce10 0x7f7f481020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.419+0000 7f7f477fe640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f481039b0 0x7f7f48101660 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7f3802a930 tx=0x7f7f38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.420+0000 7f7f44ff9640 1 -- 172.21.15.44:0/1366482369 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3803f030 con 0x7f7f481039b0 2024-08-20T03:05:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.420+0000 7f7f44ff9640 1 -- 172.21.15.44:0/1366482369 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f38003aa0 con 0x7f7f481039b0 2024-08-20T03:05:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.420+0000 7f7f4dd42640 1 -- 172.21.15.44:0/1366482369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f481a7660 con 0x7f7f481039b0 2024-08-20T03:05:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.420+0000 7f7f44ff9640 1 -- 172.21.15.44:0/1366482369 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38035070 con 0x7f7f481039b0 2024-08-20T03:05:40.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.420+0000 7f7f4dd42640 1 -- 172.21.15.44:0/1366482369 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f481a7b90 con 0x7f7f481039b0 2024-08-20T03:05:40.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.423+0000 7f7f44ff9640 1 -- 172.21.15.44:0/1366482369 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7f38042030 con 0x7f7f481039b0 2024-08-20T03:05:40.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.423+0000 7f7f5013b740 1 Processor -- start 2024-08-20T03:05:40.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.423+0000 7f7f5013b740 1 -- start start 2024-08-20T03:05:40.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f5013b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb118b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f5013b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 0x5615eb124aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f5013b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb124fe0 0x5615eb1273d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f5013b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5615eb08f930 con 0x5615eb119050 2024-08-20T03:05:40.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f5013b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5615eb07ee30 con 0x5615eb118730 2024-08-20T03:05:40.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f5013b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5615eb082e70 con 0x5615eb124fe0 2024-08-20T03:05:40.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 0x5615eb124aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f47fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb118b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 0x5615eb124aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37998/0 (socket says 172.21.15.44:37998) 2024-08-20T03:05:40.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f47fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb118b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:42974/0 (socket says 172.21.15.44:42974) 2024-08-20T03:05:40.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f47fff640 1 -- 172.21.15.44:0/293224558 learned_addr learned my addr 172.21.15.44:0/293224558 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f44ff9640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7f300b3dc0 0x7f7f300b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f46ffd640 1 --2- 172.21.15.44:0/293224558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb124fe0 0x5615eb1273d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f47fff640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7f300b3dc0 0x7f7f300b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f44ff9640 1 -- 172.21.15.44:0/1366482369 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7f381044c0 con 0x7f7f481039b0 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f46ffd640 1 -- 172.21.15.44:0/293224558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 msgr2=0x5615eb118b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f46ffd640 1 --2- 172.21.15.44:0/293224558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb118b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f46ffd640 1 -- 172.21.15.44:0/293224558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 msgr2=0x5615eb124aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f46ffd640 1 --2- 172.21.15.44:0/293224558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 0x5615eb124aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.424+0000 7f7f46ffd640 1 -- 172.21.15.44:0/293224558 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615eadde650 con 0x5615eb124fe0 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.425+0000 7f7f46ffd640 1 --2- 172.21.15.44:0/293224558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb124fe0 0x5615eb1273d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c010480 tx=0x7f7f3c010940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b93db99421d7539 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.425+0000 7f7f47fff640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7f300b3dc0 0x7f7f300b6280 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7f40012d60 tx=0x7f7f40004ab0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.425+0000 7f7f2dffb640 1 -- 172.21.15.44:0/293224558 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c011820 con 0x5615eb124fe0 2024-08-20T03:05:40.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.425+0000 7f7f2dffb640 1 -- 172.21.15.44:0/293224558 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f3c01c480 con 0x5615eb124fe0 2024-08-20T03:05:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.425+0000 7f7f2dffb640 1 -- 172.21.15.44:0/293224558 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00e690 con 0x5615eb124fe0 2024-08-20T03:05:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.426+0000 7f7f5013b740 1 -- 172.21.15.44:0/293224558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb124fe0 msgr2=0x5615eb1273d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.426+0000 7f7f5013b740 1 --2- 172.21.15.44:0/293224558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb124fe0 0x5615eb1273d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c010480 tx=0x7f7f3c010940 comp rx=0 tx=0).stop 2024-08-20T03:05:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.426+0000 7f7f5013b740 1 -- 172.21.15.44:0/293224558 shutdown_connections 2024-08-20T03:05:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.426+0000 7f7f5013b740 1 --2- 172.21.15.44:0/293224558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb124fe0 0x5615eb1273d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.426+0000 7f7f5013b740 1 --2- 172.21.15.44:0/293224558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 0x5615eb124aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.426+0000 7f7f5013b740 1 --2- 172.21.15.44:0/293224558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb118b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.426+0000 7f7f5013b740 1 -- 172.21.15.44:0/293224558 >> 172.21.15.44:0/293224558 conn(0x5615eb116560 msgr2=0x5615eb117700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.426+0000 7f7f5013b740 1 -- 172.21.15.44:0/293224558 shutdown_connections 2024-08-20T03:05:40.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.426+0000 7f7f5013b740 1 -- 172.21.15.44:0/293224558 wait complete. 2024-08-20T03:05:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.426+0000 7f7f5013b740 1 -- start start 2024-08-20T03:05:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.426+0000 7f7f5013b740 1 Processor -- start 2024-08-20T03:05:40.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f5013b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb121ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f5013b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb119050 0x5615eb122530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f5013b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb124fe0 0x5615eb122a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f5013b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5615eaee1700 con 0x5615eb124fe0 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f5013b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5615eb005810 con 0x5615eb118730 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f5013b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5615eb127c90 con 0x5615eb119050 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f5013b740 10 client.0 _ll_register_callbacks cb 0x5615eb078700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f47fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb124fe0 0x5615eb122a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f477fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb121ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f477fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb121ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:42982/0 (socket says 172.21.15.44:42982) 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f477fe640 1 -- 172.21.15.44:0/1718452974 learned_addr learned my addr 172.21.15.44:0/1718452974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f46ffd640 1 --2- 172.21.15.44:0/1718452974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb119050 0x5615eb122530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f477fe640 1 -- 172.21.15.44:0/1718452974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb119050 msgr2=0x5615eb122530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:40.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f477fe640 1 --2- 172.21.15.44:0/1718452974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb119050 0x5615eb122530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f477fe640 1 -- 172.21.15.44:0/1718452974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb124fe0 msgr2=0x5615eb122a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f477fe640 1 --2- 172.21.15.44:0/1718452974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb124fe0 0x5615eb122a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f477fe640 1 -- 172.21.15.44:0/1718452974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615eb1310c0 con 0x5615eb118730 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f477fe640 1 --2- 172.21.15.44:0/1718452974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb121ff0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7f380c7270 tx=0x7f7f380caa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d4bcf4783001c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.427+0000 7f7f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f26ffd640 1 -- 172.21.15.44:0/1718452974 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38049070 con 0x5615eb118730 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f26ffd640 1 -- 172.21.15.44:0/1718452974 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7f38101f00 con 0x5615eb118730 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f5013b740 1 -- 172.21.15.44:0/1718452974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5615eb02e150 con 0x5615eb118730 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f26ffd640 1 -- 172.21.15.44:0/1718452974 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7f380d41f0 con 0x5615eb118730 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f5013b740 10 client.25832 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f5013b740 1 -- 172.21.15.44:0/1718452974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5615eb133a70 con 0x5615eb118730 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f26ffd640 1 -- 172.21.15.44:0/1718452974 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5615eb02e150 con 0x5615eb118730 2024-08-20T03:05:40.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f26ffd640 1 -- 172.21.15.44:0/1718452974 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f7f380cb450 con 0x5615eb118730 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f5013b740 10 client.25832 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f5013b740 10 client.25832 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f5013b740 20 client.25832 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f5013b740 1 --2- 172.21.15.44:0/1718452974 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x5615eb134f20 0x5615eb137330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f5013b740 4 client.25832 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f5013b740 1 -- 172.21.15.44:0/1718452974 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615eb091880 con 0x5615eb134f20 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.428+0000 7f7f46ffd640 1 --2- 172.21.15.44:0/1718452974 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x5615eb134f20 0x5615eb137330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.429+0000 7f7f26ffd640 10 client.25832 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.429+0000 7f7f46ffd640 1 --2- 172.21.15.44:0/1718452974 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x5615eb134f20 0x5615eb137330 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f9c39e3dcc894a41 server_cookie=c02b05b0905c4c20 in_seq=0 out_seq=0 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.429+0000 7f7f26ffd640 1 -- 172.21.15.44:0/1718452974 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615eb091880 con 0x5615eb134f20 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.429+0000 7f7f26ffd640 10 client.25832 handle_command_reply: tid=0 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.429+0000 7f7f5013b740 1 client.25832 shutdown 2024-08-20T03:05:40.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.430+0000 7f7f5013b740 1 -- 172.21.15.44:0/1718452974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 msgr2=0x5615eb121ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.430+0000 7f7f5013b740 1 --2- 172.21.15.44:0/1718452974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb121ff0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7f380c7270 tx=0x7f7f380caa20 comp rx=0 tx=0).stop 2024-08-20T03:05:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.430+0000 7f7f5013b740 1 -- 172.21.15.44:0/1718452974 shutdown_connections 2024-08-20T03:05:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.430+0000 7f7f5013b740 1 --2- 172.21.15.44:0/1718452974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb124fe0 0x5615eb122a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.430+0000 7f7f5013b740 1 --2- 172.21.15.44:0/1718452974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb119050 0x5615eb122530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.430+0000 7f7f5013b740 1 --2- 172.21.15.44:0/1718452974 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x5615eb134f20 0x5615eb137330 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.430+0000 7f7f5013b740 1 --2- 172.21.15.44:0/1718452974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb121ff0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.430+0000 7f7f26ffd640 0 client.25832 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:05:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.430+0000 7f7f5013b740 1 -- 172.21.15.44:0/1718452974 >> 172.21.15.44:0/1718452974 conn(0x5615eb116560 msgr2=0x5615eb11c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.430+0000 7f7f5013b740 1 -- 172.21.15.44:0/1718452974 shutdown_connections 2024-08-20T03:05:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.430+0000 7f7f5013b740 1 -- 172.21.15.44:0/1718452974 wait complete. 2024-08-20T03:05:40.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.430+0000 7f7f5013b740 20 client.25832 trim_cache size 0 max 16384 2024-08-20T03:05:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.443+0000 7f7f5013b740 1 Processor -- start 2024-08-20T03:05:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.443+0000 7f7f5013b740 1 -- start start 2024-08-20T03:05:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.443+0000 7f7f5013b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb118730 0x5615eb1157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.443+0000 7f7f5013b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 0x5615eb11c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.443+0000 7f7f5013b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb124fe0 0x5615eb121750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.443+0000 7f7f5013b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f481b0460 con 0x5615eb119050 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.443+0000 7f7f5013b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f481b2060 con 0x5615eb124fe0 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.443+0000 7f7f5013b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f481b0b30 con 0x5615eb118730 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f46ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 0x5615eb11c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f46ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 0x5615eb11c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38022/0 (socket says 172.21.15.44:38022) 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f46ffd640 1 -- 172.21.15.44:0/3832451969 learned_addr learned my addr 172.21.15.44:0/3832451969 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f477fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb118730 0x5615eb1157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f47fff640 1 --2- 172.21.15.44:0/3832451969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb124fe0 0x5615eb121750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f477fe640 1 -- 172.21.15.44:0/3832451969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb124fe0 msgr2=0x5615eb121750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f477fe640 1 --2- 172.21.15.44:0/3832451969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb124fe0 0x5615eb121750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f477fe640 1 -- 172.21.15.44:0/3832451969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 msgr2=0x5615eb11c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:40.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f477fe640 1 --2- 172.21.15.44:0/3832451969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 0x5615eb11c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f477fe640 1 -- 172.21.15.44:0/3832451969 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615eb116fe0 con 0x5615eb118730 2024-08-20T03:05:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f46ffd640 1 --2- 172.21.15.44:0/3832451969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 0x5615eb11c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f477fe640 1 --2- 172.21.15.44:0/3832451969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb118730 0x5615eb1157a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7f380092a0 tx=0x7f7f38003e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b0ff41cfc8f34a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f2d7fa640 1 -- 172.21.15.44:0/3832451969 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38049070 con 0x5615eb118730 2024-08-20T03:05:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f2d7fa640 1 -- 172.21.15.44:0/3832451969 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f380cb1f0 con 0x5615eb118730 2024-08-20T03:05:40.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.444+0000 7f7f2d7fa640 1 -- 172.21.15.44:0/3832451969 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f380d3e30 con 0x5615eb118730 2024-08-20T03:05:40.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.445+0000 7f7f5013b740 1 -- 172.21.15.44:0/3832451969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb118730 msgr2=0x5615eb1157a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.445+0000 7f7f5013b740 1 --2- 172.21.15.44:0/3832451969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb118730 0x5615eb1157a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7f380092a0 tx=0x7f7f38003e60 comp rx=0 tx=0).stop 2024-08-20T03:05:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.445+0000 7f7f5013b740 1 -- 172.21.15.44:0/3832451969 shutdown_connections 2024-08-20T03:05:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.445+0000 7f7f5013b740 1 --2- 172.21.15.44:0/3832451969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb124fe0 0x5615eb121750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.445+0000 7f7f5013b740 1 --2- 172.21.15.44:0/3832451969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb119050 0x5615eb11c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.445+0000 7f7f5013b740 1 --2- 172.21.15.44:0/3832451969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb118730 0x5615eb1157a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.445+0000 7f7f5013b740 1 -- 172.21.15.44:0/3832451969 >> 172.21.15.44:0/3832451969 conn(0x5615eb116560 msgr2=0x5615eb123e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.445+0000 7f7f5013b740 1 -- 172.21.15.44:0/3832451969 shutdown_connections 2024-08-20T03:05:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.445+0000 7f7f5013b740 1 -- 172.21.15.44:0/3832451969 wait complete. 2024-08-20T03:05:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.445+0000 7f7f5013b740 1 -- start start 2024-08-20T03:05:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f5013b740 1 Processor -- start 2024-08-20T03:05:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f5013b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb12d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f5013b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb119050 0x5615eb1288d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f5013b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb124fe0 0x5615eb128e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f5013b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f481b18d0 con 0x5615eb124fe0 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f5013b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f481b1200 con 0x5615eb118730 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f5013b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f481ae440 con 0x5615eb119050 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f5013b740 10 client.0 _ll_register_callbacks cb 0x7f7f38005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f47fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb124fe0 0x5615eb128e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f477fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb12d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f46ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb119050 0x5615eb1288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f477fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb12d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:42994/0 (socket says 172.21.15.44:42994) 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f46ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb119050 0x5615eb1288d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33518/0 (socket says 172.21.15.44:33518) 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f47fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb124fe0 0x5615eb128e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38036/0 (socket says 172.21.15.44:38036) 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f477fe640 1 -- 172.21.15.44:0/3267559471 learned_addr learned my addr 172.21.15.44:0/3267559471 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f477fe640 1 -- 172.21.15.44:0/3267559471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb119050 msgr2=0x5615eb1288d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f477fe640 1 --2- 172.21.15.44:0/3267559471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb119050 0x5615eb1288d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f477fe640 1 -- 172.21.15.44:0/3267559471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb124fe0 msgr2=0x5615eb128e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f477fe640 1 --2- 172.21.15.44:0/3267559471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb124fe0 0x5615eb128e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f477fe640 1 -- 172.21.15.44:0/3267559471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615eb129480 con 0x5615eb118730 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.446+0000 7f7f46ffd640 1 --2- 172.21.15.44:0/3267559471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb119050 0x5615eb1288d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f477fe640 1 --2- 172.21.15.44:0/3267559471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb12d400 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7f380caea0 tx=0x7f7f380cf8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaf24f3e689ba6b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f26ffd640 1 -- 172.21.15.44:0/3267559471 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f380ea060 con 0x5615eb118730 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f26ffd640 1 -- 172.21.15.44:0/3267559471 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f7f380ec2f0 con 0x5615eb118730 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f5013b740 1 -- 172.21.15.44:0/3267559471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f380cb1f0 con 0x5615eb118730 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f26ffd640 1 -- 172.21.15.44:0/3267559471 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7f380aa050 con 0x5615eb118730 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f5013b740 10 client.25838 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f5013b740 1 -- 172.21.15.44:0/3267559471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5615eb129480 con 0x5615eb118730 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f26ffd640 1 -- 172.21.15.44:0/3267559471 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f380cb1f0 con 0x5615eb118730 2024-08-20T03:05:40.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f26ffd640 1 -- 172.21.15.44:0/3267559471 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f7f381040f0 con 0x5615eb118730 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f5013b740 10 client.25838 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f5013b740 10 client.25838 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f5013b740 20 client.25838 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f5013b740 1 --2- 172.21.15.44:0/3267559471 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x5615eb19fdb0 0x5615eb1a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f5013b740 4 client.25838 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.447+0000 7f7f5013b740 1 -- 172.21.15.44:0/3267559471 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615eb08d320 con 0x5615eb19fdb0 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f46ffd640 1 --2- 172.21.15.44:0/3267559471 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x5615eb19fdb0 0x5615eb1a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f46ffd640 1 --2- 172.21.15.44:0/3267559471 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x5615eb19fdb0 0x5615eb1a21d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=58fd4415530e1098 server_cookie=82404e200e934666 in_seq=0 out_seq=0 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f26ffd640 10 client.25838 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f26ffd640 1 -- 172.21.15.44:0/3267559471 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615eb08d320 con 0x5615eb19fdb0 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f26ffd640 10 client.25838 handle_command_reply: tid=0 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f5013b740 1 client.25838 shutdown 2024-08-20T03:05:40.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f5013b740 1 -- 172.21.15.44:0/3267559471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 msgr2=0x5615eb12d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f5013b740 1 --2- 172.21.15.44:0/3267559471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb12d400 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7f380caea0 tx=0x7f7f380cf8c0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f5013b740 1 -- 172.21.15.44:0/3267559471 shutdown_connections 2024-08-20T03:05:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f5013b740 1 --2- 172.21.15.44:0/3267559471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615eb124fe0 0x5615eb128e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f5013b740 1 --2- 172.21.15.44:0/3267559471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5615eb119050 0x5615eb1288d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f5013b740 1 --2- 172.21.15.44:0/3267559471 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x5615eb19fdb0 0x5615eb1a21d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f5013b740 1 --2- 172.21.15.44:0/3267559471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615eb118730 0x5615eb12d400 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f26ffd640 0 client.25838 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:05:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f5013b740 1 -- 172.21.15.44:0/3267559471 >> 172.21.15.44:0/3267559471 conn(0x5615eb116560 msgr2=0x5615eb121f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f5013b740 1 -- 172.21.15.44:0/3267559471 shutdown_connections 2024-08-20T03:05:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f5013b740 1 -- 172.21.15.44:0/3267559471 wait complete. 2024-08-20T03:05:40.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.448+0000 7f7f5013b740 20 client.25838 trim_cache size 0 max 16384 2024-08-20T03:05:40.457 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:05:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.449+0000 7f7f4dd42640 1 -- 172.21.15.44:0/1366482369 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7f300b3dc0 msgr2=0x7f7f300b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.449+0000 7f7f4dd42640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7f300b3dc0 0x7f7f300b6280 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7f40012d60 tx=0x7f7f40004ab0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.449+0000 7f7f4dd42640 1 -- 172.21.15.44:0/1366482369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f481039b0 msgr2=0x7f7f48101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.449+0000 7f7f4dd42640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f481039b0 0x7f7f48101660 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7f3802a930 tx=0x7f7f38004270 comp rx=0 tx=0).stop 2024-08-20T03:05:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.449+0000 7f7f4dd42640 1 -- 172.21.15.44:0/1366482369 shutdown_connections 2024-08-20T03:05:40.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.449+0000 7f7f4dd42640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7f300b3dc0 0x7f7f300b6280 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.449+0000 7f7f4dd42640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f4810ce10 0x7f7f481020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.449+0000 7f7f4dd42640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f481042d0 0x7f7f48101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.449+0000 7f7f4dd42640 1 --2- 172.21.15.44:0/1366482369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f481039b0 0x7f7f48101660 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.449+0000 7f7f4dd42640 1 -- 172.21.15.44:0/1366482369 >> 172.21.15.44:0/1366482369 conn(0x7f7f480fd430 msgr2=0x7f7f480fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.449+0000 7f7f4dd42640 1 -- 172.21.15.44:0/1366482369 shutdown_connections 2024-08-20T03:05:40.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:40.449+0000 7f7f4dd42640 1 -- 172.21.15.44:0/1366482369 wait complete. 2024-08-20T03:05:40.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:40 smithi044 ceph-mon[18790]: pgmap v415: 97 pgs: 97 active+clean; 443 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.71k op/s 2024-08-20T03:05:40.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:40 smithi165 ceph-mon[22357]: pgmap v415: 97 pgs: 97 active+clean; 443 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.71k op/s 2024-08-20T03:05:40.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:40 smithi099 ceph-mon[21984]: pgmap v415: 97 pgs: 97 active+clean; 443 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.71k op/s 2024-08-20T03:05:41.962 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-20T03:05:42.284 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:42 smithi044 ceph-mon[18790]: pgmap v416: 97 pgs: 97 active+clean; 424 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 1.48k op/s 2024-08-20T03:05:42.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:42 smithi165 ceph-mon[22357]: pgmap v416: 97 pgs: 97 active+clean; 424 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 1.48k op/s 2024-08-20T03:05:42.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:42 smithi099 ceph-mon[21984]: pgmap v416: 97 pgs: 97 active+clean; 424 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 1.48k op/s 2024-08-20T03:05:43.770 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:05:44.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.344+0000 7f77fb59e640 1 -- 172.21.15.44:0/2257526494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4095c80 msgr2=0x7f77f4096060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:44.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.344+0000 7f77fb59e640 1 --2- 172.21.15.44:0/2257526494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4095c80 0x7f77f4096060 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009900 tx=0x7f77f002a440 comp rx=0 tx=0).stop 2024-08-20T03:05:44.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.347+0000 7f77fb59e640 1 -- 172.21.15.44:0/2257526494 shutdown_connections 2024-08-20T03:05:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.347+0000 7f77fb59e640 1 --2- 172.21.15.44:0/2257526494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f4099a00 0x7f77f40a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.347+0000 7f77fb59e640 1 --2- 172.21.15.44:0/2257526494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f4099060 0x7f77f40994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.347+0000 7f77fb59e640 1 --2- 172.21.15.44:0/2257526494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4095c80 0x7f77f4096060 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.347+0000 7f77fb59e640 1 -- 172.21.15.44:0/2257526494 >> 172.21.15.44:0/2257526494 conn(0x7f77f400a0d0 msgr2=0x7f77f400daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.347+0000 7f77fb59e640 1 -- 172.21.15.44:0/2257526494 shutdown_connections 2024-08-20T03:05:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.347+0000 7f77fb59e640 1 -- 172.21.15.44:0/2257526494 wait complete. 2024-08-20T03:05:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.348+0000 7f77fb59e640 1 Processor -- start 2024-08-20T03:05:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.348+0000 7f77fb59e640 1 -- start start 2024-08-20T03:05:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.348+0000 7f77fb59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4095c80 0x7f77f40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.348+0000 7f77fb59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f4099060 0x7f77f40a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.348+0000 7f77fb59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f4099a00 0x7f77f40aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.349+0000 7f77fb59e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77f40a3710 con 0x7f77f4095c80 2024-08-20T03:05:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.349+0000 7f77fb59e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77f40a3590 con 0x7f77f4099060 2024-08-20T03:05:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.349+0000 7f77fb59e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f77f40a3890 con 0x7f77f4099a00 2024-08-20T03:05:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.349+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f4099a00 0x7f77f40aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.349+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f4099a00 0x7f77f40aa950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33538/0 (socket says 172.21.15.44:33538) 2024-08-20T03:05:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.349+0000 7f77fad9d640 1 -- 172.21.15.44:0/3378972098 learned_addr learned my addr 172.21.15.44:0/3378972098 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:44.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.349+0000 7f77fa59c640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4095c80 0x7f77f40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.350+0000 7f77fa59c640 1 -- 172.21.15.44:0/3378972098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f4099a00 msgr2=0x7f77f40aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:44.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.351+0000 7f77fa59c640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f4099a00 0x7f77f40aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.351+0000 7f77fa59c640 1 -- 172.21.15.44:0/3378972098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f4099060 msgr2=0x7f77f40a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:44.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.351+0000 7f77fa59c640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f4099060 0x7f77f40a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.351+0000 7f77fa59c640 1 -- 172.21.15.44:0/3378972098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f40ab0d0 con 0x7f77f4095c80 2024-08-20T03:05:44.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.351+0000 7f77fa59c640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4095c80 0x7f77f40a5070 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f77f0031440 tx=0x7f77f0003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:44.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.352+0000 7f77eb7fe640 1 -- 172.21.15.44:0/3378972098 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f003f070 con 0x7f77f4095c80 2024-08-20T03:05:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.352+0000 7f77fb59e640 1 -- 172.21.15.44:0/3378972098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f40ab360 con 0x7f77f4095c80 2024-08-20T03:05:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.352+0000 7f77fb59e640 1 -- 172.21.15.44:0/3378972098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f40adad0 con 0x7f77f4095c80 2024-08-20T03:05:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.352+0000 7f77eb7fe640 1 -- 172.21.15.44:0/3378972098 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77f0031d60 con 0x7f77f4095c80 2024-08-20T03:05:44.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.352+0000 7f77eb7fe640 1 -- 172.21.15.44:0/3378972098 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0035030 con 0x7f77f4095c80 2024-08-20T03:05:44.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.357+0000 7f7802f5b740 1 Processor -- start 2024-08-20T03:05:44.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.357+0000 7f7802f5b740 1 -- start start 2024-08-20T03:05:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.358+0000 7f7802f5b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d059a60 0x55769d059e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.358+0000 7f7802f5b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d05a380 0x55769d065dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.358+0000 7f7802f5b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d066310 0x55769d068700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.358+0000 7f7802f5b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55769cfd0930 con 0x55769d05a380 2024-08-20T03:05:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.358+0000 7f7802f5b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55769cfbfe30 con 0x55769d059a60 2024-08-20T03:05:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.358+0000 7f7802f5b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55769cfc3e70 con 0x55769d066310 2024-08-20T03:05:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.358+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d059a60 0x55769d059e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.359+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d059a60 0x55769d059e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:43010/0 (socket says 172.21.15.44:43010) 2024-08-20T03:05:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.359+0000 7f77f9d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d066310 0x55769d068700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.359+0000 7f77fad9d640 1 -- 172.21.15.44:0/1531456549 learned_addr learned my addr 172.21.15.44:0/1531456549 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:44.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.359+0000 7f77fad9d640 1 -- 172.21.15.44:0/1531456549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d066310 msgr2=0x55769d068700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:44.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.359+0000 7f77fad9d640 1 --2- 172.21.15.44:0/1531456549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d066310 0x55769d068700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.359+0000 7f77fad9d640 1 -- 172.21.15.44:0/1531456549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d05a380 msgr2=0x55769d065dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:05:44.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.359+0000 7f77fad9d640 1 --2- 172.21.15.44:0/1531456549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d05a380 0x55769d065dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.359+0000 7f77fad9d640 1 -- 172.21.15.44:0/1531456549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55769cd1f650 con 0x55769d059a60 2024-08-20T03:05:44.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.360+0000 7f77f9d9b640 1 --2- 172.21.15.44:0/1531456549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d066310 0x55769d068700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:44.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.360+0000 7f77eb7fe640 1 -- 172.21.15.44:0/3378972098 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f77f0042070 con 0x7f77f4095c80 2024-08-20T03:05:44.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.360+0000 7f77fad9d640 1 --2- 172.21.15.44:0/1531456549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d059a60 0x55769d059e40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f77fc051e80 tx=0x7f77fc069c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a40c8bdfd7368291 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:44.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.361+0000 7f77e8ff9640 1 -- 172.21.15.44:0/1531456549 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc07c020 con 0x55769d059a60 2024-08-20T03:05:44.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.361+0000 7f77e8ff9640 1 -- 172.21.15.44:0/1531456549 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77fc078470 con 0x55769d059a60 2024-08-20T03:05:44.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.361+0000 7f77e8ff9640 1 -- 172.21.15.44:0/1531456549 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc077630 con 0x55769d059a60 2024-08-20T03:05:44.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.362+0000 7f7802f5b740 1 -- 172.21.15.44:0/1531456549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d059a60 msgr2=0x55769d059e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:44.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.362+0000 7f7802f5b740 1 --2- 172.21.15.44:0/1531456549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d059a60 0x55769d059e40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f77fc051e80 tx=0x7f77fc069c70 comp rx=0 tx=0).stop 2024-08-20T03:05:44.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.363+0000 7f77eb7fe640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77e40b3df0 0x7f77e40b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.363+0000 7f77fad9d640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77e40b3df0 0x7f77e40b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.363+0000 7f77eb7fe640 1 -- 172.21.15.44:0/3378972098 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f77f01040e0 con 0x7f77f4095c80 2024-08-20T03:05:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.364+0000 7f77fad9d640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77e40b3df0 0x7f77e40b62b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0691d0 tx=0x7f77fc068ec0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:44.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.364+0000 7f7802f5b740 1 -- 172.21.15.44:0/1531456549 shutdown_connections 2024-08-20T03:05:44.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.364+0000 7f7802f5b740 1 --2- 172.21.15.44:0/1531456549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d066310 0x55769d068700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.364+0000 7f7802f5b740 1 --2- 172.21.15.44:0/1531456549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d05a380 0x55769d065dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.365+0000 7f7802f5b740 1 --2- 172.21.15.44:0/1531456549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d059a60 0x55769d059e40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.365+0000 7f7802f5b740 1 -- 172.21.15.44:0/1531456549 >> 172.21.15.44:0/1531456549 conn(0x55769d057890 msgr2=0x55769d058a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:44.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.365+0000 7f7802f5b740 1 -- 172.21.15.44:0/1531456549 shutdown_connections 2024-08-20T03:05:44.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.365+0000 7f7802f5b740 1 -- 172.21.15.44:0/1531456549 wait complete. 2024-08-20T03:05:44.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.366+0000 7f7802f5b740 1 -- start start 2024-08-20T03:05:44.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.366+0000 7f7802f5b740 1 Processor -- start 2024-08-20T03:05:44.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.367+0000 7f7802f5b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d059a60 0x55769d063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.367+0000 7f7802f5b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d05a380 0x55769d063770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.367+0000 7f7802f5b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 0x55769d063cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.367+0000 7f7802f5b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55769ce22700 con 0x55769d066310 2024-08-20T03:05:44.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.367+0000 7f77f9d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d05a380 0x55769d063770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.367+0000 7f77f9d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d05a380 0x55769d063770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:43022/0 (socket says 172.21.15.44:43022) 2024-08-20T03:05:44.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.367+0000 7f77f9d9b640 1 -- 172.21.15.44:0/1339231045 learned_addr learned my addr 172.21.15.44:0/1339231045 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:44.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.367+0000 7f77fa59c640 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d059a60 0x55769d063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.367+0000 7f7802f5b740 1 -- 172.21.15.44:0/1339231045 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55769cf46810 con 0x55769d05a380 2024-08-20T03:05:44.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.367+0000 7f7802f5b740 1 -- 172.21.15.44:0/1339231045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55769d068fc0 con 0x55769d059a60 2024-08-20T03:05:44.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.368+0000 7f7802f5b740 10 client.0 _ll_register_callbacks cb 0x55769cfb9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:44.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.368+0000 7f77fad9d640 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 0x55769d063cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.368+0000 7f77f9d9b640 1 -- 172.21.15.44:0/1339231045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d059a60 msgr2=0x55769d063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.368+0000 7f77f9d9b640 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d059a60 0x55769d063230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.368+0000 7f77f9d9b640 1 -- 172.21.15.44:0/1339231045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 msgr2=0x55769d063cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.368+0000 7f77fa59c640 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d059a60 0x55769d063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:05:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.368+0000 7f77f9d9b640 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 0x55769d063cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.368+0000 7f77f9d9b640 1 -- 172.21.15.44:0/1339231045 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55769d072380 con 0x55769d05a380 2024-08-20T03:05:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.368+0000 7f77fad9d640 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 0x55769d063cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:05:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.369+0000 7f77e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:05:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.369+0000 7f77f9d9b640 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d05a380 0x55769d063770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00e240 tx=0x7f77ec00e700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fc49c232abc6389 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.369+0000 7f77e1ffb640 1 -- 172.21.15.44:0/1339231045 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec017070 con 0x55769d05a380 2024-08-20T03:05:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.369+0000 7f77e1ffb640 1 -- 172.21.15.44:0/1339231045 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f77ec013680 con 0x55769d05a380 2024-08-20T03:05:44.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.369+0000 7f7802f5b740 1 -- 172.21.15.44:0/1339231045 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55769cf6f150 con 0x55769d05a380 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.369+0000 7f77e1ffb640 1 -- 172.21.15.44:0/1339231045 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f77ec013880 con 0x55769d05a380 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.369+0000 7f7802f5b740 10 client.25844 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.369+0000 7f7802f5b740 1 -- 172.21.15.44:0/1339231045 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55769d074d80 con 0x55769d05a380 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.370+0000 7f77e1ffb640 1 -- 172.21.15.44:0/1339231045 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55769cf6f150 con 0x55769d05a380 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.370+0000 7f77e1ffb640 1 -- 172.21.15.44:0/1339231045 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f77ec012ae0 con 0x55769d05a380 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.370+0000 7f7802f5b740 10 client.25844 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.370+0000 7f7802f5b740 10 client.25844 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.370+0000 7f7802f5b740 20 client.25844 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.370+0000 7f7802f5b740 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55769d076330 0x55769d078750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.370+0000 7f7802f5b740 4 client.25844 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.371+0000 7f7802f5b740 1 -- 172.21.15.44:0/1339231045 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55769cfd2880 con 0x55769d076330 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.371+0000 7f77fa59c640 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55769d076330 0x55769d078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.371+0000 7f77e1ffb640 10 client.25844 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:05:44.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.371+0000 7f77fa59c640 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55769d076330 0x55769d078750 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e839f4a4c8fde0c server_cookie=40262ae61309b1 in_seq=0 out_seq=0 2024-08-20T03:05:44.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.372+0000 7f77e1ffb640 1 -- 172.21.15.44:0/1339231045 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55769cfd2880 con 0x55769d076330 2024-08-20T03:05:44.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.372+0000 7f77e1ffb640 10 client.25844 handle_command_reply: tid=0 2024-08-20T03:05:44.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.372+0000 7f7802f5b740 1 client.25844 shutdown 2024-08-20T03:05:44.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.372+0000 7f7802f5b740 1 -- 172.21.15.44:0/1339231045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d05a380 msgr2=0x55769d063770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.373+0000 7f7802f5b740 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d05a380 0x55769d063770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00e240 tx=0x7f77ec00e700 comp rx=0 tx=0).stop 2024-08-20T03:05:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.373+0000 7f7802f5b740 1 -- 172.21.15.44:0/1339231045 shutdown_connections 2024-08-20T03:05:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.373+0000 7f7802f5b740 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55769d076330 0x55769d078750 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.373+0000 7f77e1ffb640 0 client.25844 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:05:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.373+0000 7f7802f5b740 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 0x55769d063cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.373+0000 7f7802f5b740 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d05a380 0x55769d063770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.373+0000 7f7802f5b740 1 --2- 172.21.15.44:0/1339231045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d059a60 0x55769d063230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.373+0000 7f7802f5b740 1 -- 172.21.15.44:0/1339231045 >> 172.21.15.44:0/1339231045 conn(0x55769d057890 msgr2=0x55769d05d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.373+0000 7f7802f5b740 1 -- 172.21.15.44:0/1339231045 shutdown_connections 2024-08-20T03:05:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.374+0000 7f7802f5b740 1 -- 172.21.15.44:0/1339231045 wait complete. 2024-08-20T03:05:44.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.374+0000 7f7802f5b740 20 client.25844 trim_cache size 0 max 16384 2024-08-20T03:05:44.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.401+0000 7f7802f5b740 1 Processor -- start 2024-08-20T03:05:44.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.401+0000 7f7802f5b740 1 -- start start 2024-08-20T03:05:44.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.402+0000 7f7802f5b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d059a60 0x55769d05db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.402+0000 7f7802f5b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d05a380 0x55769d07e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.402+0000 7f7802f5b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d066310 0x55769d07e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.402+0000 7f77f9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d05a380 0x55769d07e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.402+0000 7f77f9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d05a380 0x55769d07e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38080/0 (socket says 172.21.15.44:38080) 2024-08-20T03:05:44.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.402+0000 7f7802f5b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc00e630 con 0x55769d05a380 2024-08-20T03:05:44.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.402+0000 7f7802f5b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc010230 con 0x55769d066310 2024-08-20T03:05:44.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.402+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d066310 0x55769d07e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.402+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d066310 0x55769d07e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:43026/0 (socket says 172.21.15.44:43026) 2024-08-20T03:05:44.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.403+0000 7f77fad9d640 1 -- 172.21.15.44:0/919008884 learned_addr learned my addr 172.21.15.44:0/919008884 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:44.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.403+0000 7f7802f5b740 1 -- 172.21.15.44:0/919008884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc00ed00 con 0x55769d059a60 2024-08-20T03:05:44.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.407+0000 7f77fa59c640 1 --2- 172.21.15.44:0/919008884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d059a60 0x55769d05db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.407+0000 7f77fad9d640 1 -- 172.21.15.44:0/919008884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d059a60 msgr2=0x55769d05db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:44.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.407+0000 7f77fad9d640 1 --2- 172.21.15.44:0/919008884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d059a60 0x55769d05db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.407+0000 7f77fad9d640 1 -- 172.21.15.44:0/919008884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d05a380 msgr2=0x55769d07e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:44.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.407+0000 7f77fad9d640 1 --2- 172.21.15.44:0/919008884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d05a380 0x55769d07e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.407+0000 7f77fad9d640 1 -- 172.21.15.44:0/919008884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55769d061730 con 0x55769d066310 2024-08-20T03:05:44.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.407+0000 7f77fad9d640 1 --2- 172.21.15.44:0/919008884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d066310 0x55769d07e5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f77fc052160 tx=0x7f77fc078dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=662d8c4f57f8ace3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:44.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.409+0000 7f77e3fff640 1 -- 172.21.15.44:0/919008884 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc07c040 con 0x55769d066310 2024-08-20T03:05:44.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.409+0000 7f77e3fff640 1 -- 172.21.15.44:0/919008884 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77fc082e30 con 0x55769d066310 2024-08-20T03:05:44.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.409+0000 7f77e3fff640 1 -- 172.21.15.44:0/919008884 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc098460 con 0x55769d066310 2024-08-20T03:05:44.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.410+0000 7f7802f5b740 1 -- 172.21.15.44:0/919008884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d066310 msgr2=0x55769d07e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:44.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.410+0000 7f7802f5b740 1 --2- 172.21.15.44:0/919008884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d066310 0x55769d07e5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f77fc052160 tx=0x7f77fc078dc0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.415+0000 7f7802f5b740 1 -- 172.21.15.44:0/919008884 shutdown_connections 2024-08-20T03:05:44.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.415+0000 7f7802f5b740 1 --2- 172.21.15.44:0/919008884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d066310 0x55769d07e5a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.415+0000 7f7802f5b740 1 --2- 172.21.15.44:0/919008884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d05a380 0x55769d07e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.415+0000 7f7802f5b740 1 --2- 172.21.15.44:0/919008884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d059a60 0x55769d05db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.415+0000 7f7802f5b740 1 -- 172.21.15.44:0/919008884 >> 172.21.15.44:0/919008884 conn(0x55769d057890 msgr2=0x55769d065080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:44.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.426+0000 7f7802f5b740 1 -- 172.21.15.44:0/919008884 shutdown_connections 2024-08-20T03:05:44.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.426+0000 7f7802f5b740 1 -- 172.21.15.44:0/919008884 wait complete. 2024-08-20T03:05:44.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.426+0000 7f7802f5b740 1 -- start start 2024-08-20T03:05:44.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.427+0000 7f7802f5b740 1 Processor -- start 2024-08-20T03:05:44.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.427+0000 7f7802f5b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d059a60 0x55769d06b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.427+0000 7f7802f5b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d05a380 0x55769d06bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.427+0000 7f7802f5b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 0x55769d0df030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.427+0000 7f7802f5b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc00faa0 con 0x55769d066310 2024-08-20T03:05:44.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.427+0000 7f7802f5b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc00f3d0 con 0x55769d059a60 2024-08-20T03:05:44.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.427+0000 7f7802f5b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc00c610 con 0x55769d05a380 2024-08-20T03:05:44.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.427+0000 7f7802f5b740 10 client.0 _ll_register_callbacks cb 0x7f77ec009390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:44.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.427+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 0x55769d0df030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.427+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 0x55769d0df030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38088/0 (socket says 172.21.15.44:38088) 2024-08-20T03:05:44.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.427+0000 7f77fad9d640 1 -- 172.21.15.44:0/2302207119 learned_addr learned my addr 172.21.15.44:0/2302207119 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:44.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.428+0000 7f77f9d9b640 1 --2- 172.21.15.44:0/2302207119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d05a380 0x55769d06bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.428+0000 7f77fad9d640 1 -- 172.21.15.44:0/2302207119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d05a380 msgr2=0x55769d06bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:44.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.428+0000 7f77fad9d640 1 --2- 172.21.15.44:0/2302207119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d05a380 0x55769d06bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.428+0000 7f77fad9d640 1 -- 172.21.15.44:0/2302207119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d059a60 msgr2=0x55769d06b840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:05:44.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.428+0000 7f77fad9d640 1 --2- 172.21.15.44:0/2302207119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d059a60 0x55769d06b840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.428+0000 7f77fad9d640 1 -- 172.21.15.44:0/2302207119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55769d0df5a0 con 0x55769d066310 2024-08-20T03:05:44.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.428+0000 7f77fad9d640 1 --2- 172.21.15.44:0/2302207119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 0x55769d0df030 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f77fc079fd0 tx=0x7f77fc068c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66506164b53bc3a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:44.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.428+0000 7f77e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:05:44.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.429+0000 7f77e1ffb640 1 -- 172.21.15.44:0/2302207119 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc09e070 con 0x55769d066310 2024-08-20T03:05:44.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.429+0000 7f77e1ffb640 1 -- 172.21.15.44:0/2302207119 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f77fc098670 con 0x55769d066310 2024-08-20T03:05:44.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.430+0000 7f7802f5b740 1 -- 172.21.15.44:0/2302207119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77fc082e30 con 0x55769d066310 2024-08-20T03:05:44.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.430+0000 7f77e1ffb640 1 -- 172.21.15.44:0/2302207119 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f77fc07c040 con 0x55769d066310 2024-08-20T03:05:44.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.432+0000 7f7802f5b740 10 client.16374 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:44.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.432+0000 7f7802f5b740 1 -- 172.21.15.44:0/2302207119 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55769d0e08a0 con 0x55769d066310 2024-08-20T03:05:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.433+0000 7f77e1ffb640 1 -- 172.21.15.44:0/2302207119 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77fc098d60 con 0x55769d066310 2024-08-20T03:05:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.433+0000 7f77e1ffb640 1 -- 172.21.15.44:0/2302207119 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f77fc0a3880 con 0x55769d066310 2024-08-20T03:05:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.435+0000 7f7802f5b740 10 client.16374 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.435+0000 7f7802f5b740 10 client.16374 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:05:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.435+0000 7f7802f5b740 20 client.16374 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.435+0000 7f7802f5b740 1 --2- 172.21.15.44:0/2302207119 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55769d076330 0x55769d0e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.435+0000 7f7802f5b740 4 client.16374 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:05:44.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.435+0000 7f7802f5b740 1 -- 172.21.15.44:0/2302207119 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f77fc082e30 con 0x55769d076330 2024-08-20T03:05:44.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.437+0000 7f77fa59c640 1 --2- 172.21.15.44:0/2302207119 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55769d076330 0x55769d0e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:44.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.445+0000 7f77fa59c640 1 --2- 172.21.15.44:0/2302207119 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55769d076330 0x55769d0e2c30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2eb6b77480b25864 server_cookie=b8f2b71aa9de89b5 in_seq=0 out_seq=0 2024-08-20T03:05:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.447+0000 7f77e1ffb640 10 client.16374 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:05:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.449+0000 7f77e1ffb640 1 -- 172.21.15.44:0/2302207119 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f77fc082e30 con 0x55769d076330 2024-08-20T03:05:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.449+0000 7f77e1ffb640 10 client.16374 handle_command_reply: tid=0 2024-08-20T03:05:44.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.450+0000 7f7802f5b740 1 client.16374 shutdown 2024-08-20T03:05:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.457+0000 7f7802f5b740 1 -- 172.21.15.44:0/2302207119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 msgr2=0x55769d0df030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:44.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.457+0000 7f7802f5b740 1 --2- 172.21.15.44:0/2302207119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 0x55769d0df030 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f77fc079fd0 tx=0x7f77fc068c10 comp rx=0 tx=0).stop 2024-08-20T03:05:44.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.459+0000 7f7802f5b740 1 -- 172.21.15.44:0/2302207119 shutdown_connections 2024-08-20T03:05:44.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.459+0000 7f7802f5b740 1 --2- 172.21.15.44:0/2302207119 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x55769d076330 0x55769d0e2c30 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.459+0000 7f7802f5b740 1 --2- 172.21.15.44:0/2302207119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55769d066310 0x55769d0df030 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.459+0000 7f7802f5b740 1 --2- 172.21.15.44:0/2302207119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55769d05a380 0x55769d06bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.459+0000 7f7802f5b740 1 --2- 172.21.15.44:0/2302207119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55769d059a60 0x55769d06b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.459+0000 7f7802f5b740 1 -- 172.21.15.44:0/2302207119 >> 172.21.15.44:0/2302207119 conn(0x55769d057890 msgr2=0x55769d071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:44.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.460+0000 7f77e1ffb640 0 client.16374 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:05:44.464 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:44 smithi044 ceph-mon[18790]: pgmap v417: 97 pgs: 97 active+clean; 396 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.2 MiB/s wr, 1.49k op/s 2024-08-20T03:05:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.466+0000 7f7802f5b740 1 -- 172.21.15.44:0/2302207119 shutdown_connections 2024-08-20T03:05:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.468+0000 7f7802f5b740 1 -- 172.21.15.44:0/2302207119 wait complete. 2024-08-20T03:05:44.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.468+0000 7f7802f5b740 20 client.16374 trim_cache size 0 max 16384 2024-08-20T03:05:44.472 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:05:44.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.471+0000 7f77fb59e640 1 -- 172.21.15.44:0/3378972098 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77e40b3df0 msgr2=0x7f77e40b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:44.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.471+0000 7f77fb59e640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77e40b3df0 0x7f77e40b62b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0691d0 tx=0x7f77fc068ec0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.471+0000 7f77fb59e640 1 -- 172.21.15.44:0/3378972098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4095c80 msgr2=0x7f77f40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:44.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.471+0000 7f77fb59e640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4095c80 0x7f77f40a5070 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f77f0031440 tx=0x7f77f0003d60 comp rx=0 tx=0).stop 2024-08-20T03:05:44.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.471+0000 7f77fb59e640 1 -- 172.21.15.44:0/3378972098 shutdown_connections 2024-08-20T03:05:44.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.471+0000 7f77fb59e640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77e40b3df0 0x7f77e40b62b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.471+0000 7f77fb59e640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77f4099a00 0x7f77f40aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.471+0000 7f77fb59e640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f4099060 0x7f77f40a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.471+0000 7f77fb59e640 1 --2- 172.21.15.44:0/3378972098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77f4095c80 0x7f77f40a5070 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:44.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.471+0000 7f77fb59e640 1 -- 172.21.15.44:0/3378972098 >> 172.21.15.44:0/3378972098 conn(0x7f77f400a0d0 msgr2=0x7f77f40972c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:44.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.472+0000 7f77fb59e640 1 -- 172.21.15.44:0/3378972098 shutdown_connections 2024-08-20T03:05:44.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:44.472+0000 7f77fb59e640 1 -- 172.21.15.44:0/3378972098 wait complete. 2024-08-20T03:05:44.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:44 smithi165 ceph-mon[22357]: pgmap v417: 97 pgs: 97 active+clean; 396 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.2 MiB/s wr, 1.49k op/s 2024-08-20T03:05:44.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:44 smithi099 ceph-mon[21984]: pgmap v417: 97 pgs: 97 active+clean; 396 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.2 MiB/s wr, 1.49k op/s 2024-08-20T03:05:46.075 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T03:05:46.423 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:46 smithi044 ceph-mon[18790]: pgmap v418: 97 pgs: 97 active+clean; 396 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 1.23k op/s 2024-08-20T03:05:46.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:46 smithi165 ceph-mon[22357]: pgmap v418: 97 pgs: 97 active+clean; 396 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 1.23k op/s 2024-08-20T03:05:46.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:46 smithi099 ceph-mon[21984]: pgmap v418: 97 pgs: 97 active+clean; 396 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 1.23k op/s 2024-08-20T03:05:46.784 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:05:47.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.356+0000 7f42923b8640 1 -- 172.21.15.44:0/2395020038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42840965a0 msgr2=0x7f4284096980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:47.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.356+0000 7f42923b8640 1 --2- 172.21.15.44:0/2395020038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42840965a0 0x7f4284096980 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f428c068900 tx=0x7f428c0978a0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.357+0000 7f42923b8640 1 -- 172.21.15.44:0/2395020038 shutdown_connections 2024-08-20T03:05:47.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.357+0000 7f42923b8640 1 --2- 172.21.15.44:0/2395020038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4284094550 0x7f42840a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.357+0000 7f42923b8640 1 --2- 172.21.15.44:0/2395020038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4284093bd0 0x7f4284094010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.357+0000 7f42923b8640 1 --2- 172.21.15.44:0/2395020038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42840965a0 0x7f4284096980 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.357+0000 7f42923b8640 1 -- 172.21.15.44:0/2395020038 >> 172.21.15.44:0/2395020038 conn(0x7f4284009480 msgr2=0x7f428400bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:47.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.357+0000 7f42923b8640 1 -- 172.21.15.44:0/2395020038 shutdown_connections 2024-08-20T03:05:47.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.357+0000 7f42923b8640 1 -- 172.21.15.44:0/2395020038 wait complete. 2024-08-20T03:05:47.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.358+0000 7f42923b8640 1 Processor -- start 2024-08-20T03:05:47.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.358+0000 7f42923b8640 1 -- start start 2024-08-20T03:05:47.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f42923b8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4284093bd0 0x7f4284129a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f42923b8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4284094550 0x7f4284129fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f42923b8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42840965a0 0x7f428412f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f42923b8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42840a3590 con 0x7f4284093bd0 2024-08-20T03:05:47.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f42923b8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42840a3410 con 0x7f4284094550 2024-08-20T03:05:47.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f42923b8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42840a3710 con 0x7f42840965a0 2024-08-20T03:05:47.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f42913b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4284093bd0 0x7f4284129a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f42913b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4284093bd0 0x7f4284129a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38102/0 (socket says 172.21.15.44:38102) 2024-08-20T03:05:47.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f42913b6640 1 -- 172.21.15.44:0/2024902685 learned_addr learned my addr 172.21.15.44:0/2024902685 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:47.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f4290bb5640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4284094550 0x7f4284129fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f4291bb7640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42840965a0 0x7f428412f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f4290bb5640 1 -- 172.21.15.44:0/2024902685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42840965a0 msgr2=0x7f428412f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:47.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.359+0000 7f4290bb5640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42840965a0 0x7f428412f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.360+0000 7f4290bb5640 1 -- 172.21.15.44:0/2024902685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4284093bd0 msgr2=0x7f4284129a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:47.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.360+0000 7f4290bb5640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4284093bd0 0x7f4284129a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.360+0000 7f4290bb5640 1 -- 172.21.15.44:0/2024902685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428412fb30 con 0x7f4284094550 2024-08-20T03:05:47.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.360+0000 7f4290bb5640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4284094550 0x7f4284129fc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f428800e9f0 tx=0x7f428800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:47.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.360+0000 7f42827fc640 1 -- 172.21.15.44:0/2024902685 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428800ccc0 con 0x7f4284094550 2024-08-20T03:05:47.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.360+0000 7f42827fc640 1 -- 172.21.15.44:0/2024902685 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42880108e0 con 0x7f4284094550 2024-08-20T03:05:47.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.360+0000 7f42923b8640 1 -- 172.21.15.44:0/2024902685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428412fe20 con 0x7f4284094550 2024-08-20T03:05:47.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.360+0000 7f42827fc640 1 -- 172.21.15.44:0/2024902685 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288018880 con 0x7f4284094550 2024-08-20T03:05:47.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.360+0000 7f42923b8640 1 -- 172.21.15.44:0/2024902685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42841364e0 con 0x7f4284094550 2024-08-20T03:05:47.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.364+0000 7f4295a13740 1 Processor -- start 2024-08-20T03:05:47.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.364+0000 7f4295a13740 1 -- start start 2024-08-20T03:05:47.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.365+0000 7f4295a13740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb65910 0x56438bb65cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.365+0000 7f4295a13740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb71c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.365+0000 7f4295a13740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb721c0 0x56438bb745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.365+0000 7f4295a13740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56438badc930 con 0x56438bb65910 2024-08-20T03:05:47.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.365+0000 7f4295a13740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56438bacbe30 con 0x56438bb66230 2024-08-20T03:05:47.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.365+0000 7f4295a13740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56438bacfe70 con 0x56438bb721c0 2024-08-20T03:05:47.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.365+0000 7f42913b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb71c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.365+0000 7f42913b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb71c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:43072/0 (socket says 172.21.15.44:43072) 2024-08-20T03:05:47.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.365+0000 7f42913b6640 1 -- 172.21.15.44:0/2813792748 learned_addr learned my addr 172.21.15.44:0/2813792748 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:47.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.366+0000 7f42913b6640 1 -- 172.21.15.44:0/2813792748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb721c0 msgr2=0x56438bb745b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:05:47.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.366+0000 7f42913b6640 1 --2- 172.21.15.44:0/2813792748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb721c0 0x56438bb745b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.366+0000 7f42913b6640 1 -- 172.21.15.44:0/2813792748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb65910 msgr2=0x56438bb65cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:47.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.366+0000 7f42913b6640 1 --2- 172.21.15.44:0/2813792748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb65910 0x56438bb65cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.366+0000 7f42913b6640 1 -- 172.21.15.44:0/2813792748 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56438b82b650 con 0x56438bb66230 2024-08-20T03:05:47.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.366+0000 7f42827fc640 1 -- 172.21.15.44:0/2024902685 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4288010430 con 0x7f4284094550 2024-08-20T03:05:47.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.366+0000 7f42913b6640 1 --2- 172.21.15.44:0/2813792748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb71c80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f428c02e900 tx=0x7f428c077ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=879524cc02f35d96 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:47.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.367+0000 7f42777fe640 1 -- 172.21.15.44:0/2813792748 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c06c960 con 0x56438bb66230 2024-08-20T03:05:47.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.367+0000 7f42827fc640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42780b3df0 0x7f42780b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.367+0000 7f42777fe640 1 -- 172.21.15.44:0/2813792748 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428c0774b0 con 0x56438bb66230 2024-08-20T03:05:47.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.367+0000 7f42777fe640 1 -- 172.21.15.44:0/2813792748 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c09f680 con 0x56438bb66230 2024-08-20T03:05:47.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.368+0000 7f42827fc640 1 -- 172.21.15.44:0/2024902685 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f42880e4420 con 0x7f4284094550 2024-08-20T03:05:47.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.368+0000 7f4291bb7640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42780b3df0 0x7f42780b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.368+0000 7f4295a13740 1 -- 172.21.15.44:0/2813792748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 msgr2=0x56438bb71c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.368+0000 7f4295a13740 1 --2- 172.21.15.44:0/2813792748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb71c80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f428c02e900 tx=0x7f428c077ed0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.368+0000 7f4295a13740 1 -- 172.21.15.44:0/2813792748 shutdown_connections 2024-08-20T03:05:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.368+0000 7f4295a13740 1 --2- 172.21.15.44:0/2813792748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb721c0 0x56438bb745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.368+0000 7f4295a13740 1 --2- 172.21.15.44:0/2813792748 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb71c80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.368+0000 7f4295a13740 1 --2- 172.21.15.44:0/2813792748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb65910 0x56438bb65cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.368+0000 7f4295a13740 1 -- 172.21.15.44:0/2813792748 >> 172.21.15.44:0/2813792748 conn(0x56438bb63740 msgr2=0x56438bb648e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:47.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.369+0000 7f4295a13740 1 -- 172.21.15.44:0/2813792748 shutdown_connections 2024-08-20T03:05:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.369+0000 7f4295a13740 1 -- 172.21.15.44:0/2813792748 wait complete. 2024-08-20T03:05:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.370+0000 7f4291bb7640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42780b3df0 0x7f42780b62b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f427c013fd0 tx=0x7f427c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.370+0000 7f4295a13740 1 -- start start 2024-08-20T03:05:47.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.370+0000 7f4295a13740 1 Processor -- start 2024-08-20T03:05:47.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.371+0000 7f4295a13740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb65910 0x56438bb6f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.371+0000 7f4295a13740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb66230 0x56438bb6f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.371+0000 7f4295a13740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb721c0 0x56438bb6fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.371+0000 7f4295a13740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56438b92e700 con 0x56438bb66230 2024-08-20T03:05:47.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.371+0000 7f4295a13740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56438ba52810 con 0x56438bb65910 2024-08-20T03:05:47.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.371+0000 7f42913b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb65910 0x56438bb6f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.371+0000 7f42913b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb65910 0x56438bb6f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:43078/0 (socket says 172.21.15.44:43078) 2024-08-20T03:05:47.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.371+0000 7f42913b6640 1 -- 172.21.15.44:0/3502394214 learned_addr learned my addr 172.21.15.44:0/3502394214 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:47.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.371+0000 7f4295a13740 1 -- 172.21.15.44:0/3502394214 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56438bb85e20 con 0x56438bb721c0 2024-08-20T03:05:47.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.371+0000 7f4290bb5640 1 --2- 172.21.15.44:0/3502394214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb66230 0x56438bb6f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.372+0000 7f4295a13740 10 client.0 _ll_register_callbacks cb 0x56438bac5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:47.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.372+0000 7f4290bb5640 1 -- 172.21.15.44:0/3502394214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb721c0 msgr2=0x56438bb6fc70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:05:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.372+0000 7f4290bb5640 1 --2- 172.21.15.44:0/3502394214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb721c0 0x56438bb6fc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.372+0000 7f4290bb5640 1 -- 172.21.15.44:0/3502394214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb65910 msgr2=0x56438bb6f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.372+0000 7f4290bb5640 1 --2- 172.21.15.44:0/3502394214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb65910 0x56438bb6f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.372+0000 7f4290bb5640 1 -- 172.21.15.44:0/3502394214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438bb8f2e0 con 0x56438bb66230 2024-08-20T03:05:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.372+0000 7f4290bb5640 1 --2- 172.21.15.44:0/3502394214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb66230 0x56438bb6f730 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f42880e1b80 tx=0x7f42880aaaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fa44205734a90a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.373+0000 7f4274ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:05:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.373+0000 7f4274ff9640 1 -- 172.21.15.44:0/3502394214 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42880e2010 con 0x56438bb66230 2024-08-20T03:05:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.373+0000 7f4295a13740 1 -- 172.21.15.44:0/3502394214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56438ba7b150 con 0x56438bb66230 2024-08-20T03:05:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.373+0000 7f4274ff9640 1 -- 172.21.15.44:0/3502394214 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f42880aa2a0 con 0x56438bb66230 2024-08-20T03:05:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.373+0000 7f4274ff9640 1 -- 172.21.15.44:0/3502394214 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42880af640 con 0x56438bb66230 2024-08-20T03:05:47.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.373+0000 7f4295a13740 10 client.16383 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:47.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.373+0000 7f4295a13740 1 -- 172.21.15.44:0/3502394214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56438bb91cd0 con 0x56438bb66230 2024-08-20T03:05:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.374+0000 7f4274ff9640 1 -- 172.21.15.44:0/3502394214 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42880b40f0 con 0x56438bb66230 2024-08-20T03:05:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.374+0000 7f4274ff9640 1 -- 172.21.15.44:0/3502394214 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f42880af250 con 0x56438bb66230 2024-08-20T03:05:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.374+0000 7f4295a13740 10 client.16383 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.374+0000 7f4295a13740 10 client.16383 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:05:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.374+0000 7f4295a13740 20 client.16383 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.374+0000 7f4295a13740 1 --2- 172.21.15.44:0/3502394214 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x56438bb936c0 0x56438bb95ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.374+0000 7f4295a13740 4 client.16383 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:05:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.374+0000 7f4295a13740 1 -- 172.21.15.44:0/3502394214 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56438ba7b150 con 0x56438bb936c0 2024-08-20T03:05:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.375+0000 7f42913b6640 1 --2- 172.21.15.44:0/3502394214 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x56438bb936c0 0x56438bb95ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.375+0000 7f42913b6640 1 --2- 172.21.15.44:0/3502394214 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x56438bb936c0 0x56438bb95ae0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a497b443b51b5c08 server_cookie=68abf0fd3e06d3be in_seq=0 out_seq=0 2024-08-20T03:05:47.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.375+0000 7f4274ff9640 10 client.16383 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:05:47.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.376+0000 7f4274ff9640 1 -- 172.21.15.44:0/3502394214 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56438ba7b150 con 0x56438bb936c0 2024-08-20T03:05:47.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.376+0000 7f4274ff9640 10 client.16383 handle_command_reply: tid=0 2024-08-20T03:05:47.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.376+0000 7f4295a13740 1 client.16383 shutdown 2024-08-20T03:05:47.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.377+0000 7f4295a13740 1 -- 172.21.15.44:0/3502394214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb66230 msgr2=0x56438bb6f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:47.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.377+0000 7f4295a13740 1 --2- 172.21.15.44:0/3502394214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb66230 0x56438bb6f730 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f42880e1b80 tx=0x7f42880aaaa0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.377+0000 7f4295a13740 1 -- 172.21.15.44:0/3502394214 shutdown_connections 2024-08-20T03:05:47.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.377+0000 7f4295a13740 1 --2- 172.21.15.44:0/3502394214 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x56438bb936c0 0x56438bb95ae0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.377+0000 7f4295a13740 1 --2- 172.21.15.44:0/3502394214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb721c0 0x56438bb6fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.377+0000 7f4295a13740 1 --2- 172.21.15.44:0/3502394214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb66230 0x56438bb6f730 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.377+0000 7f4295a13740 1 --2- 172.21.15.44:0/3502394214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb65910 0x56438bb6f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.377+0000 7f4295a13740 1 -- 172.21.15.44:0/3502394214 >> 172.21.15.44:0/3502394214 conn(0x56438bb63740 msgr2=0x56438bb692e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:47.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.377+0000 7f4274ff9640 0 client.16383 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:05:47.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.377+0000 7f4295a13740 1 -- 172.21.15.44:0/3502394214 shutdown_connections 2024-08-20T03:05:47.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.377+0000 7f4295a13740 1 -- 172.21.15.44:0/3502394214 wait complete. 2024-08-20T03:05:47.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.377+0000 7f4295a13740 20 client.16383 trim_cache size 0 max 16384 2024-08-20T03:05:47.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.401+0000 7f4295a13740 1 Processor -- start 2024-08-20T03:05:47.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.402+0000 7f4295a13740 1 -- start start 2024-08-20T03:05:47.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.402+0000 7f4295a13740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 0x56438bb69980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.402+0000 7f4295a13740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.402+0000 7f4295a13740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb721c0 0x56438bb6f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.402+0000 7f4295a13740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f428413ed70 con 0x56438bb721c0 2024-08-20T03:05:47.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.402+0000 7f4295a13740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4284140970 con 0x56438bb66230 2024-08-20T03:05:47.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.402+0000 7f4295a13740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f428413f440 con 0x56438bb65910 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.402+0000 7f42913b6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 0x56438bb69980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.402+0000 7f4291bb7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb721c0 0x56438bb6f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.402+0000 7f4290bb5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.402+0000 7f42913b6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 0x56438bb69980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33606/0 (socket says 172.21.15.44:33606) 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.402+0000 7f42913b6640 1 -- 172.21.15.44:0/406755579 learned_addr learned my addr 172.21.15.44:0/406755579 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.403+0000 7f4291bb7640 1 -- 172.21.15.44:0/406755579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 msgr2=0x56438bb69980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.403+0000 7f4291bb7640 1 --2- 172.21.15.44:0/406755579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 0x56438bb69980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.403+0000 7f4291bb7640 1 -- 172.21.15.44:0/406755579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 msgr2=0x56438bb625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.403+0000 7f4291bb7640 1 --2- 172.21.15.44:0/406755579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.403+0000 7f4291bb7640 1 -- 172.21.15.44:0/406755579 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56438bb6e150 con 0x56438bb721c0 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.403+0000 7f4291bb7640 1 --2- 172.21.15.44:0/406755579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb721c0 0x56438bb6f6f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f427c003f20 tx=0x7f427c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3337ffc2764be71 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.403+0000 7f4276ffd640 1 -- 172.21.15.44:0/406755579 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c030070 con 0x56438bb721c0 2024-08-20T03:05:47.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.403+0000 7f4276ffd640 1 -- 172.21.15.44:0/406755579 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427c02bab0 con 0x56438bb721c0 2024-08-20T03:05:47.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.403+0000 7f4276ffd640 1 -- 172.21.15.44:0/406755579 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c01e8d0 con 0x56438bb721c0 2024-08-20T03:05:47.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.404+0000 7f4295a13740 1 -- 172.21.15.44:0/406755579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb721c0 msgr2=0x56438bb6f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:47.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.404+0000 7f4295a13740 1 --2- 172.21.15.44:0/406755579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb721c0 0x56438bb6f6f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f427c003f20 tx=0x7f427c01c000 comp rx=0 tx=0).stop 2024-08-20T03:05:47.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.404+0000 7f4295a13740 1 -- 172.21.15.44:0/406755579 shutdown_connections 2024-08-20T03:05:47.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.404+0000 7f4295a13740 1 --2- 172.21.15.44:0/406755579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb721c0 0x56438bb6f6f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.404+0000 7f4295a13740 1 --2- 172.21.15.44:0/406755579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.404+0000 7f4295a13740 1 --2- 172.21.15.44:0/406755579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 0x56438bb69980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.404+0000 7f4295a13740 1 -- 172.21.15.44:0/406755579 >> 172.21.15.44:0/406755579 conn(0x56438bb63740 msgr2=0x56438bb61ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:47.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.404+0000 7f4295a13740 1 -- 172.21.15.44:0/406755579 shutdown_connections 2024-08-20T03:05:47.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.404+0000 7f4295a13740 1 -- 172.21.15.44:0/406755579 wait complete. 2024-08-20T03:05:47.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.405+0000 7f4295a13740 1 -- start start 2024-08-20T03:05:47.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.405+0000 7f4295a13740 1 Processor -- start 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.405+0000 7f4295a13740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 0x56438bb8b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.405+0000 7f4295a13740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.405+0000 7f4295a13740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb721c0 0x56438bb86750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f42913b6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 0x56438bb8b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f42913b6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 0x56438bb8b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33622/0 (socket says 172.21.15.44:33622) 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f42913b6640 1 -- 172.21.15.44:0/28483693 learned_addr learned my addr 172.21.15.44:0/28483693 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f4291bb7640 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb721c0 0x56438bb86750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f4290bb5640 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f4295a13740 1 -- 172.21.15.44:0/28483693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42841401e0 con 0x56438bb721c0 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f4295a13740 1 -- 172.21.15.44:0/28483693 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f428413fb10 con 0x56438bb66230 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f4295a13740 1 -- 172.21.15.44:0/28483693 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f428413cd50 con 0x56438bb65910 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f42913b6640 1 -- 172.21.15.44:0/28483693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 msgr2=0x56438bb860d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f42913b6640 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb860d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f42913b6640 1 -- 172.21.15.44:0/28483693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb721c0 msgr2=0x56438bb86750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f42913b6640 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb721c0 0x56438bb86750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f42913b6640 1 -- 172.21.15.44:0/28483693 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56438bbfc370 con 0x56438bb65910 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f4295a13740 10 client.0 _ll_register_callbacks cb 0x7f42880a6f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f4295a13740 1 -- 172.21.15.44:0/28483693 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f427c02bab0 con 0x56438bb65910 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f42913b6640 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 0x56438bb8b8e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f428c052160 tx=0x7f428c068640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5c0bab6294423e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.406+0000 7f4274ff9640 10 client.25846 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4274ff9640 1 -- 172.21.15.44:0/28483693 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c06c960 con 0x56438bb65910 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4274ff9640 1 -- 172.21.15.44:0/28483693 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f428c06cb00 con 0x56438bb65910 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4274ff9640 1 -- 172.21.15.44:0/28483693 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f428c06cd80 con 0x56438bb65910 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4295a13740 10 client.25846 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4295a13740 1 -- 172.21.15.44:0/28483693 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({fsmap=30}) -- 0x56438bbfd300 con 0x56438bb65910 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4274ff9640 1 -- 172.21.15.44:0/28483693 <== mon.1 v2:172.21.15.165:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f428c0774b0 con 0x56438bb65910 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4295a13740 10 client.25846 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:47.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4295a13740 10 client.25846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4295a13740 20 client.25846 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4295a13740 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x56438bbfd580 0x56438bbff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4295a13740 4 client.25846 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4295a13740 1 -- 172.21.15.44:0/28483693 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56438bade880 con 0x56438bbfd580 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.407+0000 7f4290bb5640 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x56438bbfd580 0x56438bbff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.408+0000 7f4274ff9640 10 client.25846 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.408+0000 7f4290bb5640 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x56438bbfd580 0x56438bbff9d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2af248a46563403 server_cookie=c742862323e20bff in_seq=0 out_seq=0 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.408+0000 7f4274ff9640 1 -- 172.21.15.44:0/28483693 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56438bade880 con 0x56438bbfd580 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.408+0000 7f4274ff9640 10 client.25846 handle_command_reply: tid=0 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.408+0000 7f4295a13740 1 client.25846 shutdown 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.408+0000 7f4295a13740 1 -- 172.21.15.44:0/28483693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 msgr2=0x56438bb8b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.409+0000 7f4295a13740 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 0x56438bb8b8e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f428c052160 tx=0x7f428c068640 comp rx=0 tx=0).stop 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.409+0000 7f4295a13740 1 -- 172.21.15.44:0/28483693 shutdown_connections 2024-08-20T03:05:47.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.409+0000 7f4295a13740 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56438bb721c0 0x56438bb86750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.409+0000 7f4295a13740 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56438bb66230 0x56438bb860d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.409+0000 7f4295a13740 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x56438bbfd580 0x56438bbff9d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.409+0000 7f4274ff9640 0 client.25846 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:05:47.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.409+0000 7f4295a13740 1 --2- 172.21.15.44:0/28483693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56438bb65910 0x56438bb8b8e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.409+0000 7f4295a13740 1 -- 172.21.15.44:0/28483693 >> 172.21.15.44:0/28483693 conn(0x56438bb63740 msgr2=0x56438bb626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:47.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.409+0000 7f4295a13740 1 -- 172.21.15.44:0/28483693 shutdown_connections 2024-08-20T03:05:47.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.409+0000 7f4295a13740 1 -- 172.21.15.44:0/28483693 wait complete. 2024-08-20T03:05:47.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.409+0000 7f4295a13740 20 client.25846 trim_cache size 0 max 16384 2024-08-20T03:05:47.417 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:05:47.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.410+0000 7f42923b8640 1 -- 172.21.15.44:0/2024902685 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42780b3df0 msgr2=0x7f42780b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:47.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.410+0000 7f42923b8640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42780b3df0 0x7f42780b62b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f427c013fd0 tx=0x7f427c005f90 comp rx=0 tx=0).stop 2024-08-20T03:05:47.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.410+0000 7f42923b8640 1 -- 172.21.15.44:0/2024902685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4284094550 msgr2=0x7f4284129fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:47.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.410+0000 7f42923b8640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4284094550 0x7f4284129fc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f428800e9f0 tx=0x7f428800eeb0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.410+0000 7f42923b8640 1 -- 172.21.15.44:0/2024902685 shutdown_connections 2024-08-20T03:05:47.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.410+0000 7f42923b8640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42780b3df0 0x7f42780b62b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.410+0000 7f42923b8640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42840965a0 0x7f428412f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.410+0000 7f42923b8640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4284094550 0x7f4284129fc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.410+0000 7f42923b8640 1 --2- 172.21.15.44:0/2024902685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4284093bd0 0x7f4284129a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:47.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.410+0000 7f42923b8640 1 -- 172.21.15.44:0/2024902685 >> 172.21.15.44:0/2024902685 conn(0x7f4284009480 msgr2=0x7f42840983a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:47.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.410+0000 7f42923b8640 1 -- 172.21.15.44:0/2024902685 shutdown_connections 2024-08-20T03:05:47.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:47.410+0000 7f42923b8640 1 -- 172.21.15.44:0/2024902685 wait complete. 2024-08-20T03:05:47.488 INFO:tasks.workunit.client.0.smithi044.stdout: 115 1433196 630 985752 354 798308 1855 2024-08-20T03:05:48.393 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T03:05:48.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:48 smithi165 ceph-mon[22357]: pgmap v419: 97 pgs: 97 active+clean; 397 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 1.16k op/s 2024-08-20T03:05:48.644 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:48 smithi044 ceph-mon[18790]: pgmap v419: 97 pgs: 97 active+clean; 397 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 1.16k op/s 2024-08-20T03:05:48.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:48 smithi099 ceph-mon[21984]: pgmap v419: 97 pgs: 97 active+clean; 397 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 1.16k op/s 2024-08-20T03:05:50.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:50 smithi165 ceph-mon[22357]: pgmap v420: 97 pgs: 97 active+clean; 404 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 685 op/s 2024-08-20T03:05:50.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:50 smithi099 ceph-mon[21984]: pgmap v420: 97 pgs: 97 active+clean; 404 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 685 op/s 2024-08-20T03:05:50.788 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:50 smithi044 ceph-mon[18790]: pgmap v420: 97 pgs: 97 active+clean; 404 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 685 op/s 2024-08-20T03:05:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.862+0000 7f986b55d640 1 -- 172.21.15.44:0/2627604311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f986411bdd0 msgr2=0x7f986411e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.862+0000 7f986b55d640 1 --2- 172.21.15.44:0/2627604311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f986411bdd0 0x7f986411e1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f985c009ca0 tx=0x7f985c02a8f0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.863+0000 7f986b55d640 1 -- 172.21.15.44:0/2627604311 shutdown_connections 2024-08-20T03:05:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.863+0000 7f986b55d640 1 --2- 172.21.15.44:0/2627604311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f986411bdd0 0x7f986411e1c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.863+0000 7f986b55d640 1 --2- 172.21.15.44:0/2627604311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f986410a260 0x7f986410a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.863+0000 7f986b55d640 1 --2- 172.21.15.44:0/2627604311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864109e80 0x7f9864110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.863+0000 7f986b55d640 1 -- 172.21.15.44:0/2627604311 >> 172.21.15.44:0/2627604311 conn(0x7f986406d8b0 msgr2=0x7f986406dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.867+0000 7f986b55d640 1 -- 172.21.15.44:0/2627604311 shutdown_connections 2024-08-20T03:05:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.867+0000 7f986b55d640 1 -- 172.21.15.44:0/2627604311 wait complete. 2024-08-20T03:05:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.868+0000 7f986b55d640 1 Processor -- start 2024-08-20T03:05:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.868+0000 7f986b55d640 1 -- start start 2024-08-20T03:05:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.868+0000 7f986b55d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864109e80 0x7f9864112330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.869+0000 7f986b55d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f986410a260 0x7f9864112870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.869+0000 7f986b55d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9864117c00 0x7f98641180b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.869+0000 7f986b55d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9864121750 con 0x7f9864109e80 2024-08-20T03:05:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.869+0000 7f986b55d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98641215d0 con 0x7f986410a260 2024-08-20T03:05:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.869+0000 7f986b55d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f98641218d0 con 0x7f9864117c00 2024-08-20T03:05:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.869+0000 7f986a55b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864109e80 0x7f9864112330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.869+0000 7f9869d5a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f986410a260 0x7f9864112870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.869+0000 7f986a55b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864109e80 0x7f9864112330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35464/0 (socket says 172.21.15.44:35464) 2024-08-20T03:05:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.869+0000 7f986a55b640 1 -- 172.21.15.44:0/3656208491 learned_addr learned my addr 172.21.15.44:0/3656208491 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.869+0000 7f986ad5c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9864117c00 0x7f98641180b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.870+0000 7f986ad5c640 1 -- 172.21.15.44:0/3656208491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f986410a260 msgr2=0x7f9864112870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.870+0000 7f986ad5c640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f986410a260 0x7f9864112870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.870+0000 7f986ad5c640 1 -- 172.21.15.44:0/3656208491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864109e80 msgr2=0x7f9864112330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.870+0000 7f986ad5c640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864109e80 0x7f9864112330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.870+0000 7f986ad5c640 1 -- 172.21.15.44:0/3656208491 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9864118800 con 0x7f9864117c00 2024-08-20T03:05:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.870+0000 7f986a55b640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864109e80 0x7f9864112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.870+0000 7f986ad5c640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9864117c00 0x7f98641180b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f985c009bf0 tx=0x7f985c031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.906+0000 7f985b7fe640 1 -- 172.21.15.44:0/3656208491 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c004450 con 0x7f9864117c00 2024-08-20T03:05:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.906+0000 7f986b55d640 1 -- 172.21.15.44:0/3656208491 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9864118d70 con 0x7f9864117c00 2024-08-20T03:05:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.906+0000 7f986b55d640 1 -- 172.21.15.44:0/3656208491 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9864119340 con 0x7f9864117c00 2024-08-20T03:05:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.906+0000 7f985b7fe640 1 -- 172.21.15.44:0/3656208491 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f985c03f070 con 0x7f9864117c00 2024-08-20T03:05:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.908+0000 7f985b7fe640 1 -- 172.21.15.44:0/3656208491 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c02f800 con 0x7f9864117c00 2024-08-20T03:05:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.910+0000 7f986f68f740 1 Processor -- start 2024-08-20T03:05:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.913+0000 7f985b7fe640 1 -- 172.21.15.44:0/3656208491 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f985c0305f0 con 0x7f9864117c00 2024-08-20T03:05:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.913+0000 7f986f68f740 1 -- start start 2024-08-20T03:05:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.913+0000 7f986f68f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb957b30 0x5585eb957f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.913+0000 7f986f68f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb958450 0x5585eb963ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.913+0000 7f986f68f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb9643e0 0x5585eb9667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.913+0000 7f986f68f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5585eb8cd360 con 0x5585eb9643e0 2024-08-20T03:05:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.913+0000 7f986f68f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5585eb8bc860 con 0x5585eb958450 2024-08-20T03:05:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.913+0000 7f986f68f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5585eb8c08a0 con 0x5585eb957b30 2024-08-20T03:05:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.914+0000 7f986a55b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb958450 0x5585eb963ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.914+0000 7f986a55b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb958450 0x5585eb963ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47000/0 (socket says 172.21.15.44:47000) 2024-08-20T03:05:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.914+0000 7f986a55b640 1 -- 172.21.15.44:0/3768972505 learned_addr learned my addr 172.21.15.44:0/3768972505 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.914+0000 7f9869d5a640 1 --2- 172.21.15.44:0/3768972505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb9643e0 0x5585eb9667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.914+0000 7f986a55b640 1 -- 172.21.15.44:0/3768972505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb957b30 msgr2=0x5585eb957f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:05:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.914+0000 7f986a55b640 1 --2- 172.21.15.44:0/3768972505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb957b30 0x5585eb957f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.914+0000 7f986a55b640 1 -- 172.21.15.44:0/3768972505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb9643e0 msgr2=0x5585eb9667d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.914+0000 7f986a55b640 1 --2- 172.21.15.44:0/3768972505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb9643e0 0x5585eb9667d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.914+0000 7f986a55b640 1 -- 172.21.15.44:0/3768972505 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585eb863e40 con 0x5585eb958450 2024-08-20T03:05:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.914+0000 7f9869d5a640 1 --2- 172.21.15.44:0/3768972505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb9643e0 0x5585eb9667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.915+0000 7f986a55b640 1 --2- 172.21.15.44:0/3768972505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb958450 0x5585eb963ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9860009ed0 tx=0x7f986000e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d4eb27dfe17db33 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.915+0000 7f985b7fe640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f98500b3f50 0x7f98500b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.916+0000 7f9869d5a640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f98500b3f50 0x7f98500b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.916+0000 7f9869d5a640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f98500b3f50 0x7f98500b6410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9864118c90 tx=0x7f9854008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.917+0000 7f985b7fe640 1 -- 172.21.15.44:0/3656208491 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f985c105220 con 0x7f9864117c00 2024-08-20T03:05:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.936+0000 7f9858ff9640 1 -- 172.21.15.44:0/3768972505 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860017070 con 0x5585eb958450 2024-08-20T03:05:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.936+0000 7f9858ff9640 1 -- 172.21.15.44:0/3768972505 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9860013920 con 0x5585eb958450 2024-08-20T03:05:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.937+0000 7f986f68f740 1 -- 172.21.15.44:0/3768972505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb958450 msgr2=0x5585eb963ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.937+0000 7f986f68f740 1 --2- 172.21.15.44:0/3768972505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb958450 0x5585eb963ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9860009ed0 tx=0x7f986000e5a0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.937+0000 7f986f68f740 1 -- 172.21.15.44:0/3768972505 shutdown_connections 2024-08-20T03:05:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.937+0000 7f986f68f740 1 --2- 172.21.15.44:0/3768972505 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb9643e0 0x5585eb9667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.937+0000 7f986f68f740 1 --2- 172.21.15.44:0/3768972505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb958450 0x5585eb963ea0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.937+0000 7f986f68f740 1 --2- 172.21.15.44:0/3768972505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb957b30 0x5585eb957f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.937+0000 7f986f68f740 1 -- 172.21.15.44:0/3768972505 >> 172.21.15.44:0/3768972505 conn(0x5585eb955960 msgr2=0x5585eb956b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.938+0000 7f986f68f740 1 -- 172.21.15.44:0/3768972505 shutdown_connections 2024-08-20T03:05:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.938+0000 7f986f68f740 1 -- 172.21.15.44:0/3768972505 wait complete. 2024-08-20T03:05:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.938+0000 7f986f68f740 1 -- start start 2024-08-20T03:05:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.938+0000 7f986f68f740 1 Processor -- start 2024-08-20T03:05:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.939+0000 7f986f68f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 0x5585eb963c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.939+0000 7f986f68f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb958450 0x5585eb961040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.939+0000 7f986f68f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb9643e0 0x5585eb961580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.939+0000 7f986f68f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5585eb7f5ab0 con 0x5585eb957b30 2024-08-20T03:05:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.939+0000 7f986f68f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5585eb843780 con 0x5585eb9643e0 2024-08-20T03:05:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.939+0000 7f986f68f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5585eb967110 con 0x5585eb958450 2024-08-20T03:05:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.939+0000 7f986f68f740 10 client.0 _ll_register_callbacks cb 0x5585eb5cba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.939+0000 7f9869d5a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb958450 0x5585eb961040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.939+0000 7f9869d5a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb958450 0x5585eb961040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:43770/0 (socket says 172.21.15.44:43770) 2024-08-20T03:05:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.939+0000 7f9869d5a640 1 -- 172.21.15.44:0/1922129566 learned_addr learned my addr 172.21.15.44:0/1922129566 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.940+0000 7f986ad5c640 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb9643e0 0x5585eb961580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.940+0000 7f986a55b640 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 0x5585eb963c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.940+0000 7f9869d5a640 1 -- 172.21.15.44:0/1922129566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb9643e0 msgr2=0x5585eb961580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.940+0000 7f9869d5a640 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb9643e0 0x5585eb961580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.940+0000 7f9869d5a640 1 -- 172.21.15.44:0/1922129566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 msgr2=0x5585eb963c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.940+0000 7f9869d5a640 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 0x5585eb963c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.940+0000 7f9869d5a640 1 -- 172.21.15.44:0/1922129566 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585eb61f790 con 0x5585eb958450 2024-08-20T03:05:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.940+0000 7f986a55b640 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 0x5585eb963c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:05:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.940+0000 7f986ad5c640 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb9643e0 0x5585eb961580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:05:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.940+0000 7f9869d5a640 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb958450 0x5585eb961040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9854007fd0 tx=0x7f9854004700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0531dd8604fe0ac server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.959+0000 7f984dffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:05:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.959+0000 7f984dffb640 1 -- 172.21.15.44:0/1922129566 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854014700 con 0x5585eb958450 2024-08-20T03:05:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.959+0000 7f986f68f740 1 -- 172.21.15.44:0/1922129566 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5585eb86be00 con 0x5585eb958450 2024-08-20T03:05:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.959+0000 7f984dffb640 1 -- 172.21.15.44:0/1922129566 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f98540148a0 con 0x5585eb958450 2024-08-20T03:05:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.963+0000 7f984dffb640 1 -- 172.21.15.44:0/1922129566 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9854014aa0 con 0x5585eb958450 2024-08-20T03:05:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.963+0000 7f986f68f740 10 client.25855 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.967+0000 7f986f68f740 1 -- 172.21.15.44:0/1922129566 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5585eb698620 con 0x5585eb958450 2024-08-20T03:05:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.969+0000 7f984dffb640 1 -- 172.21.15.44:0/1922129566 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5585eb86be00 con 0x5585eb958450 2024-08-20T03:05:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.969+0000 7f984dffb640 1 -- 172.21.15.44:0/1922129566 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f985402a3e0 con 0x5585eb958450 2024-08-20T03:05:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.969+0000 7f986f68f740 10 client.25855 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.969+0000 7f986f68f740 10 client.25855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.969+0000 7f986f68f740 20 client.25855 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.969+0000 7f986f68f740 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5585eb974340 0x5585eb9767c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.969+0000 7f986f68f740 4 client.25855 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:05:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.969+0000 7f986f68f740 1 -- 172.21.15.44:0/1922129566 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585eb8cf2b0 con 0x5585eb974340 2024-08-20T03:05:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.969+0000 7f986a55b640 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5585eb974340 0x5585eb9767c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.970+0000 7f986a55b640 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5585eb974340 0x5585eb9767c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802dfa82947d803f server_cookie=385bd41e4817a96d in_seq=0 out_seq=0 2024-08-20T03:05:50.972 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.970+0000 7f984dffb640 10 client.25855 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.976+0000 7f984dffb640 1 -- 172.21.15.44:0/1922129566 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585eb8cf2b0 con 0x5585eb974340 2024-08-20T03:05:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.976+0000 7f984dffb640 10 client.25855 handle_command_reply: tid=0 2024-08-20T03:05:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.976+0000 7f986f68f740 1 client.25855 shutdown 2024-08-20T03:05:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.981+0000 7f986f68f740 1 -- 172.21.15.44:0/1922129566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb958450 msgr2=0x5585eb961040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.981+0000 7f986f68f740 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb958450 0x5585eb961040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9854007fd0 tx=0x7f9854004700 comp rx=0 tx=0).stop 2024-08-20T03:05:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.981+0000 7f986f68f740 1 -- 172.21.15.44:0/1922129566 shutdown_connections 2024-08-20T03:05:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.981+0000 7f986f68f740 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5585eb974340 0x5585eb9767c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.981+0000 7f986f68f740 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb9643e0 0x5585eb961580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.981+0000 7f986f68f740 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb958450 0x5585eb961040 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.981+0000 7f986f68f740 1 --2- 172.21.15.44:0/1922129566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 0x5585eb963c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.981+0000 7f986f68f740 1 -- 172.21.15.44:0/1922129566 >> 172.21.15.44:0/1922129566 conn(0x5585eb955960 msgr2=0x5585eb95b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.982+0000 7f984dffb640 0 client.25855 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.982+0000 7f986f68f740 1 -- 172.21.15.44:0/1922129566 shutdown_connections 2024-08-20T03:05:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.982+0000 7f986f68f740 1 -- 172.21.15.44:0/1922129566 wait complete. 2024-08-20T03:05:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:50.982+0000 7f986f68f740 20 client.25855 trim_cache size 0 max 16384 2024-08-20T03:05:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.142+0000 7f986f68f740 1 Processor -- start 2024-08-20T03:05:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.142+0000 7f986f68f740 1 -- start start 2024-08-20T03:05:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.142+0000 7f986f68f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb957b30 0x5585eb970310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986f68f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb958450 0x5585eb955230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986f68f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb9643e0 0x5585eb9569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986a55b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb957b30 0x5585eb970310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986a55b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb957b30 0x5585eb970310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47018/0 (socket says 172.21.15.44:47018) 2024-08-20T03:05:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986f68f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f984800e780 con 0x5585eb958450 2024-08-20T03:05:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986f68f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9848010380 con 0x5585eb957b30 2024-08-20T03:05:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986f68f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f984800ee50 con 0x5585eb9643e0 2024-08-20T03:05:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986a55b640 1 -- 172.21.15.44:0/3145460289 learned_addr learned my addr 172.21.15.44:0/3145460289 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986ad5c640 1 --2- 172.21.15.44:0/3145460289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb9643e0 0x5585eb9569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f9869d5a640 1 --2- 172.21.15.44:0/3145460289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb958450 0x5585eb955230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986a55b640 1 -- 172.21.15.44:0/3145460289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb9643e0 msgr2=0x5585eb9569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:51.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986a55b640 1 --2- 172.21.15.44:0/3145460289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb9643e0 0x5585eb9569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986a55b640 1 -- 172.21.15.44:0/3145460289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb958450 msgr2=0x5585eb955230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986a55b640 1 --2- 172.21.15.44:0/3145460289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb958450 0x5585eb955230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f986a55b640 1 -- 172.21.15.44:0/3145460289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585eb94fd20 con 0x5585eb957b30 2024-08-20T03:05:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.143+0000 7f9869d5a640 1 --2- 172.21.15.44:0/3145460289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb958450 0x5585eb955230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:05:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.144+0000 7f986a55b640 1 --2- 172.21.15.44:0/3145460289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb957b30 0x5585eb970310 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9860009ed0 tx=0x7f9860009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e76c0bab2f3d85e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.171+0000 7f984ffff640 1 -- 172.21.15.44:0/3145460289 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860017040 con 0x5585eb957b30 2024-08-20T03:05:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.171+0000 7f984ffff640 1 -- 172.21.15.44:0/3145460289 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98600036b0 con 0x5585eb957b30 2024-08-20T03:05:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.171+0000 7f984ffff640 1 -- 172.21.15.44:0/3145460289 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860024660 con 0x5585eb957b30 2024-08-20T03:05:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.175+0000 7f986f68f740 1 -- 172.21.15.44:0/3145460289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb957b30 msgr2=0x5585eb970310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.175+0000 7f986f68f740 1 --2- 172.21.15.44:0/3145460289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb957b30 0x5585eb970310 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9860009ed0 tx=0x7f9860009190 comp rx=0 tx=0).stop 2024-08-20T03:05:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.182+0000 7f986f68f740 1 -- 172.21.15.44:0/3145460289 shutdown_connections 2024-08-20T03:05:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.182+0000 7f986f68f740 1 --2- 172.21.15.44:0/3145460289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb9643e0 0x5585eb9569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.182+0000 7f986f68f740 1 --2- 172.21.15.44:0/3145460289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb958450 0x5585eb955230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:51.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.182+0000 7f986f68f740 1 --2- 172.21.15.44:0/3145460289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb957b30 0x5585eb970310 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.182+0000 7f986f68f740 1 -- 172.21.15.44:0/3145460289 >> 172.21.15.44:0/3145460289 conn(0x5585eb955960 msgr2=0x5585eb95c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.182+0000 7f986f68f740 1 -- 172.21.15.44:0/3145460289 shutdown_connections 2024-08-20T03:05:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.182+0000 7f986f68f740 1 -- 172.21.15.44:0/3145460289 wait complete. 2024-08-20T03:05:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.183+0000 7f986f68f740 1 -- start start 2024-08-20T03:05:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.183+0000 7f986f68f740 1 Processor -- start 2024-08-20T03:05:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.184+0000 7f986f68f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 0x5585eb967440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.184+0000 7f986f68f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb958450 0x5585eb967980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.184+0000 7f986f68f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb9643e0 0x5585eb967ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.184+0000 7f986a55b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 0x5585eb967440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.184+0000 7f986f68f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f984800fbf0 con 0x5585eb957b30 2024-08-20T03:05:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.184+0000 7f986a55b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 0x5585eb967440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35504/0 (socket says 172.21.15.44:35504) 2024-08-20T03:05:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.184+0000 7f986a55b640 1 -- 172.21.15.44:0/1739179975 learned_addr learned my addr 172.21.15.44:0/1739179975 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.184+0000 7f9869d5a640 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb958450 0x5585eb967980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.185+0000 7f986ad5c640 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb9643e0 0x5585eb967ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.185+0000 7f986f68f740 1 -- 172.21.15.44:0/1739179975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f984800f520 con 0x5585eb9643e0 2024-08-20T03:05:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.185+0000 7f986f68f740 1 -- 172.21.15.44:0/1739179975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f984800c760 con 0x5585eb958450 2024-08-20T03:05:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.186+0000 7f986a55b640 1 -- 172.21.15.44:0/1739179975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb958450 msgr2=0x5585eb967980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.186+0000 7f986a55b640 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb958450 0x5585eb967980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.186+0000 7f986a55b640 1 -- 172.21.15.44:0/1739179975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb9643e0 msgr2=0x5585eb967ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.186+0000 7f986a55b640 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb9643e0 0x5585eb967ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.186+0000 7f986a55b640 1 -- 172.21.15.44:0/1739179975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585eb9dd0d0 con 0x5585eb957b30 2024-08-20T03:05:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.186+0000 7f986a55b640 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 0x5585eb967440 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9860009800 tx=0x7f9860012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d77b49f860896f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.187+0000 7f986f68f740 10 client.0 _ll_register_callbacks cb 0x7f9854007610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.187+0000 7f986f68f740 1 -- 172.21.15.44:0/1739179975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98600036b0 con 0x5585eb957b30 2024-08-20T03:05:51.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.214+0000 7f984dffb640 10 client.16404 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:05:51.223 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:05:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.216+0000 7f984dffb640 1 -- 172.21.15.44:0/1739179975 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860017030 con 0x5585eb957b30 2024-08-20T03:05:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.227+0000 7f984dffb640 1 -- 172.21.15.44:0/1739179975 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f9860003c70 con 0x5585eb957b30 2024-08-20T03:05:51.229 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.227+0000 7f984dffb640 1 -- 172.21.15.44:0/1739179975 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9860036050 con 0x5585eb957b30 2024-08-20T03:05:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.229+0000 7f986f68f740 10 client.16404 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:51.231 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.229+0000 7f986f68f740 1 -- 172.21.15.44:0/1739179975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5585eb9de020 con 0x5585eb957b30 2024-08-20T03:05:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.230+0000 7f984dffb640 1 -- 172.21.15.44:0/1739179975 <== mon.0 v2:172.21.15.44:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f9860004040 con 0x5585eb957b30 2024-08-20T03:05:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.230+0000 7f986f68f740 10 client.16404 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.230+0000 7f986f68f740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.231+0000 7f986f68f740 20 client.16404 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.231+0000 7f986f68f740 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5585eb9de470 0x5585eb9e0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.231+0000 7f986f68f740 4 client.16404 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:05:51.233 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.231+0000 7f986f68f740 1 -- 172.21.15.44:0/1739179975 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f98600036b0 con 0x5585eb9de470 2024-08-20T03:05:51.234 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.232+0000 7f9869d5a640 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5585eb9de470 0x5585eb9e0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.233+0000 7f9869d5a640 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5585eb9de470 0x5585eb9e0900 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66ad25e34871ce2e server_cookie=d5763728b269584d in_seq=0 out_seq=0 2024-08-20T03:05:51.235 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:51.233+0000 7f984dffb640 10 client.16404 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:51.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:51 smithi165 ceph-mon[22357]: pgmap v421: 97 pgs: 97 active+clean; 408 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-08-20T03:05:51.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:51 smithi099 ceph-mon[21984]: pgmap v421: 97 pgs: 97 active+clean; 408 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-08-20T03:05:51.752 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:51 smithi044 ceph-mon[18790]: pgmap v421: 97 pgs: 97 active+clean; 408 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-08-20T03:05:52.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.109+0000 7f586f59e640 1 -- 172.21.15.44:0/3396506178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587006dde0 msgr2=0x7f587006c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:52.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.109+0000 7f586f59e640 1 --2- 172.21.15.44:0/3396506178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587006dde0 0x7f587006c0e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5860009860 tx=0x7f586002a360 comp rx=0 tx=0).stop 2024-08-20T03:05:52.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.111+0000 7f586f59e640 1 -- 172.21.15.44:0/3396506178 shutdown_connections 2024-08-20T03:05:52.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.111+0000 7f586f59e640 1 --2- 172.21.15.44:0/3396506178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587011bad0 0x7f587011dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.111+0000 7f586f59e640 1 --2- 172.21.15.44:0/3396506178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f587006c620 0x7f587006ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.111+0000 7f586f59e640 1 --2- 172.21.15.44:0/3396506178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587006dde0 0x7f587006c0e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.111+0000 7f586f59e640 1 -- 172.21.15.44:0/3396506178 >> 172.21.15.44:0/3396506178 conn(0x7f587006ef80 msgr2=0x7f58700713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:52.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.113+0000 7f586f59e640 1 -- 172.21.15.44:0/3396506178 shutdown_connections 2024-08-20T03:05:52.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.113+0000 7f586f59e640 1 -- 172.21.15.44:0/3396506178 wait complete. 2024-08-20T03:05:52.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.116+0000 7f586f59e640 1 Processor -- start 2024-08-20T03:05:52.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.116+0000 7f586f59e640 1 -- start start 2024-08-20T03:05:52.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.117+0000 7f586f59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587006c620 0x7f58701a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.117+0000 7f586f59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f587006dde0 0x7f58701a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.117+0000 7f586f59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587011bad0 0x7f58701ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.117+0000 7f586f59e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5870121770 con 0x7f587006c620 2024-08-20T03:05:52.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.117+0000 7f586f59e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58701215f0 con 0x7f587006dde0 2024-08-20T03:05:52.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.117+0000 7f586f59e640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f58701218f0 con 0x7f587011bad0 2024-08-20T03:05:52.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.117+0000 7f586ed9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587011bad0 0x7f58701ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.117+0000 7f586ed9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587011bad0 0x7f58701ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:43788/0 (socket says 172.21.15.44:43788) 2024-08-20T03:05:52.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.117+0000 7f586ed9d640 1 -- 172.21.15.44:0/1403278202 learned_addr learned my addr 172.21.15.44:0/1403278202 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:52.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.119+0000 7f586ed9d640 1 -- 172.21.15.44:0/1403278202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f587006dde0 msgr2=0x7f58701a82e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-20T03:05:52.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.119+0000 7f586ed9d640 1 --2- 172.21.15.44:0/1403278202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f587006dde0 0x7f58701a82e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.119+0000 7f586ed9d640 1 -- 172.21.15.44:0/1403278202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587006c620 msgr2=0x7f58701a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:52.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.119+0000 7f586ed9d640 1 --2- 172.21.15.44:0/1403278202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587006c620 0x7f58701a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.119+0000 7f586ed9d640 1 -- 172.21.15.44:0/1403278202 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58701ade50 con 0x7f587011bad0 2024-08-20T03:05:52.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.120+0000 7f586ed9d640 1 --2- 172.21.15.44:0/1403278202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587011bad0 0x7f58701ad680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f586800b990 tx=0x7f586800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:52.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.121+0000 7f585f7fe640 1 -- 172.21.15.44:0/1403278202 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868019070 con 0x7f587011bad0 2024-08-20T03:05:52.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.121+0000 7f585f7fe640 1 -- 172.21.15.44:0/1403278202 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586800cba0 con 0x7f587011bad0 2024-08-20T03:05:52.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.121+0000 7f586f59e640 1 -- 172.21.15.44:0/1403278202 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58701ae140 con 0x7f587011bad0 2024-08-20T03:05:52.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.121+0000 7f585f7fe640 1 -- 172.21.15.44:0/1403278202 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680075d0 con 0x7f587011bad0 2024-08-20T03:05:52.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.121+0000 7f586f59e640 1 -- 172.21.15.44:0/1403278202 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58701b4800 con 0x7f587011bad0 2024-08-20T03:05:52.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.127+0000 7f5876c75740 1 Processor -- start 2024-08-20T03:05:52.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.128+0000 7f5876c75740 1 -- start start 2024-08-20T03:05:52.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.129+0000 7f585f7fe640 1 -- 172.21.15.44:0/1403278202 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f586801c070 con 0x7f587011bad0 2024-08-20T03:05:52.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.129+0000 7f5876c75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 0x55dbc0359e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.129+0000 7f5876c75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc035a350 0x55dbc0365da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.129+0000 7f5876c75740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc03662e0 0x55dbc03686d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.129+0000 7f5876c75740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55dbc02d0930 con 0x55dbc03662e0 2024-08-20T03:05:52.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.129+0000 7f586e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc035a350 0x55dbc0365da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.129+0000 7f586e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc035a350 0x55dbc0365da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47028/0 (socket says 172.21.15.44:47028) 2024-08-20T03:05:52.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.129+0000 7f586e59c640 1 -- 172.21.15.44:0/593533291 learned_addr learned my addr 172.21.15.44:0/593533291 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:52.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.129+0000 7f5876c75740 1 -- 172.21.15.44:0/593533291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dbc02bfe30 con 0x55dbc035a350 2024-08-20T03:05:52.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.129+0000 7f5876c75740 1 -- 172.21.15.44:0/593533291 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dbc02c3e70 con 0x55dbc0359a30 2024-08-20T03:05:52.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.130+0000 7f586dd9b640 1 --2- 172.21.15.44:0/593533291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 0x55dbc0359e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.130+0000 7f586dd9b640 1 --2- 172.21.15.44:0/593533291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc03662e0 0x55dbc03686d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.130+0000 7f586e59c640 1 -- 172.21.15.44:0/593533291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 msgr2=0x55dbc0359e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:52.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.130+0000 7f586e59c640 1 --2- 172.21.15.44:0/593533291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 0x55dbc0359e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.130+0000 7f586e59c640 1 -- 172.21.15.44:0/593533291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc03662e0 msgr2=0x55dbc03686d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:52.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.130+0000 7f586e59c640 1 --2- 172.21.15.44:0/593533291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc03662e0 0x55dbc03686d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.130+0000 7f586e59c640 1 -- 172.21.15.44:0/593533291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbc001f650 con 0x55dbc035a350 2024-08-20T03:05:52.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.131+0000 7f586e59c640 1 --2- 172.21.15.44:0/593533291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc035a350 0x55dbc0365da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f586002a8d0 tx=0x7f5860030340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=479f3f93b1922b33 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:52.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.131+0000 7f585cff9640 1 -- 172.21.15.44:0/593533291 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860040020 con 0x55dbc035a350 2024-08-20T03:05:52.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.132+0000 7f585cff9640 1 -- 172.21.15.44:0/593533291 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5860038040 con 0x55dbc035a350 2024-08-20T03:05:52.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.132+0000 7f585f7fe640 1 --2- 172.21.15.44:0/1403278202 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f58580b4010 0x7f58580b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.132+0000 7f585f7fe640 1 -- 172.21.15.44:0/1403278202 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f58680e3340 con 0x7f587011bad0 2024-08-20T03:05:52.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.133+0000 7f5876c75740 1 -- 172.21.15.44:0/593533291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc035a350 msgr2=0x55dbc0365da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:52.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.133+0000 7f5876c75740 1 --2- 172.21.15.44:0/593533291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc035a350 0x55dbc0365da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f586002a8d0 tx=0x7f5860030340 comp rx=0 tx=0).stop 2024-08-20T03:05:52.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.134+0000 7f5876c75740 1 -- 172.21.15.44:0/593533291 shutdown_connections 2024-08-20T03:05:52.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.134+0000 7f5876c75740 1 --2- 172.21.15.44:0/593533291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc03662e0 0x55dbc03686d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.134+0000 7f5876c75740 1 --2- 172.21.15.44:0/593533291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc035a350 0x55dbc0365da0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.134+0000 7f5876c75740 1 --2- 172.21.15.44:0/593533291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 0x55dbc0359e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.134+0000 7f5876c75740 1 -- 172.21.15.44:0/593533291 >> 172.21.15.44:0/593533291 conn(0x55dbc0357860 msgr2=0x55dbc0358a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:52.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.134+0000 7f586dd9b640 1 --2- 172.21.15.44:0/1403278202 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f58580b4010 0x7f58580b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.135+0000 7f5876c75740 1 -- 172.21.15.44:0/593533291 shutdown_connections 2024-08-20T03:05:52.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.135+0000 7f5876c75740 1 -- 172.21.15.44:0/593533291 wait complete. 2024-08-20T03:05:52.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.135+0000 7f5876c75740 1 -- start start 2024-08-20T03:05:52.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.135+0000 7f5876c75740 1 Processor -- start 2024-08-20T03:05:52.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.136+0000 7f5876c75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 0x55dbc0363290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.136+0000 7f5876c75740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc035a350 0x55dbc03637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.136+0000 7f5876c75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc03662e0 0x55dbc0363d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.136+0000 7f5876c75740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55dbc0122700 con 0x55dbc035a350 2024-08-20T03:05:52.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.136+0000 7f5876c75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dbc0246810 con 0x55dbc03662e0 2024-08-20T03:05:52.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.136+0000 7f5876c75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dbc0368fd0 con 0x55dbc0359a30 2024-08-20T03:05:52.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.136+0000 7f5876c75740 10 client.0 _ll_register_callbacks cb 0x55dbc02b9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:52.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.137+0000 7f586dd9b640 1 --2- 172.21.15.44:0/1403278202 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f58580b4010 0x7f58580b64d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55dbc03599e0 tx=0x7f5864009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:52.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.137+0000 7f586dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc035a350 0x55dbc03637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.137+0000 7f586dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc035a350 0x55dbc03637d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35538/0 (socket says 172.21.15.44:35538) 2024-08-20T03:05:52.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.137+0000 7f586dd9b640 1 -- 172.21.15.44:0/2735971280 learned_addr learned my addr 172.21.15.44:0/2735971280 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:52.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.137+0000 7f586ed9d640 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc03662e0 0x55dbc0363d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.138+0000 7f586e59c640 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 0x55dbc0363290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.138+0000 7f586ed9d640 1 -- 172.21.15.44:0/2735971280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 msgr2=0x55dbc0363290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:52.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.138+0000 7f586ed9d640 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 0x55dbc0363290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.138+0000 7f586ed9d640 1 -- 172.21.15.44:0/2735971280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc035a350 msgr2=0x55dbc03637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:52.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.138+0000 7f586ed9d640 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc035a350 0x55dbc03637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.138+0000 7f586ed9d640 1 -- 172.21.15.44:0/2735971280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbc0372490 con 0x55dbc03662e0 2024-08-20T03:05:52.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.143+0000 7f586ed9d640 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc03662e0 0x55dbc0363d10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f58680e0bb0 tx=0x7f58680a9ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677009793b3d5a52 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:52.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.143+0000 7f5855ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:05:52.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.144+0000 7f5855ffb640 1 -- 172.21.15.44:0/2735971280 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680e0f30 con 0x55dbc03662e0 2024-08-20T03:05:52.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.145+0000 7f5876c75740 1 -- 172.21.15.44:0/2735971280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dbc026f150 con 0x55dbc03662e0 2024-08-20T03:05:52.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.145+0000 7f5855ffb640 1 -- 172.21.15.44:0/2735971280 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f58680b2c90 con 0x55dbc03662e0 2024-08-20T03:05:52.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.145+0000 7f5855ffb640 1 -- 172.21.15.44:0/2735971280 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58680b3120 con 0x55dbc03662e0 2024-08-20T03:05:52.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.146+0000 7f5876c75740 10 client.25889 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:52.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.146+0000 7f5876c75740 1 -- 172.21.15.44:0/2735971280 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dbc0374f10 con 0x55dbc03662e0 2024-08-20T03:05:52.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.152+0000 7f5855ffb640 1 -- 172.21.15.44:0/2735971280 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dbc026f150 con 0x55dbc03662e0 2024-08-20T03:05:52.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.152+0000 7f5855ffb640 1 -- 172.21.15.44:0/2735971280 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f58680a5b40 con 0x55dbc03662e0 2024-08-20T03:05:52.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.153+0000 7f5876c75740 10 client.25889 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:52.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.153+0000 7f5876c75740 10 client.25889 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:05:52.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.153+0000 7f5876c75740 20 client.25889 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:52.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.153+0000 7f5876c75740 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55dbc0376420 0x55dbc0378830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.153+0000 7f5876c75740 4 client.25889 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:05:52.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.153+0000 7f5876c75740 1 -- 172.21.15.44:0/2735971280 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbc02d2880 con 0x55dbc0376420 2024-08-20T03:05:52.156 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.154+0000 7f586e59c640 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55dbc0376420 0x55dbc0378830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.163+0000 7f586e59c640 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55dbc0376420 0x55dbc0378830 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2eb20dcd1c889774 server_cookie=ae6cf8bfaecca5f6 in_seq=0 out_seq=0 2024-08-20T03:05:52.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.163+0000 7f5855ffb640 10 client.25889 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:05:52.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.169+0000 7f5855ffb640 1 -- 172.21.15.44:0/2735971280 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbc02d2880 con 0x55dbc0376420 2024-08-20T03:05:52.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.169+0000 7f5855ffb640 10 client.25889 handle_command_reply: tid=0 2024-08-20T03:05:52.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.170+0000 7f5876c75740 1 client.25889 shutdown 2024-08-20T03:05:52.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.170+0000 7f5876c75740 1 -- 172.21.15.44:0/2735971280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc03662e0 msgr2=0x55dbc0363d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:52.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.170+0000 7f5876c75740 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc03662e0 0x55dbc0363d10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f58680e0bb0 tx=0x7f58680a9ce0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.173+0000 7f5876c75740 1 -- 172.21.15.44:0/2735971280 shutdown_connections 2024-08-20T03:05:52.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.173+0000 7f5876c75740 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55dbc0376420 0x55dbc0378830 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.173+0000 7f5876c75740 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc03662e0 0x55dbc0363d10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.173+0000 7f5876c75740 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc035a350 0x55dbc03637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.173+0000 7f5876c75740 1 --2- 172.21.15.44:0/2735971280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 0x55dbc0363290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.173+0000 7f5876c75740 1 -- 172.21.15.44:0/2735971280 >> 172.21.15.44:0/2735971280 conn(0x55dbc0357860 msgr2=0x55dbc035d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:52.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.174+0000 7f5855ffb640 0 client.25889 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:05:52.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.174+0000 7f5876c75740 1 -- 172.21.15.44:0/2735971280 shutdown_connections 2024-08-20T03:05:52.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.176+0000 7f5876c75740 1 -- 172.21.15.44:0/2735971280 wait complete. 2024-08-20T03:05:52.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.179+0000 7f5876c75740 20 client.25889 trim_cache size 0 max 16384 2024-08-20T03:05:52.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.209+0000 7f5876c75740 1 Processor -- start 2024-08-20T03:05:52.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.209+0000 7f5876c75740 1 -- start start 2024-08-20T03:05:52.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.209+0000 7f5876c75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 0x55dbc037e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.209+0000 7f5876c75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc035a350 0x55dbc037e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.209+0000 7f5876c75740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc03662e0 0x55dbc035cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.209+0000 7f5876c75740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f585000e790 con 0x55dbc03662e0 2024-08-20T03:05:52.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.209+0000 7f5876c75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5850010390 con 0x55dbc035a350 2024-08-20T03:05:52.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.209+0000 7f5876c75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f585000ee60 con 0x55dbc0359a30 2024-08-20T03:05:52.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.209+0000 7f586ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc03662e0 0x55dbc035cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.210+0000 7f586ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc03662e0 0x55dbc035cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35540/0 (socket says 172.21.15.44:35540) 2024-08-20T03:05:52.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.210+0000 7f586ed9d640 1 -- 172.21.15.44:0/3116154077 learned_addr learned my addr 172.21.15.44:0/3116154077 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:52.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.210+0000 7f586dd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc035a350 0x55dbc037e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.210+0000 7f586ed9d640 1 -- 172.21.15.44:0/3116154077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 msgr2=0x55dbc037e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:52.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.210+0000 7f586ed9d640 1 --2- 172.21.15.44:0/3116154077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 0x55dbc037e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.210+0000 7f586ed9d640 1 -- 172.21.15.44:0/3116154077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc035a350 msgr2=0x55dbc037e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:52.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.210+0000 7f586ed9d640 1 --2- 172.21.15.44:0/3116154077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc035a350 0x55dbc037e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.210+0000 7f586ed9d640 1 -- 172.21.15.44:0/3116154077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbc0361790 con 0x55dbc03662e0 2024-08-20T03:05:52.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.211+0000 7f586ed9d640 1 --2- 172.21.15.44:0/3116154077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc03662e0 0x55dbc035cbe0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f58680aa000 tx=0x7f58680aa1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10309cec0e5a727a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:52.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.215+0000 7f5857fff640 1 -- 172.21.15.44:0/3116154077 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680e0f30 con 0x55dbc03662e0 2024-08-20T03:05:52.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.215+0000 7f5857fff640 1 -- 172.21.15.44:0/3116154077 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58680cb280 con 0x55dbc03662e0 2024-08-20T03:05:52.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.215+0000 7f5857fff640 1 -- 172.21.15.44:0/3116154077 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680a99c0 con 0x55dbc03662e0 2024-08-20T03:05:52.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.219+0000 7f5876c75740 1 -- 172.21.15.44:0/3116154077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc03662e0 msgr2=0x55dbc035cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:52.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.219+0000 7f5876c75740 1 --2- 172.21.15.44:0/3116154077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc03662e0 0x55dbc035cbe0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f58680aa000 tx=0x7f58680aa1f0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.220+0000 7f5876c75740 1 -- 172.21.15.44:0/3116154077 shutdown_connections 2024-08-20T03:05:52.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.220+0000 7f5876c75740 1 --2- 172.21.15.44:0/3116154077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc03662e0 0x55dbc035cbe0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.220+0000 7f5876c75740 1 --2- 172.21.15.44:0/3116154077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc035a350 0x55dbc037e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.220+0000 7f5876c75740 1 --2- 172.21.15.44:0/3116154077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc0359a30 0x55dbc037e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.220+0000 7f5876c75740 1 -- 172.21.15.44:0/3116154077 >> 172.21.15.44:0/3116154077 conn(0x55dbc0357860 msgr2=0x55dbc0362780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:52.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.221+0000 7f5876c75740 1 -- 172.21.15.44:0/3116154077 shutdown_connections 2024-08-20T03:05:52.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.221+0000 7f5876c75740 1 -- 172.21.15.44:0/3116154077 wait complete. 2024-08-20T03:05:52.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.222+0000 7f5876c75740 1 -- start start 2024-08-20T03:05:52.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.222+0000 7f5876c75740 1 Processor -- start 2024-08-20T03:05:52.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.222+0000 7f5876c75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc0359a30 0x55dbc03698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.222+0000 7f5876c75740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc035a350 0x55dbc0369e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.222+0000 7f5876c75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc03662e0 0x55dbc03df100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.222+0000 7f5876c75740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f585000fc00 con 0x55dbc035a350 2024-08-20T03:05:52.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.222+0000 7f5876c75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f585000f530 con 0x55dbc0359a30 2024-08-20T03:05:52.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.222+0000 7f5876c75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f585000c770 con 0x55dbc03662e0 2024-08-20T03:05:52.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.222+0000 7f5876c75740 10 client.0 _ll_register_callbacks cb 0x7f58680a5fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:52.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.223+0000 7f586e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc0359a30 0x55dbc03698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.223+0000 7f586e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc0359a30 0x55dbc03698b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47048/0 (socket says 172.21.15.44:47048) 2024-08-20T03:05:52.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.223+0000 7f586e59c640 1 -- 172.21.15.44:0/3827839940 learned_addr learned my addr 172.21.15.44:0/3827839940 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:52.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.223+0000 7f586dd9b640 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc035a350 0x55dbc0369e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.223+0000 7f586ed9d640 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc03662e0 0x55dbc03df100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.224+0000 7f586e59c640 1 -- 172.21.15.44:0/3827839940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc03662e0 msgr2=0x55dbc03df100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:52.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.224+0000 7f586e59c640 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc03662e0 0x55dbc03df100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.224+0000 7f586e59c640 1 -- 172.21.15.44:0/3827839940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc035a350 msgr2=0x55dbc0369e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:52.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.224+0000 7f586e59c640 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc035a350 0x55dbc0369e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.224+0000 7f586e59c640 1 -- 172.21.15.44:0/3827839940 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbc03df640 con 0x55dbc0359a30 2024-08-20T03:05:52.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.225+0000 7f586e59c640 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc0359a30 0x55dbc03698b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55dbc035cbb0 tx=0x7f5860005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41f1fdc1e4d0f79 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:52.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.225+0000 7f5855ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:05:52.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.230+0000 7f5855ffb640 1 -- 172.21.15.44:0/3827839940 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860040040 con 0x55dbc0359a30 2024-08-20T03:05:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.230+0000 7f5855ffb640 1 -- 172.21.15.44:0/3827839940 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5860030dc0 con 0x55dbc0359a30 2024-08-20T03:05:52.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.231+0000 7f5876c75740 1 -- 172.21.15.44:0/3827839940 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58680cb280 con 0x55dbc0359a30 2024-08-20T03:05:52.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.235+0000 7f5855ffb640 1 -- 172.21.15.44:0/3827839940 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5860055050 con 0x55dbc0359a30 2024-08-20T03:05:52.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.235+0000 7f5876c75740 10 client.25895 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:52.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.235+0000 7f5876c75740 1 -- 172.21.15.44:0/3827839940 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dbc03e0940 con 0x55dbc0359a30 2024-08-20T03:05:52.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.237+0000 7f5855ffb640 1 -- 172.21.15.44:0/3827839940 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58680cb280 con 0x55dbc0359a30 2024-08-20T03:05:52.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.237+0000 7f5855ffb640 1 -- 172.21.15.44:0/3827839940 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f58600046c0 con 0x55dbc0359a30 2024-08-20T03:05:52.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.238+0000 7f5876c75740 10 client.25895 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:52.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.238+0000 7f5876c75740 10 client.25895 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:05:52.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.238+0000 7f5876c75740 20 client.25895 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:52.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.238+0000 7f5876c75740 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55dbc03e0cb0 0x55dbc03e30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:52.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.238+0000 7f5876c75740 4 client.25895 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:05:52.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.238+0000 7f5876c75740 1 -- 172.21.15.44:0/3827839940 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dbc02ce320 con 0x55dbc03e0cb0 2024-08-20T03:05:52.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.238+0000 7f586dd9b640 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55dbc03e0cb0 0x55dbc03e30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:52.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.240+0000 7f586dd9b640 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55dbc03e0cb0 0x55dbc03e30a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba3b779643656139 server_cookie=f90ebd99894c2c7a in_seq=0 out_seq=0 2024-08-20T03:05:52.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.243+0000 7f5855ffb640 10 client.25895 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:05:52.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.243+0000 7f5855ffb640 1 -- 172.21.15.44:0/3827839940 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dbc02ce320 con 0x55dbc03e0cb0 2024-08-20T03:05:52.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.243+0000 7f5855ffb640 10 client.25895 handle_command_reply: tid=0 2024-08-20T03:05:52.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.243+0000 7f5876c75740 1 client.25895 shutdown 2024-08-20T03:05:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.249+0000 7f5876c75740 1 -- 172.21.15.44:0/3827839940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc0359a30 msgr2=0x55dbc03698b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:52.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.249+0000 7f5876c75740 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc0359a30 0x55dbc03698b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55dbc035cbb0 tx=0x7f5860005b20 comp rx=0 tx=0).stop 2024-08-20T03:05:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.249+0000 7f5876c75740 1 -- 172.21.15.44:0/3827839940 shutdown_connections 2024-08-20T03:05:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.249+0000 7f5876c75740 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55dbc03e0cb0 0x55dbc03e30a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.249+0000 7f5876c75740 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dbc03662e0 0x55dbc03df100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.249+0000 7f5876c75740 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dbc035a350 0x55dbc0369e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.249+0000 7f5876c75740 1 --2- 172.21.15.44:0/3827839940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dbc0359a30 0x55dbc03698b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.249+0000 7f5876c75740 1 -- 172.21.15.44:0/3827839940 >> 172.21.15.44:0/3827839940 conn(0x55dbc0357860 msgr2=0x55dbc03648c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.255+0000 7f5855ffb640 0 client.25895 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:05:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.255+0000 7f5876c75740 1 -- 172.21.15.44:0/3827839940 shutdown_connections 2024-08-20T03:05:52.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.255+0000 7f5876c75740 1 -- 172.21.15.44:0/3827839940 wait complete. 2024-08-20T03:05:52.259 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:05:52.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.256+0000 7f5876c75740 20 client.25895 trim_cache size 0 max 16384 2024-08-20T03:05:52.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.259+0000 7f586f59e640 1 -- 172.21.15.44:0/1403278202 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f58580b4010 msgr2=0x7f58580b64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:52.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.259+0000 7f586f59e640 1 --2- 172.21.15.44:0/1403278202 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f58580b4010 0x7f58580b64d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55dbc03599e0 tx=0x7f5864009260 comp rx=0 tx=0).stop 2024-08-20T03:05:52.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.259+0000 7f586f59e640 1 -- 172.21.15.44:0/1403278202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587011bad0 msgr2=0x7f58701ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:52.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.259+0000 7f586f59e640 1 --2- 172.21.15.44:0/1403278202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587011bad0 0x7f58701ad680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f586800b990 tx=0x7f586800be50 comp rx=0 tx=0).stop 2024-08-20T03:05:52.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.259+0000 7f586f59e640 1 -- 172.21.15.44:0/1403278202 shutdown_connections 2024-08-20T03:05:52.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.259+0000 7f586f59e640 1 --2- 172.21.15.44:0/1403278202 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f58580b4010 0x7f58580b64d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.259+0000 7f586f59e640 1 --2- 172.21.15.44:0/1403278202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f587011bad0 0x7f58701ad680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.259+0000 7f586f59e640 1 --2- 172.21.15.44:0/1403278202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f587006dde0 0x7f58701a82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.259+0000 7f586f59e640 1 --2- 172.21.15.44:0/1403278202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f587006c620 0x7f58701a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:52.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.259+0000 7f586f59e640 1 -- 172.21.15.44:0/1403278202 >> 172.21.15.44:0/1403278202 conn(0x7f587006ef80 msgr2=0x7f5870070750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:52.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.260+0000 7f586f59e640 1 -- 172.21.15.44:0/1403278202 shutdown_connections 2024-08-20T03:05:52.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:52.260+0000 7f586f59e640 1 -- 172.21.15.44:0/1403278202 wait complete. 2024-08-20T03:05:53.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T03:05:53.426 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:05:53.808 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:53 smithi044 ceph-mon[18790]: pgmap v422: 97 pgs: 97 active+clean; 421 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 273 op/s 2024-08-20T03:05:53.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:53 smithi165 ceph-mon[22357]: pgmap v422: 97 pgs: 97 active+clean; 421 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 273 op/s 2024-08-20T03:05:53.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:53 smithi099 ceph-mon[21984]: pgmap v422: 97 pgs: 97 active+clean; 421 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 273 op/s 2024-08-20T03:05:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.025+0000 7f213df41640 1 -- 172.21.15.44:0/2352282789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2138108600 msgr2=0x7f2138108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.025+0000 7f213df41640 1 --2- 172.21.15.44:0/2352282789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2138108600 0x7f2138108a40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2120009930 tx=0x7f212002a420 comp rx=0 tx=0).stop 2024-08-20T03:05:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.025+0000 7f213df41640 1 -- 172.21.15.44:0/2352282789 shutdown_connections 2024-08-20T03:05:54.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.025+0000 7f213df41640 1 --2- 172.21.15.44:0/2352282789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2138108f80 0x7f213810f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.025+0000 7f213df41640 1 --2- 172.21.15.44:0/2352282789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2138108600 0x7f2138108a40 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.025+0000 7f213df41640 1 --2- 172.21.15.44:0/2352282789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2138101500 0x7f21381018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.025+0000 7f213df41640 1 -- 172.21.15.44:0/2352282789 >> 172.21.15.44:0/2352282789 conn(0x7f2138076cc0 msgr2=0x7f21380770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.025+0000 7f213df41640 1 -- 172.21.15.44:0/2352282789 shutdown_connections 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.025+0000 7f213df41640 1 -- 172.21.15.44:0/2352282789 wait complete. 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.025+0000 7f213df41640 1 Processor -- start 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.025+0000 7f213df41640 1 -- start start 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f213df41640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2138101500 0x7f21381024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f213df41640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2138108600 0x7f2138102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f213df41640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2138108f80 0x7f2138102f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f213df41640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2138112af0 con 0x7f2138108600 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f213df41640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2138112970 con 0x7f2138101500 2024-08-20T03:05:54.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f213df41640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2138112c70 con 0x7f2138108f80 2024-08-20T03:05:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f212effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2138108600 0x7f2138102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f21377fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2138101500 0x7f21381024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f212effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2138108600 0x7f2138102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35574/0 (socket says 172.21.15.44:35574) 2024-08-20T03:05:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f212effd640 1 -- 172.21.15.44:0/257962479 learned_addr learned my addr 172.21.15.44:0/257962479 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f2137fff640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2138108f80 0x7f2138102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f21377fe640 1 -- 172.21.15.44:0/257962479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2138108f80 msgr2=0x7f2138102f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f21377fe640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2138108f80 0x7f2138102f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f21377fe640 1 -- 172.21.15.44:0/257962479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2138108600 msgr2=0x7f2138102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f21377fe640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2138108600 0x7f2138102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f21377fe640 1 -- 172.21.15.44:0/257962479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21381a7400 con 0x7f2138101500 2024-08-20T03:05:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.026+0000 7f212effd640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2138108600 0x7f2138102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:54.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.027+0000 7f21377fe640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2138101500 0x7f21381024d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f212800d6d0 tx=0x7f212800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.027+0000 7f21357fa640 1 -- 172.21.15.44:0/257962479 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128017070 con 0x7f2138101500 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.027+0000 7f213df41640 1 -- 172.21.15.44:0/257962479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21381a76f0 con 0x7f2138101500 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.027+0000 7f21357fa640 1 -- 172.21.15.44:0/257962479 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2128009820 con 0x7f2138101500 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.027+0000 7f21357fa640 1 -- 172.21.15.44:0/257962479 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2128004c40 con 0x7f2138101500 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.027+0000 7f213df41640 1 -- 172.21.15.44:0/257962479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21381a7c00 con 0x7f2138101500 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.029+0000 7f21357fa640 1 -- 172.21.15.44:0/257962479 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2128005220 con 0x7f2138101500 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.029+0000 7f214033a740 1 Processor -- start 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.029+0000 7f214033a740 1 -- start start 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f214033a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820d880 0x55841820dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f214033a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841820e1a0 0x558418219bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f214033a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841821a130 0x55841821c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f214033a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558418184930 con 0x55841820d880 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f214033a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558418173e30 con 0x55841820e1a0 2024-08-20T03:05:54.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f214033a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558418177e70 con 0x55841821a130 2024-08-20T03:05:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f2137fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820d880 0x55841820dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f21377fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841820e1a0 0x558418219bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f2137fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820d880 0x55841820dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35590/0 (socket says 172.21.15.44:35590) 2024-08-20T03:05:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f2137fff640 1 -- 172.21.15.44:0/2148607697 learned_addr learned my addr 172.21.15.44:0/2148607697 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f212effd640 1 --2- 172.21.15.44:0/2148607697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841821a130 0x55841821c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f2137fff640 1 -- 172.21.15.44:0/2148607697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841821a130 msgr2=0x55841821c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f2137fff640 1 --2- 172.21.15.44:0/2148607697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841821a130 0x55841821c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f2137fff640 1 -- 172.21.15.44:0/2148607697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841820e1a0 msgr2=0x558418219bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f2137fff640 1 --2- 172.21.15.44:0/2148607697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841820e1a0 0x558418219bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f21357fa640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f21180b3d40 0x7f21180b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f2137fff640 1 -- 172.21.15.44:0/2148607697 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558417ed3650 con 0x55841820d880 2024-08-20T03:05:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f21377fe640 1 --2- 172.21.15.44:0/2148607697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841820e1a0 0x558418219bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f212effd640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f21180b3d40 0x7f21180b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f2137fff640 1 --2- 172.21.15.44:0/2148607697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820d880 0x55841820dc60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2130010370 tx=0x7f2130010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6a98a438f02114 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f21357fa640 1 -- 172.21.15.44:0/257962479 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f21280e3a50 con 0x7f2138101500 2024-08-20T03:05:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.030+0000 7f212dffb640 1 -- 172.21.15.44:0/2148607697 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130016070 con 0x55841820d880 2024-08-20T03:05:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.031+0000 7f212dffb640 1 -- 172.21.15.44:0/2148607697 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2130012bc0 con 0x55841820d880 2024-08-20T03:05:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.031+0000 7f212dffb640 1 -- 172.21.15.44:0/2148607697 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130011be0 con 0x55841820d880 2024-08-20T03:05:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.031+0000 7f214033a740 1 -- 172.21.15.44:0/2148607697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820d880 msgr2=0x55841820dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.031+0000 7f214033a740 1 --2- 172.21.15.44:0/2148607697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820d880 0x55841820dc60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2130010370 tx=0x7f2130010830 comp rx=0 tx=0).stop 2024-08-20T03:05:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.031+0000 7f212effd640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f21180b3d40 0x7f21180b6200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2120000c00 tx=0x7f21200058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:54.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.031+0000 7f214033a740 1 -- 172.21.15.44:0/2148607697 shutdown_connections 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.031+0000 7f214033a740 1 --2- 172.21.15.44:0/2148607697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841821a130 0x55841821c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.031+0000 7f214033a740 1 --2- 172.21.15.44:0/2148607697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841820e1a0 0x558418219bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.031+0000 7f214033a740 1 --2- 172.21.15.44:0/2148607697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820d880 0x55841820dc60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.031+0000 7f214033a740 1 -- 172.21.15.44:0/2148607697 >> 172.21.15.44:0/2148607697 conn(0x55841820b6b0 msgr2=0x55841820c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.031+0000 7f214033a740 1 -- 172.21.15.44:0/2148607697 shutdown_connections 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.031+0000 7f214033a740 1 -- 172.21.15.44:0/2148607697 wait complete. 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f214033a740 1 -- start start 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f214033a740 1 Processor -- start 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f214033a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 0x558418217120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f214033a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820e1a0 0x558418217660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f214033a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841821a130 0x558418217ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f214033a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558417fd6700 con 0x55841820e1a0 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f214033a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5584180fa810 con 0x55841821a130 2024-08-20T03:05:54.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f214033a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55841821cde0 con 0x55841820d880 2024-08-20T03:05:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f214033a740 10 client.0 _ll_register_callbacks cb 0x55841816d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f212effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820e1a0 0x558418217660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f21377fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 0x558418217120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f212effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820e1a0 0x558418217660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35598/0 (socket says 172.21.15.44:35598) 2024-08-20T03:05:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f21377fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 0x558418217120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:43844/0 (socket says 172.21.15.44:43844) 2024-08-20T03:05:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f212effd640 1 -- 172.21.15.44:0/3497781423 learned_addr learned my addr 172.21.15.44:0/3497781423 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.032+0000 7f2137fff640 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841821a130 0x558418217ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f212effd640 1 -- 172.21.15.44:0/3497781423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 msgr2=0x558418217120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f212effd640 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 0x558418217120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f212effd640 1 -- 172.21.15.44:0/3497781423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841821a130 msgr2=0x558418217ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f212effd640 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841821a130 0x558418217ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f212effd640 1 -- 172.21.15.44:0/3497781423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584182261e0 con 0x55841820e1a0 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f2137fff640 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841821a130 0x558418217ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f21377fe640 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 0x558418217120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f212effd640 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820e1a0 0x558418217660 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f21200096c0 tx=0x7f2120009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379e71a727f64d45 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f2126ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f2126ffd640 1 -- 172.21.15.44:0/3497781423 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212004d070 con 0x55841820e1a0 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f214033a740 1 -- 172.21.15.44:0/3497781423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558418123150 con 0x55841820e1a0 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f2126ffd640 1 -- 172.21.15.44:0/3497781423 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f2120005020 con 0x55841820e1a0 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f2126ffd640 1 -- 172.21.15.44:0/3497781423 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2120043070 con 0x55841820e1a0 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f214033a740 10 client.16431 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f214033a740 1 -- 172.21.15.44:0/3497781423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558418228b90 con 0x55841820e1a0 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.033+0000 7f2126ffd640 1 -- 172.21.15.44:0/3497781423 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558418123150 con 0x55841820e1a0 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.034+0000 7f2126ffd640 1 -- 172.21.15.44:0/3497781423 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2120059020 con 0x55841820e1a0 2024-08-20T03:05:54.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.034+0000 7f214033a740 10 client.16431 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.034+0000 7f214033a740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.034+0000 7f214033a740 20 client.16431 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.034+0000 7f214033a740 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55841822a040 0x55841822c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.034+0000 7f214033a740 4 client.16431 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:05:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.034+0000 7f214033a740 1 -- 172.21.15.44:0/3497781423 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558418186880 con 0x55841822a040 2024-08-20T03:05:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.034+0000 7f21377fe640 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55841822a040 0x55841822c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.034+0000 7f2126ffd640 10 client.16431 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:54.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.034+0000 7f21377fe640 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55841822a040 0x55841822c4c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef1fd8706b16a21 server_cookie=95a2f99fa6db2ba3 in_seq=0 out_seq=0 2024-08-20T03:05:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.223+0000 7f2126ffd640 1 -- 172.21.15.44:0/3497781423 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558418186880 con 0x55841822a040 2024-08-20T03:05:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.223+0000 7f2126ffd640 10 client.16431 handle_command_reply: tid=0 2024-08-20T03:05:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.223+0000 7f214033a740 1 client.16431 shutdown 2024-08-20T03:05:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.223+0000 7f214033a740 1 -- 172.21.15.44:0/3497781423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820e1a0 msgr2=0x558418217660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:54.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.223+0000 7f214033a740 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820e1a0 0x558418217660 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f21200096c0 tx=0x7f2120009260 comp rx=0 tx=0).stop 2024-08-20T03:05:54.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.224+0000 7f214033a740 1 -- 172.21.15.44:0/3497781423 shutdown_connections 2024-08-20T03:05:54.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.224+0000 7f214033a740 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55841822a040 0x55841822c4c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.224+0000 7f214033a740 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841821a130 0x558418217ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.224+0000 7f214033a740 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820e1a0 0x558418217660 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.224+0000 7f2126ffd640 0 client.16431 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:54.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.224+0000 7f214033a740 1 --2- 172.21.15.44:0/3497781423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 0x558418217120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.224+0000 7f214033a740 1 -- 172.21.15.44:0/3497781423 >> 172.21.15.44:0/3497781423 conn(0x55841820b6b0 msgr2=0x558418211250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:54.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.224+0000 7f214033a740 1 -- 172.21.15.44:0/3497781423 shutdown_connections 2024-08-20T03:05:54.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.224+0000 7f214033a740 1 -- 172.21.15.44:0/3497781423 wait complete. 2024-08-20T03:05:54.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.224+0000 7f214033a740 20 client.16431 trim_cache size 0 max 16384 2024-08-20T03:05:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.252+0000 7f214033a740 1 Processor -- start 2024-08-20T03:05:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.254+0000 7f214033a740 1 -- start start 2024-08-20T03:05:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.254+0000 7f214033a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841820d880 0x5584182063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.254+0000 7f214033a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820e1a0 0x558418231d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.254+0000 7f214033a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841821a130 0x5584182322d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.254+0000 7f214033a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f21381b0500 con 0x55841821a130 2024-08-20T03:05:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.254+0000 7f214033a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21381b2100 con 0x55841820d880 2024-08-20T03:05:54.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.254+0000 7f214033a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f21381b0bd0 con 0x55841820e1a0 2024-08-20T03:05:54.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.260+0000 7f2137fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841821a130 0x5584182322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.260+0000 7f2137fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841821a130 0x5584182322d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35608/0 (socket says 172.21.15.44:35608) 2024-08-20T03:05:54.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.260+0000 7f2137fff640 1 -- 172.21.15.44:0/3778791634 learned_addr learned my addr 172.21.15.44:0/3778791634 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:54.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.260+0000 7f212effd640 1 --2- 172.21.15.44:0/3778791634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820e1a0 0x558418231d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.260+0000 7f21377fe640 1 --2- 172.21.15.44:0/3778791634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841820d880 0x5584182063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.262+0000 7f212effd640 1 -- 172.21.15.44:0/3778791634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841820d880 msgr2=0x5584182063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:54.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.262+0000 7f212effd640 1 --2- 172.21.15.44:0/3778791634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841820d880 0x5584182063d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.262+0000 7f212effd640 1 -- 172.21.15.44:0/3778791634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841821a130 msgr2=0x5584182322d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:54.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.262+0000 7f212effd640 1 --2- 172.21.15.44:0/3778791634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841821a130 0x5584182322d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.262+0000 7f212effd640 1 -- 172.21.15.44:0/3778791634 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55841820c130 con 0x55841820e1a0 2024-08-20T03:05:54.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.262+0000 7f212effd640 1 --2- 172.21.15.44:0/3778791634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820e1a0 0x558418231d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2120005ec0 tx=0x7f2120004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84b2ebfaea7bb991 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.266+0000 7f212d7fa640 1 -- 172.21.15.44:0/3778791634 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212004d040 con 0x55841820e1a0 2024-08-20T03:05:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.266+0000 7f212d7fa640 1 -- 172.21.15.44:0/3778791634 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2120005000 con 0x55841820e1a0 2024-08-20T03:05:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.266+0000 7f212d7fa640 1 -- 172.21.15.44:0/3778791634 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120035040 con 0x55841820e1a0 2024-08-20T03:05:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.269+0000 7f214033a740 1 -- 172.21.15.44:0/3778791634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820e1a0 msgr2=0x558418231d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.269+0000 7f214033a740 1 --2- 172.21.15.44:0/3778791634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820e1a0 0x558418231d90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2120005ec0 tx=0x7f2120004f90 comp rx=0 tx=0).stop 2024-08-20T03:05:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.273+0000 7f214033a740 1 -- 172.21.15.44:0/3778791634 shutdown_connections 2024-08-20T03:05:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.273+0000 7f214033a740 1 --2- 172.21.15.44:0/3778791634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841821a130 0x5584182322d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.273+0000 7f214033a740 1 --2- 172.21.15.44:0/3778791634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820e1a0 0x558418231d90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.273+0000 7f214033a740 1 --2- 172.21.15.44:0/3778791634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841820d880 0x5584182063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.273+0000 7f214033a740 1 -- 172.21.15.44:0/3778791634 >> 172.21.15.44:0/3778791634 conn(0x55841820b6b0 msgr2=0x5584182159a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.283+0000 7f214033a740 1 -- 172.21.15.44:0/3778791634 shutdown_connections 2024-08-20T03:05:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.283+0000 7f214033a740 1 -- 172.21.15.44:0/3778791634 wait complete. 2024-08-20T03:05:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.283+0000 7f214033a740 1 -- start start 2024-08-20T03:05:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.284+0000 7f214033a740 1 Processor -- start 2024-08-20T03:05:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.284+0000 7f214033a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 0x55841821d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.284+0000 7f214033a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820e1a0 0x55841821dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.284+0000 7f214033a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841821a130 0x558418292d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.284+0000 7f214033a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f21381b1970 con 0x55841820e1a0 2024-08-20T03:05:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.284+0000 7f214033a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21381b12a0 con 0x55841821a130 2024-08-20T03:05:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.284+0000 7f214033a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f21381ae4e0 con 0x55841820d880 2024-08-20T03:05:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.284+0000 7f214033a740 10 client.0 _ll_register_callbacks cb 0x7f21200317d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.287+0000 7f21377fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 0x55841821d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.287+0000 7f21377fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 0x55841821d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:43862/0 (socket says 172.21.15.44:43862) 2024-08-20T03:05:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.287+0000 7f21377fe640 1 -- 172.21.15.44:0/105660949 learned_addr learned my addr 172.21.15.44:0/105660949 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:54.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.287+0000 7f212effd640 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820e1a0 0x55841821dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.288+0000 7f2137fff640 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841821a130 0x558418292d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.291+0000 7f21377fe640 1 -- 172.21.15.44:0/105660949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841821a130 msgr2=0x558418292d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:54.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.291+0000 7f21377fe640 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841821a130 0x558418292d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.291+0000 7f21377fe640 1 -- 172.21.15.44:0/105660949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820e1a0 msgr2=0x55841821dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:54.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.291+0000 7f21377fe640 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820e1a0 0x55841821dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.291+0000 7f21377fe640 1 -- 172.21.15.44:0/105660949 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841820d260 con 0x55841820d880 2024-08-20T03:05:54.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.292+0000 7f21377fe640 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 0x55841821d820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f21280e1270 tx=0x7f21280e7590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18c90fa58d20916a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:54.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.292+0000 7f2126ffd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:05:54.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.293+0000 7f2126ffd640 1 -- 172.21.15.44:0/105660949 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21280ea070 con 0x55841820d880 2024-08-20T03:05:54.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.293+0000 7f2126ffd640 1 -- 172.21.15.44:0/105660949 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f21280099d0 con 0x55841820d880 2024-08-20T03:05:54.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.293+0000 7f214033a740 1 -- 172.21.15.44:0/105660949 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2120005000 con 0x55841820d880 2024-08-20T03:05:54.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.297+0000 7f2126ffd640 1 -- 172.21.15.44:0/105660949 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2128020040 con 0x55841820d880 2024-08-20T03:05:54.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.299+0000 7f214033a740 10 client.25888 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:54.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.299+0000 7f214033a740 1 -- 172.21.15.44:0/105660949 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558418294590 con 0x55841820d880 2024-08-20T03:05:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.306+0000 7f2126ffd640 1 -- 172.21.15.44:0/105660949 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2120005000 con 0x55841820d880 2024-08-20T03:05:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.306+0000 7f2126ffd640 1 -- 172.21.15.44:0/105660949 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f21280cd0b0 con 0x55841820d880 2024-08-20T03:05:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.308+0000 7f214033a740 10 client.25888 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.308+0000 7f214033a740 10 client.25888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.308+0000 7f214033a740 20 client.25888 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.308+0000 7f214033a740 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558418294d00 0x5584182b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.308+0000 7f214033a740 4 client.25888 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:05:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.308+0000 7f214033a740 1 -- 172.21.15.44:0/105660949 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558418182320 con 0x558418294d00 2024-08-20T03:05:54.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.365+0000 7f212effd640 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558418294d00 0x5584182b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:54.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.375+0000 7f212effd640 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558418294d00 0x5584182b9560 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c1dbd0798547d47 server_cookie=a121fd4af8730981 in_seq=0 out_seq=0 2024-08-20T03:05:54.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.375+0000 7f2126ffd640 10 client.25888 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:54.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.401+0000 7f2126ffd640 1 -- 172.21.15.44:0/105660949 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558418182320 con 0x558418294d00 2024-08-20T03:05:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.401+0000 7f2126ffd640 10 client.25888 handle_command_reply: tid=0 2024-08-20T03:05:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.401+0000 7f214033a740 1 client.25888 shutdown 2024-08-20T03:05:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.402+0000 7f214033a740 1 -- 172.21.15.44:0/105660949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 msgr2=0x55841821d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:54.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.402+0000 7f214033a740 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 0x55841821d820 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f21280e1270 tx=0x7f21280e7590 comp rx=0 tx=0).stop 2024-08-20T03:05:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.402+0000 7f214033a740 1 -- 172.21.15.44:0/105660949 shutdown_connections 2024-08-20T03:05:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.402+0000 7f214033a740 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558418294d00 0x5584182b9560 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.402+0000 7f214033a740 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55841821a130 0x558418292d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.402+0000 7f214033a740 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55841820e1a0 0x55841821dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.402+0000 7f214033a740 1 --2- 172.21.15.44:0/105660949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55841820d880 0x55841821d820 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f21280e1270 tx=0x7f21280e7590 comp rx=0 tx=0).stop 2024-08-20T03:05:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.402+0000 7f214033a740 1 -- 172.21.15.44:0/105660949 >> 172.21.15.44:0/105660949 conn(0x55841820b6b0 msgr2=0x558418218f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.402+0000 7f2126ffd640 0 client.25888 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.402+0000 7f214033a740 1 -- 172.21.15.44:0/105660949 shutdown_connections 2024-08-20T03:05:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.402+0000 7f214033a740 1 -- 172.21.15.44:0/105660949 wait complete. 2024-08-20T03:05:54.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.402+0000 7f214033a740 20 client.25888 trim_cache size 0 max 16384 2024-08-20T03:05:54.406 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:05:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.403+0000 7f213df41640 1 -- 172.21.15.44:0/257962479 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f21180b3d40 msgr2=0x7f21180b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.403+0000 7f213df41640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f21180b3d40 0x7f21180b6200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2120000c00 tx=0x7f21200058d0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.403+0000 7f213df41640 1 -- 172.21.15.44:0/257962479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2138101500 msgr2=0x7f21381024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.403+0000 7f213df41640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2138101500 0x7f21381024d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f212800d6d0 tx=0x7f212800db90 comp rx=0 tx=0).stop 2024-08-20T03:05:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.403+0000 7f213df41640 1 -- 172.21.15.44:0/257962479 shutdown_connections 2024-08-20T03:05:54.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.403+0000 7f213df41640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f21180b3d40 0x7f21180b6200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.403+0000 7f213df41640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2138108f80 0x7f2138102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.403+0000 7f213df41640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2138108600 0x7f2138102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.403+0000 7f213df41640 1 --2- 172.21.15.44:0/257962479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2138101500 0x7f21381024d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.403+0000 7f213df41640 1 -- 172.21.15.44:0/257962479 >> 172.21.15.44:0/257962479 conn(0x7f2138076cc0 msgr2=0x7f21380fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.404+0000 7f213df41640 1 -- 172.21.15.44:0/257962479 shutdown_connections 2024-08-20T03:05:54.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:54.404+0000 7f213df41640 1 -- 172.21.15.44:0/257962479 wait complete. 2024-08-20T03:05:55.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:55 smithi044 ceph-mon[18790]: pgmap v423: 97 pgs: 97 active+clean; 437 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 334 op/s 2024-08-20T03:05:55.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:55 smithi165 ceph-mon[22357]: pgmap v423: 97 pgs: 97 active+clean; 437 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 334 op/s 2024-08-20T03:05:55.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:55 smithi099 ceph-mon[21984]: pgmap v423: 97 pgs: 97 active+clean; 437 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 334 op/s 2024-08-20T03:05:56.393 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T03:05:56.394 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T03:05:56.661 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:05:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f984dffb640 1 -- 172.21.15.44:0/1739179975 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f98600036b0 con 0x5585eb9de470 2024-08-20T03:05:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f984dffb640 10 client.16404 handle_command_reply: tid=0 2024-08-20T03:05:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f986f68f740 1 client.16404 shutdown 2024-08-20T03:05:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f986f68f740 1 -- 172.21.15.44:0/1739179975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 msgr2=0x5585eb967440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f986f68f740 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 0x5585eb967440 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9860009800 tx=0x7f9860012730 comp rx=0 tx=0).stop 2024-08-20T03:05:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f986f68f740 1 -- 172.21.15.44:0/1739179975 shutdown_connections 2024-08-20T03:05:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f986f68f740 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5585eb9de470 0x5585eb9e0900 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f986f68f740 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585eb9643e0 0x5585eb967ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f986f68f740 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5585eb958450 0x5585eb967980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f986f68f740 1 --2- 172.21.15.44:0/1739179975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5585eb957b30 0x5585eb967440 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f984dffb640 0 client.16404 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f986f68f740 1 -- 172.21.15.44:0/1739179975 >> 172.21.15.44:0/1739179975 conn(0x5585eb955960 msgr2=0x5585eb962670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f986f68f740 1 -- 172.21.15.44:0/1739179975 shutdown_connections 2024-08-20T03:05:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f986f68f740 1 -- 172.21.15.44:0/1739179975 wait complete. 2024-08-20T03:05:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.832+0000 7f986f68f740 20 client.16404 trim_cache size 0 max 16384 2024-08-20T03:05:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:05:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:05:56.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:05:56.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:05:56.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.833+0000 7f986b55d640 1 -- 172.21.15.44:0/3656208491 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f98500b3f50 msgr2=0x7f98500b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:56.837 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.833+0000 7f986b55d640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f98500b3f50 0x7f98500b6410 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9864118c90 tx=0x7f9854008040 comp rx=0 tx=0).stop 2024-08-20T03:05:56.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.833+0000 7f986b55d640 1 -- 172.21.15.44:0/3656208491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9864117c00 msgr2=0x7f98641180b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:56.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.833+0000 7f986b55d640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9864117c00 0x7f98641180b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f985c009bf0 tx=0x7f985c031a20 comp rx=0 tx=0).stop 2024-08-20T03:05:56.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.833+0000 7f986b55d640 1 -- 172.21.15.44:0/3656208491 shutdown_connections 2024-08-20T03:05:56.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.833+0000 7f986b55d640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f98500b3f50 0x7f98500b6410 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:56.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.833+0000 7f986b55d640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9864117c00 0x7f98641180b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:56.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.833+0000 7f986b55d640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f986410a260 0x7f9864112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:56.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.833+0000 7f986b55d640 1 --2- 172.21.15.44:0/3656208491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9864109e80 0x7f9864112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:56.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.833+0000 7f986b55d640 1 -- 172.21.15.44:0/3656208491 >> 172.21.15.44:0/3656208491 conn(0x7f986406d8b0 msgr2=0x7f986411c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:56.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.833+0000 7f986b55d640 1 -- 172.21.15.44:0/3656208491 shutdown_connections 2024-08-20T03:05:56.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:05:56.833+0000 7f986b55d640 1 -- 172.21.15.44:0/3656208491 wait complete. 2024-08-20T03:05:57.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.250+0000 7f24edc23640 1 -- 172.21.15.44:0/1791828085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e8103870 msgr2=0x7f24e8103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:57.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.250+0000 7f24edc23640 1 --2- 172.21.15.44:0/1791828085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e8103870 0x7f24e8103c50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009900 tx=0x7f24d802a410 comp rx=0 tx=0).stop 2024-08-20T03:05:57.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.250+0000 7f24edc23640 1 -- 172.21.15.44:0/1791828085 shutdown_connections 2024-08-20T03:05:57.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.250+0000 7f24edc23640 1 --2- 172.21.15.44:0/1791828085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e810ce20 0x7f24e810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.250+0000 7f24edc23640 1 --2- 172.21.15.44:0/1791828085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e8104270 0x7f24e810c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.250+0000 7f24edc23640 1 --2- 172.21.15.44:0/1791828085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e8103870 0x7f24e8103c50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.250+0000 7f24edc23640 1 -- 172.21.15.44:0/1791828085 >> 172.21.15.44:0/1791828085 conn(0x7f24e80fd400 msgr2=0x7f24e80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:57.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.251+0000 7f24edc23640 1 -- 172.21.15.44:0/1791828085 shutdown_connections 2024-08-20T03:05:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.251+0000 7f24edc23640 1 -- 172.21.15.44:0/1791828085 wait complete. 2024-08-20T03:05:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.251+0000 7f24edc23640 1 Processor -- start 2024-08-20T03:05:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.251+0000 7f24edc23640 1 -- start start 2024-08-20T03:05:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.251+0000 7f24edc23640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e8103870 0x7f24e8198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.251+0000 7f24edc23640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e8104270 0x7f24e8199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24edc23640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e810ce20 0x7f24e819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24edc23640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8112a60 con 0x7f24e8104270 2024-08-20T03:05:57.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24edc23640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81128e0 con 0x7f24e8103870 2024-08-20T03:05:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24edc23640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8112be0 con 0x7f24e810ce20 2024-08-20T03:05:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e8104270 0x7f24e8199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e8103870 0x7f24e8198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e8104270 0x7f24e8199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35628/0 (socket says 172.21.15.44:35628) 2024-08-20T03:05:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e6ffd640 1 -- 172.21.15.44:0/2566881049 learned_addr learned my addr 172.21.15.44:0/2566881049 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e7fff640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e810ce20 0x7f24e819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e77fe640 1 -- 172.21.15.44:0/2566881049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e810ce20 msgr2=0x7f24e819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e77fe640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e810ce20 0x7f24e819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e77fe640 1 -- 172.21.15.44:0/2566881049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e8104270 msgr2=0x7f24e8199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:57.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e77fe640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e8104270 0x7f24e8199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e77fe640 1 -- 172.21.15.44:0/2566881049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e819f0a0 con 0x7f24e8103870 2024-08-20T03:05:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e7fff640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e810ce20 0x7f24e819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e6ffd640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e8104270 0x7f24e8199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e77fe640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e8103870 0x7f24e8198ff0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f24d802a930 tx=0x7f24d8031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e4ff9640 1 -- 172.21.15.44:0/2566881049 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d803f030 con 0x7f24e8103870 2024-08-20T03:05:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e4ff9640 1 -- 172.21.15.44:0/2566881049 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d8004490 con 0x7f24e8103870 2024-08-20T03:05:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24e4ff9640 1 -- 172.21.15.44:0/2566881049 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8035070 con 0x7f24e8103870 2024-08-20T03:05:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.252+0000 7f24edc23640 1 -- 172.21.15.44:0/2566881049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e819f330 con 0x7f24e8103870 2024-08-20T03:05:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.253+0000 7f24edc23640 1 -- 172.21.15.44:0/2566881049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e81a5a50 con 0x7f24e8103870 2024-08-20T03:05:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24f001c740 1 Processor -- start 2024-08-20T03:05:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24e4ff9640 1 -- 172.21.15.44:0/2566881049 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f24d8042030 con 0x7f24e8103870 2024-08-20T03:05:57.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24f001c740 1 -- start start 2024-08-20T03:05:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24f001c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99f3b0 0x5621ec99f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24f001c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 0x5621ec9ab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24f001c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec9abc60 0x5621ec9ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24f001c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5621ec915940 con 0x5621ec99fcd0 2024-08-20T03:05:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24f001c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5621ec904e40 con 0x5621ec99f3b0 2024-08-20T03:05:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24f001c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5621ec908e80 con 0x5621ec9abc60 2024-08-20T03:05:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 0x5621ec9ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 0x5621ec9ab720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35640/0 (socket says 172.21.15.44:35640) 2024-08-20T03:05:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24e77fe640 1 -- 172.21.15.44:0/2726261523 learned_addr learned my addr 172.21.15.44:0/2726261523 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.255+0000 7f24e7fff640 1 --2- 172.21.15.44:0/2726261523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99f3b0 0x5621ec99f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24e6ffd640 1 --2- 172.21.15.44:0/2726261523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec9abc60 0x5621ec9ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24e6ffd640 1 -- 172.21.15.44:0/2726261523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99f3b0 msgr2=0x5621ec99f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24e6ffd640 1 --2- 172.21.15.44:0/2726261523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99f3b0 0x5621ec99f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24e6ffd640 1 -- 172.21.15.44:0/2726261523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 msgr2=0x5621ec9ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24e6ffd640 1 --2- 172.21.15.44:0/2726261523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 0x5621ec9ab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24e6ffd640 1 -- 172.21.15.44:0/2726261523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621ec8a1520 con 0x5621ec9abc60 2024-08-20T03:05:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24e4ff9640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f24d00b3d40 0x7f24d00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24e6ffd640 1 --2- 172.21.15.44:0/2726261523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec9abc60 0x5621ec9ae050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f24dc013800 tx=0x7f24dc013cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a3e1e887fe0c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24e7fff640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f24d00b3d40 0x7f24d00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24e4ff9640 1 -- 172.21.15.44:0/2566881049 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f24d81041f0 con 0x7f24e8103870 2024-08-20T03:05:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24cdffb640 1 -- 172.21.15.44:0/2726261523 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc004030 con 0x5621ec9abc60 2024-08-20T03:05:57.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24cdffb640 1 -- 172.21.15.44:0/2726261523 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24dc00e990 con 0x5621ec9abc60 2024-08-20T03:05:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.256+0000 7f24cdffb640 1 -- 172.21.15.44:0/2726261523 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc003c50 con 0x5621ec9abc60 2024-08-20T03:05:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.257+0000 7f24e7fff640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f24d00b3d40 0x7f24d00b6200 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009fd0 tx=0x7f24e0009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.257+0000 7f24f001c740 1 -- 172.21.15.44:0/2726261523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec9abc60 msgr2=0x5621ec9ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.257+0000 7f24f001c740 1 --2- 172.21.15.44:0/2726261523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec9abc60 0x5621ec9ae050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f24dc013800 tx=0x7f24dc013cc0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.257+0000 7f24f001c740 1 -- 172.21.15.44:0/2726261523 shutdown_connections 2024-08-20T03:05:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.257+0000 7f24f001c740 1 --2- 172.21.15.44:0/2726261523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec9abc60 0x5621ec9ae050 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.257+0000 7f24f001c740 1 --2- 172.21.15.44:0/2726261523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 0x5621ec9ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.257+0000 7f24f001c740 1 --2- 172.21.15.44:0/2726261523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99f3b0 0x5621ec99f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.257+0000 7f24f001c740 1 -- 172.21.15.44:0/2726261523 >> 172.21.15.44:0/2726261523 conn(0x5621ec99d1e0 msgr2=0x5621ec99e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:57.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.257+0000 7f24f001c740 1 -- 172.21.15.44:0/2726261523 shutdown_connections 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.257+0000 7f24f001c740 1 -- 172.21.15.44:0/2726261523 wait complete. 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24f001c740 1 -- start start 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24f001c740 1 Processor -- start 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24f001c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99f3b0 0x5621ec9a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24f001c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99fcd0 0x5621ec9a9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24f001c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 0x5621ec9a9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24f001c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5621ec767890 con 0x5621ec9abc60 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24f001c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5621ec88b700 con 0x5621ec99f3b0 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24f001c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5621ec9ae9b0 con 0x5621ec99fcd0 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24f001c740 10 client.0 _ll_register_callbacks cb 0x5621ec9006f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24e6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99fcd0 0x5621ec9a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 0x5621ec9a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24e6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99fcd0 0x5621ec9a9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:43906/0 (socket says 172.21.15.44:43906) 2024-08-20T03:05:57.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 0x5621ec9a9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35648/0 (socket says 172.21.15.44:35648) 2024-08-20T03:05:57.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24e6ffd640 1 -- 172.21.15.44:0/2496832957 learned_addr learned my addr 172.21.15.44:0/2496832957 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:57.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.258+0000 7f24e77fe640 1 --2- 172.21.15.44:0/2496832957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99f3b0 0x5621ec9a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24e7fff640 1 -- 172.21.15.44:0/2496832957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99fcd0 msgr2=0x5621ec9a9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:57.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24e7fff640 1 --2- 172.21.15.44:0/2496832957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99fcd0 0x5621ec9a9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24e7fff640 1 -- 172.21.15.44:0/2496832957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99f3b0 msgr2=0x5621ec9a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:57.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24e7fff640 1 --2- 172.21.15.44:0/2496832957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99f3b0 0x5621ec9a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24e7fff640 1 -- 172.21.15.44:0/2496832957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621ec664930 con 0x5621ec9abc60 2024-08-20T03:05:57.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24e7fff640 1 --2- 172.21.15.44:0/2496832957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 0x5621ec9a9680 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f24e0003f20 tx=0x7f24e0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff83e25c590d206 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:57.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:05:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24c6ffd640 1 -- 172.21.15.44:0/2496832957 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e00045c0 con 0x5621ec9abc60 2024-08-20T03:05:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24c6ffd640 1 -- 172.21.15.44:0/2496832957 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f24e0004760 con 0x5621ec9abc60 2024-08-20T03:05:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24f001c740 1 -- 172.21.15.44:0/2496832957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24dc00e990 con 0x5621ec9abc60 2024-08-20T03:05:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24c6ffd640 1 -- 172.21.15.44:0/2496832957 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24e0012020 con 0x5621ec9abc60 2024-08-20T03:05:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24f001c740 10 client.16449 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.259+0000 7f24f001c740 1 -- 172.21.15.44:0/2496832957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5621ec9ba5d0 con 0x5621ec9abc60 2024-08-20T03:05:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.260+0000 7f24c6ffd640 1 -- 172.21.15.44:0/2496832957 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24dc00e990 con 0x5621ec9abc60 2024-08-20T03:05:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.260+0000 7f24c6ffd640 1 -- 172.21.15.44:0/2496832957 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f24e0027070 con 0x5621ec9abc60 2024-08-20T03:05:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.260+0000 7f24f001c740 10 client.16449 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.260+0000 7f24f001c740 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.260+0000 7f24f001c740 20 client.16449 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:57.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.260+0000 7f24f001c740 1 --2- 172.21.15.44:0/2496832957 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5621ec9bbc00 0x5621ec9be060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.260+0000 7f24f001c740 4 client.16449 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:05:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.260+0000 7f24f001c740 1 -- 172.21.15.44:0/2496832957 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621ec917890 con 0x5621ec9bbc00 2024-08-20T03:05:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.260+0000 7f24e77fe640 1 --2- 172.21.15.44:0/2496832957 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5621ec9bbc00 0x5621ec9be060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.260+0000 7f24e77fe640 1 --2- 172.21.15.44:0/2496832957 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5621ec9bbc00 0x5621ec9be060 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc17460f87a9d65 server_cookie=7174a6a1fcfecd73 in_seq=0 out_seq=0 2024-08-20T03:05:57.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.260+0000 7f24c6ffd640 10 client.16449 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.262+0000 7f24c6ffd640 1 -- 172.21.15.44:0/2496832957 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621ec917890 con 0x5621ec9bbc00 2024-08-20T03:05:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.262+0000 7f24c6ffd640 10 client.16449 handle_command_reply: tid=0 2024-08-20T03:05:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.262+0000 7f24f001c740 1 client.16449 shutdown 2024-08-20T03:05:57.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.262+0000 7f24f001c740 1 -- 172.21.15.44:0/2496832957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 msgr2=0x5621ec9a9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.262+0000 7f24f001c740 1 --2- 172.21.15.44:0/2496832957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 0x5621ec9a9680 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f24e0003f20 tx=0x7f24e0004ab0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.262+0000 7f24f001c740 1 -- 172.21.15.44:0/2496832957 shutdown_connections 2024-08-20T03:05:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.262+0000 7f24f001c740 1 --2- 172.21.15.44:0/2496832957 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5621ec9bbc00 0x5621ec9be060 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.262+0000 7f24f001c740 1 --2- 172.21.15.44:0/2496832957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 0x5621ec9a9680 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.262+0000 7f24f001c740 1 --2- 172.21.15.44:0/2496832957 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99fcd0 0x5621ec9a9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.262+0000 7f24f001c740 1 --2- 172.21.15.44:0/2496832957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99f3b0 0x5621ec9a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.262+0000 7f24f001c740 1 -- 172.21.15.44:0/2496832957 >> 172.21.15.44:0/2496832957 conn(0x5621ec99d1e0 msgr2=0x5621ec9a2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.262+0000 7f24c6ffd640 0 client.16449 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.263+0000 7f24f001c740 1 -- 172.21.15.44:0/2496832957 shutdown_connections 2024-08-20T03:05:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.263+0000 7f24f001c740 1 -- 172.21.15.44:0/2496832957 wait complete. 2024-08-20T03:05:57.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.263+0000 7f24f001c740 20 client.16449 trim_cache size 0 max 16384 2024-08-20T03:05:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24f001c740 1 Processor -- start 2024-08-20T03:05:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24f001c740 1 -- start start 2024-08-20T03:05:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24f001c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99f3b0 0x5621ec9a3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24f001c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99fcd0 0x5621ec9c3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24f001c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 0x5621ec9c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24f001c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ac0a0 con 0x5621ec9abc60 2024-08-20T03:05:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24f001c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81adca0 con 0x5621ec99fcd0 2024-08-20T03:05:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24f001c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ac770 con 0x5621ec99f3b0 2024-08-20T03:05:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 0x5621ec9c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24e7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 0x5621ec9c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35654/0 (socket says 172.21.15.44:35654) 2024-08-20T03:05:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24e77fe640 1 --2- 172.21.15.44:0/2974499632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99f3b0 0x5621ec9a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24e7fff640 1 -- 172.21.15.44:0/2974499632 learned_addr learned my addr 172.21.15.44:0/2974499632 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24e6ffd640 1 --2- 172.21.15.44:0/2974499632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99fcd0 0x5621ec9c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24e7fff640 1 -- 172.21.15.44:0/2974499632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99f3b0 msgr2=0x5621ec9a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24e7fff640 1 --2- 172.21.15.44:0/2974499632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99f3b0 0x5621ec9a3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24e7fff640 1 -- 172.21.15.44:0/2974499632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99fcd0 msgr2=0x5621ec9c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:57.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24e7fff640 1 --2- 172.21.15.44:0/2974499632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99fcd0 0x5621ec9c3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.281+0000 7f24e7fff640 1 -- 172.21.15.44:0/2974499632 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621ec9aad70 con 0x5621ec9abc60 2024-08-20T03:05:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.282+0000 7f24e7fff640 1 --2- 172.21.15.44:0/2974499632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 0x5621ec9c3e70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f24e000ffd0 tx=0x7f24e0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d2107e0b2c9ce7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.282+0000 7f24cd7fa640 1 -- 172.21.15.44:0/2974499632 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0035030 con 0x5621ec9abc60 2024-08-20T03:05:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.282+0000 7f24cd7fa640 1 -- 172.21.15.44:0/2974499632 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e0012240 con 0x5621ec9abc60 2024-08-20T03:05:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.282+0000 7f24cd7fa640 1 -- 172.21.15.44:0/2974499632 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0011830 con 0x5621ec9abc60 2024-08-20T03:05:57.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.283+0000 7f24f001c740 1 -- 172.21.15.44:0/2974499632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 msgr2=0x5621ec9c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:57.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.283+0000 7f24f001c740 1 --2- 172.21.15.44:0/2974499632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 0x5621ec9c3e70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f24e000ffd0 tx=0x7f24e0005c50 comp rx=0 tx=0).stop 2024-08-20T03:05:57.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.283+0000 7f24f001c740 1 -- 172.21.15.44:0/2974499632 shutdown_connections 2024-08-20T03:05:57.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.283+0000 7f24f001c740 1 --2- 172.21.15.44:0/2974499632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec9abc60 0x5621ec9c3e70 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.283+0000 7f24f001c740 1 --2- 172.21.15.44:0/2974499632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec99fcd0 0x5621ec9c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.283+0000 7f24f001c740 1 --2- 172.21.15.44:0/2974499632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99f3b0 0x5621ec9a3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.283+0000 7f24f001c740 1 -- 172.21.15.44:0/2974499632 >> 172.21.15.44:0/2974499632 conn(0x5621ec99d1e0 msgr2=0x5621ec99c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:57.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.283+0000 7f24f001c740 1 -- 172.21.15.44:0/2974499632 shutdown_connections 2024-08-20T03:05:57.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.283+0000 7f24f001c740 1 -- 172.21.15.44:0/2974499632 wait complete. 2024-08-20T03:05:57.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.283+0000 7f24f001c740 1 -- start start 2024-08-20T03:05:57.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.283+0000 7f24f001c740 1 Processor -- start 2024-08-20T03:05:57.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24f001c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99f3b0 0x5621ec9af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24f001c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 0x5621ec9af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24f001c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec9abc60 0x5621eca2d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24f001c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ad510 con 0x5621ec99fcd0 2024-08-20T03:05:57.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24f001c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81ace40 con 0x5621ec9abc60 2024-08-20T03:05:57.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24f001c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24e81aa080 con 0x5621ec99f3b0 2024-08-20T03:05:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24f001c740 10 client.0 _ll_register_callbacks cb 0x7f24e00125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:05:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24e6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 0x5621ec9af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24e77fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99f3b0 0x5621ec9af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24e6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 0x5621ec9af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35666/0 (socket says 172.21.15.44:35666) 2024-08-20T03:05:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24e6ffd640 1 -- 172.21.15.44:0/3161714277 learned_addr learned my addr 172.21.15.44:0/3161714277 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:05:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24e7fff640 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec9abc60 0x5621eca2d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24e6ffd640 1 -- 172.21.15.44:0/3161714277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99f3b0 msgr2=0x5621ec9af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24e6ffd640 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99f3b0 0x5621ec9af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24e6ffd640 1 -- 172.21.15.44:0/3161714277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec9abc60 msgr2=0x5621eca2d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:05:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24e6ffd640 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec9abc60 0x5621eca2d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24e6ffd640 1 -- 172.21.15.44:0/3161714277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621eca2d7a0 con 0x5621ec99fcd0 2024-08-20T03:05:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24e77fe640 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99f3b0 0x5621ec9af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:05:57.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24e6ffd640 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 0x5621ec9af8c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009380 tx=0x7f24dc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc56aa03ea10933f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:05:57.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:05:57.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.284+0000 7f24c6ffd640 1 -- 172.21.15.44:0/3161714277 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24dc02b060 con 0x5621ec99fcd0 2024-08-20T03:05:57.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24f001c740 1 -- 172.21.15.44:0/3161714277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e0012240 con 0x5621ec99fcd0 2024-08-20T03:05:57.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24c6ffd640 1 -- 172.21.15.44:0/3161714277 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f24dc0052e0 con 0x5621ec99fcd0 2024-08-20T03:05:57.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24c6ffd640 1 -- 172.21.15.44:0/3161714277 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24dc0054e0 con 0x5621ec99fcd0 2024-08-20T03:05:57.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24f001c740 10 client.16455 fetch_fsmap learned FSMap version 30 2024-08-20T03:05:57.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24f001c740 1 -- 172.21.15.44:0/3161714277 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5621eca2eac0 con 0x5621ec99fcd0 2024-08-20T03:05:57.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24c6ffd640 1 -- 172.21.15.44:0/3161714277 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e0012240 con 0x5621ec99fcd0 2024-08-20T03:05:57.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24c6ffd640 1 -- 172.21.15.44:0/3161714277 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f24dc02e020 con 0x5621ec99fcd0 2024-08-20T03:05:57.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24f001c740 10 client.16455 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:05:57.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24f001c740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:05:57.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24f001c740 20 client.16455 populate_metadata read hostname 'smithi044' 2024-08-20T03:05:57.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24f001c740 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5621eca2ef50 0x5621eca31370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:05:57.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24f001c740 4 client.16455 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T03:05:57.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.285+0000 7f24f001c740 1 -- 172.21.15.44:0/3161714277 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5621ec913330 con 0x5621eca2ef50 2024-08-20T03:05:57.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.286+0000 7f24e77fe640 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5621eca2ef50 0x5621eca31370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:05:57.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.286+0000 7f24c6ffd640 10 client.16455 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:57.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.286+0000 7f24e77fe640 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5621eca2ef50 0x5621eca31370 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca5713d10cb89b server_cookie=b12992627bef7ae3 in_seq=0 out_seq=0 2024-08-20T03:05:57.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24c6ffd640 1 -- 172.21.15.44:0/3161714277 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5621ec913330 con 0x5621eca2ef50 2024-08-20T03:05:57.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24c6ffd640 10 client.16455 handle_command_reply: tid=0 2024-08-20T03:05:57.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24f001c740 1 client.16455 shutdown 2024-08-20T03:05:57.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24f001c740 1 -- 172.21.15.44:0/3161714277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 msgr2=0x5621ec9af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:57.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24f001c740 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 0x5621ec9af8c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009380 tx=0x7f24dc003820 comp rx=0 tx=0).stop 2024-08-20T03:05:57.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24f001c740 1 -- 172.21.15.44:0/3161714277 shutdown_connections 2024-08-20T03:05:57.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24f001c740 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5621eca2ef50 0x5621eca31370 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24c6ffd640 0 client.16455 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:05:57.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24f001c740 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621ec9abc60 0x5621eca2d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24f001c740 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5621ec99fcd0 0x5621ec9af8c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24f001c740 1 --2- 172.21.15.44:0/3161714277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621ec99f3b0 0x5621ec9af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24f001c740 1 -- 172.21.15.44:0/3161714277 >> 172.21.15.44:0/3161714277 conn(0x5621ec99d1e0 msgr2=0x5621ec99bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:57.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24f001c740 1 -- 172.21.15.44:0/3161714277 shutdown_connections 2024-08-20T03:05:57.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24f001c740 1 -- 172.21.15.44:0/3161714277 wait complete. 2024-08-20T03:05:57.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.290+0000 7f24f001c740 20 client.16455 trim_cache size 0 max 16384 2024-08-20T03:05:57.296 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:05:57.296 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-08-20T03:05:57.296 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "a24f9f81-df91-4f22-a871-09cc9455fee8", 2024-08-20T03:05:57.297 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-08-20T03:05:57.297 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:05:57.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.291+0000 7f24edc23640 1 -- 172.21.15.44:0/2566881049 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f24d00b3d40 msgr2=0x7f24d00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:57.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.291+0000 7f24edc23640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f24d00b3d40 0x7f24d00b6200 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009fd0 tx=0x7f24e0009500 comp rx=0 tx=0).stop 2024-08-20T03:05:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.291+0000 7f24edc23640 1 -- 172.21.15.44:0/2566881049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e8103870 msgr2=0x7f24e8198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:05:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.291+0000 7f24edc23640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e8103870 0x7f24e8198ff0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f24d802a930 tx=0x7f24d8031a20 comp rx=0 tx=0).stop 2024-08-20T03:05:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.291+0000 7f24edc23640 1 -- 172.21.15.44:0/2566881049 shutdown_connections 2024-08-20T03:05:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.291+0000 7f24edc23640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f24d00b3d40 0x7f24d00b6200 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.291+0000 7f24edc23640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24e810ce20 0x7f24e819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.291+0000 7f24edc23640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24e8104270 0x7f24e8199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.291+0000 7f24edc23640 1 --2- 172.21.15.44:0/2566881049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e8103870 0x7f24e8198ff0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:05:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.291+0000 7f24edc23640 1 -- 172.21.15.44:0/2566881049 >> 172.21.15.44:0/2566881049 conn(0x7f24e80fd400 msgr2=0x7f24e80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:05:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.291+0000 7f24edc23640 1 -- 172.21.15.44:0/2566881049 shutdown_connections 2024-08-20T03:05:57.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:05:57.291+0000 7f24edc23640 1 -- 172.21.15.44:0/2566881049 wait complete. 2024-08-20T03:05:57.477 INFO:tasks.workunit.client.0.smithi044.stdout: 118 1081302 300 740067 277 611274 772 2024-08-20T03:05:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:57 smithi165 ceph-mon[22357]: pgmap v424: 97 pgs: 97 active+clean; 471 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.0 MiB/s wr, 928 op/s 2024-08-20T03:05:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:57 smithi165 ceph-mon[22357]: scrub queued for path: / 2024-08-20T03:05:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:57 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:05:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:57 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:05:57.905 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:57 smithi044 ceph-mon[18790]: pgmap v424: 97 pgs: 97 active+clean; 471 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.0 MiB/s wr, 928 op/s 2024-08-20T03:05:57.905 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:57 smithi044 ceph-mon[18790]: scrub queued for path: / 2024-08-20T03:05:57.905 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:57 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:05:57.905 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:57 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:05:57.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:57 smithi099 ceph-mon[21984]: pgmap v424: 97 pgs: 97 active+clean; 471 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.0 MiB/s wr, 928 op/s 2024-08-20T03:05:57.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:57 smithi099 ceph-mon[21984]: scrub queued for path: / 2024-08-20T03:05:57.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:57 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:05:57.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:57 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:05:58.991 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:05:59.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:59 smithi165 ceph-mon[22357]: pgmap v425: 97 pgs: 97 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.8 MiB/s wr, 1.51k op/s 2024-08-20T03:05:59.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:59 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:05:59.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:05:59 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:05:59.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:59 smithi099 ceph-mon[21984]: pgmap v425: 97 pgs: 97 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.8 MiB/s wr, 1.51k op/s 2024-08-20T03:05:59.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:59 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:05:59.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:05:59 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:00.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:59 smithi044 ceph-mon[18790]: pgmap v425: 97 pgs: 97 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.8 MiB/s wr, 1.51k op/s 2024-08-20T03:06:00.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:59 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:06:00.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:05:59 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:00.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:00 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:06:00.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:00 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:00.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:00 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:06:00.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:00 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:00.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:00 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:06:00.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:00 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:00.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:00 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:06:00.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:00 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:00 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:06:01.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:00 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:00 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:06:01.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:00 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:06:01.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:06:01.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:06:01.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:06:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:06:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: pgmap v426: 97 pgs: 97 active+clean; 468 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.5 MiB/s wr, 1.56k op/s 2024-08-20T03:06:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:06:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:06:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:06:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:01 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:06:01.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: pgmap v426: 97 pgs: 97 active+clean; 468 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.5 MiB/s wr, 1.56k op/s 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:06:01.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:01.916 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:01 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:02.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:06:02.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:02.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:06:02.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:06:02.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:06:02.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:02.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:02.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:02.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:06:02.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:02.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: pgmap v426: 97 pgs: 97 active+clean; 468 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.5 MiB/s wr, 1.56k op/s 2024-08-20T03:06:02.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:06:02.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:02.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:06:02.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:06:02.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:02.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:01 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:06:02.506 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:06:03.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.553+0000 7f6664341640 1 -- 172.21.15.44:0/1901708223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f665c10a260 msgr2=0x7f665c10a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:03.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.553+0000 7f6664341640 1 --2- 172.21.15.44:0/1901708223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f665c10a260 0x7f665c10a6a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6654009ca0 tx=0x7f665402a980 comp rx=0 tx=0).stop 2024-08-20T03:06:03.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.559+0000 7f6664341640 1 -- 172.21.15.44:0/1901708223 shutdown_connections 2024-08-20T03:06:03.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.559+0000 7f6664341640 1 --2- 172.21.15.44:0/1901708223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f665c11bdd0 0x7f665c11e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.559+0000 7f6664341640 1 --2- 172.21.15.44:0/1901708223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f665c10a260 0x7f665c10a6a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.559+0000 7f6664341640 1 --2- 172.21.15.44:0/1901708223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f665c109e80 0x7f665c110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.559+0000 7f6664341640 1 -- 172.21.15.44:0/1901708223 >> 172.21.15.44:0/1901708223 conn(0x7f665c06dcc0 msgr2=0x7f665c06e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:03.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.564+0000 7f6664341640 1 -- 172.21.15.44:0/1901708223 shutdown_connections 2024-08-20T03:06:03.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.565+0000 7f6664341640 1 -- 172.21.15.44:0/1901708223 wait complete. 2024-08-20T03:06:03.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.566+0000 7f6664341640 1 Processor -- start 2024-08-20T03:06:03.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.566+0000 7f6664341640 1 -- start start 2024-08-20T03:06:03.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.566+0000 7f6664341640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f665c109e80 0x7f665c1a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.566+0000 7f6664341640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f665c10a260 0x7f665c1a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.566+0000 7f6664341640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f665c11bdd0 0x7f665c1ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.566+0000 7f6664341640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f665c1219b0 con 0x7f665c109e80 2024-08-20T03:06:03.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.566+0000 7f6664341640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f665c121830 con 0x7f665c10a260 2024-08-20T03:06:03.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.566+0000 7f6664341640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f665c121b30 con 0x7f665c11bdd0 2024-08-20T03:06:03.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.567+0000 7f66618dc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f665c10a260 0x7f665c1a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.567+0000 7f66620dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f665c109e80 0x7f665c1a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.567+0000 7f66618dc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f665c10a260 0x7f665c1a8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:59232/0 (socket says 172.21.15.44:59232) 2024-08-20T03:06:03.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.567+0000 7f66618dc640 1 -- 172.21.15.44:0/3722477664 learned_addr learned my addr 172.21.15.44:0/3722477664 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:03.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.568+0000 7f66618dc640 1 -- 172.21.15.44:0/3722477664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f665c11bdd0 msgr2=0x7f665c1ad820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-20T03:06:03.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.568+0000 7f66618dc640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f665c11bdd0 0x7f665c1ad820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.568+0000 7f66618dc640 1 -- 172.21.15.44:0/3722477664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f665c109e80 msgr2=0x7f665c1a7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:03.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.568+0000 7f66618dc640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f665c109e80 0x7f665c1a7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.568+0000 7f66618dc640 1 -- 172.21.15.44:0/3722477664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c1adff0 con 0x7f665c10a260 2024-08-20T03:06:03.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.568+0000 7f66620dd640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f665c109e80 0x7f665c1a7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:06:03.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.568+0000 7f66618dc640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f665c10a260 0x7f665c1a8480 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f665402af70 tx=0x7f66540308b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:03.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.569+0000 7f66537fe640 1 -- 172.21.15.44:0/3722477664 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654034030 con 0x7f665c10a260 2024-08-20T03:06:03.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.569+0000 7f6664341640 1 -- 172.21.15.44:0/3722477664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c1ae280 con 0x7f665c10a260 2024-08-20T03:06:03.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.569+0000 7f66537fe640 1 -- 172.21.15.44:0/3722477664 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6654004040 con 0x7f665c10a260 2024-08-20T03:06:03.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.569+0000 7f66537fe640 1 -- 172.21.15.44:0/3722477664 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654039660 con 0x7f665c10a260 2024-08-20T03:06:03.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.569+0000 7f6664341640 1 -- 172.21.15.44:0/3722477664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c1b49a0 con 0x7f665c10a260 2024-08-20T03:06:03.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.576+0000 7f666673a740 1 Processor -- start 2024-08-20T03:06:03.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.577+0000 7f66537fe640 1 -- 172.21.15.44:0/3722477664 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6654037020 con 0x7f665c10a260 2024-08-20T03:06:03.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.578+0000 7f666673a740 1 -- start start 2024-08-20T03:06:03.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.578+0000 7f666673a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21336830 0x55da21336c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.578+0000 7f666673a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21337150 0x55da21342ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.578+0000 7f666673a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da213454d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.578+0000 7f666673a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55da212ad8e0 con 0x55da213430e0 2024-08-20T03:06:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.578+0000 7f666673a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55da2129cde0 con 0x55da21337150 2024-08-20T03:06:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.578+0000 7f666673a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55da212a0e20 con 0x55da21336830 2024-08-20T03:06:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.579+0000 7f66618dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da213454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.579+0000 7f66618dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da213454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60878/0 (socket says 172.21.15.44:60878) 2024-08-20T03:06:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.579+0000 7f66618dc640 1 -- 172.21.15.44:0/2639537220 learned_addr learned my addr 172.21.15.44:0/2639537220 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.579+0000 7f66620dd640 1 --2- 172.21.15.44:0/2639537220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21337150 0x55da21342ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.580+0000 7f66537fe640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f66480b3e70 0x7f66480b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.580+0000 7f66537fe640 1 -- 172.21.15.44:0/3722477664 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6654105ec0 con 0x7f665c10a260 2024-08-20T03:06:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.580+0000 7f66628de640 1 --2- 172.21.15.44:0/2639537220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21336830 0x55da21336c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.583+0000 7f66628de640 1 -- 172.21.15.44:0/2639537220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21337150 msgr2=0x55da21342ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:03.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.583+0000 7f66628de640 1 --2- 172.21.15.44:0/2639537220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21337150 0x55da21342ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.583+0000 7f66628de640 1 -- 172.21.15.44:0/2639537220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 msgr2=0x55da213454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:03.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.583+0000 7f66628de640 1 --2- 172.21.15.44:0/2639537220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da213454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.583+0000 7f66628de640 1 -- 172.21.15.44:0/2639537220 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da20ffc650 con 0x55da21336830 2024-08-20T03:06:03.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.583+0000 7f66628de640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f66480b3e70 0x7f66480b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.584+0000 7f66628de640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f66480b3e70 0x7f66480b6330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f664c009fd0 tx=0x7f664c008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:03.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.584+0000 7f66628de640 1 --2- 172.21.15.44:0/2639537220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21336830 0x55da21336c10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f664c009800 tx=0x7f664c009cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8977021da27e0d93 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:03.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.585+0000 7f6650de6640 1 -- 172.21.15.44:0/2639537220 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c00f040 con 0x55da21336830 2024-08-20T03:06:03.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.585+0000 7f6650de6640 1 -- 172.21.15.44:0/2639537220 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f664c020070 con 0x55da21336830 2024-08-20T03:06:03.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.586+0000 7f666673a740 1 -- 172.21.15.44:0/2639537220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21336830 msgr2=0x55da21336c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:03.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.586+0000 7f666673a740 1 --2- 172.21.15.44:0/2639537220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21336830 0x55da21336c10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f664c009800 tx=0x7f664c009cc0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.588+0000 7f666673a740 1 -- 172.21.15.44:0/2639537220 shutdown_connections 2024-08-20T03:06:03.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.588+0000 7f666673a740 1 --2- 172.21.15.44:0/2639537220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da213454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.588+0000 7f666673a740 1 --2- 172.21.15.44:0/2639537220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21337150 0x55da21342ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.588+0000 7f666673a740 1 --2- 172.21.15.44:0/2639537220 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21336830 0x55da21336c10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.588+0000 7f666673a740 1 -- 172.21.15.44:0/2639537220 >> 172.21.15.44:0/2639537220 conn(0x55da21334660 msgr2=0x55da21335800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:03.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.588+0000 7f66620dd640 1 --2- 172.21.15.44:0/2639537220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21337150 0x55da21342ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:06:03.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.592+0000 7f666673a740 1 -- 172.21.15.44:0/2639537220 shutdown_connections 2024-08-20T03:06:03.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.593+0000 7f666673a740 1 -- 172.21.15.44:0/2639537220 wait complete. 2024-08-20T03:06:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.593+0000 7f666673a740 1 -- start start 2024-08-20T03:06:03.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.593+0000 7f666673a740 1 Processor -- start 2024-08-20T03:06:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.594+0000 7f666673a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21336830 0x55da21340070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.594+0000 7f666673a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da213405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.594+0000 7f666673a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 0x55da2134f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.594+0000 7f666673a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55da210ff700 con 0x55da213430e0 2024-08-20T03:06:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.594+0000 7f666673a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55da212237c0 con 0x55da21336830 2024-08-20T03:06:03.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.594+0000 7f666673a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55da21345e30 con 0x55da21340af0 2024-08-20T03:06:03.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.594+0000 7f666673a740 10 client.0 _ll_register_callbacks cb 0x55da212966b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:06:03.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.595+0000 7f66618dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da213405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.595+0000 7f66618dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da213405b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60894/0 (socket says 172.21.15.44:60894) 2024-08-20T03:06:03.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.595+0000 7f66618dc640 1 -- 172.21.15.44:0/4263630780 learned_addr learned my addr 172.21.15.44:0/4263630780 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:03.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.595+0000 7f66620dd640 1 --2- 172.21.15.44:0/4263630780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21336830 0x55da21340070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.595+0000 7f66618dc640 1 -- 172.21.15.44:0/4263630780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 msgr2=0x55da2134f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:03.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.595+0000 7f66618dc640 1 --2- 172.21.15.44:0/4263630780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 0x55da2134f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.595+0000 7f66618dc640 1 -- 172.21.15.44:0/4263630780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21336830 msgr2=0x55da21340070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:03.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.595+0000 7f66618dc640 1 --2- 172.21.15.44:0/4263630780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21336830 0x55da21340070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.595+0000 7f66618dc640 1 -- 172.21.15.44:0/4263630780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da21360760 con 0x55da213430e0 2024-08-20T03:06:03.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.596+0000 7f66618dc640 1 --2- 172.21.15.44:0/4263630780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da213405b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f66540c8b60 tx=0x7f665403a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3956aabb6c6c0ad0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.596+0000 7f6645ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:06:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.596+0000 7f6645ffb640 1 -- 172.21.15.44:0/4263630780 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66540cc430 con 0x55da213430e0 2024-08-20T03:06:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.596+0000 7f666673a740 1 -- 172.21.15.44:0/4263630780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da2124c100 con 0x55da213430e0 2024-08-20T03:06:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.596+0000 7f6645ffb640 1 -- 172.21.15.44:0/4263630780 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f66540b2b80 con 0x55da213430e0 2024-08-20T03:06:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.597+0000 7f6645ffb640 1 -- 172.21.15.44:0/4263630780 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f66540cc250 con 0x55da213430e0 2024-08-20T03:06:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.597+0000 7f666673a740 10 client.16467 fetch_fsmap learned FSMap version 30 2024-08-20T03:06:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.597+0000 7f666673a740 1 -- 172.21.15.44:0/4263630780 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55da21363020 con 0x55da213430e0 2024-08-20T03:06:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.597+0000 7f6645ffb640 1 -- 172.21.15.44:0/4263630780 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66540e3030 con 0x55da213430e0 2024-08-20T03:06:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.598+0000 7f6645ffb640 1 -- 172.21.15.44:0/4263630780 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6654041070 con 0x55da213430e0 2024-08-20T03:06:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.598+0000 7f666673a740 10 client.16467 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:06:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.598+0000 7f666673a740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:06:03.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.598+0000 7f666673a740 20 client.16467 populate_metadata read hostname 'smithi044' 2024-08-20T03:06:03.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.598+0000 7f666673a740 1 --2- 172.21.15.44:0/4263630780 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55da213646e0 0x55da21366b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.598+0000 7f666673a740 4 client.16467 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:06:03.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.598+0000 7f666673a740 1 -- 172.21.15.44:0/4263630780 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da212af830 con 0x55da213646e0 2024-08-20T03:06:03.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.598+0000 7f66620dd640 1 --2- 172.21.15.44:0/4263630780 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55da213646e0 0x55da21366b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.599+0000 7f66620dd640 1 --2- 172.21.15.44:0/4263630780 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55da213646e0 0x55da21366b80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=227a286779fd4b15 server_cookie=77bcdd5841de0826 in_seq=0 out_seq=0 2024-08-20T03:06:03.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.600+0000 7f6645ffb640 10 client.16467 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:03.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.606+0000 7f6645ffb640 1 -- 172.21.15.44:0/4263630780 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da212af830 con 0x55da213646e0 2024-08-20T03:06:03.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.606+0000 7f6645ffb640 10 client.16467 handle_command_reply: tid=0 2024-08-20T03:06:03.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.606+0000 7f666673a740 1 client.16467 shutdown 2024-08-20T03:06:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.612+0000 7f666673a740 1 -- 172.21.15.44:0/4263630780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 msgr2=0x55da213405b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.612+0000 7f666673a740 1 --2- 172.21.15.44:0/4263630780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da213405b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f66540c8b60 tx=0x7f665403a000 comp rx=0 tx=0).stop 2024-08-20T03:06:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.613+0000 7f666673a740 1 -- 172.21.15.44:0/4263630780 shutdown_connections 2024-08-20T03:06:03.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.613+0000 7f666673a740 1 --2- 172.21.15.44:0/4263630780 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55da213646e0 0x55da21366b80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.613+0000 7f666673a740 1 --2- 172.21.15.44:0/4263630780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 0x55da2134f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.613+0000 7f666673a740 1 --2- 172.21.15.44:0/4263630780 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da213405b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.613+0000 7f666673a740 1 --2- 172.21.15.44:0/4263630780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21336830 0x55da21340070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.613+0000 7f666673a740 1 -- 172.21.15.44:0/4263630780 >> 172.21.15.44:0/4263630780 conn(0x55da21334660 msgr2=0x55da2133a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:03.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.614+0000 7f6645ffb640 0 client.16467 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:03.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.614+0000 7f666673a740 1 -- 172.21.15.44:0/4263630780 shutdown_connections 2024-08-20T03:06:03.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.615+0000 7f666673a740 1 -- 172.21.15.44:0/4263630780 wait complete. 2024-08-20T03:06:03.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.618+0000 7f666673a740 20 client.16467 trim_cache size 0 max 16384 2024-08-20T03:06:03.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.663+0000 7f666673a740 1 Processor -- start 2024-08-20T03:06:03.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.663+0000 7f666673a740 1 -- start start 2024-08-20T03:06:03.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.663+0000 7f666673a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21336830 0x55da21335d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.664+0000 7f666673a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 0x55da2134f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.664+0000 7f666673a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da2133f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.664+0000 7f666673a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f664000e720 con 0x55da213430e0 2024-08-20T03:06:03.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.664+0000 7f666673a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6640010320 con 0x55da21336830 2024-08-20T03:06:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.664+0000 7f666673a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f664000edf0 con 0x55da21340af0 2024-08-20T03:06:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.664+0000 7f66618dc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 0x55da2134f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.664+0000 7f66618dc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 0x55da2134f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:36004/0 (socket says 172.21.15.44:36004) 2024-08-20T03:06:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.664+0000 7f66618dc640 1 -- 172.21.15.44:0/1950948093 learned_addr learned my addr 172.21.15.44:0/1950948093 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.665+0000 7f66620dd640 1 --2- 172.21.15.44:0/1950948093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21336830 0x55da21335d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.665+0000 7f66628de640 1 --2- 172.21.15.44:0/1950948093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da2133f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.665+0000 7f66618dc640 1 -- 172.21.15.44:0/1950948093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21336830 msgr2=0x55da21335d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.665+0000 7f66618dc640 1 --2- 172.21.15.44:0/1950948093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21336830 0x55da21335d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.665+0000 7f66618dc640 1 -- 172.21.15.44:0/1950948093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 msgr2=0x55da2133f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.665+0000 7f66618dc640 1 --2- 172.21.15.44:0/1950948093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da2133f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.665+0000 7f66618dc640 1 -- 172.21.15.44:0/1950948093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da2134f540 con 0x55da21340af0 2024-08-20T03:06:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.665+0000 7f66628de640 1 --2- 172.21.15.44:0/1950948093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da2133f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:06:03.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.667+0000 7f66618dc640 1 --2- 172.21.15.44:0/1950948093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 0x55da2134f000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f66540cd140 tx=0x7f665402fa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c412efa85e1f982c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:03.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.668+0000 7f6647fff640 1 -- 172.21.15.44:0/1950948093 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66540cc430 con 0x55da21340af0 2024-08-20T03:06:03.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.668+0000 7f6647fff640 1 -- 172.21.15.44:0/1950948093 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66540ec070 con 0x55da21340af0 2024-08-20T03:06:03.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.668+0000 7f6647fff640 1 -- 172.21.15.44:0/1950948093 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66540e73e0 con 0x55da21340af0 2024-08-20T03:06:03.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.672+0000 7f666673a740 1 -- 172.21.15.44:0/1950948093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 msgr2=0x55da2134f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:03.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.672+0000 7f666673a740 1 --2- 172.21.15.44:0/1950948093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 0x55da2134f000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f66540cd140 tx=0x7f665402fa40 comp rx=0 tx=0).stop 2024-08-20T03:06:03.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.677+0000 7f666673a740 1 -- 172.21.15.44:0/1950948093 shutdown_connections 2024-08-20T03:06:03.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.677+0000 7f666673a740 1 --2- 172.21.15.44:0/1950948093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da213430e0 0x55da2133f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.677+0000 7f666673a740 1 --2- 172.21.15.44:0/1950948093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 0x55da2134f000 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.677+0000 7f666673a740 1 --2- 172.21.15.44:0/1950948093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da21336830 0x55da21335d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.677+0000 7f666673a740 1 -- 172.21.15.44:0/1950948093 >> 172.21.15.44:0/1950948093 conn(0x55da21334660 msgr2=0x55da21341ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:03.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.680+0000 7f666673a740 1 -- 172.21.15.44:0/1950948093 shutdown_connections 2024-08-20T03:06:03.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.681+0000 7f666673a740 1 -- 172.21.15.44:0/1950948093 wait complete. 2024-08-20T03:06:03.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.681+0000 7f666673a740 1 -- start start 2024-08-20T03:06:03.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.682+0000 7f666673a740 1 Processor -- start 2024-08-20T03:06:03.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.682+0000 7f666673a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da21336830 0x55da21346720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f66620dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da21336830 0x55da21346720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f666673a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 0x55da21346c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f66620dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da21336830 0x55da21346720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60914/0 (socket says 172.21.15.44:60914) 2024-08-20T03:06:03.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f666673a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da213430e0 0x55da213cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f666673a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f664000fb90 con 0x55da21336830 2024-08-20T03:06:03.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f666673a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f664000f4c0 con 0x55da213430e0 2024-08-20T03:06:03.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f666673a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f664000c700 con 0x55da21340af0 2024-08-20T03:06:03.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f66620dd640 1 -- 172.21.15.44:0/2389675004 learned_addr learned my addr 172.21.15.44:0/2389675004 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:03.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f666673a740 10 client.0 _ll_register_callbacks cb 0x7f66540c8f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:06:03.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f66620dd640 1 -- 172.21.15.44:0/2389675004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 msgr2=0x55da21346c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:06:03.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f66620dd640 1 --2- 172.21.15.44:0/2389675004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 0x55da21346c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f66620dd640 1 -- 172.21.15.44:0/2389675004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da213430e0 msgr2=0x55da213cd410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:06:03.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f66620dd640 1 --2- 172.21.15.44:0/2389675004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da213430e0 0x55da213cd410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.683+0000 7f66620dd640 1 -- 172.21.15.44:0/2389675004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da213cd9f0 con 0x55da21336830 2024-08-20T03:06:03.694 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:03 smithi044 ceph-mon[18790]: pgmap v427: 97 pgs: 97 active+clean; 456 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 1.66k op/s 2024-08-20T03:06:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.695+0000 7f66620dd640 1 --2- 172.21.15.44:0/2389675004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da21336830 0x55da21346720 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6658006fd0 tx=0x7f6658018820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5882088f9ff98b03 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.695+0000 7f6645ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:06:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.696+0000 7f6645ffb640 1 -- 172.21.15.44:0/2389675004 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6658016630 con 0x55da21336830 2024-08-20T03:06:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.696+0000 7f6645ffb640 1 -- 172.21.15.44:0/2389675004 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f66580167d0 con 0x55da21336830 2024-08-20T03:06:03.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.696+0000 7f666673a740 1 -- 172.21.15.44:0/2389675004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66540ec070 con 0x55da21336830 2024-08-20T03:06:03.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.698+0000 7f6645ffb640 1 -- 172.21.15.44:0/2389675004 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f665802b050 con 0x55da21336830 2024-08-20T03:06:03.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.699+0000 7f666673a740 10 client.16473 fetch_fsmap learned FSMap version 30 2024-08-20T03:06:03.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.699+0000 7f666673a740 1 -- 172.21.15.44:0/2389675004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55da213ced70 con 0x55da21336830 2024-08-20T03:06:03.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.701+0000 7f6645ffb640 1 -- 172.21.15.44:0/2389675004 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66540ec070 con 0x55da21336830 2024-08-20T03:06:03.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.701+0000 7f6645ffb640 1 -- 172.21.15.44:0/2389675004 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f66580269f0 con 0x55da21336830 2024-08-20T03:06:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.701+0000 7f666673a740 10 client.16473 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:06:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.701+0000 7f666673a740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:06:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.701+0000 7f666673a740 20 client.16473 populate_metadata read hostname 'smithi044' 2024-08-20T03:06:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.701+0000 7f666673a740 1 --2- 172.21.15.44:0/2389675004 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55da213cf5e0 0x55da213d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.701+0000 7f666673a740 4 client.16473 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:06:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.701+0000 7f666673a740 1 -- 172.21.15.44:0/2389675004 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da212ab2d0 con 0x55da213cf5e0 2024-08-20T03:06:03.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.702+0000 7f66618dc640 1 --2- 172.21.15.44:0/2389675004 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55da213cf5e0 0x55da213d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:03.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.705+0000 7f66618dc640 1 --2- 172.21.15.44:0/2389675004 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55da213cf5e0 0x55da213d1a00 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5e7f53ac78cdbd6 server_cookie=1a573ddc41e013e9 in_seq=0 out_seq=0 2024-08-20T03:06:03.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.707+0000 7f6645ffb640 10 client.16473 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:03.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.712+0000 7f6645ffb640 1 -- 172.21.15.44:0/2389675004 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55da212ab2d0 con 0x55da213cf5e0 2024-08-20T03:06:03.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.712+0000 7f6645ffb640 10 client.16473 handle_command_reply: tid=0 2024-08-20T03:06:03.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.712+0000 7f666673a740 1 client.16473 shutdown 2024-08-20T03:06:03.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.714+0000 7f666673a740 1 -- 172.21.15.44:0/2389675004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da21336830 msgr2=0x55da21346720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:03.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.714+0000 7f666673a740 1 --2- 172.21.15.44:0/2389675004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da21336830 0x55da21346720 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6658006fd0 tx=0x7f6658018820 comp rx=0 tx=0).stop 2024-08-20T03:06:03.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.716+0000 7f666673a740 1 -- 172.21.15.44:0/2389675004 shutdown_connections 2024-08-20T03:06:03.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.716+0000 7f666673a740 1 --2- 172.21.15.44:0/2389675004 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55da213cf5e0 0x55da213d1a00 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.716+0000 7f666673a740 1 --2- 172.21.15.44:0/2389675004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da213430e0 0x55da213cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.716+0000 7f666673a740 1 --2- 172.21.15.44:0/2389675004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55da21340af0 0x55da21346c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.716+0000 7f666673a740 1 --2- 172.21.15.44:0/2389675004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da21336830 0x55da21346720 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.716+0000 7f666673a740 1 -- 172.21.15.44:0/2389675004 >> 172.21.15.44:0/2389675004 conn(0x55da21334660 msgr2=0x55da213399e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:03.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.716+0000 7f6645ffb640 0 client.16473 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:03.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.716+0000 7f666673a740 1 -- 172.21.15.44:0/2389675004 shutdown_connections 2024-08-20T03:06:03.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.716+0000 7f666673a740 1 -- 172.21.15.44:0/2389675004 wait complete. 2024-08-20T03:06:03.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.716+0000 7f666673a740 20 client.16473 trim_cache size 0 max 16384 2024-08-20T03:06:03.720 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:06:03.721 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "scrub active (17155 inodes in the stack)", 2024-08-20T03:06:03.721 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-08-20T03:06:03.721 INFO:teuthology.orchestra.run.smithi044.stdout: "a24f9f81-df91-4f22-a871-09cc9455fee8": { 2024-08-20T03:06:03.721 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-08-20T03:06:03.721 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "a24f9f81-df91-4f22-a871-09cc9455fee8", 2024-08-20T03:06:03.721 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-08-20T03:06:03.721 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:06:03.721 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:06:03.721 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:06:03.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.717+0000 7f6664341640 1 -- 172.21.15.44:0/3722477664 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f66480b3e70 msgr2=0x7f66480b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:03.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.717+0000 7f6664341640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f66480b3e70 0x7f66480b6330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f664c009fd0 tx=0x7f664c008040 comp rx=0 tx=0).stop 2024-08-20T03:06:03.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.717+0000 7f6664341640 1 -- 172.21.15.44:0/3722477664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f665c10a260 msgr2=0x7f665c1a8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:03.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.717+0000 7f6664341640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f665c10a260 0x7f665c1a8480 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f665402af70 tx=0x7f66540308b0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.717+0000 7f6664341640 1 -- 172.21.15.44:0/3722477664 shutdown_connections 2024-08-20T03:06:03.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.717+0000 7f6664341640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f66480b3e70 0x7f66480b6330 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.717+0000 7f6664341640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f665c11bdd0 0x7f665c1ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.717+0000 7f6664341640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f665c10a260 0x7f665c1a8480 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.717+0000 7f6664341640 1 --2- 172.21.15.44:0/3722477664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f665c109e80 0x7f665c1a7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:03.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.717+0000 7f6664341640 1 -- 172.21.15.44:0/3722477664 >> 172.21.15.44:0/3722477664 conn(0x7f665c06dcc0 msgr2=0x7f665c11c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:03.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.717+0000 7f6664341640 1 -- 172.21.15.44:0/3722477664 shutdown_connections 2024-08-20T03:06:03.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:03.717+0000 7f6664341640 1 -- 172.21.15.44:0/3722477664 wait complete. 2024-08-20T03:06:03.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:03 smithi165 ceph-mon[22357]: pgmap v427: 97 pgs: 97 active+clean; 456 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 1.66k op/s 2024-08-20T03:06:03.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:03 smithi099 ceph-mon[21984]: pgmap v427: 97 pgs: 97 active+clean; 456 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.5 MiB/s wr, 1.66k op/s 2024-08-20T03:06:04.687 INFO:tasks.cephfs.filesystem:scrub status for tag:a24f9f81-df91-4f22-a871-09cc9455fee8 - {'path': '/', 'tag': 'a24f9f81-df91-4f22-a871-09cc9455fee8', 'options': 'recursive,force'} 2024-08-20T03:06:05.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:05 smithi165 ceph-mon[22357]: pgmap v428: 97 pgs: 97 active+clean; 460 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.7 MiB/s wr, 1.62k op/s 2024-08-20T03:06:05.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:05 smithi099 ceph-mon[21984]: pgmap v428: 97 pgs: 97 active+clean; 460 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.7 MiB/s wr, 1.62k op/s 2024-08-20T03:06:06.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:05 smithi044 ceph-mon[18790]: pgmap v428: 97 pgs: 97 active+clean; 460 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.7 MiB/s wr, 1.62k op/s 2024-08-20T03:06:07.478 INFO:tasks.workunit.client.0.smithi044.stdout: 125 2170547 428 1511885 488 1217974 1702 2024-08-20T03:06:07.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:07 smithi165 ceph-mon[22357]: pgmap v429: 97 pgs: 97 active+clean; 468 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.0 MiB/s wr, 1.61k op/s 2024-08-20T03:06:07.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:07 smithi099 ceph-mon[21984]: pgmap v429: 97 pgs: 97 active+clean; 468 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.0 MiB/s wr, 1.61k op/s 2024-08-20T03:06:08.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:07 smithi044 ceph-mon[18790]: pgmap v429: 97 pgs: 97 active+clean; 468 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.0 MiB/s wr, 1.61k op/s 2024-08-20T03:06:09.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:09 smithi165 ceph-mon[22357]: pgmap v430: 97 pgs: 97 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.3 MiB/s wr, 1.12k op/s 2024-08-20T03:06:09.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:09 smithi099 ceph-mon[21984]: pgmap v430: 97 pgs: 97 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.3 MiB/s wr, 1.12k op/s 2024-08-20T03:06:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.944+0000 7fb71054a640 1 -- 172.21.15.44:0/390337100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb708104230 msgr2=0x7fb70810c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.944+0000 7fb71054a640 1 --2- 172.21.15.44:0/390337100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb708104230 0x7fb70810c710 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009900 tx=0x7fb6f802a410 comp rx=0 tx=0).stop 2024-08-20T03:06:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.944+0000 7fb71054a640 1 -- 172.21.15.44:0/390337100 shutdown_connections 2024-08-20T03:06:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.944+0000 7fb71054a640 1 --2- 172.21.15.44:0/390337100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb70810cd30 0x7fb70810f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.944+0000 7fb71054a640 1 --2- 172.21.15.44:0/390337100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb708104230 0x7fb70810c710 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.944+0000 7fb71054a640 1 --2- 172.21.15.44:0/390337100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb708103910 0x7fb708103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.944+0000 7fb71054a640 1 -- 172.21.15.44:0/390337100 >> 172.21.15.44:0/390337100 conn(0x7fb708078620 msgr2=0x7fb7080ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.944+0000 7fb71054a640 1 -- 172.21.15.44:0/390337100 shutdown_connections 2024-08-20T03:06:09.947 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.944+0000 7fb71054a640 1 -- 172.21.15.44:0/390337100 wait complete. 2024-08-20T03:06:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.945+0000 7fb71054a640 1 Processor -- start 2024-08-20T03:06:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.945+0000 7fb71054a640 1 -- start start 2024-08-20T03:06:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb71054a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb708103910 0x7fb708198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb71054a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb708104230 0x7fb7081994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb71054a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70810cd30 0x7fb70819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb71054a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081129c0 con 0x7fb708103910 2024-08-20T03:06:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb71054a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb708112840 con 0x7fb708104230 2024-08-20T03:06:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb71054a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb708112b40 con 0x7fb70810cd30 2024-08-20T03:06:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70e2e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb708103910 0x7fb708198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70eae7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70810cd30 0x7fb70819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70dae5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb708104230 0x7fb7081994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70eae7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70810cd30 0x7fb70819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46100/0 (socket says 172.21.15.44:46100) 2024-08-20T03:06:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70e2e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb708103910 0x7fb708198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47476/0 (socket says 172.21.15.44:47476) 2024-08-20T03:06:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70eae7640 1 -- 172.21.15.44:0/2979216340 learned_addr learned my addr 172.21.15.44:0/2979216340 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70dae5640 1 -- 172.21.15.44:0/2979216340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70810cd30 msgr2=0x7fb70819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70dae5640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70810cd30 0x7fb70819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70dae5640 1 -- 172.21.15.44:0/2979216340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb708103910 msgr2=0x7fb708198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70dae5640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb708103910 0x7fb708198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70dae5640 1 -- 172.21.15.44:0/2979216340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70819f000 con 0x7fb708104230 2024-08-20T03:06:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70e2e6640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb708103910 0x7fb708198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:06:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.946+0000 7fb70dae5640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb708104230 0x7fb7081994a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6f802a930 tx=0x7fb6f8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.947+0000 7fb6ff7fe640 1 -- 172.21.15.44:0/2979216340 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f803f070 con 0x7fb708104230 2024-08-20T03:06:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.947+0000 7fb71054a640 1 -- 172.21.15.44:0/2979216340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70819f290 con 0x7fb708104230 2024-08-20T03:06:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.947+0000 7fb6ff7fe640 1 -- 172.21.15.44:0/2979216340 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f8003ca0 con 0x7fb708104230 2024-08-20T03:06:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.947+0000 7fb6ff7fe640 1 -- 172.21.15.44:0/2979216340 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8035070 con 0x7fb708104230 2024-08-20T03:06:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.947+0000 7fb71054a640 1 -- 172.21.15.44:0/2979216340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081a59b0 con 0x7fb708104230 2024-08-20T03:06:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.950+0000 7fb71341b740 1 Processor -- start 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.950+0000 7fb71341b740 1 -- start start 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.950+0000 7fb6ff7fe640 1 -- 172.21.15.44:0/2979216340 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb6f803b460 con 0x7fb708104230 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.950+0000 7fb71341b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e8bf0 0x559b660e8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.950+0000 7fb71341b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.950+0000 7fb71341b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660f98b0 0x559b660fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.950+0000 7fb71341b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559b6605e360 con 0x559b660f98b0 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.950+0000 7fb71341b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559b6604d860 con 0x559b660e8bf0 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.950+0000 7fb71341b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559b660518a0 con 0x559b660e9510 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70dae5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660f98b0 0x559b660fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70dae5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660f98b0 0x559b660fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47478/0 (socket says 172.21.15.44:47478) 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70dae5640 1 -- 172.21.15.44:0/3885016622 learned_addr learned my addr 172.21.15.44:0/3885016622 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70eae7640 1 --2- 172.21.15.44:0/3885016622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e8bf0 0x559b660e8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70e2e6640 1 --2- 172.21.15.44:0/3885016622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70dae5640 1 -- 172.21.15.44:0/3885016622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 msgr2=0x559b660f9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70dae5640 1 --2- 172.21.15.44:0/3885016622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70dae5640 1 -- 172.21.15.44:0/3885016622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e8bf0 msgr2=0x559b660e8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70dae5640 1 --2- 172.21.15.44:0/3885016622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e8bf0 0x559b660e8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70dae5640 1 -- 172.21.15.44:0/3885016622 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b65ff4e40 con 0x559b660f98b0 2024-08-20T03:06:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70eae7640 1 --2- 172.21.15.44:0/3885016622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e8bf0 0x559b660e8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:06:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70e2e6640 1 --2- 172.21.15.44:0/3885016622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:06:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.951+0000 7fb70dae5640 1 --2- 172.21.15.44:0/3885016622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660f98b0 0x559b660fbca0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8101af0 tx=0x7fb6f810a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801627d3dccb0c90 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.952+0000 7fb6fcbfe640 1 -- 172.21.15.44:0/3885016622 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8104870 con 0x559b660f98b0 2024-08-20T03:06:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.952+0000 7fb6fcbfe640 1 -- 172.21.15.44:0/3885016622 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f8111030 con 0x559b660f98b0 2024-08-20T03:06:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.952+0000 7fb6fcbfe640 1 -- 172.21.15.44:0/3885016622 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8116640 con 0x559b660f98b0 2024-08-20T03:06:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.952+0000 7fb6ff7fe640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb6f40b3d40 0x7fb6f40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.952+0000 7fb70eae7640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb6f40b3d40 0x7fb6f40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.953+0000 7fb6ff7fe640 1 -- 172.21.15.44:0/2979216340 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb6f81043c0 con 0x7fb708104230 2024-08-20T03:06:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.953+0000 7fb71341b740 1 -- 172.21.15.44:0/3885016622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660f98b0 msgr2=0x559b660fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.953+0000 7fb71341b740 1 --2- 172.21.15.44:0/3885016622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660f98b0 0x559b660fbca0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8101af0 tx=0x7fb6f810a660 comp rx=0 tx=0).stop 2024-08-20T03:06:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.953+0000 7fb71341b740 1 -- 172.21.15.44:0/3885016622 shutdown_connections 2024-08-20T03:06:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.953+0000 7fb71341b740 1 --2- 172.21.15.44:0/3885016622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660f98b0 0x559b660fbca0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.953+0000 7fb71341b740 1 --2- 172.21.15.44:0/3885016622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.953+0000 7fb71341b740 1 --2- 172.21.15.44:0/3885016622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e8bf0 0x559b660e8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.953+0000 7fb71341b740 1 -- 172.21.15.44:0/3885016622 >> 172.21.15.44:0/3885016622 conn(0x559b660e6a20 msgr2=0x559b660e7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.953+0000 7fb70eae7640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb6f40b3d40 0x7fb6f40b6200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb700005c40 tx=0x7fb700009760 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.953+0000 7fb71341b740 1 -- 172.21.15.44:0/3885016622 shutdown_connections 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.953+0000 7fb71341b740 1 -- 172.21.15.44:0/3885016622 wait complete. 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.953+0000 7fb71341b740 1 -- start start 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb71341b740 1 Processor -- start 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb71341b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 0x559b660f9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb71341b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb71341b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660f98b0 0x559b660f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb71341b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559b65f86ab0 con 0x559b660e8bf0 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb71341b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559b65fd4780 con 0x559b660f98b0 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb71341b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559b660fc350 con 0x559b660e9510 2024-08-20T03:06:09.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb71341b740 10 client.0 _ll_register_callbacks cb 0x559b65d5ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb70e2e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 0x559b660f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb70e2e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 0x559b660f9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47492/0 (socket says 172.21.15.44:47492) 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb70e2e6640 1 -- 172.21.15.44:0/2860675310 learned_addr learned my addr 172.21.15.44:0/2860675310 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb70dae5640 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb70eae7640 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660f98b0 0x559b660f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb70e2e6640 1 -- 172.21.15.44:0/2860675310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 msgr2=0x559b660f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb70e2e6640 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb70e2e6640 1 -- 172.21.15.44:0/2860675310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660f98b0 msgr2=0x559b660f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb70e2e6640 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660f98b0 0x559b660f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb70e2e6640 1 -- 172.21.15.44:0/2860675310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b65db0790 con 0x559b660e8bf0 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.954+0000 7fb70dae5640 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.955+0000 7fb70e2e6640 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 0x559b660f9110 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb704012d20 tx=0x7fb70400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afeb6e1a538eddc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.955+0000 7fb6f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.955+0000 7fb6f1ffb640 1 -- 172.21.15.44:0/2860675310 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7040120d0 con 0x559b660e8bf0 2024-08-20T03:06:09.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.955+0000 7fb6f1ffb640 1 -- 172.21.15.44:0/2860675310 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb704010430 con 0x559b660e8bf0 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.955+0000 7fb71341b740 1 -- 172.21.15.44:0/2860675310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559b65ffce00 con 0x559b660e8bf0 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.955+0000 7fb6f1ffb640 1 -- 172.21.15.44:0/2860675310 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb704012270 con 0x559b660e8bf0 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.955+0000 7fb71341b740 10 client.16485 fetch_fsmap learned FSMap version 30 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.955+0000 7fb71341b740 1 -- 172.21.15.44:0/2860675310 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b65e29620 con 0x559b660e8bf0 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.955+0000 7fb6f1ffb640 1 -- 172.21.15.44:0/2860675310 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559b65ffce00 con 0x559b660e8bf0 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.955+0000 7fb6f1ffb640 1 -- 172.21.15.44:0/2860675310 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fb70401c760 con 0x559b660e8bf0 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.956+0000 7fb71341b740 10 client.16485 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.956+0000 7fb71341b740 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.956+0000 7fb71341b740 20 client.16485 populate_metadata read hostname 'smithi044' 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.956+0000 7fb71341b740 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559b66109060 0x559b6610b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.956+0000 7fb71341b740 4 client.16485 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.956+0000 7fb71341b740 1 -- 172.21.15.44:0/2860675310 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b660602b0 con 0x559b66109060 2024-08-20T03:06:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.956+0000 7fb70dae5640 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559b66109060 0x559b6610b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.956+0000 7fb6f1ffb640 10 client.16485 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.956+0000 7fb70dae5640 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559b66109060 0x559b6610b4c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be51eaa16ec14798 server_cookie=9e6627020f7ffc3f in_seq=0 out_seq=0 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb6f1ffb640 1 -- 172.21.15.44:0/2860675310 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b660602b0 con 0x559b66109060 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb6f1ffb640 10 client.16485 handle_command_reply: tid=0 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb71341b740 1 client.16485 shutdown 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb71341b740 1 -- 172.21.15.44:0/2860675310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 msgr2=0x559b660f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb71341b740 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 0x559b660f9110 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb704012d20 tx=0x7fb70400f790 comp rx=0 tx=0).stop 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb71341b740 1 -- 172.21.15.44:0/2860675310 shutdown_connections 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb71341b740 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559b66109060 0x559b6610b4c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb71341b740 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660f98b0 0x559b660f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb71341b740 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb71341b740 1 --2- 172.21.15.44:0/2860675310 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 0x559b660f9110 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb71341b740 1 -- 172.21.15.44:0/2860675310 >> 172.21.15.44:0/2860675310 conn(0x559b660e6a20 msgr2=0x559b660ec5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb6f1ffb640 0 client.16485 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb71341b740 1 -- 172.21.15.44:0/2860675310 shutdown_connections 2024-08-20T03:06:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb71341b740 1 -- 172.21.15.44:0/2860675310 wait complete. 2024-08-20T03:06:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.961+0000 7fb71341b740 20 client.16485 trim_cache size 0 max 16384 2024-08-20T03:06:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb71341b740 1 Processor -- start 2024-08-20T03:06:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb71341b740 1 -- start start 2024-08-20T03:06:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb71341b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e6a20 0x559b66105430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb71341b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 0x559b660f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb71341b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb71341b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081abfc0 con 0x559b660e8bf0 2024-08-20T03:06:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb71341b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081adbc0 con 0x559b660e6a20 2024-08-20T03:06:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb71341b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081ac690 con 0x559b660e9510 2024-08-20T03:06:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb70dae5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 0x559b660f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb70dae5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 0x559b660f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47504/0 (socket says 172.21.15.44:47504) 2024-08-20T03:06:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb70dae5640 1 -- 172.21.15.44:0/1710934410 learned_addr learned my addr 172.21.15.44:0/1710934410 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb70eae7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb70e2e6640 1 --2- 172.21.15.44:0/1710934410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e6a20 0x559b66105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb70dae5640 1 -- 172.21.15.44:0/1710934410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 msgr2=0x559b660f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb70dae5640 1 --2- 172.21.15.44:0/1710934410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb70dae5640 1 -- 172.21.15.44:0/1710934410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e6a20 msgr2=0x559b66105430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb70dae5640 1 --2- 172.21.15.44:0/1710934410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e6a20 0x559b66105430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.977+0000 7fb70dae5640 1 -- 172.21.15.44:0/1710934410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b660f6330 con 0x559b660e8bf0 2024-08-20T03:06:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.978+0000 7fb70dae5640 1 --2- 172.21.15.44:0/1710934410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 0x559b660f4db0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8101c30 tx=0x7fb6f8106580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c5b8c502b4f36d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.978+0000 7fb6f3fff640 1 -- 172.21.15.44:0/1710934410 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8048070 con 0x559b660e8bf0 2024-08-20T03:06:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.978+0000 7fb6f3fff640 1 -- 172.21.15.44:0/1710934410 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f8111040 con 0x559b660e8bf0 2024-08-20T03:06:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.978+0000 7fb6f3fff640 1 -- 172.21.15.44:0/1710934410 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f80cb0f0 con 0x559b660e8bf0 2024-08-20T03:06:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.978+0000 7fb71341b740 1 -- 172.21.15.44:0/1710934410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 msgr2=0x559b660f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.978+0000 7fb71341b740 1 --2- 172.21.15.44:0/1710934410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 0x559b660f4db0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8101c30 tx=0x7fb6f8106580 comp rx=0 tx=0).stop 2024-08-20T03:06:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 -- 172.21.15.44:0/1710934410 shutdown_connections 2024-08-20T03:06:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 --2- 172.21.15.44:0/1710934410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e9510 0x559b660f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 --2- 172.21.15.44:0/1710934410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e8bf0 0x559b660f4db0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 --2- 172.21.15.44:0/1710934410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e6a20 0x559b66105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 -- 172.21.15.44:0/1710934410 >> 172.21.15.44:0/1710934410 conn(0x559b660f9270 msgr2=0x559b660e7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 -- 172.21.15.44:0/1710934410 shutdown_connections 2024-08-20T03:06:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 -- 172.21.15.44:0/1710934410 wait complete. 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 -- start start 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 Processor -- start 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e6a20 0x559b66101790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e8bf0 0x559b66101d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e9510 0x559b66110450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081ad430 con 0x559b660e9510 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081acd60 con 0x559b660e6a20 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081a9fa0 con 0x559b660e8bf0 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb71341b740 10 client.0 _ll_register_callbacks cb 0x7fb704009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb70eae7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e9510 0x559b66110450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.979+0000 7fb70eae7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e9510 0x559b66110450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47520/0 (socket says 172.21.15.44:47520) 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb70e2e6640 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e6a20 0x559b66101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb70eae7640 1 -- 172.21.15.44:0/1693375151 learned_addr learned my addr 172.21.15.44:0/1693375151 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb70eae7640 1 -- 172.21.15.44:0/1693375151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e8bf0 msgr2=0x559b66101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb70eae7640 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e8bf0 0x559b66101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb70eae7640 1 -- 172.21.15.44:0/1693375151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e6a20 msgr2=0x559b66101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb70eae7640 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e6a20 0x559b66101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb70eae7640 1 -- 172.21.15.44:0/1693375151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b660e9340 con 0x559b660e9510 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb70e2e6640 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e6a20 0x559b66101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb70eae7640 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e9510 0x559b66110450 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb700010ba0 tx=0x7fb700004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6480b8f66bdd0871 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb6f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb6f1ffb640 1 -- 172.21.15.44:0/1693375151 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70001e9a0 con 0x559b660e9510 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb6f1ffb640 1 -- 172.21.15.44:0/1693375151 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb70001eb40 con 0x559b660e9510 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb71341b740 1 -- 172.21.15.44:0/1693375151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6f8111040 con 0x559b660e9510 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb6f1ffb640 1 -- 172.21.15.44:0/1693375151 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb70001ed40 con 0x559b660e9510 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb71341b740 10 client.16491 fetch_fsmap learned FSMap version 30 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.980+0000 7fb71341b740 1 -- 172.21.15.44:0/1693375151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559b661737b0 con 0x559b660e9510 2024-08-20T03:06:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.981+0000 7fb6f1ffb640 1 -- 172.21.15.44:0/1693375151 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f8111040 con 0x559b660e9510 2024-08-20T03:06:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.981+0000 7fb6f1ffb640 1 -- 172.21.15.44:0/1693375151 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fb700031030 con 0x559b660e9510 2024-08-20T03:06:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.981+0000 7fb71341b740 10 client.16491 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:06:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.981+0000 7fb71341b740 10 client.16491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:06:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.981+0000 7fb71341b740 20 client.16491 populate_metadata read hostname 'smithi044' 2024-08-20T03:06:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.981+0000 7fb71341b740 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559b660f98b0 0x559b661760e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.981+0000 7fb71341b740 4 client.16491 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:06:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.981+0000 7fb71341b740 1 -- 172.21.15.44:0/1693375151 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b6605bd50 con 0x559b660f98b0 2024-08-20T03:06:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.981+0000 7fb70e2e6640 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559b660f98b0 0x559b661760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.981+0000 7fb6f1ffb640 10 client.16491 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:09.981+0000 7fb70e2e6640 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559b660f98b0 0x559b661760e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2601c37be3b9459 server_cookie=43c9054a75d1f32a in_seq=0 out_seq=0 2024-08-20T03:06:10.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:09 smithi044 ceph-mon[18790]: pgmap v430: 97 pgs: 97 active+clean; 472 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.3 MiB/s wr, 1.12k op/s 2024-08-20T03:06:11.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:11 smithi165 ceph-mon[22357]: pgmap v431: 97 pgs: 97 active+clean; 473 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 523 op/s 2024-08-20T03:06:11.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:11 smithi099 ceph-mon[21984]: pgmap v431: 97 pgs: 97 active+clean; 473 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 523 op/s 2024-08-20T03:06:12.039 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:11 smithi044 ceph-mon[18790]: pgmap v431: 97 pgs: 97 active+clean; 473 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 523 op/s 2024-08-20T03:06:13.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:13 smithi165 ceph-mon[22357]: pgmap v432: 97 pgs: 97 active+clean; 485 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 735 op/s 2024-08-20T03:06:13.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:13 smithi099 ceph-mon[21984]: pgmap v432: 97 pgs: 97 active+clean; 485 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 735 op/s 2024-08-20T03:06:14.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:13 smithi044 ceph-mon[18790]: pgmap v432: 97 pgs: 97 active+clean; 485 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 735 op/s 2024-08-20T03:06:15.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:15 smithi165 ceph-mon[22357]: pgmap v433: 97 pgs: 97 active+clean; 497 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 875 op/s 2024-08-20T03:06:15.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:15 smithi099 ceph-mon[21984]: pgmap v433: 97 pgs: 97 active+clean; 497 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 875 op/s 2024-08-20T03:06:16.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:15 smithi044 ceph-mon[18790]: pgmap v433: 97 pgs: 97 active+clean; 497 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 875 op/s 2024-08-20T03:06:17.478 INFO:tasks.workunit.client.0.smithi044.stdout: 128 1394199 150 974382 257 771344 1047 2024-08-20T03:06:17.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:17 smithi165 ceph-mon[22357]: pgmap v434: 97 pgs: 97 active+clean; 533 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.7 MiB/s wr, 1.31k op/s 2024-08-20T03:06:17.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:17 smithi099 ceph-mon[21984]: pgmap v434: 97 pgs: 97 active+clean; 533 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.7 MiB/s wr, 1.31k op/s 2024-08-20T03:06:18.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:17 smithi044 ceph-mon[18790]: pgmap v434: 97 pgs: 97 active+clean; 533 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 6.7 MiB/s wr, 1.31k op/s 2024-08-20T03:06:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.877+0000 7fb6f1ffb640 1 -- 172.21.15.44:0/1693375151 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b6605bd50 con 0x559b660f98b0 2024-08-20T03:06:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.877+0000 7fb6f1ffb640 10 client.16491 handle_command_reply: tid=0 2024-08-20T03:06:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.878+0000 7fb71341b740 1 client.16491 shutdown 2024-08-20T03:06:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.878+0000 7fb71341b740 1 -- 172.21.15.44:0/1693375151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e9510 msgr2=0x559b66110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.878+0000 7fb71341b740 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e9510 0x559b66110450 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb700010ba0 tx=0x7fb700004a40 comp rx=0 tx=0).stop 2024-08-20T03:06:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.878+0000 7fb71341b740 1 -- 172.21.15.44:0/1693375151 shutdown_connections 2024-08-20T03:06:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.878+0000 7fb71341b740 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559b660f98b0 0x559b661760e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.878+0000 7fb71341b740 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559b660e9510 0x559b66110450 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.878+0000 7fb71341b740 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559b660e8bf0 0x559b66101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.878+0000 7fb71341b740 1 --2- 172.21.15.44:0/1693375151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559b660e6a20 0x559b66101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.878+0000 7fb71341b740 1 -- 172.21.15.44:0/1693375151 >> 172.21.15.44:0/1693375151 conn(0x559b660f9270 msgr2=0x559b660e7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.878+0000 7fb6f1ffb640 0 client.16491 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.879+0000 7fb71341b740 1 -- 172.21.15.44:0/1693375151 shutdown_connections 2024-08-20T03:06:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.879+0000 7fb71341b740 1 -- 172.21.15.44:0/1693375151 wait complete. 2024-08-20T03:06:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.879+0000 7fb71341b740 20 client.16491 trim_cache size 0 max 16384 2024-08-20T03:06:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:06:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:06:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:06:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:06:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.912+0000 7fb71054a640 1 -- 172.21.15.44:0/2979216340 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb6f40b3d40 msgr2=0x7fb6f40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.912+0000 7fb71054a640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb6f40b3d40 0x7fb6f40b6200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb700005c40 tx=0x7fb700009760 comp rx=0 tx=0).stop 2024-08-20T03:06:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.912+0000 7fb71054a640 1 -- 172.21.15.44:0/2979216340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb708104230 msgr2=0x7fb7081994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.912+0000 7fb71054a640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb708104230 0x7fb7081994a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6f802a930 tx=0x7fb6f8004270 comp rx=0 tx=0).stop 2024-08-20T03:06:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.912+0000 7fb71054a640 1 -- 172.21.15.44:0/2979216340 shutdown_connections 2024-08-20T03:06:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.912+0000 7fb71054a640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb6f40b3d40 0x7fb6f40b6200 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.913+0000 7fb71054a640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb70810cd30 0x7fb70819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.913+0000 7fb71054a640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb708104230 0x7fb7081994a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.913+0000 7fb71054a640 1 --2- 172.21.15.44:0/2979216340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb708103910 0x7fb708198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.913+0000 7fb71054a640 1 -- 172.21.15.44:0/2979216340 >> 172.21.15.44:0/2979216340 conn(0x7fb708078620 msgr2=0x7fb7080ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.913+0000 7fb71054a640 1 -- 172.21.15.44:0/2979216340 shutdown_connections 2024-08-20T03:06:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:18.913+0000 7fb71054a640 1 -- 172.21.15.44:0/2979216340 wait complete. 2024-08-20T03:06:19.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:19 smithi165 ceph-mon[22357]: pgmap v435: 97 pgs: 97 active+clean; 547 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 1.39k op/s 2024-08-20T03:06:19.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:19 smithi099 ceph-mon[21984]: pgmap v435: 97 pgs: 97 active+clean; 547 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 1.39k op/s 2024-08-20T03:06:20.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:19 smithi044 ceph-mon[18790]: pgmap v435: 97 pgs: 97 active+clean; 547 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 1.39k op/s 2024-08-20T03:06:21.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:21 smithi165 ceph-mon[22357]: pgmap v436: 97 pgs: 97 active+clean; 539 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 1.29k op/s 2024-08-20T03:06:21.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:21 smithi099 ceph-mon[21984]: pgmap v436: 97 pgs: 97 active+clean; 539 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 1.29k op/s 2024-08-20T03:06:22.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:21 smithi044 ceph-mon[18790]: pgmap v436: 97 pgs: 97 active+clean; 539 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.9 MiB/s wr, 1.29k op/s 2024-08-20T03:06:23.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:23 smithi099 ceph-mon[21984]: pgmap v437: 97 pgs: 97 active+clean; 525 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.2 MiB/s wr, 1.43k op/s 2024-08-20T03:06:24.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:23 smithi044 ceph-mon[18790]: pgmap v437: 97 pgs: 97 active+clean; 525 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.2 MiB/s wr, 1.43k op/s 2024-08-20T03:06:24.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:23 smithi165 ceph-mon[22357]: pgmap v437: 97 pgs: 97 active+clean; 525 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 8.2 MiB/s wr, 1.43k op/s 2024-08-20T03:06:25.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:25 smithi099 ceph-mon[21984]: pgmap v438: 97 pgs: 97 active+clean; 517 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 1.19k op/s 2024-08-20T03:06:26.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:25 smithi044 ceph-mon[18790]: pgmap v438: 97 pgs: 97 active+clean; 517 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 1.19k op/s 2024-08-20T03:06:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:25 smithi165 ceph-mon[22357]: pgmap v438: 97 pgs: 97 active+clean; 517 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 1.19k op/s 2024-08-20T03:06:27.478 INFO:tasks.workunit.client.0.smithi044.stdout: 136 2134885 516 1520131 520 1202032 1693 2024-08-20T03:06:27.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:27 smithi099 ceph-mon[21984]: pgmap v439: 97 pgs: 97 active+clean; 522 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.4 MiB/s wr, 978 op/s 2024-08-20T03:06:28.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:27 smithi044 ceph-mon[18790]: pgmap v439: 97 pgs: 97 active+clean; 522 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.4 MiB/s wr, 978 op/s 2024-08-20T03:06:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:27 smithi165 ceph-mon[22357]: pgmap v439: 97 pgs: 97 active+clean; 522 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.4 MiB/s wr, 978 op/s 2024-08-20T03:06:29.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:29 smithi099 ceph-mon[21984]: pgmap v440: 97 pgs: 97 active+clean; 528 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 541 op/s 2024-08-20T03:06:30.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:29 smithi044 ceph-mon[18790]: pgmap v440: 97 pgs: 97 active+clean; 528 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 541 op/s 2024-08-20T03:06:30.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:29 smithi165 ceph-mon[22357]: pgmap v440: 97 pgs: 97 active+clean; 528 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 541 op/s 2024-08-20T03:06:31.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:30 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:06:31.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:30 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:06:31.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:30 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:06:32.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:31 smithi044 ceph-mon[18790]: pgmap v441: 97 pgs: 97 active+clean; 532 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 355 op/s 2024-08-20T03:06:32.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:31 smithi165 ceph-mon[22357]: pgmap v441: 97 pgs: 97 active+clean; 532 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 355 op/s 2024-08-20T03:06:32.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:31 smithi099 ceph-mon[21984]: pgmap v441: 97 pgs: 97 active+clean; 532 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 355 op/s 2024-08-20T03:06:33.039 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:32 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:06:33.039 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:32 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:06:33.039 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:32 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:06:33.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:32 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:06:33.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:32 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:06:33.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:32 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:06:33.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:32 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:06:33.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:32 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:06:33.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:32 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:06:34.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:33 smithi044 ceph-mon[18790]: pgmap v442: 97 pgs: 97 active+clean; 538 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 417 op/s 2024-08-20T03:06:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:33 smithi165 ceph-mon[22357]: pgmap v442: 97 pgs: 97 active+clean; 538 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 417 op/s 2024-08-20T03:06:34.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:33 smithi099 ceph-mon[21984]: pgmap v442: 97 pgs: 97 active+clean; 538 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 417 op/s 2024-08-20T03:06:34.688 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:06:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.771+0000 7f4292085640 1 -- 172.21.15.44:0/1854809710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f428c079b90 msgr2=0x7f428c07c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.771+0000 7f4292085640 1 --2- 172.21.15.44:0/1854809710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f428c079b90 0x7f428c07c050 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4284009ca0 tx=0x7f428402ba00 comp rx=0 tx=0).stop 2024-08-20T03:06:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.771+0000 7f4292085640 1 -- 172.21.15.44:0/1854809710 shutdown_connections 2024-08-20T03:06:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.771+0000 7f4292085640 1 --2- 172.21.15.44:0/1854809710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f428c079b90 0x7f428c07c050 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.771+0000 7f4292085640 1 --2- 172.21.15.44:0/1854809710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f428c074bb0 0x7f428c074ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.771+0000 7f4292085640 1 --2- 172.21.15.44:0/1854809710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f428c10a540 0x7f428c10a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.771+0000 7f4292085640 1 -- 172.21.15.44:0/1854809710 >> 172.21.15.44:0/1854809710 conn(0x7f428c06d8e0 msgr2=0x7f428c06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.777+0000 7f4292085640 1 -- 172.21.15.44:0/1854809710 shutdown_connections 2024-08-20T03:06:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.796+0000 7f4292085640 1 -- 172.21.15.44:0/1854809710 wait complete. 2024-08-20T03:06:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.796+0000 7f4292085640 1 Processor -- start 2024-08-20T03:06:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.797+0000 7f4292085640 1 -- start start 2024-08-20T03:06:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.797+0000 7f4292085640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f428c074bb0 0x7f428c080840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.797+0000 7f4292085640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f428c079b90 0x7f428c080d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.797+0000 7f4292085640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f428c10a540 0x7f428c084fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.797+0000 7f4292085640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f428c07ec80 con 0x7f428c074bb0 2024-08-20T03:06:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.797+0000 7f4292085640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f428c07eb00 con 0x7f428c079b90 2024-08-20T03:06:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.797+0000 7f4292085640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f428c07ee00 con 0x7f428c10a540 2024-08-20T03:06:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.798+0000 7f428bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f428c10a540 0x7f428c084fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.798+0000 7f428bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f428c10a540 0x7f428c084fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59082/0 (socket says 172.21.15.44:59082) 2024-08-20T03:06:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.798+0000 7f428bfff640 1 -- 172.21.15.44:0/479678273 learned_addr learned my addr 172.21.15.44:0/479678273 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.798+0000 7f428affd640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f428c079b90 0x7f428c080d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.799+0000 7f428bfff640 1 -- 172.21.15.44:0/479678273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f428c079b90 msgr2=0x7f428c080d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.799+0000 7f428bfff640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f428c079b90 0x7f428c080d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.799+0000 7f428bfff640 1 -- 172.21.15.44:0/479678273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f428c074bb0 msgr2=0x7f428c080840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.799+0000 7f428bfff640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f428c074bb0 0x7f428c080840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.799+0000 7f428bfff640 1 -- 172.21.15.44:0/479678273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c0857b0 con 0x7f428c10a540 2024-08-20T03:06:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.799+0000 7f428bfff640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f428c10a540 0x7f428c084fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f428402f950 tx=0x7f428402f980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.816+0000 7f4288ff9640 1 -- 172.21.15.44:0/479678273 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284047070 con 0x7f428c10a540 2024-08-20T03:06:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.816+0000 7f4292085640 1 -- 172.21.15.44:0/479678273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c085a40 con 0x7f428c10a540 2024-08-20T03:06:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.816+0000 7f4292085640 1 -- 172.21.15.44:0/479678273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c130ee0 con 0x7f428c10a540 2024-08-20T03:06:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.816+0000 7f4288ff9640 1 -- 172.21.15.44:0/479678273 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4284030870 con 0x7f428c10a540 2024-08-20T03:06:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.816+0000 7f4288ff9640 1 -- 172.21.15.44:0/479678273 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284034040 con 0x7f428c10a540 2024-08-20T03:06:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.821+0000 7f4294e75740 1 Processor -- start 2024-08-20T03:06:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.823+0000 7f4294e75740 1 -- start start 2024-08-20T03:06:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.823+0000 7f4294e75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d96eb30 0x556b7d96ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.823+0000 7f4294e75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96f450 0x556b7d97aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.823+0000 7f4294e75740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d97d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.823+0000 7f4294e75740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556b7d8e4360 con 0x556b7d97b3e0 2024-08-20T03:06:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.823+0000 7f4294e75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556b7d8d3860 con 0x556b7d96f450 2024-08-20T03:06:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.823+0000 7f4294e75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556b7d8d78a0 con 0x556b7d96eb30 2024-08-20T03:06:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.831+0000 7f428b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96f450 0x556b7d97aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.832+0000 7f428b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96f450 0x556b7d97aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34520/0 (socket says 172.21.15.44:34520) 2024-08-20T03:06:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.832+0000 7f428b7fe640 1 -- 172.21.15.44:0/760904085 learned_addr learned my addr 172.21.15.44:0/760904085 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.832+0000 7f428affd640 1 --2- 172.21.15.44:0/760904085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d97d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.832+0000 7f428affd640 1 --2- 172.21.15.44:0/760904085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d96eb30 0x556b7d96ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.832+0000 7f428b7fe640 1 -- 172.21.15.44:0/760904085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d96eb30 msgr2=0x556b7d96ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.832+0000 7f428b7fe640 1 --2- 172.21.15.44:0/760904085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d96eb30 0x556b7d96ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.832+0000 7f428b7fe640 1 -- 172.21.15.44:0/760904085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 msgr2=0x556b7d97d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.832+0000 7f428b7fe640 1 --2- 172.21.15.44:0/760904085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d97d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.836+0000 7f4288ff9640 1 -- 172.21.15.44:0/479678273 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4284037070 con 0x7f428c10a540 2024-08-20T03:06:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.837+0000 7f4288ff9640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42740b3f50 0x7f42740b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.838+0000 7f4288ff9640 1 -- 172.21.15.44:0/479678273 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4284105af0 con 0x7f428c10a540 2024-08-20T03:06:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.844+0000 7f428affd640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42740b3f50 0x7f42740b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.856+0000 7f428b7fe640 1 -- 172.21.15.44:0/760904085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b7d87ae40 con 0x556b7d96f450 2024-08-20T03:06:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.857+0000 7f428b7fe640 1 --2- 172.21.15.44:0/760904085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96f450 0x556b7d97aea0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f427c00bb10 tx=0x7f427c00d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f927c03f5ce36c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.885+0000 7f4271ffb640 1 -- 172.21.15.44:0/760904085 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c016070 con 0x556b7d96f450 2024-08-20T03:06:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.886+0000 7f4271ffb640 1 -- 172.21.15.44:0/760904085 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427c00dce0 con 0x556b7d96f450 2024-08-20T03:06:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.888+0000 7f4294e75740 1 -- 172.21.15.44:0/760904085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96f450 msgr2=0x556b7d97aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.888+0000 7f4294e75740 1 --2- 172.21.15.44:0/760904085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96f450 0x556b7d97aea0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f427c00bb10 tx=0x7f427c00d2e0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.888+0000 7f4294e75740 1 -- 172.21.15.44:0/760904085 shutdown_connections 2024-08-20T03:06:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.888+0000 7f4294e75740 1 --2- 172.21.15.44:0/760904085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d97d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.888+0000 7f4294e75740 1 --2- 172.21.15.44:0/760904085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96f450 0x556b7d97aea0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.888+0000 7f4294e75740 1 --2- 172.21.15.44:0/760904085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d96eb30 0x556b7d96ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.888+0000 7f4294e75740 1 -- 172.21.15.44:0/760904085 >> 172.21.15.44:0/760904085 conn(0x556b7d96c960 msgr2=0x556b7d96db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.891+0000 7f4294e75740 1 -- 172.21.15.44:0/760904085 shutdown_connections 2024-08-20T03:06:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.891+0000 7f4294e75740 1 -- 172.21.15.44:0/760904085 wait complete. 2024-08-20T03:06:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.892+0000 7f4294e75740 1 -- start start 2024-08-20T03:06:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.892+0000 7f4294e75740 1 Processor -- start 2024-08-20T03:06:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.892+0000 7f4294e75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 0x556b7d97aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.892+0000 7f4294e75740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d977f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.893+0000 7f4294e75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d978440 0x556b7d9788c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.893+0000 7f4294e75740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556b7d80cab0 con 0x556b7d97b3e0 2024-08-20T03:06:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.893+0000 7f4294e75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556b7d85a780 con 0x556b7d96eb30 2024-08-20T03:06:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.893+0000 7f4294e75740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556b7d97e110 con 0x556b7d978440 2024-08-20T03:06:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.893+0000 7f4294e75740 10 client.0 _ll_register_callbacks cb 0x556b7d5e2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:06:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.893+0000 7f428b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 0x556b7d97aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.893+0000 7f428bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d978440 0x556b7d9788c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.893+0000 7f428bfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d978440 0x556b7d9788c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59102/0 (socket says 172.21.15.44:59102) 2024-08-20T03:06:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.893+0000 7f428bfff640 1 -- 172.21.15.44:0/3757857434 learned_addr learned my addr 172.21.15.44:0/3757857434 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.894+0000 7f428affd640 1 --2- 172.21.15.44:0/3757857434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d977f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.894+0000 7f428b7fe640 1 -- 172.21.15.44:0/3757857434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d978440 msgr2=0x556b7d9788c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.894+0000 7f428b7fe640 1 --2- 172.21.15.44:0/3757857434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d978440 0x556b7d9788c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.894+0000 7f428b7fe640 1 -- 172.21.15.44:0/3757857434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 msgr2=0x556b7d977f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.894+0000 7f428b7fe640 1 --2- 172.21.15.44:0/3757857434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d977f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.894+0000 7f428b7fe640 1 -- 172.21.15.44:0/3757857434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b7d636790 con 0x556b7d96eb30 2024-08-20T03:06:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.898+0000 7f428affd640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42740b3f50 0x7f42740b6410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f42800058c0 tx=0x7f4280005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.918+0000 7f428b7fe640 1 --2- 172.21.15.44:0/3757857434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 0x556b7d97aac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f427c00bae0 tx=0x7f427c00b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5190991e1227f6ef server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.924+0000 7f426affd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:06:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.925+0000 7f426affd640 1 -- 172.21.15.44:0/3757857434 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c016070 con 0x556b7d96eb30 2024-08-20T03:06:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.925+0000 7f426affd640 1 -- 172.21.15.44:0/3757857434 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f427c00dce0 con 0x556b7d96eb30 2024-08-20T03:06:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.925+0000 7f4294e75740 1 -- 172.21.15.44:0/3757857434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b7d882e00 con 0x556b7d96eb30 2024-08-20T03:06:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.930+0000 7f426affd640 1 -- 172.21.15.44:0/3757857434 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f427c004af0 con 0x556b7d96eb30 2024-08-20T03:06:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.930+0000 7f4294e75740 10 client.25970 fetch_fsmap learned FSMap version 30 2024-08-20T03:06:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.930+0000 7f4294e75740 1 -- 172.21.15.44:0/3757857434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b7d6af620 con 0x556b7d96eb30 2024-08-20T03:06:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.931+0000 7f426affd640 1 -- 172.21.15.44:0/3757857434 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556b7d882e00 con 0x556b7d96eb30 2024-08-20T03:06:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.931+0000 7f426affd640 1 -- 172.21.15.44:0/3757857434 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f427c02b050 con 0x556b7d96eb30 2024-08-20T03:06:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.931+0000 7f4294e75740 10 client.25970 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:06:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.931+0000 7f4294e75740 10 client.25970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:06:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.931+0000 7f4294e75740 20 client.25970 populate_metadata read hostname 'smithi044' 2024-08-20T03:06:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.931+0000 7f4294e75740 1 --2- 172.21.15.44:0/3757857434 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556b7d98b4d0 0x556b7d98d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.931+0000 7f4294e75740 4 client.25970 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:06:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.931+0000 7f4294e75740 1 -- 172.21.15.44:0/3757857434 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b7d8e62b0 con 0x556b7d98b4d0 2024-08-20T03:06:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.931+0000 7f428affd640 1 --2- 172.21.15.44:0/3757857434 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556b7d98b4d0 0x556b7d98d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.933+0000 7f428affd640 1 --2- 172.21.15.44:0/3757857434 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556b7d98b4d0 0x556b7d98d930 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e8854a296b84117 server_cookie=b1dc6cbbb55e2454 in_seq=0 out_seq=0 2024-08-20T03:06:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.933+0000 7f426affd640 10 client.25970 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.937+0000 7f426affd640 1 -- 172.21.15.44:0/3757857434 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b7d8e62b0 con 0x556b7d98b4d0 2024-08-20T03:06:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.937+0000 7f426affd640 10 client.25970 handle_command_reply: tid=0 2024-08-20T03:06:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.937+0000 7f4294e75740 1 client.25970 shutdown 2024-08-20T03:06:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.937+0000 7f4294e75740 1 -- 172.21.15.44:0/3757857434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 msgr2=0x556b7d97aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.937+0000 7f4294e75740 1 --2- 172.21.15.44:0/3757857434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 0x556b7d97aac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f427c00bae0 tx=0x7f427c00b7b0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.937+0000 7f4294e75740 1 -- 172.21.15.44:0/3757857434 shutdown_connections 2024-08-20T03:06:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.937+0000 7f4294e75740 1 --2- 172.21.15.44:0/3757857434 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556b7d98b4d0 0x556b7d98d930 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.937+0000 7f4294e75740 1 --2- 172.21.15.44:0/3757857434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d978440 0x556b7d9788c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.937+0000 7f4294e75740 1 --2- 172.21.15.44:0/3757857434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d977f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.937+0000 7f4294e75740 1 --2- 172.21.15.44:0/3757857434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 0x556b7d97aac0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.937+0000 7f4294e75740 1 -- 172.21.15.44:0/3757857434 >> 172.21.15.44:0/3757857434 conn(0x556b7d96c960 msgr2=0x556b7d971ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.938+0000 7f426affd640 0 client.25970 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.954+0000 7f4294e75740 1 -- 172.21.15.44:0/3757857434 shutdown_connections 2024-08-20T03:06:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.954+0000 7f4294e75740 1 -- 172.21.15.44:0/3757857434 wait complete. 2024-08-20T03:06:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:34.960+0000 7f4294e75740 20 client.25970 trim_cache size 0 max 16384 2024-08-20T03:06:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.178+0000 7f4294e75740 1 Processor -- start 2024-08-20T03:06:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.178+0000 7f4294e75740 1 -- start start 2024-08-20T03:06:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.179+0000 7f4294e75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 0x556b7d96ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.179+0000 7f4294e75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d978440 0x556b7d976760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.179+0000 7f428b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 0x556b7d96ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.179+0000 7f428b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 0x556b7d96ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34538/0 (socket says 172.21.15.44:34538) 2024-08-20T03:06:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.179+0000 7f428b7fe640 1 -- 172.21.15.44:0/291474171 learned_addr learned my addr 172.21.15.44:0/291474171 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.179+0000 7f428affd640 1 --2- 172.21.15.44:0/291474171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d978440 0x556b7d976760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.179+0000 7f4294e75740 1 --2- 172.21.15.44:0/291474171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d976ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.180+0000 7f428bfff640 1 --2- 172.21.15.44:0/291474171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d976ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.180+0000 7f4294e75740 1 -- 172.21.15.44:0/291474171 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f427800e760 con 0x556b7d97b3e0 2024-08-20T03:06:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.180+0000 7f4294e75740 1 -- 172.21.15.44:0/291474171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4278010360 con 0x556b7d96eb30 2024-08-20T03:06:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.180+0000 7f4294e75740 1 -- 172.21.15.44:0/291474171 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f427800ee30 con 0x556b7d978440 2024-08-20T03:06:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.181+0000 7f428b7fe640 1 -- 172.21.15.44:0/291474171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d978440 msgr2=0x556b7d976760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.181+0000 7f428b7fe640 1 --2- 172.21.15.44:0/291474171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d978440 0x556b7d976760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.181+0000 7f428b7fe640 1 -- 172.21.15.44:0/291474171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 msgr2=0x556b7d976ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.181+0000 7f428b7fe640 1 --2- 172.21.15.44:0/291474171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d976ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.181+0000 7f428b7fe640 1 -- 172.21.15.44:0/291474171 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b7d978c30 con 0x556b7d96eb30 2024-08-20T03:06:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.181+0000 7f428b7fe640 1 --2- 172.21.15.44:0/291474171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 0x556b7d96ef10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f427c005820 tx=0x7f427c010600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f8e0af80eb06f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.210+0000 7f42717fa640 1 -- 172.21.15.44:0/291474171 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c031070 con 0x556b7d96eb30 2024-08-20T03:06:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.210+0000 7f42717fa640 1 -- 172.21.15.44:0/291474171 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427c01e030 con 0x556b7d96eb30 2024-08-20T03:06:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.210+0000 7f42717fa640 1 -- 172.21.15.44:0/291474171 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c016070 con 0x556b7d96eb30 2024-08-20T03:06:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.211+0000 7f4294e75740 1 -- 172.21.15.44:0/291474171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 msgr2=0x556b7d96ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.211+0000 7f4294e75740 1 --2- 172.21.15.44:0/291474171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 0x556b7d96ef10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f427c005820 tx=0x7f427c010600 comp rx=0 tx=0).stop 2024-08-20T03:06:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.218+0000 7f4294e75740 1 -- 172.21.15.44:0/291474171 shutdown_connections 2024-08-20T03:06:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.218+0000 7f4294e75740 1 --2- 172.21.15.44:0/291474171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d976ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.218+0000 7f4294e75740 1 --2- 172.21.15.44:0/291474171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d978440 0x556b7d976760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.218+0000 7f4294e75740 1 --2- 172.21.15.44:0/291474171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d96eb30 0x556b7d96ef10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.218+0000 7f4294e75740 1 -- 172.21.15.44:0/291474171 >> 172.21.15.44:0/291474171 conn(0x556b7d96c960 msgr2=0x556b7d973190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.218+0000 7f4294e75740 1 -- 172.21.15.44:0/291474171 shutdown_connections 2024-08-20T03:06:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.218+0000 7f4294e75740 1 -- 172.21.15.44:0/291474171 wait complete. 2024-08-20T03:06:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.219+0000 7f4294e75740 1 -- start start 2024-08-20T03:06:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.220+0000 7f4294e75740 1 Processor -- start 2024-08-20T03:06:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.236+0000 7f4294e75740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d96eb30 0x556b7d983860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.236+0000 7f4294e75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d978440 0x556b7d974100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.236+0000 7f428b7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d96eb30 0x556b7d983860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.236+0000 7f428b7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d96eb30 0x556b7d983860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59108/0 (socket says 172.21.15.44:59108) 2024-08-20T03:06:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.236+0000 7f428b7fe640 1 -- 172.21.15.44:0/2864597134 learned_addr learned my addr 172.21.15.44:0/2864597134 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.236+0000 7f428affd640 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d978440 0x556b7d974100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.237+0000 7f4294e75740 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d974640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.237+0000 7f4294e75740 1 -- 172.21.15.44:0/2864597134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f427800fbd0 con 0x556b7d97b3e0 2024-08-20T03:06:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.237+0000 7f4294e75740 1 -- 172.21.15.44:0/2864597134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f427800f500 con 0x556b7d978440 2024-08-20T03:06:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.237+0000 7f4294e75740 1 -- 172.21.15.44:0/2864597134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f427800c740 con 0x556b7d96eb30 2024-08-20T03:06:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.237+0000 7f4294e75740 10 client.0 _ll_register_callbacks cb 0x7f428000e6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:06:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.237+0000 7f428affd640 1 -- 172.21.15.44:0/2864597134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d96eb30 msgr2=0x556b7d983860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.237+0000 7f428affd640 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d96eb30 0x556b7d983860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.237+0000 7f428affd640 1 -- 172.21.15.44:0/2864597134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 msgr2=0x556b7d974640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.237+0000 7f428affd640 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d974640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.237+0000 7f428affd640 1 -- 172.21.15.44:0/2864597134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b7d974bb0 con 0x556b7d978440 2024-08-20T03:06:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.240+0000 7f428affd640 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d978440 0x556b7d974100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x556b7d96dcf0 tx=0x7f4280004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e28463c5396aa2b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.240+0000 7f426affd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:06:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.240+0000 7f426affd640 1 -- 172.21.15.44:0/2864597134 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428001fa20 con 0x556b7d978440 2024-08-20T03:06:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.241+0000 7f426affd640 1 -- 172.21.15.44:0/2864597134 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f428001fbc0 con 0x556b7d978440 2024-08-20T03:06:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.243+0000 7f4294e75740 1 -- 172.21.15.44:0/2864597134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f427c01e030 con 0x556b7d978440 2024-08-20T03:06:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.249+0000 7f426affd640 1 -- 172.21.15.44:0/2864597134 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f428001fdc0 con 0x556b7d978440 2024-08-20T03:06:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.249+0000 7f4294e75740 10 client.25976 fetch_fsmap learned FSMap version 30 2024-08-20T03:06:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.249+0000 7f4294e75740 1 -- 172.21.15.44:0/2864597134 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b7d975e80 con 0x556b7d978440 2024-08-20T03:06:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.249+0000 7f426affd640 1 -- 172.21.15.44:0/2864597134 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4280039910 con 0x556b7d978440 2024-08-20T03:06:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.250+0000 7f426affd640 1 -- 172.21.15.44:0/2864597134 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4280037020 con 0x556b7d978440 2024-08-20T03:06:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.250+0000 7f4294e75740 10 client.25976 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:06:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.250+0000 7f4294e75740 10 client.25976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:06:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.250+0000 7f4294e75740 20 client.25976 populate_metadata read hostname 'smithi044' 2024-08-20T03:06:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.250+0000 7f4294e75740 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556b7d9f5710 0x556b7d9f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.250+0000 7f4294e75740 4 client.25976 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:06:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.250+0000 7f4294e75740 1 -- 172.21.15.44:0/2864597134 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f427c01e030 con 0x556b7d9f5710 2024-08-20T03:06:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.253+0000 7f428b7fe640 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556b7d9f5710 0x556b7d9f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:35.256 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.253+0000 7f428b7fe640 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556b7d9f5710 0x556b7d9f7bd0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70cdabb1da517368 server_cookie=7031e331f5affdc8 in_seq=0 out_seq=0 2024-08-20T03:06:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:35.256+0000 7f426affd640 10 client.25976 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:35.721 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:35 smithi044 ceph-mon[18790]: pgmap v443: 97 pgs: 97 active+clean; 554 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 344 op/s 2024-08-20T03:06:36.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:35 smithi165 ceph-mon[22357]: pgmap v443: 97 pgs: 97 active+clean; 554 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 344 op/s 2024-08-20T03:06:36.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:35 smithi099 ceph-mon[21984]: pgmap v443: 97 pgs: 97 active+clean; 554 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 344 op/s 2024-08-20T03:06:36.295 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:06:37.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.076+0000 7fb04fa3b640 1 -- 172.21.15.44:0/804164730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb04811bdd0 msgr2=0x7fb04811e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:37.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.076+0000 7fb04fa3b640 1 --2- 172.21.15.44:0/804164730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb04811bdd0 0x7fb04811e1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb040009be0 tx=0x7fb04002a770 comp rx=0 tx=0).stop 2024-08-20T03:06:37.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.079+0000 7fb04fa3b640 1 -- 172.21.15.44:0/804164730 shutdown_connections 2024-08-20T03:06:37.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.079+0000 7fb04fa3b640 1 --2- 172.21.15.44:0/804164730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb04811bdd0 0x7fb04811e1c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.079+0000 7fb04fa3b640 1 --2- 172.21.15.44:0/804164730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb04810a260 0x7fb04810a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.079+0000 7fb04fa3b640 1 --2- 172.21.15.44:0/804164730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb048109e80 0x7fb048110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.079+0000 7fb04fa3b640 1 -- 172.21.15.44:0/804164730 >> 172.21.15.44:0/804164730 conn(0x7fb04806dcc0 msgr2=0x7fb04806e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:37.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.082+0000 7fb04fa3b640 1 -- 172.21.15.44:0/804164730 shutdown_connections 2024-08-20T03:06:37.087 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.082+0000 7fb04fa3b640 1 -- 172.21.15.44:0/804164730 wait complete. 2024-08-20T03:06:37.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.083+0000 7fb04fa3b640 1 Processor -- start 2024-08-20T03:06:37.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.083+0000 7fb04fa3b640 1 -- start start 2024-08-20T03:06:37.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.084+0000 7fb04fa3b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb048109e80 0x7fb048117550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.084+0000 7fb04fa3b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb04810a260 0x7fb048117a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.088 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.084+0000 7fb04fa3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb04811bdd0 0x7fb0481a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.084+0000 7fb04fa3b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb048113df0 con 0x7fb048109e80 2024-08-20T03:06:37.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.084+0000 7fb04fa3b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb048113c70 con 0x7fb04811bdd0 2024-08-20T03:06:37.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.084+0000 7fb04fa3b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb048113f70 con 0x7fb04810a260 2024-08-20T03:06:37.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.085+0000 7fb04dfd8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb04811bdd0 0x7fb0481a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.085+0000 7fb04dfd8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb04811bdd0 0x7fb0481a4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34556/0 (socket says 172.21.15.44:34556) 2024-08-20T03:06:37.089 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.085+0000 7fb04dfd8640 1 -- 172.21.15.44:0/710940344 learned_addr learned my addr 172.21.15.44:0/710940344 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:37.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.089+0000 7fb04dfd8640 1 -- 172.21.15.44:0/710940344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb04810a260 msgr2=0x7fb048117a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:37.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.089+0000 7fb04dfd8640 1 --2- 172.21.15.44:0/710940344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb04810a260 0x7fb048117a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.089+0000 7fb04dfd8640 1 -- 172.21.15.44:0/710940344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb048109e80 msgr2=0x7fb048117550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:37.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.089+0000 7fb04dfd8640 1 --2- 172.21.15.44:0/710940344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb048109e80 0x7fb048117550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.089+0000 7fb04dfd8640 1 -- 172.21.15.44:0/710940344 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0481a5580 con 0x7fb04811bdd0 2024-08-20T03:06:37.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.093+0000 7fb04dfd8640 1 --2- 172.21.15.44:0/710940344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb04811bdd0 0x7fb0481a4db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb040034040 tx=0x7fb040003c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:37.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.093+0000 7fb03e7fc640 1 -- 172.21.15.44:0/710940344 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040041020 con 0x7fb04811bdd0 2024-08-20T03:06:37.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.093+0000 7fb03e7fc640 1 -- 172.21.15.44:0/710940344 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb040030970 con 0x7fb04811bdd0 2024-08-20T03:06:37.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.094+0000 7fb03e7fc640 1 -- 172.21.15.44:0/710940344 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040036040 con 0x7fb04811bdd0 2024-08-20T03:06:37.106 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.094+0000 7fb04fa3b640 1 -- 172.21.15.44:0/710940344 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0481a5810 con 0x7fb04811bdd0 2024-08-20T03:06:37.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.094+0000 7fb04fa3b640 1 -- 172.21.15.44:0/710940344 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0481abf10 con 0x7fb04811bdd0 2024-08-20T03:06:37.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.111+0000 7fb03e7fc640 1 -- 172.21.15.44:0/710940344 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb040044020 con 0x7fb04811bdd0 2024-08-20T03:06:37.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.124+0000 7fb03e7fc640 1 --2- 172.21.15.44:0/710940344 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0340b3f10 0x7fb0340b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.124+0000 7fb03e7fc640 1 -- 172.21.15.44:0/710940344 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb0401045c0 con 0x7fb04811bdd0 2024-08-20T03:06:37.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.126+0000 7fb051e34740 1 Processor -- start 2024-08-20T03:06:37.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.126+0000 7fb051e34740 1 -- start start 2024-08-20T03:06:37.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.126+0000 7fb051e34740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958cae830 0x558958caec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.126+0000 7fb051e34740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958caf150 0x558958cbaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.127+0000 7fb051e34740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cbd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.127+0000 7fb051e34740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558958c258e0 con 0x558958cae830 2024-08-20T03:06:37.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.127+0000 7fb051e34740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558958c14de0 con 0x558958cbb0e0 2024-08-20T03:06:37.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.127+0000 7fb051e34740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558958c18e20 con 0x558958caf150 2024-08-20T03:06:37.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.128+0000 7fb04cfd6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958caf150 0x558958cbaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.128+0000 7fb04d7d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958cae830 0x558958caec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.128+0000 7fb04d7d7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958cae830 0x558958caec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54456/0 (socket says 172.21.15.44:54456) 2024-08-20T03:06:37.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.128+0000 7fb04d7d7640 1 -- 172.21.15.44:0/397025932 learned_addr learned my addr 172.21.15.44:0/397025932 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:37.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.129+0000 7fb04dfd8640 1 --2- 172.21.15.44:0/397025932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cbd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.129+0000 7fb04d7d7640 1 --2- 172.21.15.44:0/710940344 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0340b3f10 0x7fb0340b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.130+0000 7fb04dfd8640 1 -- 172.21.15.44:0/397025932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958caf150 msgr2=0x558958cbaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:37.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.130+0000 7fb04dfd8640 1 --2- 172.21.15.44:0/397025932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958caf150 0x558958cbaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.130+0000 7fb04dfd8640 1 -- 172.21.15.44:0/397025932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958cae830 msgr2=0x558958caec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:37.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.130+0000 7fb04dfd8640 1 --2- 172.21.15.44:0/397025932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958cae830 0x558958caec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.130+0000 7fb04dfd8640 1 -- 172.21.15.44:0/397025932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558958974650 con 0x558958cbb0e0 2024-08-20T03:06:37.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.134+0000 7fb04dfd8640 1 --2- 172.21.15.44:0/397025932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cbd4d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb040101d20 tx=0x7fb0400caa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2da18960da34621 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:37.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.135+0000 7fb0337fe640 1 -- 172.21.15.44:0/397025932 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0400e5070 con 0x558958cbb0e0 2024-08-20T03:06:37.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.135+0000 7fb0337fe640 1 -- 172.21.15.44:0/397025932 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0400cf7e0 con 0x558958cbb0e0 2024-08-20T03:06:37.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.135+0000 7fb0337fe640 1 -- 172.21.15.44:0/397025932 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0400d4060 con 0x558958cbb0e0 2024-08-20T03:06:37.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.137+0000 7fb051e34740 1 -- 172.21.15.44:0/397025932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 msgr2=0x558958cbd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:37.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.137+0000 7fb051e34740 1 --2- 172.21.15.44:0/397025932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cbd4d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb040101d20 tx=0x7fb0400caa10 comp rx=0 tx=0).stop 2024-08-20T03:06:37.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.140+0000 7fb051e34740 1 -- 172.21.15.44:0/397025932 shutdown_connections 2024-08-20T03:06:37.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.140+0000 7fb051e34740 1 --2- 172.21.15.44:0/397025932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cbd4d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.140+0000 7fb051e34740 1 --2- 172.21.15.44:0/397025932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958caf150 0x558958cbaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.140+0000 7fb051e34740 1 --2- 172.21.15.44:0/397025932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958cae830 0x558958caec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.140+0000 7fb051e34740 1 -- 172.21.15.44:0/397025932 >> 172.21.15.44:0/397025932 conn(0x558958cac660 msgr2=0x558958cad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.144+0000 7fb051e34740 1 -- 172.21.15.44:0/397025932 shutdown_connections 2024-08-20T03:06:37.147 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.144+0000 7fb051e34740 1 -- 172.21.15.44:0/397025932 wait complete. 2024-08-20T03:06:37.151 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.145+0000 7fb051e34740 1 -- start start 2024-08-20T03:06:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.145+0000 7fb04d7d7640 1 --2- 172.21.15.44:0/710940344 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0340b3f10 0x7fb0340b63d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb044005dc0 tx=0x7fb044005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.145+0000 7fb051e34740 1 Processor -- start 2024-08-20T03:06:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.146+0000 7fb051e34740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cae830 0x558958cb8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.146+0000 7fb051e34740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 0x558958cb8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.146+0000 7fb051e34740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cbb0e0 0x558958cb8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.146+0000 7fb04d7d7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cae830 0x558958cb8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.146+0000 7fb04d7d7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cae830 0x558958cb8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:34588/0 (socket says 172.21.15.44:34588) 2024-08-20T03:06:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.146+0000 7fb04d7d7640 1 -- 172.21.15.44:0/957292734 learned_addr learned my addr 172.21.15.44:0/957292734 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.146+0000 7fb051e34740 1 -- 172.21.15.44:0/957292734 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558958a77700 con 0x558958caf150 2024-08-20T03:06:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.146+0000 7fb051e34740 1 -- 172.21.15.44:0/957292734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558958b9b7c0 con 0x558958cae830 2024-08-20T03:06:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.146+0000 7fb051e34740 1 -- 172.21.15.44:0/957292734 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558958c14de0 con 0x558958cbb0e0 2024-08-20T03:06:37.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.147+0000 7fb051e34740 10 client.0 _ll_register_callbacks cb 0x558958c0e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:06:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.147+0000 7fb04d7d7640 1 -- 172.21.15.44:0/957292734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cbb0e0 msgr2=0x558958cb8a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:06:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.147+0000 7fb04d7d7640 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cbb0e0 0x558958cb8a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.147+0000 7fb04d7d7640 1 -- 172.21.15.44:0/957292734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 msgr2=0x558958cb8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.147+0000 7fb04cfd6640 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 0x558958cb8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.147+0000 7fb04d7d7640 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 0x558958cb8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.147+0000 7fb04d7d7640 1 -- 172.21.15.44:0/957292734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558958cc6cf0 con 0x558958cae830 2024-08-20T03:06:37.153 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.147+0000 7fb04cfd6640 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 0x558958cb8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:06:37.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.154+0000 7fb04d7d7640 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cae830 0x558958cb8010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb04401ed50 tx=0x7fb0440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3563fa939858cc4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:37.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.158+0000 7fb030ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:06:37.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.158+0000 7fb030ff9640 1 -- 172.21.15.44:0/957292734 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044028070 con 0x558958cae830 2024-08-20T03:06:37.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.158+0000 7fb030ff9640 1 -- 172.21.15.44:0/957292734 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb044020d60 con 0x558958cae830 2024-08-20T03:06:37.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.158+0000 7fb051e34740 1 -- 172.21.15.44:0/957292734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558958bc4100 con 0x558958cae830 2024-08-20T03:06:37.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.163+0000 7fb030ff9640 1 -- 172.21.15.44:0/957292734 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb044037400 con 0x558958cae830 2024-08-20T03:06:37.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.163+0000 7fb051e34740 10 client.25988 fetch_fsmap learned FSMap version 30 2024-08-20T03:06:37.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.163+0000 7fb051e34740 1 -- 172.21.15.44:0/957292734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558958cc9710 con 0x558958cae830 2024-08-20T03:06:37.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.169+0000 7fb030ff9640 1 -- 172.21.15.44:0/957292734 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558958bc4100 con 0x558958cae830 2024-08-20T03:06:37.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.169+0000 7fb030ff9640 1 -- 172.21.15.44:0/957292734 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fb04401f040 con 0x558958cae830 2024-08-20T03:06:37.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.169+0000 7fb051e34740 10 client.25988 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:06:37.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.169+0000 7fb051e34740 10 client.25988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:06:37.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.169+0000 7fb051e34740 20 client.25988 populate_metadata read hostname 'smithi044' 2024-08-20T03:06:37.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.169+0000 7fb051e34740 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558958ccac70 0x558958ccd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.169+0000 7fb051e34740 4 client.25988 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:06:37.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.170+0000 7fb051e34740 1 -- 172.21.15.44:0/957292734 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558958c27830 con 0x558958ccac70 2024-08-20T03:06:37.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.173+0000 7fb04cfd6640 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558958ccac70 0x558958ccd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.187+0000 7fb030ff9640 10 client.25988 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:37.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.187+0000 7fb04cfd6640 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558958ccac70 0x558958ccd0c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d919c9cef3df924 server_cookie=a8deabb0423b6b48 in_seq=0 out_seq=0 2024-08-20T03:06:37.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.201+0000 7fb030ff9640 1 -- 172.21.15.44:0/957292734 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558958c27830 con 0x558958ccac70 2024-08-20T03:06:37.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.201+0000 7fb030ff9640 10 client.25988 handle_command_reply: tid=0 2024-08-20T03:06:37.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.201+0000 7fb051e34740 1 client.25988 shutdown 2024-08-20T03:06:37.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.202+0000 7fb051e34740 1 -- 172.21.15.44:0/957292734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cae830 msgr2=0x558958cb8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:37.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.202+0000 7fb051e34740 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cae830 0x558958cb8010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb04401ed50 tx=0x7fb0440046c0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.202+0000 7fb051e34740 1 -- 172.21.15.44:0/957292734 shutdown_connections 2024-08-20T03:06:37.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.202+0000 7fb051e34740 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558958ccac70 0x558958ccd0c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.202+0000 7fb051e34740 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cbb0e0 0x558958cb8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.202+0000 7fb051e34740 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 0x558958cb8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.202+0000 7fb051e34740 1 --2- 172.21.15.44:0/957292734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cae830 0x558958cb8010 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.202+0000 7fb051e34740 1 -- 172.21.15.44:0/957292734 >> 172.21.15.44:0/957292734 conn(0x558958cac660 msgr2=0x558958cb2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:37.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.202+0000 7fb030ff9640 0 client.25988 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:37.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.202+0000 7fb051e34740 1 -- 172.21.15.44:0/957292734 shutdown_connections 2024-08-20T03:06:37.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.202+0000 7fb051e34740 1 -- 172.21.15.44:0/957292734 wait complete. 2024-08-20T03:06:37.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.202+0000 7fb051e34740 20 client.25988 trim_cache size 0 max 16384 2024-08-20T03:06:37.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.242+0000 7fb051e34740 1 Processor -- start 2024-08-20T03:06:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.242+0000 7fb051e34740 1 -- start start 2024-08-20T03:06:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.242+0000 7fb051e34740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cae830 0x558958cb29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.242+0000 7fb051e34740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 0x558958cb9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.243+0000 7fb04cfd6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 0x558958cb9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.243+0000 7fb04cfd6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 0x558958cb9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54474/0 (socket says 172.21.15.44:54474) 2024-08-20T03:06:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.243+0000 7fb04cfd6640 1 -- 172.21.15.44:0/2343589995 learned_addr learned my addr 172.21.15.44:0/2343589995 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.243+0000 7fb051e34740 1 --2- 172.21.15.44:0/2343589995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cd2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.243+0000 7fb051e34740 1 -- 172.21.15.44:0/2343589995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c00e750 con 0x558958caf150 2024-08-20T03:06:37.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.243+0000 7fb051e34740 1 -- 172.21.15.44:0/2343589995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c010350 con 0x558958cbb0e0 2024-08-20T03:06:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.243+0000 7fb051e34740 1 -- 172.21.15.44:0/2343589995 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c00ee20 con 0x558958cae830 2024-08-20T03:06:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.243+0000 7fb04dfd8640 1 --2- 172.21.15.44:0/2343589995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cd2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.243+0000 7fb04d7d7640 1 --2- 172.21.15.44:0/2343589995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cae830 0x558958cb29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.244+0000 7fb04cfd6640 1 -- 172.21.15.44:0/2343589995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cae830 msgr2=0x558958cb29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.244+0000 7fb04cfd6640 1 --2- 172.21.15.44:0/2343589995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cae830 0x558958cb29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.244+0000 7fb04cfd6640 1 -- 172.21.15.44:0/2343589995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 msgr2=0x558958cd2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.244+0000 7fb04cfd6640 1 --2- 172.21.15.44:0/2343589995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cd2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.244+0000 7fb04cfd6640 1 -- 172.21.15.44:0/2343589995 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558958caef80 con 0x558958caf150 2024-08-20T03:06:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.244+0000 7fb04cfd6640 1 --2- 172.21.15.44:0/2343589995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 0x558958cb9d50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb0480742a0 tx=0x7fb038005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e5796e597b691c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.245+0000 7fb032ffd640 1 -- 172.21.15.44:0/2343589995 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038004050 con 0x558958caf150 2024-08-20T03:06:37.249 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.245+0000 7fb032ffd640 1 -- 172.21.15.44:0/2343589995 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0380054d0 con 0x558958caf150 2024-08-20T03:06:37.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.246+0000 7fb051e34740 1 -- 172.21.15.44:0/2343589995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 msgr2=0x558958cb9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.246+0000 7fb051e34740 1 --2- 172.21.15.44:0/2343589995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 0x558958cb9d50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb0480742a0 tx=0x7fb038005ab0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.246+0000 7fb051e34740 1 -- 172.21.15.44:0/2343589995 shutdown_connections 2024-08-20T03:06:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.246+0000 7fb051e34740 1 --2- 172.21.15.44:0/2343589995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cd2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.246+0000 7fb051e34740 1 --2- 172.21.15.44:0/2343589995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958caf150 0x558958cb9d50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.246+0000 7fb051e34740 1 --2- 172.21.15.44:0/2343589995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cae830 0x558958cb29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.251 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.246+0000 7fb051e34740 1 -- 172.21.15.44:0/2343589995 >> 172.21.15.44:0/2343589995 conn(0x558958cac660 msgr2=0x558958cb9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:37.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.252+0000 7fb051e34740 1 -- 172.21.15.44:0/2343589995 shutdown_connections 2024-08-20T03:06:37.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.254+0000 7fb051e34740 1 -- 172.21.15.44:0/2343589995 wait complete. 2024-08-20T03:06:37.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.255+0000 7fb051e34740 1 -- start start 2024-08-20T03:06:37.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.255+0000 7fb051e34740 1 Processor -- start 2024-08-20T03:06:37.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.255+0000 7fb051e34740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cc2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.255+0000 7fb051e34740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cc3340 0x558958cb4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.255+0000 7fb051e34740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958cb4d80 0x558958cb5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.255+0000 7fb051e34740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c00fbc0 con 0x558958cb4d80 2024-08-20T03:06:37.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.255+0000 7fb051e34740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c00f4f0 con 0x558958cbb0e0 2024-08-20T03:06:37.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.255+0000 7fb051e34740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c00c730 con 0x558958cc3340 2024-08-20T03:06:37.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.255+0000 7fb051e34740 10 client.0 _ll_register_callbacks cb 0x7fb04401bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:06:37.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.256+0000 7fb04cfd6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cc3340 0x558958cb4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.256+0000 7fb04dfd8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958cb4d80 0x558958cb5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.256+0000 7fb04dfd8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958cb4d80 0x558958cb5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54482/0 (socket says 172.21.15.44:54482) 2024-08-20T03:06:37.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.256+0000 7fb04dfd8640 1 -- 172.21.15.44:0/3766301202 learned_addr learned my addr 172.21.15.44:0/3766301202 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.266+0000 7fb04d7d7640 1 --2- 172.21.15.44:0/3766301202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cc2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.266+0000 7fb04cfd6640 1 -- 172.21.15.44:0/3766301202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 msgr2=0x558958cc2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.266+0000 7fb04cfd6640 1 --2- 172.21.15.44:0/3766301202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cc2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.266+0000 7fb04cfd6640 1 -- 172.21.15.44:0/3766301202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958cb4d80 msgr2=0x558958cb5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.266+0000 7fb04cfd6640 1 --2- 172.21.15.44:0/3766301202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958cb4d80 0x558958cb5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.266+0000 7fb04cfd6640 1 -- 172.21.15.44:0/3766301202 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558958d34960 con 0x558958cc3340 2024-08-20T03:06:37.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.277+0000 7fb04cfd6640 1 --2- 172.21.15.44:0/3766301202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cc3340 0x558958cb4840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb03800b490 tx=0x7fb03800bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c50c9c51e9a63eab server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:37.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.277+0000 7fb030ff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:06:37.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.281+0000 7fb030ff9640 1 -- 172.21.15.44:0/3766301202 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038004050 con 0x558958cc3340 2024-08-20T03:06:37.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.281+0000 7fb030ff9640 1 -- 172.21.15.44:0/3766301202 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fb0380041f0 con 0x558958cc3340 2024-08-20T03:06:37.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.282+0000 7fb051e34740 1 -- 172.21.15.44:0/3766301202 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558958c232d0 con 0x558958cc3340 2024-08-20T03:06:37.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.289+0000 7fb030ff9640 1 -- 172.21.15.44:0/3766301202 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb038031050 con 0x558958cc3340 2024-08-20T03:06:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.290+0000 7fb051e34740 10 client.25957 fetch_fsmap learned FSMap version 30 2024-08-20T03:06:37.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.290+0000 7fb051e34740 1 -- 172.21.15.44:0/3766301202 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558958d35c70 con 0x558958cc3340 2024-08-20T03:06:37.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.296+0000 7fb030ff9640 1 -- 172.21.15.44:0/3766301202 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558958c232d0 con 0x558958cc3340 2024-08-20T03:06:37.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.297+0000 7fb030ff9640 1 -- 172.21.15.44:0/3766301202 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fb03800f040 con 0x558958cc3340 2024-08-20T03:06:37.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.298+0000 7fb051e34740 10 client.25957 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:06:37.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.298+0000 7fb051e34740 10 client.25957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:06:37.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.298+0000 7fb051e34740 20 client.25957 populate_metadata read hostname 'smithi044' 2024-08-20T03:06:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.298+0000 7fb051e34740 1 --2- 172.21.15.44:0/3766301202 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558958d36370 0x558958d38790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.298+0000 7fb051e34740 4 client.25957 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:06:37.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.298+0000 7fb051e34740 1 -- 172.21.15.44:0/3766301202 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558958c20a10 con 0x558958d36370 2024-08-20T03:06:37.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.303+0000 7fb04d7d7640 1 --2- 172.21.15.44:0/3766301202 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558958d36370 0x558958d38790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:37.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.314+0000 7fb030ff9640 10 client.25957 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:37.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.314+0000 7fb04d7d7640 1 --2- 172.21.15.44:0/3766301202 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558958d36370 0x558958d38790 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4507beb4eb12151 server_cookie=3d61cebd4b117d8a in_seq=0 out_seq=0 2024-08-20T03:06:37.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb030ff9640 1 -- 172.21.15.44:0/3766301202 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x558958c20a10 con 0x558958d36370 2024-08-20T03:06:37.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb030ff9640 10 client.25957 handle_command_reply: tid=0 2024-08-20T03:06:37.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb051e34740 1 client.25957 shutdown 2024-08-20T03:06:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb051e34740 1 -- 172.21.15.44:0/3766301202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cc3340 msgr2=0x558958cb4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb051e34740 1 --2- 172.21.15.44:0/3766301202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cc3340 0x558958cb4840 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb03800b490 tx=0x7fb03800bcc0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb051e34740 1 -- 172.21.15.44:0/3766301202 shutdown_connections 2024-08-20T03:06:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb051e34740 1 --2- 172.21.15.44:0/3766301202 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x558958d36370 0x558958d38790 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb051e34740 1 --2- 172.21.15.44:0/3766301202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558958cb4d80 0x558958cb5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb051e34740 1 --2- 172.21.15.44:0/3766301202 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558958cc3340 0x558958cb4840 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb030ff9640 0 client.25957 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb051e34740 1 --2- 172.21.15.44:0/3766301202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558958cbb0e0 0x558958cc2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb051e34740 1 -- 172.21.15.44:0/3766301202 >> 172.21.15.44:0/3766301202 conn(0x558958cac660 msgr2=0x558958cc62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb051e34740 1 -- 172.21.15.44:0/3766301202 shutdown_connections 2024-08-20T03:06:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.331+0000 7fb051e34740 1 -- 172.21.15.44:0/3766301202 wait complete. 2024-08-20T03:06:37.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.332+0000 7fb051e34740 20 client.25957 trim_cache size 0 max 16384 2024-08-20T03:06:37.336 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:06:37.336 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "scrub active (11903 inodes in the stack)", 2024-08-20T03:06:37.336 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-08-20T03:06:37.336 INFO:teuthology.orchestra.run.smithi044.stdout: "a24f9f81-df91-4f22-a871-09cc9455fee8": { 2024-08-20T03:06:37.336 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-08-20T03:06:37.336 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "a24f9f81-df91-4f22-a871-09cc9455fee8", 2024-08-20T03:06:37.336 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-08-20T03:06:37.336 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:06:37.337 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:06:37.337 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:06:37.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.332+0000 7fb04fa3b640 1 -- 172.21.15.44:0/710940344 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0340b3f10 msgr2=0x7fb0340b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:37.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.332+0000 7fb04fa3b640 1 --2- 172.21.15.44:0/710940344 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0340b3f10 0x7fb0340b63d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb044005dc0 tx=0x7fb044005850 comp rx=0 tx=0).stop 2024-08-20T03:06:37.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.332+0000 7fb04fa3b640 1 -- 172.21.15.44:0/710940344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb04811bdd0 msgr2=0x7fb0481a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:37.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.332+0000 7fb04fa3b640 1 --2- 172.21.15.44:0/710940344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb04811bdd0 0x7fb0481a4db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb040034040 tx=0x7fb040003c90 comp rx=0 tx=0).stop 2024-08-20T03:06:37.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.333+0000 7fb04fa3b640 1 -- 172.21.15.44:0/710940344 shutdown_connections 2024-08-20T03:06:37.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.333+0000 7fb04fa3b640 1 --2- 172.21.15.44:0/710940344 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb0340b3f10 0x7fb0340b63d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.333+0000 7fb04fa3b640 1 --2- 172.21.15.44:0/710940344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb04811bdd0 0x7fb0481a4db0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.333+0000 7fb04fa3b640 1 --2- 172.21.15.44:0/710940344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb04810a260 0x7fb048117a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.333+0000 7fb04fa3b640 1 --2- 172.21.15.44:0/710940344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb048109e80 0x7fb048117550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:37.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.333+0000 7fb04fa3b640 1 -- 172.21.15.44:0/710940344 >> 172.21.15.44:0/710940344 conn(0x7fb04806dcc0 msgr2=0x7fb048071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:37.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.333+0000 7fb04fa3b640 1 -- 172.21.15.44:0/710940344 shutdown_connections 2024-08-20T03:06:37.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:06:37.333+0000 7fb04fa3b640 1 -- 172.21.15.44:0/710940344 wait complete. 2024-08-20T03:06:37.478 INFO:tasks.workunit.client.0.smithi044.stdout: 141 1856854 336 1322249 293 1019126 766 2024-08-20T03:06:38.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:37 smithi044 ceph-mon[18790]: pgmap v444: 97 pgs: 97 active+clean; 584 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.1 MiB/s wr, 487 op/s 2024-08-20T03:06:38.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:37 smithi165 ceph-mon[22357]: pgmap v444: 97 pgs: 97 active+clean; 584 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.1 MiB/s wr, 487 op/s 2024-08-20T03:06:38.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:37 smithi099 ceph-mon[21984]: pgmap v444: 97 pgs: 97 active+clean; 584 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.1 MiB/s wr, 487 op/s 2024-08-20T03:06:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.802+0000 7f426affd640 1 -- 172.21.15.44:0/2864597134 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f427c01e030 con 0x556b7d9f5710 2024-08-20T03:06:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.802+0000 7f426affd640 10 client.25976 handle_command_reply: tid=0 2024-08-20T03:06:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.802+0000 7f4294e75740 1 client.25976 shutdown 2024-08-20T03:06:38.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.803+0000 7f4294e75740 1 -- 172.21.15.44:0/2864597134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d978440 msgr2=0x556b7d974100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.803+0000 7f4294e75740 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d978440 0x556b7d974100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x556b7d96dcf0 tx=0x7f4280004730 comp rx=0 tx=0).stop 2024-08-20T03:06:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.803+0000 7f4294e75740 1 -- 172.21.15.44:0/2864597134 shutdown_connections 2024-08-20T03:06:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.803+0000 7f4294e75740 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556b7d9f5710 0x556b7d9f7bd0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.803+0000 7f426affd640 0 client.25976 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.803+0000 7f4294e75740 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556b7d97b3e0 0x556b7d974640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.803+0000 7f4294e75740 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556b7d978440 0x556b7d974100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.803+0000 7f4294e75740 1 --2- 172.21.15.44:0/2864597134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556b7d96eb30 0x556b7d983860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.803+0000 7f4294e75740 1 -- 172.21.15.44:0/2864597134 >> 172.21.15.44:0/2864597134 conn(0x556b7d96c960 msgr2=0x556b7d976720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.803+0000 7f4294e75740 1 -- 172.21.15.44:0/2864597134 shutdown_connections 2024-08-20T03:06:38.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.803+0000 7f4294e75740 1 -- 172.21.15.44:0/2864597134 wait complete. 2024-08-20T03:06:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.804+0000 7f4294e75740 20 client.25976 trim_cache size 0 max 16384 2024-08-20T03:06:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:06:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:06:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:06:38.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:06:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.850+0000 7f4292085640 1 -- 172.21.15.44:0/479678273 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42740b3f50 msgr2=0x7f42740b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.850+0000 7f4292085640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42740b3f50 0x7f42740b6410 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f42800058c0 tx=0x7f4280005850 comp rx=0 tx=0).stop 2024-08-20T03:06:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.850+0000 7f4292085640 1 -- 172.21.15.44:0/479678273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f428c10a540 msgr2=0x7f428c084fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.850+0000 7f4292085640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f428c10a540 0x7f428c084fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f428402f950 tx=0x7f428402f980 comp rx=0 tx=0).stop 2024-08-20T03:06:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.850+0000 7f4292085640 1 -- 172.21.15.44:0/479678273 shutdown_connections 2024-08-20T03:06:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.850+0000 7f4292085640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42740b3f50 0x7f42740b6410 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.850+0000 7f4292085640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f428c10a540 0x7f428c084fe0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.850+0000 7f4292085640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f428c079b90 0x7f428c080d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.851+0000 7f4292085640 1 --2- 172.21.15.44:0/479678273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f428c074bb0 0x7f428c080840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.851+0000 7f4292085640 1 -- 172.21.15.44:0/479678273 >> 172.21.15.44:0/479678273 conn(0x7f428c06d8e0 msgr2=0x7f428c10ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.851+0000 7f4292085640 1 -- 172.21.15.44:0/479678273 shutdown_connections 2024-08-20T03:06:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:38.851+0000 7f4292085640 1 -- 172.21.15.44:0/479678273 wait complete. 2024-08-20T03:06:39.046 INFO:tasks.cephfs.filesystem:scrub status for tag:a24f9f81-df91-4f22-a871-09cc9455fee8 - {'path': '/', 'tag': 'a24f9f81-df91-4f22-a871-09cc9455fee8', 'options': 'recursive,force'} 2024-08-20T03:06:40.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:39 smithi044 ceph-mon[18790]: pgmap v445: 97 pgs: 97 active+clean; 611 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.2 MiB/s wr, 1.04k op/s 2024-08-20T03:06:40.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:39 smithi165 ceph-mon[22357]: pgmap v445: 97 pgs: 97 active+clean; 611 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.2 MiB/s wr, 1.04k op/s 2024-08-20T03:06:40.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:39 smithi099 ceph-mon[21984]: pgmap v445: 97 pgs: 97 active+clean; 611 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.2 MiB/s wr, 1.04k op/s 2024-08-20T03:06:42.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:41 smithi044 ceph-mon[18790]: pgmap v446: 97 pgs: 97 active+clean; 603 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.4 MiB/s wr, 1.17k op/s 2024-08-20T03:06:42.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:41 smithi165 ceph-mon[22357]: pgmap v446: 97 pgs: 97 active+clean; 603 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.4 MiB/s wr, 1.17k op/s 2024-08-20T03:06:42.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:41 smithi099 ceph-mon[21984]: pgmap v446: 97 pgs: 97 active+clean; 603 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.4 MiB/s wr, 1.17k op/s 2024-08-20T03:06:44.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:43 smithi044 ceph-mon[18790]: pgmap v447: 97 pgs: 97 active+clean; 565 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.5 MiB/s wr, 1.76k op/s 2024-08-20T03:06:44.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:43 smithi165 ceph-mon[22357]: pgmap v447: 97 pgs: 97 active+clean; 565 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.5 MiB/s wr, 1.76k op/s 2024-08-20T03:06:44.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:43 smithi099 ceph-mon[21984]: pgmap v447: 97 pgs: 97 active+clean; 565 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.5 MiB/s wr, 1.76k op/s 2024-08-20T03:06:46.037 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:45 smithi044 ceph-mon[18790]: pgmap v448: 97 pgs: 97 active+clean; 564 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 1.72k op/s 2024-08-20T03:06:46.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:45 smithi165 ceph-mon[22357]: pgmap v448: 97 pgs: 97 active+clean; 564 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 1.72k op/s 2024-08-20T03:06:46.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:45 smithi099 ceph-mon[21984]: pgmap v448: 97 pgs: 97 active+clean; 564 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 1.72k op/s 2024-08-20T03:06:47.478 INFO:tasks.workunit.client.0.smithi044.stdout: 152 2105070 604 1493286 463 1192427 1674 2024-08-20T03:06:48.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:47 smithi044 ceph-mon[18790]: pgmap v449: 97 pgs: 97 active+clean; 567 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.5 MiB/s wr, 1.69k op/s 2024-08-20T03:06:48.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:47 smithi165 ceph-mon[22357]: pgmap v449: 97 pgs: 97 active+clean; 567 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.5 MiB/s wr, 1.69k op/s 2024-08-20T03:06:48.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:47 smithi099 ceph-mon[21984]: pgmap v449: 97 pgs: 97 active+clean; 567 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.5 MiB/s wr, 1.69k op/s 2024-08-20T03:06:50.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:49 smithi044 ceph-mon[18790]: pgmap v450: 97 pgs: 97 active+clean; 566 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 1.56k op/s 2024-08-20T03:06:50.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:49 smithi165 ceph-mon[22357]: pgmap v450: 97 pgs: 97 active+clean; 566 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 1.56k op/s 2024-08-20T03:06:50.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:49 smithi099 ceph-mon[21984]: pgmap v450: 97 pgs: 97 active+clean; 566 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 1.56k op/s 2024-08-20T03:06:52.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:51 smithi044 ceph-mon[18790]: pgmap v451: 97 pgs: 97 active+clean; 568 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 979 op/s 2024-08-20T03:06:52.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:51 smithi165 ceph-mon[22357]: pgmap v451: 97 pgs: 97 active+clean; 568 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 979 op/s 2024-08-20T03:06:52.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:51 smithi099 ceph-mon[21984]: pgmap v451: 97 pgs: 97 active+clean; 568 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 979 op/s 2024-08-20T03:06:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.532+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2766122769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d1011bdd0 msgr2=0x7f1d1011e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.532+0000 7f1d0ffff640 1 --2- 172.21.15.44:0/2766122769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d1011bdd0 0x7f1d1011e1c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009860 tx=0x7f1d0002a340 comp rx=0 tx=0).stop 2024-08-20T03:06:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.532+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2766122769 shutdown_connections 2024-08-20T03:06:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.532+0000 7f1d0ffff640 1 --2- 172.21.15.44:0/2766122769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d1011bdd0 0x7f1d1011e1c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.532+0000 7f1d0ffff640 1 --2- 172.21.15.44:0/2766122769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1010a260 0x7f1d1010a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.532+0000 7f1d0ffff640 1 --2- 172.21.15.44:0/2766122769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d10109e80 0x7f1d10110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.532+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2766122769 >> 172.21.15.44:0/2766122769 conn(0x7f1d1006d8b0 msgr2=0x7f1d1006dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.532+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2766122769 shutdown_connections 2024-08-20T03:06:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.533+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2766122769 wait complete. 2024-08-20T03:06:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.534+0000 7f1d0ffff640 1 Processor -- start 2024-08-20T03:06:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.535+0000 7f1d0ffff640 1 -- start start 2024-08-20T03:06:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.535+0000 7f1d0ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d10109e80 0x7f1d101a7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.535+0000 7f1d0ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1010a260 0x7f1d101a8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.535+0000 7f1d0ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d1011bdd0 0x7f1d101ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.535+0000 7f1d0ffff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101215e0 con 0x7f1d10109e80 2024-08-20T03:06:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.535+0000 7f1d0ffff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d10121460 con 0x7f1d1010a260 2024-08-20T03:06:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.535+0000 7f1d0ffff640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d10121760 con 0x7f1d1011bdd0 2024-08-20T03:06:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.535+0000 7f1d0effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d10109e80 0x7f1d101a7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.535+0000 7f1d0effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d10109e80 0x7f1d101a7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42846/0 (socket says 172.21.15.44:42846) 2024-08-20T03:06:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.535+0000 7f1d0effd640 1 -- 172.21.15.44:0/2220495091 learned_addr learned my addr 172.21.15.44:0/2220495091 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.535+0000 7f1d0f7fe640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d1011bdd0 0x7f1d101ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.536+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1010a260 0x7f1d101a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.536+0000 7f1d0e7fc640 1 -- 172.21.15.44:0/2220495091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d1011bdd0 msgr2=0x7f1d101ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.536+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d1011bdd0 0x7f1d101ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.536+0000 7f1d0e7fc640 1 -- 172.21.15.44:0/2220495091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d10109e80 msgr2=0x7f1d101a7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.536+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d10109e80 0x7f1d101a7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.536+0000 7f1d0e7fc640 1 -- 172.21.15.44:0/2220495091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d101adce0 con 0x7f1d1010a260 2024-08-20T03:06:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.536+0000 7f1d0f7fe640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d1011bdd0 0x7f1d101ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:06:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.536+0000 7f1d0effd640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d10109e80 0x7f1d101a7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:06:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.536+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1010a260 0x7f1d101a8180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400e770 tx=0x7f1d0400ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.536+0000 7f1cf7fff640 1 -- 172.21.15.44:0/2220495091 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0400ccc0 con 0x7f1d1010a260 2024-08-20T03:06:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.537+0000 7f1cf7fff640 1 -- 172.21.15.44:0/2220495091 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d0400ee80 con 0x7f1d1010a260 2024-08-20T03:06:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.537+0000 7f1cf7fff640 1 -- 172.21.15.44:0/2220495091 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04003e40 con 0x7f1d1010a260 2024-08-20T03:06:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.538+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2220495091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d101adfd0 con 0x7f1d1010a260 2024-08-20T03:06:52.540 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.538+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2220495091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d101b4690 con 0x7f1d1010a260 2024-08-20T03:06:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.542+0000 7f1cf7fff640 1 -- 172.21.15.44:0/2220495091 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1d04003fe0 con 0x7f1d1010a260 2024-08-20T03:06:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.543+0000 7f1d180c4740 1 Processor -- start 2024-08-20T03:06:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.543+0000 7f1d180c4740 1 -- start start 2024-08-20T03:06:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.543+0000 7f1d180c4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 0x55c8d6782120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.543+0000 7f1d180c4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 0x55c8d678e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.543+0000 7f1d180c4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d678e5f0 0x55c8d67909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.543+0000 7f1d180c4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c8d66f7360 con 0x55c8d678e5f0 2024-08-20T03:06:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.543+0000 7f1d180c4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c8d66e6860 con 0x55c8d6782660 2024-08-20T03:06:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.543+0000 7f1d180c4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c8d66ea8a0 con 0x55c8d6781d40 2024-08-20T03:06:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.544+0000 7f1d0f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 0x55c8d6782120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.544+0000 7f1d0f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 0x55c8d6782120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44660/0 (socket says 172.21.15.44:44660) 2024-08-20T03:06:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.544+0000 7f1d0f7fe640 1 -- 172.21.15.44:0/2807545480 learned_addr learned my addr 172.21.15.44:0/2807545480 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.544+0000 7f1d0effd640 1 --2- 172.21.15.44:0/2807545480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 0x55c8d678e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.544+0000 7f1d0f7fe640 1 -- 172.21.15.44:0/2807545480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 msgr2=0x55c8d678e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.544+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/2807545480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d678e5f0 0x55c8d67909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.544+0000 7f1d0f7fe640 1 --2- 172.21.15.44:0/2807545480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 0x55c8d678e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.544+0000 7f1d0f7fe640 1 -- 172.21.15.44:0/2807545480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d678e5f0 msgr2=0x55c8d67909e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.544+0000 7f1d0f7fe640 1 --2- 172.21.15.44:0/2807545480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d678e5f0 0x55c8d67909e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.544+0000 7f1d0f7fe640 1 -- 172.21.15.44:0/2807545480 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8d668de40 con 0x55c8d6781d40 2024-08-20T03:06:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.544+0000 7f1cf7fff640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1cf80b3e70 0x7f1cf80b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.544+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/2807545480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d678e5f0 0x55c8d67909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:06:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.545+0000 7f1d0f7fe640 1 --2- 172.21.15.44:0/2807545480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 0x55c8d6782120 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009860 tx=0x7f1d00009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32cb655c43aa59a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.545+0000 7f1d0effd640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1cf80b3e70 0x7f1cf80b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.545+0000 7f1cf7fff640 1 -- 172.21.15.44:0/2220495091 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1d040e41c0 con 0x7f1d1010a260 2024-08-20T03:06:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.546+0000 7f1cf57fa640 1 -- 172.21.15.44:0/2807545480 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00037040 con 0x55c8d6781d40 2024-08-20T03:06:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.546+0000 7f1cf57fa640 1 -- 172.21.15.44:0/2807545480 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d00042070 con 0x55c8d6781d40 2024-08-20T03:06:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.546+0000 7f1cf57fa640 1 -- 172.21.15.44:0/2807545480 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d000052a0 con 0x55c8d6781d40 2024-08-20T03:06:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.547+0000 7f1d180c4740 1 -- 172.21.15.44:0/2807545480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 msgr2=0x55c8d6782120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.547+0000 7f1d0effd640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1cf80b3e70 0x7f1cf80b6330 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009a60 tx=0x7f1d080095c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.547+0000 7f1d180c4740 1 --2- 172.21.15.44:0/2807545480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 0x55c8d6782120 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009860 tx=0x7f1d00009490 comp rx=0 tx=0).stop 2024-08-20T03:06:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.547+0000 7f1d180c4740 1 -- 172.21.15.44:0/2807545480 shutdown_connections 2024-08-20T03:06:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.547+0000 7f1d180c4740 1 --2- 172.21.15.44:0/2807545480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d678e5f0 0x55c8d67909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.547+0000 7f1d180c4740 1 --2- 172.21.15.44:0/2807545480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 0x55c8d678e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.547+0000 7f1d180c4740 1 --2- 172.21.15.44:0/2807545480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 0x55c8d6782120 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.547+0000 7f1d180c4740 1 -- 172.21.15.44:0/2807545480 >> 172.21.15.44:0/2807545480 conn(0x55c8d677fb70 msgr2=0x55c8d6780d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.547+0000 7f1d180c4740 1 -- 172.21.15.44:0/2807545480 shutdown_connections 2024-08-20T03:06:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.547+0000 7f1d180c4740 1 -- 172.21.15.44:0/2807545480 wait complete. 2024-08-20T03:06:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.548+0000 7f1d180c4740 1 -- start start 2024-08-20T03:06:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.548+0000 7f1d180c4740 1 Processor -- start 2024-08-20T03:06:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.548+0000 7f1d180c4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 0x55c8d678dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.548+0000 7f1d180c4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 0x55c8d678b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.548+0000 7f1d180c4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d678e5f0 0x55c8d678b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.548+0000 7f1d180c4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c8d661fab0 con 0x55c8d678e5f0 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.548+0000 7f1d180c4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c8d666d780 con 0x55c8d6782660 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.548+0000 7f1d180c4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c8d6791270 con 0x55c8d6781d40 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.548+0000 7f1d180c4740 10 client.0 _ll_register_callbacks cb 0x55c8d63f5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.548+0000 7f1d0effd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 0x55c8d678dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.548+0000 7f1d0effd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 0x55c8d678dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44676/0 (socket says 172.21.15.44:44676) 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.548+0000 7f1d0effd640 1 -- 172.21.15.44:0/1033795565 learned_addr learned my addr 172.21.15.44:0/1033795565 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.549+0000 7f1d0f7fe640 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d678e5f0 0x55c8d678b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.549+0000 7f1d0effd640 1 -- 172.21.15.44:0/1033795565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 msgr2=0x55c8d678b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.549+0000 7f1d0effd640 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 0x55c8d678b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.549+0000 7f1d0effd640 1 -- 172.21.15.44:0/1033795565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d678e5f0 msgr2=0x55c8d678b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.549+0000 7f1d0effd640 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d678e5f0 0x55c8d678b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.549+0000 7f1d0effd640 1 -- 172.21.15.44:0/1033795565 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8d6449790 con 0x55c8d6781d40 2024-08-20T03:06:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.549+0000 7f1d0f7fe640 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d678e5f0 0x55c8d678b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:06:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.549+0000 7f1d0effd640 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 0x55c8d678dee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800bfd0 tx=0x7f1d08004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a16ef2337004ce85 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.549+0000 7f1cf2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:06:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.549+0000 7f1cf2ff5640 1 -- 172.21.15.44:0/1033795565 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0802a060 con 0x55c8d6781d40 2024-08-20T03:06:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.549+0000 7f1cf2ff5640 1 -- 172.21.15.44:0/1033795565 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1d08018910 con 0x55c8d6781d40 2024-08-20T03:06:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.549+0000 7f1d180c4740 1 -- 172.21.15.44:0/1033795565 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c8d6695e00 con 0x55c8d6781d40 2024-08-20T03:06:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.550+0000 7f1cf2ff5640 1 -- 172.21.15.44:0/1033795565 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d08018b10 con 0x55c8d6781d40 2024-08-20T03:06:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.550+0000 7f1d180c4740 10 client.25969 fetch_fsmap learned FSMap version 30 2024-08-20T03:06:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.550+0000 7f1d180c4740 1 -- 172.21.15.44:0/1033795565 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8d64c2620 con 0x55c8d6781d40 2024-08-20T03:06:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.550+0000 7f1cf2ff5640 1 -- 172.21.15.44:0/1033795565 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c8d6695e00 con 0x55c8d6781d40 2024-08-20T03:06:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.551+0000 7f1cf2ff5640 1 -- 172.21.15.44:0/1033795565 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1d0802d030 con 0x55c8d6781d40 2024-08-20T03:06:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.551+0000 7f1d180c4740 10 client.25969 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:06:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.551+0000 7f1d180c4740 10 client.25969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:06:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.551+0000 7f1d180c4740 20 client.25969 populate_metadata read hostname 'smithi044' 2024-08-20T03:06:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.551+0000 7f1d180c4740 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55c8d679dfa0 0x55c8d67a0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.551+0000 7f1d180c4740 4 client.25969 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:06:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.551+0000 7f1d180c4740 1 -- 172.21.15.44:0/1033795565 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8d66f92b0 con 0x55c8d679dfa0 2024-08-20T03:06:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.551+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55c8d679dfa0 0x55c8d67a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.559 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.551+0000 7f1cf2ff5640 10 client.25969 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.551+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55c8d679dfa0 0x55c8d67a0460 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb5956817f1aa702 server_cookie=ce3ec6b63dd7f6a7 in_seq=0 out_seq=0 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.556+0000 7f1cf2ff5640 1 -- 172.21.15.44:0/1033795565 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8d66f92b0 con 0x55c8d679dfa0 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.556+0000 7f1cf2ff5640 10 client.25969 handle_command_reply: tid=0 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.556+0000 7f1d180c4740 1 client.25969 shutdown 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.556+0000 7f1d180c4740 1 -- 172.21.15.44:0/1033795565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 msgr2=0x55c8d678dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.556+0000 7f1d180c4740 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 0x55c8d678dee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800bfd0 tx=0x7f1d08004a40 comp rx=0 tx=0).stop 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.556+0000 7f1d180c4740 1 -- 172.21.15.44:0/1033795565 shutdown_connections 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.556+0000 7f1d180c4740 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55c8d679dfa0 0x55c8d67a0460 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.556+0000 7f1d180c4740 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d678e5f0 0x55c8d678b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.556+0000 7f1d180c4740 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 0x55c8d678b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.556+0000 7f1d180c4740 1 --2- 172.21.15.44:0/1033795565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d6781d40 0x55c8d678dee0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.556+0000 7f1cf2ff5640 0 client.25969 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.556+0000 7f1d180c4740 1 -- 172.21.15.44:0/1033795565 >> 172.21.15.44:0/1033795565 conn(0x55c8d677fb70 msgr2=0x55c8d6785710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.557+0000 7f1d180c4740 1 -- 172.21.15.44:0/1033795565 shutdown_connections 2024-08-20T03:06:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.557+0000 7f1d180c4740 1 -- 172.21.15.44:0/1033795565 wait complete. 2024-08-20T03:06:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.557+0000 7f1d180c4740 20 client.25969 trim_cache size 0 max 16384 2024-08-20T03:06:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.573+0000 7f1d180c4740 1 Processor -- start 2024-08-20T03:06:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.573+0000 7f1d180c4740 1 -- start start 2024-08-20T03:06:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.573+0000 7f1d180c4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6781d40 0x55c8d678ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d180c4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6782660 0x55c8d677f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d0effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6781d40 0x55c8d678ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d0effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6781d40 0x55c8d678ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52722/0 (socket says 172.21.15.44:52722) 2024-08-20T03:06:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d0e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6782660 0x55c8d677f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d0e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6782660 0x55c8d677f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42874/0 (socket says 172.21.15.44:42874) 2024-08-20T03:06:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d0effd640 1 -- 172.21.15.44:0/3096648847 learned_addr learned my addr 172.21.15.44:0/3096648847 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d180c4740 1 --2- 172.21.15.44:0/3096648847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d678e5f0 0x55c8d678f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d180c4740 1 -- 172.21.15.44:0/3096648847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101bcf30 con 0x55c8d6782660 2024-08-20T03:06:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d180c4740 1 -- 172.21.15.44:0/3096648847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101beb30 con 0x55c8d6781d40 2024-08-20T03:06:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d180c4740 1 -- 172.21.15.44:0/3096648847 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101bd600 con 0x55c8d678e5f0 2024-08-20T03:06:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d0f7fe640 1 --2- 172.21.15.44:0/3096648847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d678e5f0 0x55c8d678f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d0effd640 1 -- 172.21.15.44:0/3096648847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d678e5f0 msgr2=0x55c8d678f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d0effd640 1 --2- 172.21.15.44:0/3096648847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d678e5f0 0x55c8d678f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.574+0000 7f1d0effd640 1 -- 172.21.15.44:0/3096648847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6782660 msgr2=0x55c8d677f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.575+0000 7f1d0effd640 1 --2- 172.21.15.44:0/3096648847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6782660 0x55c8d677f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.575+0000 7f1d0effd640 1 -- 172.21.15.44:0/3096648847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8d6782a40 con 0x55c8d6781d40 2024-08-20T03:06:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.575+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/3096648847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6782660 0x55c8d677f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:06:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.575+0000 7f1d0effd640 1 --2- 172.21.15.44:0/3096648847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6781d40 0x55c8d678ab90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800b7a0 tx=0x7f1d080094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d92fb1162f84033 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.575+0000 7f1cf4ff9640 1 -- 172.21.15.44:0/3096648847 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0802a020 con 0x55c8d6781d40 2024-08-20T03:06:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.575+0000 7f1cf4ff9640 1 -- 172.21.15.44:0/3096648847 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d08018710 con 0x55c8d6781d40 2024-08-20T03:06:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.575+0000 7f1cf4ff9640 1 -- 172.21.15.44:0/3096648847 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08025400 con 0x55c8d6781d40 2024-08-20T03:06:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.576+0000 7f1d180c4740 1 -- 172.21.15.44:0/3096648847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6781d40 msgr2=0x55c8d678ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.576+0000 7f1d180c4740 1 --2- 172.21.15.44:0/3096648847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6781d40 0x55c8d678ab90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1d0800b7a0 tx=0x7f1d080094a0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.576+0000 7f1d180c4740 1 -- 172.21.15.44:0/3096648847 shutdown_connections 2024-08-20T03:06:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.576+0000 7f1d180c4740 1 --2- 172.21.15.44:0/3096648847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d678e5f0 0x55c8d678f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.576+0000 7f1d180c4740 1 --2- 172.21.15.44:0/3096648847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6782660 0x55c8d677f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.576+0000 7f1d180c4740 1 --2- 172.21.15.44:0/3096648847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6781d40 0x55c8d678ab90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.576+0000 7f1d180c4740 1 -- 172.21.15.44:0/3096648847 >> 172.21.15.44:0/3096648847 conn(0x55c8d677fb70 msgr2=0x55c8d678d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.576+0000 7f1d180c4740 1 -- 172.21.15.44:0/3096648847 shutdown_connections 2024-08-20T03:06:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.577+0000 7f1d180c4740 1 -- 172.21.15.44:0/3096648847 wait complete. 2024-08-20T03:06:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.577+0000 7f1d180c4740 1 -- start start 2024-08-20T03:06:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.577+0000 7f1d180c4740 1 Processor -- start 2024-08-20T03:06:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.578+0000 7f1d180c4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6781d40 0x55c8d6796d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.578+0000 7f1d0effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6781d40 0x55c8d6796d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.578+0000 7f1d0effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6781d40 0x55c8d6796d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42882/0 (socket says 172.21.15.44:42882) 2024-08-20T03:06:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.578+0000 7f1d0effd640 1 -- 172.21.15.44:0/919624056 learned_addr learned my addr 172.21.15.44:0/919624056 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:06:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.578+0000 7f1d180c4740 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 0x55c8d678f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.578+0000 7f1d180c4740 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d678e5f0 0x55c8d678fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.578+0000 7f1d180c4740 1 -- 172.21.15.44:0/919624056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101be3a0 con 0x55c8d6781d40 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1d180c4740 1 -- 172.21.15.44:0/919624056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101bdcd0 con 0x55c8d6782660 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 0x55c8d678f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1d0f7fe640 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d678e5f0 0x55c8d678fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1d180c4740 1 -- 172.21.15.44:0/919624056 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1d101baf10 con 0x55c8d678e5f0 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1d0e7fc640 1 -- 172.21.15.44:0/919624056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d678e5f0 msgr2=0x55c8d678fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1d180c4740 10 client.0 _ll_register_callbacks cb 0x7f1d08009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d678e5f0 0x55c8d678fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1d0e7fc640 1 -- 172.21.15.44:0/919624056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6781d40 msgr2=0x55c8d6796d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6781d40 0x55c8d6796d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1d0e7fc640 1 -- 172.21.15.44:0/919624056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8d67803f0 con 0x55c8d6782660 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1d180c4740 1 -- 172.21.15.44:0/919624056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d08018710 con 0x55c8d6782660 2024-08-20T03:06:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1d0e7fc640 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 0x55c8d678f560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1d040e1920 tx=0x7f1d0401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a12338499a0b250 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:06:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.579+0000 7f1cf2ff5640 10 client.26009 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:06:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.580+0000 7f1cf2ff5640 1 -- 172.21.15.44:0/919624056 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d040af1c0 con 0x55c8d6782660 2024-08-20T03:06:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.580+0000 7f1cf2ff5640 1 -- 172.21.15.44:0/919624056 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1d040042d0 con 0x55c8d6782660 2024-08-20T03:06:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.580+0000 7f1cf2ff5640 1 -- 172.21.15.44:0/919624056 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d040aa8f0 con 0x55c8d6782660 2024-08-20T03:06:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.580+0000 7f1d180c4740 10 client.26009 fetch_fsmap learned FSMap version 30 2024-08-20T03:06:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.580+0000 7f1d180c4740 1 -- 172.21.15.44:0/919624056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55c8d6808d00 con 0x55c8d6782660 2024-08-20T03:06:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.581+0000 7f1cf2ff5640 1 -- 172.21.15.44:0/919624056 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1d040e60e0 con 0x55c8d6782660 2024-08-20T03:06:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.581+0000 7f1d180c4740 10 client.26009 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:06:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.581+0000 7f1d180c4740 10 client.26009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:06:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.581+0000 7f1d180c4740 20 client.26009 populate_metadata read hostname 'smithi044' 2024-08-20T03:06:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.581+0000 7f1d180c4740 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55c8d68093c0 0x55c8d680b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:06:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.581+0000 7f1d180c4740 4 client.26009 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:06:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.581+0000 7f1d180c4740 1 -- 172.21.15.44:0/919624056 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8d66f4d50 con 0x55c8d68093c0 2024-08-20T03:06:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.581+0000 7f1d0effd640 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55c8d68093c0 0x55c8d680b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:06:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.582+0000 7f1cf2ff5640 10 client.26009 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:52.582+0000 7f1d0effd640 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55c8d68093c0 0x55c8d680b7e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f1934b683445a75 server_cookie=4c4021cfdbd88af0 in_seq=0 out_seq=0 2024-08-20T03:06:54.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:53 smithi165 ceph-mon[22357]: pgmap v452: 97 pgs: 97 active+clean; 578 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 884 op/s 2024-08-20T03:06:54.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:53 smithi099 ceph-mon[21984]: pgmap v452: 97 pgs: 97 active+clean; 578 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 884 op/s 2024-08-20T03:06:54.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:53 smithi044 ceph-mon[18790]: pgmap v452: 97 pgs: 97 active+clean; 578 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 884 op/s 2024-08-20T03:06:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.816+0000 7f1cf2ff5640 1 -- 172.21.15.44:0/919624056 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8d66f4d50 con 0x55c8d68093c0 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.816+0000 7f1cf2ff5640 10 client.26009 handle_command_reply: tid=0 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.816+0000 7f1d180c4740 1 client.26009 shutdown 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d180c4740 1 -- 172.21.15.44:0/919624056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 msgr2=0x55c8d678f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d180c4740 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 0x55c8d678f560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1d040e1920 tx=0x7f1d0401f000 comp rx=0 tx=0).stop 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d180c4740 1 -- 172.21.15.44:0/919624056 shutdown_connections 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d180c4740 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55c8d68093c0 0x55c8d680b7e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d180c4740 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c8d678e5f0 0x55c8d678fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d180c4740 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c8d6782660 0x55c8d678f560 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d180c4740 1 --2- 172.21.15.44:0/919624056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8d6781d40 0x55c8d6796d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1cf2ff5640 0 client.26009 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d180c4740 1 -- 172.21.15.44:0/919624056 >> 172.21.15.44:0/919624056 conn(0x55c8d677fb70 msgr2=0x55c8d6784ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d180c4740 1 -- 172.21.15.44:0/919624056 shutdown_connections 2024-08-20T03:06:54.820 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d180c4740 1 -- 172.21.15.44:0/919624056 wait complete. 2024-08-20T03:06:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d180c4740 20 client.26009 trim_cache size 0 max 16384 2024-08-20T03:06:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:06:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:06:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:06:54.821 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:06:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2220495091 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1cf80b3e70 msgr2=0x7f1cf80b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d0ffff640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1cf80b3e70 0x7f1cf80b6330 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009a60 tx=0x7f1d080095c0 comp rx=0 tx=0).stop 2024-08-20T03:06:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2220495091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1010a260 msgr2=0x7f1d101a8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:06:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.817+0000 7f1d0ffff640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1010a260 0x7f1d101a8180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400e770 tx=0x7f1d0400ec30 comp rx=0 tx=0).stop 2024-08-20T03:06:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.818+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2220495091 shutdown_connections 2024-08-20T03:06:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.818+0000 7f1d0ffff640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1cf80b3e70 0x7f1cf80b6330 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:54.822 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.818+0000 7f1d0ffff640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d1011bdd0 0x7f1d101ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.818+0000 7f1d0ffff640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d1010a260 0x7f1d101a8180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.818+0000 7f1d0ffff640 1 --2- 172.21.15.44:0/2220495091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d10109e80 0x7f1d101a7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:06:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.818+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2220495091 >> 172.21.15.44:0/2220495091 conn(0x7f1d1006d8b0 msgr2=0x7f1d1011c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:06:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.818+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2220495091 shutdown_connections 2024-08-20T03:06:54.823 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:06:54.818+0000 7f1d0ffff640 1 -- 172.21.15.44:0/2220495091 wait complete. 2024-08-20T03:06:56.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:55 smithi165 ceph-mon[22357]: pgmap v453: 97 pgs: 97 active+clean; 578 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 409 op/s 2024-08-20T03:06:56.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:55 smithi099 ceph-mon[21984]: pgmap v453: 97 pgs: 97 active+clean; 578 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 409 op/s 2024-08-20T03:06:56.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:55 smithi044 ceph-mon[18790]: pgmap v453: 97 pgs: 97 active+clean; 578 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 409 op/s 2024-08-20T03:06:57.481 INFO:tasks.workunit.client.0.smithi044.stdout: 159 1586930 410 1117373 502 892321 1433 2024-08-20T03:06:58.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:57 smithi165 ceph-mon[22357]: pgmap v454: 97 pgs: 97 active+clean; 566 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 879 op/s 2024-08-20T03:06:58.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:57 smithi099 ceph-mon[21984]: pgmap v454: 97 pgs: 97 active+clean; 566 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 879 op/s 2024-08-20T03:06:58.257 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:57 smithi044 ceph-mon[18790]: pgmap v454: 97 pgs: 97 active+clean; 566 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 879 op/s 2024-08-20T03:07:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:06:59 smithi165 ceph-mon[22357]: pgmap v455: 97 pgs: 97 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.7 MiB/s wr, 1.20k op/s 2024-08-20T03:07:00.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:06:59 smithi099 ceph-mon[21984]: pgmap v455: 97 pgs: 97 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.7 MiB/s wr, 1.20k op/s 2024-08-20T03:07:00.299 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:06:59 smithi044 ceph-mon[18790]: pgmap v455: 97 pgs: 97 active+clean; 558 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.7 MiB/s wr, 1.20k op/s 2024-08-20T03:07:02.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:01 smithi165 ceph-mon[22357]: pgmap v456: 97 pgs: 97 active+clean; 566 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.0 MiB/s wr, 1.38k op/s 2024-08-20T03:07:02.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:01 smithi099 ceph-mon[21984]: pgmap v456: 97 pgs: 97 active+clean; 566 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.0 MiB/s wr, 1.38k op/s 2024-08-20T03:07:02.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:01 smithi044 ceph-mon[18790]: pgmap v456: 97 pgs: 97 active+clean; 566 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.0 MiB/s wr, 1.38k op/s 2024-08-20T03:07:04.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:03 smithi165 ceph-mon[22357]: pgmap v457: 97 pgs: 97 active+clean; 606 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.3 MiB/s wr, 1.67k op/s 2024-08-20T03:07:04.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:03 smithi099 ceph-mon[21984]: pgmap v457: 97 pgs: 97 active+clean; 606 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.3 MiB/s wr, 1.67k op/s 2024-08-20T03:07:04.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:03 smithi044 ceph-mon[18790]: pgmap v457: 97 pgs: 97 active+clean; 606 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.3 MiB/s wr, 1.67k op/s 2024-08-20T03:07:06.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:05 smithi165 ceph-mon[22357]: pgmap v458: 97 pgs: 97 active+clean; 624 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.9 MiB/s wr, 1.67k op/s 2024-08-20T03:07:06.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:05 smithi099 ceph-mon[21984]: pgmap v458: 97 pgs: 97 active+clean; 624 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.9 MiB/s wr, 1.67k op/s 2024-08-20T03:07:06.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:05 smithi044 ceph-mon[18790]: pgmap v458: 97 pgs: 97 active+clean; 624 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.9 MiB/s wr, 1.67k op/s 2024-08-20T03:07:07.480 INFO:tasks.workunit.client.0.smithi044.stdout: 165 1101463 431 776951 225 627310 974 2024-08-20T03:07:08.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:07 smithi165 ceph-mon[22357]: pgmap v459: 97 pgs: 97 active+clean; 639 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.3 MiB/s wr, 1.63k op/s 2024-08-20T03:07:08.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:07 smithi099 ceph-mon[21984]: pgmap v459: 97 pgs: 97 active+clean; 639 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.3 MiB/s wr, 1.63k op/s 2024-08-20T03:07:08.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:07 smithi044 ceph-mon[18790]: pgmap v459: 97 pgs: 97 active+clean; 639 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.3 MiB/s wr, 1.63k op/s 2024-08-20T03:07:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:08.990+0000 7f23993ba640 1 -- 172.21.15.44:0/3973239873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2394109e80 msgr2=0x7f2394110ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:08.993 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:08.990+0000 7f23993ba640 1 --2- 172.21.15.44:0/3973239873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2394109e80 0x7f2394110ee0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f238c009be0 tx=0x7f238c02a6f0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.001+0000 7f23993ba640 1 -- 172.21.15.44:0/3973239873 shutdown_connections 2024-08-20T03:07:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.001+0000 7f23993ba640 1 --2- 172.21.15.44:0/3973239873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f239411bdd0 0x7f239411e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.001+0000 7f23993ba640 1 --2- 172.21.15.44:0/3973239873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239410a260 0x7f239410a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.001+0000 7f23993ba640 1 --2- 172.21.15.44:0/3973239873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2394109e80 0x7f2394110ee0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.001+0000 7f23993ba640 1 -- 172.21.15.44:0/3973239873 >> 172.21.15.44:0/3973239873 conn(0x7f239406d8e0 msgr2=0x7f239406dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.001+0000 7f23993ba640 1 -- 172.21.15.44:0/3973239873 shutdown_connections 2024-08-20T03:07:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.001+0000 7f23993ba640 1 -- 172.21.15.44:0/3973239873 wait complete. 2024-08-20T03:07:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.003+0000 7f23993ba640 1 Processor -- start 2024-08-20T03:07:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.003+0000 7f23993ba640 1 -- start start 2024-08-20T03:07:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.005+0000 7f23993ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2394109e80 0x7f2394115230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.005+0000 7f23993ba640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f239410a260 0x7f2394115770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.005+0000 7f23993ba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239411bdd0 0x7f239411aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.005+0000 7f23993ba640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2394111940 con 0x7f239411bdd0 2024-08-20T03:07:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.005+0000 7f23993ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23941117c0 con 0x7f2394109e80 2024-08-20T03:07:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.005+0000 7f23993ba640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2394111ac0 con 0x7f239410a260 2024-08-20T03:07:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.005+0000 7f23937fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239411bdd0 0x7f239411aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.005+0000 7f23937fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239411bdd0 0x7f239411aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44212/0 (socket says 172.21.15.44:44212) 2024-08-20T03:07:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.005+0000 7f23937fe640 1 -- 172.21.15.44:0/229953945 learned_addr learned my addr 172.21.15.44:0/229953945 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.005+0000 7f23937fe640 1 -- 172.21.15.44:0/229953945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f239410a260 msgr2=0x7f2394115770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-20T03:07:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.005+0000 7f23937fe640 1 --2- 172.21.15.44:0/229953945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f239410a260 0x7f2394115770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.006+0000 7f23937fe640 1 -- 172.21.15.44:0/229953945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2394109e80 msgr2=0x7f2394115230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-20T03:07:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.006+0000 7f23937fe640 1 --2- 172.21.15.44:0/229953945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2394109e80 0x7f2394115230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.006+0000 7f23937fe640 1 -- 172.21.15.44:0/229953945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239411b0d0 con 0x7f239411bdd0 2024-08-20T03:07:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.006+0000 7f23937fe640 1 --2- 172.21.15.44:0/229953945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239411bdd0 0x7f239411aa90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f238400e970 tx=0x7f238400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.032+0000 7f2383fff640 1 -- 172.21.15.44:0/229953945 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238400ccc0 con 0x7f239411bdd0 2024-08-20T03:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.033+0000 7f23993ba640 1 -- 172.21.15.44:0/229953945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239411b3c0 con 0x7f239411bdd0 2024-08-20T03:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.033+0000 7f23993ba640 1 -- 172.21.15.44:0/229953945 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23941272f0 con 0x7f239411bdd0 2024-08-20T03:07:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.033+0000 7f2383fff640 1 -- 172.21.15.44:0/229953945 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23840054d0 con 0x7f239411bdd0 2024-08-20T03:07:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.034+0000 7f2383fff640 1 -- 172.21.15.44:0/229953945 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384010430 con 0x7f239411bdd0 2024-08-20T03:07:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.038+0000 7f239c288740 1 Processor -- start 2024-08-20T03:07:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.039+0000 7f239c288740 1 -- start start 2024-08-20T03:07:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.039+0000 7f2383fff640 1 -- 172.21.15.44:0/229953945 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2384004680 con 0x7f239411bdd0 2024-08-20T03:07:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.039+0000 7f239c288740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3add40 0x556cfe3ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.039+0000 7f239c288740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3ba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.039+0000 7f239c288740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3ba5f0 0x556cfe3bc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.039+0000 7f239c288740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556cfe323360 con 0x556cfe3ba5f0 2024-08-20T03:07:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.039+0000 7f239c288740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cfe312860 con 0x556cfe3add40 2024-08-20T03:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.039+0000 7f239c288740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556cfe3168a0 con 0x556cfe3ae660 2024-08-20T03:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.040+0000 7f23927fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3ba5f0 0x556cfe3bc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.040+0000 7f2392ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3ba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.040+0000 7f2392ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3ba0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:42342/0 (socket says 172.21.15.44:42342) 2024-08-20T03:07:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.040+0000 7f2392ffd640 1 -- 172.21.15.44:0/518281256 learned_addr learned my addr 172.21.15.44:0/518281256 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.040+0000 7f2392ffd640 1 -- 172.21.15.44:0/518281256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3add40 msgr2=0x556cfe3ae120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:07:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.040+0000 7f2392ffd640 1 --2- 172.21.15.44:0/518281256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3add40 0x556cfe3ae120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.040+0000 7f2392ffd640 1 -- 172.21.15.44:0/518281256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3ba5f0 msgr2=0x556cfe3bc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.041+0000 7f2392ffd640 1 --2- 172.21.15.44:0/518281256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3ba5f0 0x556cfe3bc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.041+0000 7f2392ffd640 1 -- 172.21.15.44:0/518281256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cfe2b9e40 con 0x556cfe3ae660 2024-08-20T03:07:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.041+0000 7f23927fc640 1 --2- 172.21.15.44:0/518281256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3ba5f0 0x556cfe3bc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:07:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.045+0000 7f2383fff640 1 --2- 172.21.15.44:0/229953945 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f237c0b3da0 0x7f237c0b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.045+0000 7f2392ffd640 1 --2- 172.21.15.44:0/518281256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3ba0b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f238c02ace0 tx=0x7f238c030a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d411711339eb1428 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.045+0000 7f23927fc640 1 --2- 172.21.15.44:0/229953945 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f237c0b3da0 0x7f237c0b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.050 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:07:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.050+0000 7f2383fff640 1 -- 172.21.15.44:0/229953945 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f23840e4370 con 0x7f239411bdd0 2024-08-20T03:07:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.050+0000 7f23927fc640 1 --2- 172.21.15.44:0/229953945 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f237c0b3da0 0x7f237c0b6260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2394074050 tx=0x7f238800b310 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.051+0000 7f23817fa640 1 -- 172.21.15.44:0/518281256 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c040020 con 0x556cfe3ae660 2024-08-20T03:07:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.051+0000 7f23817fa640 1 -- 172.21.15.44:0/518281256 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f238c030d10 con 0x556cfe3ae660 2024-08-20T03:07:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.052+0000 7f239c288740 1 -- 172.21.15.44:0/518281256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 msgr2=0x556cfe3ba0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.052+0000 7f239c288740 1 --2- 172.21.15.44:0/518281256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3ba0b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f238c02ace0 tx=0x7f238c030a20 comp rx=0 tx=0).stop 2024-08-20T03:07:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.067+0000 7f239c288740 1 -- 172.21.15.44:0/518281256 shutdown_connections 2024-08-20T03:07:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.067+0000 7f239c288740 1 --2- 172.21.15.44:0/518281256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3ba5f0 0x556cfe3bc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.067+0000 7f239c288740 1 --2- 172.21.15.44:0/518281256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3ba0b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.067+0000 7f239c288740 1 --2- 172.21.15.44:0/518281256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3add40 0x556cfe3ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.067+0000 7f239c288740 1 -- 172.21.15.44:0/518281256 >> 172.21.15.44:0/518281256 conn(0x556cfe3abb70 msgr2=0x556cfe3acd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.067+0000 7f239c288740 1 -- 172.21.15.44:0/518281256 shutdown_connections 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.067+0000 7f239c288740 1 -- 172.21.15.44:0/518281256 wait complete. 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.068+0000 7f239c288740 1 -- start start 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.068+0000 7f239c288740 1 Processor -- start 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.068+0000 7f239c288740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3add40 0x556cfe3b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.068+0000 7f239c288740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3b7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.068+0000 7f239c288740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3ba5f0 0x556cfe3b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.068+0000 7f239c288740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556cfe24bab0 con 0x556cfe3ba5f0 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.068+0000 7f239c288740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cfe299780 con 0x556cfe3add40 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.068+0000 7f239c288740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556cfe3bd2c0 con 0x556cfe3ae660 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.068+0000 7f239c288740 10 client.0 _ll_register_callbacks cb 0x556cfe021a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:07:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.069+0000 7f2392ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3add40 0x556cfe3b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.068+0000 7f23927fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.069+0000 7f2392ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3add40 0x556cfe3b9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47900/0 (socket says 172.21.15.44:47900) 2024-08-20T03:07:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.069+0000 7f2392ffd640 1 -- 172.21.15.44:0/4067107623 learned_addr learned my addr 172.21.15.44:0/4067107623 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.069+0000 7f23937fe640 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3ba5f0 0x556cfe3b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.069+0000 7f23927fc640 1 -- 172.21.15.44:0/4067107623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3add40 msgr2=0x556cfe3b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.069+0000 7f23927fc640 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3add40 0x556cfe3b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.069+0000 7f23927fc640 1 -- 172.21.15.44:0/4067107623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3ba5f0 msgr2=0x556cfe3b77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.069+0000 7f23927fc640 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3ba5f0 0x556cfe3b77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.069+0000 7f23927fc640 1 -- 172.21.15.44:0/4067107623 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cfe075790 con 0x556cfe3ae660 2024-08-20T03:07:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.070+0000 7f2392ffd640 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3add40 0x556cfe3b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:07:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.070+0000 7f23937fe640 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3ba5f0 0x556cfe3b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:07:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.070+0000 7f23927fc640 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3b7270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f238800bf40 tx=0x7f2388004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=119f5b32a3bb87d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.084+0000 7f237a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:07:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.084+0000 7f237a7fc640 1 -- 172.21.15.44:0/4067107623 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238801e830 con 0x556cfe3ae660 2024-08-20T03:07:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.085+0000 7f239c288740 1 -- 172.21.15.44:0/4067107623 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556cfe2c1e00 con 0x556cfe3ae660 2024-08-20T03:07:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.085+0000 7f237a7fc640 1 -- 172.21.15.44:0/4067107623 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f238801e9d0 con 0x556cfe3ae660 2024-08-20T03:07:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.085+0000 7f237a7fc640 1 -- 172.21.15.44:0/4067107623 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2388011e50 con 0x556cfe3ae660 2024-08-20T03:07:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.085+0000 7f239c288740 10 client.25984 fetch_fsmap learned FSMap version 30 2024-08-20T03:07:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.085+0000 7f239c288740 1 -- 172.21.15.44:0/4067107623 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556cfe0ee620 con 0x556cfe3ae660 2024-08-20T03:07:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.086+0000 7f237a7fc640 1 -- 172.21.15.44:0/4067107623 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556cfe2c1e00 con 0x556cfe3ae660 2024-08-20T03:07:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.086+0000 7f237a7fc640 1 -- 172.21.15.44:0/4067107623 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f238802e050 con 0x556cfe3ae660 2024-08-20T03:07:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.086+0000 7f239c288740 10 client.25984 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:07:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.086+0000 7f239c288740 10 client.25984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:07:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.086+0000 7f239c288740 20 client.25984 populate_metadata read hostname 'smithi044' 2024-08-20T03:07:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.086+0000 7f239c288740 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556cfe3ca500 0x556cfe3cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.086+0000 7f239c288740 4 client.25984 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:07:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.086+0000 7f239c288740 1 -- 172.21.15.44:0/4067107623 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cfe3252b0 con 0x556cfe3ca500 2024-08-20T03:07:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.086+0000 7f2392ffd640 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556cfe3ca500 0x556cfe3cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.087+0000 7f237a7fc640 10 client.25984 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.088+0000 7f2392ffd640 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556cfe3ca500 0x556cfe3cc940 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af1642bb00983f3 server_cookie=ba8ffa741acacfbc in_seq=0 out_seq=0 2024-08-20T03:07:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.092+0000 7f237a7fc640 1 -- 172.21.15.44:0/4067107623 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cfe3252b0 con 0x556cfe3ca500 2024-08-20T03:07:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.092+0000 7f237a7fc640 10 client.25984 handle_command_reply: tid=0 2024-08-20T03:07:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.092+0000 7f239c288740 1 client.25984 shutdown 2024-08-20T03:07:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.092+0000 7f239c288740 1 -- 172.21.15.44:0/4067107623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 msgr2=0x556cfe3b7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.092+0000 7f239c288740 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3b7270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f238800bf40 tx=0x7f2388004730 comp rx=0 tx=0).stop 2024-08-20T03:07:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.093+0000 7f239c288740 1 -- 172.21.15.44:0/4067107623 shutdown_connections 2024-08-20T03:07:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.093+0000 7f239c288740 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556cfe3ca500 0x556cfe3cc940 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.093+0000 7f239c288740 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3ba5f0 0x556cfe3b77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.093+0000 7f239c288740 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3b7270 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.093+0000 7f239c288740 1 --2- 172.21.15.44:0/4067107623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3add40 0x556cfe3b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.093+0000 7f239c288740 1 -- 172.21.15.44:0/4067107623 >> 172.21.15.44:0/4067107623 conn(0x556cfe3abb70 msgr2=0x556cfe3b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.093+0000 7f237a7fc640 0 client.25984 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.115+0000 7f239c288740 1 -- 172.21.15.44:0/4067107623 shutdown_connections 2024-08-20T03:07:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.115+0000 7f239c288740 1 -- 172.21.15.44:0/4067107623 wait complete. 2024-08-20T03:07:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.115+0000 7f239c288740 20 client.25984 trim_cache size 0 max 16384 2024-08-20T03:07:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.259+0000 7f239c288740 1 Processor -- start 2024-08-20T03:07:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.260+0000 7f239c288740 1 -- start start 2024-08-20T03:07:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.260+0000 7f239c288740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 0x556cfe3c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.261+0000 7f239c288740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3ab420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.261+0000 7f2392ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 0x556cfe3c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.261+0000 7f239c288740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3ba5f0 0x556cfe3d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.261+0000 7f2392ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 0x556cfe3c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:44248/0 (socket says 172.21.15.44:44248) 2024-08-20T03:07:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.261+0000 7f2392ffd640 1 -- 172.21.15.44:0/3842074982 learned_addr learned my addr 172.21.15.44:0/3842074982 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.261+0000 7f23927fc640 1 --2- 172.21.15.44:0/3842074982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.263 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.261+0000 7f239c288740 1 -- 172.21.15.44:0/3842074982 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f237400e760 con 0x556cfe3add40 2024-08-20T03:07:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.261+0000 7f239c288740 1 -- 172.21.15.44:0/3842074982 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2374010360 con 0x556cfe3ba5f0 2024-08-20T03:07:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.261+0000 7f239c288740 1 -- 172.21.15.44:0/3842074982 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f237400ee30 con 0x556cfe3ae660 2024-08-20T03:07:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.261+0000 7f23937fe640 1 --2- 172.21.15.44:0/3842074982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3ba5f0 0x556cfe3d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.262+0000 7f23927fc640 1 -- 172.21.15.44:0/3842074982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3ba5f0 msgr2=0x556cfe3d2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.262+0000 7f23927fc640 1 --2- 172.21.15.44:0/3842074982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3ba5f0 0x556cfe3d2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.262+0000 7f23927fc640 1 -- 172.21.15.44:0/3842074982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 msgr2=0x556cfe3c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.262+0000 7f23927fc640 1 --2- 172.21.15.44:0/3842074982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 0x556cfe3c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.264 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.262+0000 7f23927fc640 1 -- 172.21.15.44:0/3842074982 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cfe3a71a0 con 0x556cfe3ae660 2024-08-20T03:07:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.262+0000 7f2392ffd640 1 --2- 172.21.15.44:0/3842074982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 0x556cfe3c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:07:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.262+0000 7f23927fc640 1 --2- 172.21.15.44:0/3842074982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3ab420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f238802ffd0 tx=0x7f238800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3367b5b2ee304ea server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.283+0000 7f2380ff9640 1 -- 172.21.15.44:0/3842074982 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388026450 con 0x556cfe3ae660 2024-08-20T03:07:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.283+0000 7f2380ff9640 1 -- 172.21.15.44:0/3842074982 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23880265f0 con 0x556cfe3ae660 2024-08-20T03:07:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.283+0000 7f2380ff9640 1 -- 172.21.15.44:0/3842074982 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388026e00 con 0x556cfe3ae660 2024-08-20T03:07:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.284+0000 7f239c288740 1 -- 172.21.15.44:0/3842074982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 msgr2=0x556cfe3ab420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.284+0000 7f239c288740 1 --2- 172.21.15.44:0/3842074982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3ab420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f238802ffd0 tx=0x7f238800bab0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.285+0000 7f239c288740 1 -- 172.21.15.44:0/3842074982 shutdown_connections 2024-08-20T03:07:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.285+0000 7f239c288740 1 --2- 172.21.15.44:0/3842074982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3ba5f0 0x556cfe3d2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.285+0000 7f239c288740 1 --2- 172.21.15.44:0/3842074982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ae660 0x556cfe3ab420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.285+0000 7f239c288740 1 --2- 172.21.15.44:0/3842074982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 0x556cfe3c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.285+0000 7f239c288740 1 -- 172.21.15.44:0/3842074982 >> 172.21.15.44:0/3842074982 conn(0x556cfe3abb70 msgr2=0x556cfe3b5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.291+0000 7f239c288740 1 -- 172.21.15.44:0/3842074982 shutdown_connections 2024-08-20T03:07:09.294 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.293+0000 7f239c288740 1 -- 172.21.15.44:0/3842074982 wait complete. 2024-08-20T03:07:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.293+0000 7f239c288740 1 -- start start 2024-08-20T03:07:09.295 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.294+0000 7f239c288740 1 Processor -- start 2024-08-20T03:07:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.297+0000 7f239c288740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 0x556cfe3c2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.297+0000 7f239c288740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3ae660 0x556cfe3c1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.297+0000 7f239c288740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ba5f0 0x556cfe3c1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.297+0000 7f239c288740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f237400fbd0 con 0x556cfe3add40 2024-08-20T03:07:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.297+0000 7f239c288740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f237400f500 con 0x556cfe3ae660 2024-08-20T03:07:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.297+0000 7f239c288740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f237400c740 con 0x556cfe3ba5f0 2024-08-20T03:07:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.297+0000 7f239c288740 10 client.0 _ll_register_callbacks cb 0x7f2388010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:07:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.297+0000 7f23937fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ba5f0 0x556cfe3c1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.297+0000 7f23937fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ba5f0 0x556cfe3c1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:42358/0 (socket says 172.21.15.44:42358) 2024-08-20T03:07:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.297+0000 7f23937fe640 1 -- 172.21.15.44:0/212689410 learned_addr learned my addr 172.21.15.44:0/212689410 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.297+0000 7f23927fc640 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3ae660 0x556cfe3c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.298+0000 7f2392ffd640 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 0x556cfe3c2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.298+0000 7f23937fe640 1 -- 172.21.15.44:0/212689410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3ae660 msgr2=0x556cfe3c1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.298+0000 7f23937fe640 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3ae660 0x556cfe3c1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.298+0000 7f23937fe640 1 -- 172.21.15.44:0/212689410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 msgr2=0x556cfe3c2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.298+0000 7f23937fe640 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 0x556cfe3c2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.298+0000 7f23937fe640 1 -- 172.21.15.44:0/212689410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cfe3c2230 con 0x556cfe3ba5f0 2024-08-20T03:07:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.298+0000 7f23927fc640 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3ae660 0x556cfe3c1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:07:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.298+0000 7f2392ffd640 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 0x556cfe3c2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:07:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.298+0000 7f23937fe640 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ba5f0 0x556cfe3c1b90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f23840095e0 tx=0x7f23840b3e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f641c9fad973ca3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.314+0000 7f237a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:07:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.314+0000 7f237a7fc640 1 -- 172.21.15.44:0/212689410 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23840ab690 con 0x556cfe3ba5f0 2024-08-20T03:07:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.315+0000 7f237a7fc640 1 -- 172.21.15.44:0/212689410 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f23840a6b10 con 0x556cfe3ba5f0 2024-08-20T03:07:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.316+0000 7f239c288740 1 -- 172.21.15.44:0/212689410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23880265f0 con 0x556cfe3ba5f0 2024-08-20T03:07:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.318+0000 7f237a7fc640 1 -- 172.21.15.44:0/212689410 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f23840aaf90 con 0x556cfe3ba5f0 2024-08-20T03:07:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.318+0000 7f239c288740 10 client.25990 fetch_fsmap learned FSMap version 30 2024-08-20T03:07:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.318+0000 7f239c288740 1 -- 172.21.15.44:0/212689410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556cfe434110 con 0x556cfe3ba5f0 2024-08-20T03:07:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.318+0000 7f237a7fc640 1 -- 172.21.15.44:0/212689410 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23880265f0 con 0x556cfe3ba5f0 2024-08-20T03:07:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.320+0000 7f237a7fc640 1 -- 172.21.15.44:0/212689410 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f23840a9e60 con 0x556cfe3ba5f0 2024-08-20T03:07:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.320+0000 7f239c288740 10 client.25990 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:07:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.320+0000 7f239c288740 10 client.25990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:07:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.320+0000 7f239c288740 20 client.25990 populate_metadata read hostname 'smithi044' 2024-08-20T03:07:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.320+0000 7f239c288740 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556cfe434920 0x556cfe436d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.320+0000 7f239c288740 4 client.25990 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:07:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.320+0000 7f239c288740 1 -- 172.21.15.44:0/212689410 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556cfe320d50 con 0x556cfe434920 2024-08-20T03:07:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.320+0000 7f2392ffd640 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556cfe434920 0x556cfe436d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.321+0000 7f2392ffd640 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556cfe434920 0x556cfe436d40 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82d0b3a91c45bb94 server_cookie=296f77c48482da1a in_seq=0 out_seq=0 2024-08-20T03:07:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:09.321+0000 7f237a7fc640 10 client.25990 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:09 smithi165 ceph-mon[22357]: pgmap v460: 97 pgs: 97 active+clean; 647 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 1.15k op/s 2024-08-20T03:07:10.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:09 smithi099 ceph-mon[21984]: pgmap v460: 97 pgs: 97 active+clean; 647 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 1.15k op/s 2024-08-20T03:07:10.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:09 smithi044 ceph-mon[18790]: pgmap v460: 97 pgs: 97 active+clean; 647 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.2 MiB/s wr, 1.15k op/s 2024-08-20T03:07:11.118 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:07:11.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.799+0000 7fac2f9a0640 1 -- 172.21.15.44:0/3789759272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac2811bdd0 msgr2=0x7fac2811e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:11.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.799+0000 7fac2f9a0640 1 --2- 172.21.15.44:0/3789759272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac2811bdd0 0x7fac2811e1c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fac20009ca0 tx=0x7fac2002a980 comp rx=0 tx=0).stop 2024-08-20T03:07:11.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.800+0000 7fac2f9a0640 1 -- 172.21.15.44:0/3789759272 shutdown_connections 2024-08-20T03:07:11.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.800+0000 7fac2f9a0640 1 --2- 172.21.15.44:0/3789759272 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac2811bdd0 0x7fac2811e1c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.800+0000 7fac2f9a0640 1 --2- 172.21.15.44:0/3789759272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2810a260 0x7fac2810a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.800+0000 7fac2f9a0640 1 --2- 172.21.15.44:0/3789759272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac28109e80 0x7fac28110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.800+0000 7fac2f9a0640 1 -- 172.21.15.44:0/3789759272 >> 172.21.15.44:0/3789759272 conn(0x7fac2806dcc0 msgr2=0x7fac2806e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:11.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.802+0000 7fac2f9a0640 1 -- 172.21.15.44:0/3789759272 shutdown_connections 2024-08-20T03:07:11.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.802+0000 7fac2f9a0640 1 -- 172.21.15.44:0/3789759272 wait complete. 2024-08-20T03:07:11.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.803+0000 7fac2f9a0640 1 Processor -- start 2024-08-20T03:07:11.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.803+0000 7fac2f9a0640 1 -- start start 2024-08-20T03:07:11.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.804+0000 7fac2f9a0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac28109e80 0x7fac281a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.804+0000 7fac2f9a0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2810a260 0x7fac281a8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.804+0000 7fac2f9a0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2811bdd0 0x7fac281ad630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.804+0000 7fac2f9a0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fac28121860 con 0x7fac2810a260 2024-08-20T03:07:11.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.804+0000 7fac2f9a0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac281216e0 con 0x7fac28109e80 2024-08-20T03:07:11.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.804+0000 7fac2f9a0640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fac281219e0 con 0x7fac2811bdd0 2024-08-20T03:07:11.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.805+0000 7fac2d73c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac28109e80 0x7fac281a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.805+0000 7fac2d73c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac28109e80 0x7fac281a7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47938/0 (socket says 172.21.15.44:47938) 2024-08-20T03:07:11.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.805+0000 7fac2d73c640 1 -- 172.21.15.44:0/1737496881 learned_addr learned my addr 172.21.15.44:0/1737496881 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:11.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.807+0000 7fac2d73c640 1 -- 172.21.15.44:0/1737496881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2811bdd0 msgr2=0x7fac281ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:11.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.807+0000 7fac2d73c640 1 --2- 172.21.15.44:0/1737496881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2811bdd0 0x7fac281ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.807+0000 7fac2d73c640 1 -- 172.21.15.44:0/1737496881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2810a260 msgr2=0x7fac281a8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:11.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.807+0000 7fac2d73c640 1 --2- 172.21.15.44:0/1737496881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2810a260 0x7fac281a8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.807+0000 7fac2d73c640 1 -- 172.21.15.44:0/1737496881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac281ade00 con 0x7fac28109e80 2024-08-20T03:07:11.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.807+0000 7fac2d73c640 1 --2- 172.21.15.44:0/1737496881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac28109e80 0x7fac281a7d50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fac2400e9f0 tx=0x7fac2400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:11.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.808+0000 7fac1e7fc640 1 -- 172.21.15.44:0/1737496881 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2400cd80 con 0x7fac28109e80 2024-08-20T03:07:11.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.808+0000 7fac1e7fc640 1 -- 172.21.15.44:0/1737496881 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac240108e0 con 0x7fac28109e80 2024-08-20T03:07:11.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.808+0000 7fac1e7fc640 1 -- 172.21.15.44:0/1737496881 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac240188e0 con 0x7fac28109e80 2024-08-20T03:07:11.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.808+0000 7fac2f9a0640 1 -- 172.21.15.44:0/1737496881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac281ae0f0 con 0x7fac28109e80 2024-08-20T03:07:11.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.808+0000 7fac2f9a0640 1 -- 172.21.15.44:0/1737496881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac281115b0 con 0x7fac28109e80 2024-08-20T03:07:11.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.812+0000 7fac31d99740 1 Processor -- start 2024-08-20T03:07:11.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.816+0000 7fac31d99740 1 -- start start 2024-08-20T03:07:11.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.816+0000 7fac31d99740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b58dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.816+0000 7fac31d99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b58e150 0x55e03b599ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.816+0000 7fac31d99740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b59a0e0 0x55e03b59c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.816+0000 7fac31d99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e03b5048e0 con 0x55e03b58e150 2024-08-20T03:07:11.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.816+0000 7fac31d99740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e03b4f3de0 con 0x55e03b59a0e0 2024-08-20T03:07:11.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.816+0000 7fac31d99740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e03b4f7e20 con 0x55e03b58d830 2024-08-20T03:07:11.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.816+0000 7fac2df3d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b58dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.817+0000 7fac2cf3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b59a0e0 0x55e03b59c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.817+0000 7fac2df3d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b58dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:42384/0 (socket says 172.21.15.44:42384) 2024-08-20T03:07:11.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.817+0000 7fac2df3d640 1 -- 172.21.15.44:0/3772076187 learned_addr learned my addr 172.21.15.44:0/3772076187 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:11.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.818+0000 7fac2cf3b640 1 -- 172.21.15.44:0/3772076187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 msgr2=0x55e03b58dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:11.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.819+0000 7fac2cf3b640 1 --2- 172.21.15.44:0/3772076187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b58dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.819+0000 7fac2cf3b640 1 -- 172.21.15.44:0/3772076187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b58e150 msgr2=0x55e03b599ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:07:11.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.819+0000 7fac2cf3b640 1 --2- 172.21.15.44:0/3772076187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b58e150 0x55e03b599ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.819+0000 7fac2cf3b640 1 -- 172.21.15.44:0/3772076187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e03b253650 con 0x55e03b59a0e0 2024-08-20T03:07:11.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.819+0000 7fac2cf3b640 1 --2- 172.21.15.44:0/3772076187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b59a0e0 0x55e03b59c4d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fac1800a900 tx=0x7fac1800adc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f566e79018416d7b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:11.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.823+0000 7fac1e7fc640 1 -- 172.21.15.44:0/1737496881 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fac24010430 con 0x7fac28109e80 2024-08-20T03:07:11.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.823+0000 7fac137fe640 1 -- 172.21.15.44:0/3772076187 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1800cd80 con 0x55e03b59a0e0 2024-08-20T03:07:11.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.823+0000 7fac137fe640 1 -- 172.21.15.44:0/3772076187 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1800f900 con 0x55e03b59a0e0 2024-08-20T03:07:11.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.823+0000 7fac137fe640 1 -- 172.21.15.44:0/3772076187 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac180188c0 con 0x55e03b59a0e0 2024-08-20T03:07:11.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.824+0000 7fac31d99740 1 -- 172.21.15.44:0/3772076187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b59a0e0 msgr2=0x55e03b59c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:11.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.824+0000 7fac31d99740 1 --2- 172.21.15.44:0/3772076187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b59a0e0 0x55e03b59c4d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fac1800a900 tx=0x7fac1800adc0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.824+0000 7fac1e7fc640 1 --2- 172.21.15.44:0/1737496881 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fac140b3f50 0x7fac140b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.824+0000 7fac1e7fc640 1 -- 172.21.15.44:0/1737496881 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fac240e41e0 con 0x7fac28109e80 2024-08-20T03:07:11.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.838+0000 7fac31d99740 1 -- 172.21.15.44:0/3772076187 shutdown_connections 2024-08-20T03:07:11.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.838+0000 7fac31d99740 1 --2- 172.21.15.44:0/3772076187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b59a0e0 0x55e03b59c4d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.838+0000 7fac31d99740 1 --2- 172.21.15.44:0/3772076187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b58e150 0x55e03b599ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.838+0000 7fac31d99740 1 --2- 172.21.15.44:0/3772076187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b58dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.838+0000 7fac31d99740 1 -- 172.21.15.44:0/3772076187 >> 172.21.15.44:0/3772076187 conn(0x55e03b58b660 msgr2=0x55e03b58c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:11.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.841+0000 7fac2df3d640 1 --2- 172.21.15.44:0/1737496881 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fac140b3f50 0x7fac140b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.841+0000 7fac31d99740 1 -- 172.21.15.44:0/3772076187 shutdown_connections 2024-08-20T03:07:11.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.842+0000 7fac31d99740 1 -- 172.21.15.44:0/3772076187 wait complete. 2024-08-20T03:07:11.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.842+0000 7fac31d99740 1 -- start start 2024-08-20T03:07:11.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.842+0000 7fac31d99740 1 Processor -- start 2024-08-20T03:07:11.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.843+0000 7fac31d99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b58d830 0x55e03b5970d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.843+0000 7fac31d99740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b597610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.843+0000 7fac31d99740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b59a0e0 0x55e03b597b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.843+0000 7fac31d99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e03b356700 con 0x55e03b58d830 2024-08-20T03:07:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.843+0000 7fac31d99740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e03b47a7c0 con 0x55e03b58e150 2024-08-20T03:07:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.843+0000 7fac31d99740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e03b59cd90 con 0x55e03b59a0e0 2024-08-20T03:07:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.843+0000 7fac2df3d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b59a0e0 0x55e03b597b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.843+0000 7fac2df3d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b59a0e0 0x55e03b597b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:42390/0 (socket says 172.21.15.44:42390) 2024-08-20T03:07:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.843+0000 7fac2df3d640 1 -- 172.21.15.44:0/667199251 learned_addr learned my addr 172.21.15.44:0/667199251 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.843+0000 7fac31d99740 10 client.0 _ll_register_callbacks cb 0x55e03b4ed6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:07:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.844+0000 7fac2cf3b640 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b597610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.844+0000 7fac2d73c640 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b58d830 0x55e03b5970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.844+0000 7fac2df3d640 1 -- 172.21.15.44:0/667199251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 msgr2=0x55e03b597610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:11.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.844+0000 7fac2df3d640 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b597610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.844+0000 7fac2df3d640 1 -- 172.21.15.44:0/667199251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b58d830 msgr2=0x55e03b5970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:11.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.844+0000 7fac2df3d640 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b58d830 0x55e03b5970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.844+0000 7fac2df3d640 1 -- 172.21.15.44:0/667199251 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e03b5a6220 con 0x55e03b59a0e0 2024-08-20T03:07:11.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.850+0000 7fac2df3d640 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b59a0e0 0x55e03b597b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fac20003f60 tx=0x7fac20003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6daa821ef3e132b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:11.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.852+0000 7fac10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:07:11.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.852+0000 7fac10ff9640 1 -- 172.21.15.44:0/667199251 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20050060 con 0x55e03b59a0e0 2024-08-20T03:07:11.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.852+0000 7fac31d99740 1 -- 172.21.15.44:0/667199251 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e03b4a3100 con 0x55e03b59a0e0 2024-08-20T03:07:11.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.852+0000 7fac10ff9640 1 -- 172.21.15.44:0/667199251 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fac20030680 con 0x55e03b59a0e0 2024-08-20T03:07:11.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.856+0000 7fac10ff9640 1 -- 172.21.15.44:0/667199251 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fac20043050 con 0x55e03b59a0e0 2024-08-20T03:07:11.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.856+0000 7fac31d99740 10 client.25996 fetch_fsmap learned FSMap version 30 2024-08-20T03:07:11.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.856+0000 7fac31d99740 1 -- 172.21.15.44:0/667199251 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e03b5cac70 con 0x55e03b59a0e0 2024-08-20T03:07:11.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.858+0000 7fac10ff9640 1 -- 172.21.15.44:0/667199251 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e03b4a3100 con 0x55e03b59a0e0 2024-08-20T03:07:11.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.858+0000 7fac10ff9640 1 -- 172.21.15.44:0/667199251 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fac2004a6a0 con 0x55e03b59a0e0 2024-08-20T03:07:11.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.858+0000 7fac31d99740 10 client.25996 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:07:11.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.858+0000 7fac31d99740 10 client.25996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:07:11.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.858+0000 7fac31d99740 20 client.25996 populate_metadata read hostname 'smithi044' 2024-08-20T03:07:11.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.858+0000 7fac31d99740 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e03b5cc5b0 0x55e03b5cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.858+0000 7fac31d99740 4 client.25996 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:07:11.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.858+0000 7fac31d99740 1 -- 172.21.15.44:0/667199251 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e03b506830 con 0x55e03b5cc5b0 2024-08-20T03:07:11.865 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.863+0000 7fac2df3d640 1 --2- 172.21.15.44:0/1737496881 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fac140b3f50 0x7fac140b6410 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55e03b5cabe0 tx=0x7fac2004a000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:11.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.865+0000 7fac2d73c640 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e03b5cc5b0 0x55e03b5cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.874+0000 7fac10ff9640 10 client.25996 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:11.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.874+0000 7fac2d73c640 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e03b5cc5b0 0x55e03b5cea30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c1ef3a7f7a1c50f server_cookie=639f7780e640fb13 in_seq=0 out_seq=0 2024-08-20T03:07:11.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.900+0000 7fac10ff9640 1 -- 172.21.15.44:0/667199251 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e03b506830 con 0x55e03b5cc5b0 2024-08-20T03:07:11.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac10ff9640 10 client.25996 handle_command_reply: tid=0 2024-08-20T03:07:11.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac31d99740 1 client.25996 shutdown 2024-08-20T03:07:11.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac31d99740 1 -- 172.21.15.44:0/667199251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b59a0e0 msgr2=0x55e03b597b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:11.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac31d99740 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b59a0e0 0x55e03b597b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fac20003f60 tx=0x7fac20003f90 comp rx=0 tx=0).stop 2024-08-20T03:07:11.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac31d99740 1 -- 172.21.15.44:0/667199251 shutdown_connections 2024-08-20T03:07:11.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac31d99740 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e03b5cc5b0 0x55e03b5cea30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac31d99740 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b59a0e0 0x55e03b597b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac31d99740 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b597610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac31d99740 1 --2- 172.21.15.44:0/667199251 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b58d830 0x55e03b5970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac31d99740 1 -- 172.21.15.44:0/667199251 >> 172.21.15.44:0/667199251 conn(0x55e03b58b660 msgr2=0x55e03b591200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:11.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac10ff9640 0 client.25996 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:11.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac31d99740 1 -- 172.21.15.44:0/667199251 shutdown_connections 2024-08-20T03:07:11.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.901+0000 7fac31d99740 1 -- 172.21.15.44:0/667199251 wait complete. 2024-08-20T03:07:11.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.902+0000 7fac31d99740 20 client.25996 trim_cache size 0 max 16384 2024-08-20T03:07:11.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.938+0000 7fac31d99740 1 Processor -- start 2024-08-20T03:07:11.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.947+0000 7fac31d99740 1 -- start start 2024-08-20T03:07:11.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.947+0000 7fac31d99740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b597750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.947+0000 7fac31d99740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b5d4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.947+0000 7fac31d99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b59a0e0 0x55e03b5d4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.947+0000 7fac31d99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c00e710 con 0x55e03b59a0e0 2024-08-20T03:07:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.947+0000 7fac31d99740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c010310 con 0x55e03b58e150 2024-08-20T03:07:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.947+0000 7fac31d99740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c00ede0 con 0x55e03b58d830 2024-08-20T03:07:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.949+0000 7fac2d73c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b597750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.949+0000 7fac2cf3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b5d4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.949+0000 7fac2cf3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b5d4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47968/0 (socket says 172.21.15.44:47968) 2024-08-20T03:07:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.949+0000 7fac2cf3b640 1 -- 172.21.15.44:0/568083572 learned_addr learned my addr 172.21.15.44:0/568083572 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:11.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.950+0000 7fac2df3d640 1 --2- 172.21.15.44:0/568083572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b59a0e0 0x55e03b5d4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.952+0000 7fac2d73c640 1 -- 172.21.15.44:0/568083572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 msgr2=0x55e03b5d4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:11.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.952+0000 7fac2d73c640 1 --2- 172.21.15.44:0/568083572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b5d4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.952+0000 7fac2d73c640 1 -- 172.21.15.44:0/568083572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b59a0e0 msgr2=0x55e03b5d4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:11.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.952+0000 7fac2d73c640 1 --2- 172.21.15.44:0/568083572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b59a0e0 0x55e03b5d4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.952+0000 7fac2d73c640 1 -- 172.21.15.44:0/568083572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e03b58df80 con 0x55e03b58d830 2024-08-20T03:07:11.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.953+0000 7fac2d73c640 1 --2- 172.21.15.44:0/568083572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b597750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fac240e1940 tx=0x7fac24009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1184af0ff61af19 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:11.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.958+0000 7fac12ffd640 1 -- 172.21.15.44:0/568083572 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac240bf070 con 0x55e03b58d830 2024-08-20T03:07:11.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.958+0000 7fac12ffd640 1 -- 172.21.15.44:0/568083572 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac240aae00 con 0x55e03b58d830 2024-08-20T03:07:11.960 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.958+0000 7fac12ffd640 1 -- 172.21.15.44:0/568083572 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac240e1bc0 con 0x55e03b58d830 2024-08-20T03:07:11.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.959+0000 7fac31d99740 1 -- 172.21.15.44:0/568083572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 msgr2=0x55e03b597750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:11.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.959+0000 7fac31d99740 1 --2- 172.21.15.44:0/568083572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b597750 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fac240e1940 tx=0x7fac24009310 comp rx=0 tx=0).stop 2024-08-20T03:07:11.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.962+0000 7fac31d99740 1 -- 172.21.15.44:0/568083572 shutdown_connections 2024-08-20T03:07:11.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.962+0000 7fac31d99740 1 --2- 172.21.15.44:0/568083572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b59a0e0 0x55e03b5d4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.962+0000 7fac31d99740 1 --2- 172.21.15.44:0/568083572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b5d4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.962+0000 7fac31d99740 1 --2- 172.21.15.44:0/568083572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b597750 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.962+0000 7fac31d99740 1 -- 172.21.15.44:0/568083572 >> 172.21.15.44:0/568083572 conn(0x55e03b58b660 msgr2=0x55e03b597040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:11.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.968+0000 7fac31d99740 1 -- 172.21.15.44:0/568083572 shutdown_connections 2024-08-20T03:07:11.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.968+0000 7fac31d99740 1 -- 172.21.15.44:0/568083572 wait complete. 2024-08-20T03:07:11.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.968+0000 7fac31d99740 1 -- start start 2024-08-20T03:07:11.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.968+0000 7fac31d99740 1 Processor -- start 2024-08-20T03:07:11.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.971+0000 7fac31d99740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b59d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.971+0000 7fac31d99740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b59d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.971+0000 7fac31d99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b59a0e0 0x55e03b59daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.971+0000 7fac31d99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c00fb80 con 0x55e03b59a0e0 2024-08-20T03:07:11.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.971+0000 7fac31d99740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c00f4b0 con 0x55e03b58e150 2024-08-20T03:07:11.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.971+0000 7fac31d99740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fac0c00c6f0 con 0x55e03b58d830 2024-08-20T03:07:11.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.971+0000 7fac31d99740 10 client.0 _ll_register_callbacks cb 0x7fac24005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:07:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.972+0000 7fac2cf3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b59d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.972+0000 7fac2cf3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b59d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:47978/0 (socket says 172.21.15.44:47978) 2024-08-20T03:07:11.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.972+0000 7fac2cf3b640 1 -- 172.21.15.44:0/4179013388 learned_addr learned my addr 172.21.15.44:0/4179013388 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.973+0000 7fac2df3d640 1 --2- 172.21.15.44:0/4179013388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b59a0e0 0x55e03b59daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.974+0000 7fac2cf3b640 1 -- 172.21.15.44:0/4179013388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 msgr2=0x55e03b59d070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:07:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.974+0000 7fac2cf3b640 1 --2- 172.21.15.44:0/4179013388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b59d070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.974+0000 7fac2cf3b640 1 -- 172.21.15.44:0/4179013388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b59a0e0 msgr2=0x55e03b59daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.974+0000 7fac2cf3b640 1 --2- 172.21.15.44:0/4179013388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b59a0e0 0x55e03b59daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:11.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.974+0000 7fac2cf3b640 1 -- 172.21.15.44:0/4179013388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e03b6353f0 con 0x55e03b58e150 2024-08-20T03:07:11.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.974+0000 7fac2cf3b640 1 --2- 172.21.15.44:0/4179013388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b59d5b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fac1800b010 tx=0x7fac1800a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b381d5ab53f81e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:11.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.975+0000 7fac10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:07:11.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.975+0000 7fac10ff9640 1 -- 172.21.15.44:0/4179013388 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18027030 con 0x55e03b58e150 2024-08-20T03:07:11.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.975+0000 7fac10ff9640 1 -- 172.21.15.44:0/4179013388 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fac18018d80 con 0x55e03b58e150 2024-08-20T03:07:11.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.975+0000 7fac31d99740 1 -- 172.21.15.44:0/4179013388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac240aae00 con 0x55e03b58e150 2024-08-20T03:07:11.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.975+0000 7fac10ff9640 1 -- 172.21.15.44:0/4179013388 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fac18005b30 con 0x55e03b58e150 2024-08-20T03:07:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.975+0000 7fac31d99740 10 client.26036 fetch_fsmap learned FSMap version 30 2024-08-20T03:07:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.975+0000 7fac31d99740 1 -- 172.21.15.44:0/4179013388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e03b636710 con 0x55e03b58e150 2024-08-20T03:07:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.976+0000 7fac10ff9640 1 -- 172.21.15.44:0/4179013388 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac240aae00 con 0x55e03b58e150 2024-08-20T03:07:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.976+0000 7fac10ff9640 1 -- 172.21.15.44:0/4179013388 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fac1800f4b0 con 0x55e03b58e150 2024-08-20T03:07:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.976+0000 7fac31d99740 10 client.26036 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:07:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.976+0000 7fac31d99740 10 client.26036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:07:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.976+0000 7fac31d99740 20 client.26036 populate_metadata read hostname 'smithi044' 2024-08-20T03:07:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.976+0000 7fac31d99740 1 --2- 172.21.15.44:0/4179013388 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e03b636b50 0x55e03b638f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.976+0000 7fac31d99740 4 client.26036 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:07:11.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.976+0000 7fac31d99740 1 -- 172.21.15.44:0/4179013388 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e03b5022d0 con 0x55e03b636b50 2024-08-20T03:07:11.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.981+0000 7fac2d73c640 1 --2- 172.21.15.44:0/4179013388 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e03b636b50 0x55e03b638f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:12.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.998+0000 7fac2d73c640 1 --2- 172.21.15.44:0/4179013388 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e03b636b50 0x55e03b638f70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aea3ad78bd178ea8 server_cookie=7063e02e98985478 in_seq=0 out_seq=0 2024-08-20T03:07:12.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:11.998+0000 7fac10ff9640 10 client.26036 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:12.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.006+0000 7fac10ff9640 1 -- 172.21.15.44:0/4179013388 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55e03b5022d0 con 0x55e03b636b50 2024-08-20T03:07:12.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.006+0000 7fac10ff9640 10 client.26036 handle_command_reply: tid=0 2024-08-20T03:07:12.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.006+0000 7fac31d99740 1 client.26036 shutdown 2024-08-20T03:07:12.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.008+0000 7fac31d99740 1 -- 172.21.15.44:0/4179013388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 msgr2=0x55e03b59d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:12.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.008+0000 7fac31d99740 1 --2- 172.21.15.44:0/4179013388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b59d5b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fac1800b010 tx=0x7fac1800a2c0 comp rx=0 tx=0).stop 2024-08-20T03:07:12.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.008+0000 7fac31d99740 1 -- 172.21.15.44:0/4179013388 shutdown_connections 2024-08-20T03:07:12.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.008+0000 7fac31d99740 1 --2- 172.21.15.44:0/4179013388 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e03b636b50 0x55e03b638f70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:12.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.008+0000 7fac31d99740 1 --2- 172.21.15.44:0/4179013388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e03b59a0e0 0x55e03b59daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:12.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.008+0000 7fac31d99740 1 --2- 172.21.15.44:0/4179013388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e03b58e150 0x55e03b59d5b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.008+0000 7fac31d99740 1 --2- 172.21.15.44:0/4179013388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e03b58d830 0x55e03b59d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.008+0000 7fac31d99740 1 -- 172.21.15.44:0/4179013388 >> 172.21.15.44:0/4179013388 conn(0x55e03b58b660 msgr2=0x55e03b591140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.008+0000 7fac10ff9640 0 client.26036 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.008+0000 7fac31d99740 1 -- 172.21.15.44:0/4179013388 shutdown_connections 2024-08-20T03:07:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.008+0000 7fac31d99740 1 -- 172.21.15.44:0/4179013388 wait complete. 2024-08-20T03:07:12.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.009+0000 7fac31d99740 20 client.26036 trim_cache size 0 max 16384 2024-08-20T03:07:12.011 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:07:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "scrub active (8292 inodes in the stack)", 2024-08-20T03:07:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-08-20T03:07:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "a24f9f81-df91-4f22-a871-09cc9455fee8": { 2024-08-20T03:07:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-08-20T03:07:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "a24f9f81-df91-4f22-a871-09cc9455fee8", 2024-08-20T03:07:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-08-20T03:07:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:07:12.012 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:07:12.012 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:07:12.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.009+0000 7fac2f9a0640 1 -- 172.21.15.44:0/1737496881 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fac140b3f50 msgr2=0x7fac140b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:12.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.010+0000 7fac2f9a0640 1 --2- 172.21.15.44:0/1737496881 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fac140b3f50 0x7fac140b6410 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55e03b5cabe0 tx=0x7fac2004a000 comp rx=0 tx=0).stop 2024-08-20T03:07:12.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.010+0000 7fac2f9a0640 1 -- 172.21.15.44:0/1737496881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac28109e80 msgr2=0x7fac281a7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:12.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.010+0000 7fac2f9a0640 1 --2- 172.21.15.44:0/1737496881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac28109e80 0x7fac281a7d50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fac2400e9f0 tx=0x7fac2400eeb0 comp rx=0 tx=0).stop 2024-08-20T03:07:12.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.010+0000 7fac2f9a0640 1 -- 172.21.15.44:0/1737496881 shutdown_connections 2024-08-20T03:07:12.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.010+0000 7fac2f9a0640 1 --2- 172.21.15.44:0/1737496881 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fac140b3f50 0x7fac140b6410 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.010+0000 7fac2f9a0640 1 --2- 172.21.15.44:0/1737496881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fac2811bdd0 0x7fac281ad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.010+0000 7fac2f9a0640 1 --2- 172.21.15.44:0/1737496881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac2810a260 0x7fac281a8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.010+0000 7fac2f9a0640 1 --2- 172.21.15.44:0/1737496881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac28109e80 0x7fac281a7d50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.010+0000 7fac2f9a0640 1 -- 172.21.15.44:0/1737496881 >> 172.21.15.44:0/1737496881 conn(0x7fac2806dcc0 msgr2=0x7fac2811c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.010+0000 7fac2f9a0640 1 -- 172.21.15.44:0/1737496881 shutdown_connections 2024-08-20T03:07:12.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:12.010+0000 7fac2f9a0640 1 -- 172.21.15.44:0/1737496881 wait complete. 2024-08-20T03:07:12.053 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:11 smithi044 ceph-mon[18790]: pgmap v461: 97 pgs: 97 active+clean; 651 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.0 MiB/s wr, 785 op/s 2024-08-20T03:07:12.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:11 smithi099 ceph-mon[21984]: pgmap v461: 97 pgs: 97 active+clean; 651 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.0 MiB/s wr, 785 op/s 2024-08-20T03:07:12.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:11 smithi165 ceph-mon[22357]: pgmap v461: 97 pgs: 97 active+clean; 651 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.0 MiB/s wr, 785 op/s 2024-08-20T03:07:12.980 INFO:tasks.cephfs.filesystem:scrub status for tag:a24f9f81-df91-4f22-a871-09cc9455fee8 - {'path': '/', 'tag': 'a24f9f81-df91-4f22-a871-09cc9455fee8', 'options': 'recursive,force'} 2024-08-20T03:07:14.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:13 smithi044 ceph-mon[18790]: pgmap v462: 97 pgs: 97 active+clean; 658 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.4 MiB/s wr, 919 op/s 2024-08-20T03:07:14.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:13 smithi099 ceph-mon[21984]: pgmap v462: 97 pgs: 97 active+clean; 658 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.4 MiB/s wr, 919 op/s 2024-08-20T03:07:14.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:13 smithi165 ceph-mon[22357]: pgmap v462: 97 pgs: 97 active+clean; 658 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.4 MiB/s wr, 919 op/s 2024-08-20T03:07:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.914+0000 7f237a7fc640 1 -- 172.21.15.44:0/212689410 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556cfe320d50 con 0x556cfe434920 2024-08-20T03:07:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.914+0000 7f237a7fc640 10 client.25990 handle_command_reply: tid=0 2024-08-20T03:07:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.914+0000 7f239c288740 1 client.25990 shutdown 2024-08-20T03:07:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.915+0000 7f239c288740 1 -- 172.21.15.44:0/212689410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ba5f0 msgr2=0x556cfe3c1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.915+0000 7f239c288740 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ba5f0 0x556cfe3c1b90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f23840095e0 tx=0x7f23840b3e30 comp rx=0 tx=0).stop 2024-08-20T03:07:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.915+0000 7f239c288740 1 -- 172.21.15.44:0/212689410 shutdown_connections 2024-08-20T03:07:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.915+0000 7f239c288740 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x556cfe434920 0x556cfe436d40 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.915+0000 7f239c288740 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556cfe3ba5f0 0x556cfe3c1b90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.915+0000 7f239c288740 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cfe3ae660 0x556cfe3c1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.915+0000 7f239c288740 1 --2- 172.21.15.44:0/212689410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556cfe3add40 0x556cfe3c2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.915+0000 7f239c288740 1 -- 172.21.15.44:0/212689410 >> 172.21.15.44:0/212689410 conn(0x556cfe3abb70 msgr2=0x556cfe3b88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.915+0000 7f237a7fc640 0 client.25990 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.915+0000 7f239c288740 1 -- 172.21.15.44:0/212689410 shutdown_connections 2024-08-20T03:07:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.915+0000 7f239c288740 1 -- 172.21.15.44:0/212689410 wait complete. 2024-08-20T03:07:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.915+0000 7f239c288740 20 client.25990 trim_cache size 0 max 16384 2024-08-20T03:07:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:07:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:07:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:07:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:07:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.930+0000 7f23993ba640 1 -- 172.21.15.44:0/229953945 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f237c0b3da0 msgr2=0x7f237c0b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.930+0000 7f23993ba640 1 --2- 172.21.15.44:0/229953945 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f237c0b3da0 0x7f237c0b6260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2394074050 tx=0x7f238800b310 comp rx=0 tx=0).stop 2024-08-20T03:07:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.930+0000 7f23993ba640 1 -- 172.21.15.44:0/229953945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239411bdd0 msgr2=0x7f239411aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.931+0000 7f23993ba640 1 --2- 172.21.15.44:0/229953945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239411bdd0 0x7f239411aa90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f238400e970 tx=0x7f238400ee30 comp rx=0 tx=0).stop 2024-08-20T03:07:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.933+0000 7f23993ba640 1 -- 172.21.15.44:0/229953945 shutdown_connections 2024-08-20T03:07:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.933+0000 7f23993ba640 1 --2- 172.21.15.44:0/229953945 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f237c0b3da0 0x7f237c0b6260 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.933+0000 7f23993ba640 1 --2- 172.21.15.44:0/229953945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f239411bdd0 0x7f239411aa90 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.933+0000 7f23993ba640 1 --2- 172.21.15.44:0/229953945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f239410a260 0x7f2394115770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.933+0000 7f23993ba640 1 --2- 172.21.15.44:0/229953945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2394109e80 0x7f2394115230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.933+0000 7f23993ba640 1 -- 172.21.15.44:0/229953945 >> 172.21.15.44:0/229953945 conn(0x7f239406d8e0 msgr2=0x7f23940712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.933+0000 7f23993ba640 1 -- 172.21.15.44:0/229953945 shutdown_connections 2024-08-20T03:07:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:14.934+0000 7f23993ba640 1 -- 172.21.15.44:0/229953945 wait complete. 2024-08-20T03:07:16.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:15 smithi099 ceph-mon[21984]: pgmap v463: 97 pgs: 97 active+clean; 654 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 795 op/s 2024-08-20T03:07:16.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:15 smithi044 ceph-mon[18790]: pgmap v463: 97 pgs: 97 active+clean; 654 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 795 op/s 2024-08-20T03:07:16.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:15 smithi165 ceph-mon[22357]: pgmap v463: 97 pgs: 97 active+clean; 654 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 795 op/s 2024-08-20T03:07:17.480 INFO:tasks.workunit.client.0.smithi044.stdout: 172 1812180 429 1259515 428 1032024 1449 2024-08-20T03:07:18.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:17 smithi099 ceph-mon[21984]: pgmap v464: 97 pgs: 97 active+clean; 642 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 918 op/s 2024-08-20T03:07:18.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:17 smithi044 ceph-mon[18790]: pgmap v464: 97 pgs: 97 active+clean; 642 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 918 op/s 2024-08-20T03:07:18.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:17 smithi165 ceph-mon[22357]: pgmap v464: 97 pgs: 97 active+clean; 642 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 918 op/s 2024-08-20T03:07:20.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:19 smithi099 ceph-mon[21984]: pgmap v465: 97 pgs: 97 active+clean; 639 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 930 op/s 2024-08-20T03:07:20.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:19 smithi044 ceph-mon[18790]: pgmap v465: 97 pgs: 97 active+clean; 639 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 930 op/s 2024-08-20T03:07:20.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:19 smithi165 ceph-mon[22357]: pgmap v465: 97 pgs: 97 active+clean; 639 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 930 op/s 2024-08-20T03:07:22.188 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:21 smithi165 ceph-mon[22357]: pgmap v466: 97 pgs: 97 active+clean; 631 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 985 op/s 2024-08-20T03:07:22.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:21 smithi044 ceph-mon[18790]: pgmap v466: 97 pgs: 97 active+clean; 631 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 985 op/s 2024-08-20T03:07:22.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:21 smithi099 ceph-mon[21984]: pgmap v466: 97 pgs: 97 active+clean; 631 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 985 op/s 2024-08-20T03:07:24.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:23 smithi044 ceph-mon[18790]: pgmap v467: 97 pgs: 97 active+clean; 644 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.3 MiB/s wr, 1.23k op/s 2024-08-20T03:07:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:23 smithi165 ceph-mon[22357]: pgmap v467: 97 pgs: 97 active+clean; 644 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.3 MiB/s wr, 1.23k op/s 2024-08-20T03:07:24.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:23 smithi099 ceph-mon[21984]: pgmap v467: 97 pgs: 97 active+clean; 644 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.3 MiB/s wr, 1.23k op/s 2024-08-20T03:07:26.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:25 smithi044 ceph-mon[18790]: pgmap v468: 97 pgs: 97 active+clean; 648 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.8 MiB/s wr, 894 op/s 2024-08-20T03:07:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:25 smithi165 ceph-mon[22357]: pgmap v468: 97 pgs: 97 active+clean; 648 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.8 MiB/s wr, 894 op/s 2024-08-20T03:07:26.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:25 smithi099 ceph-mon[21984]: pgmap v468: 97 pgs: 97 active+clean; 648 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.8 MiB/s wr, 894 op/s 2024-08-20T03:07:27.480 INFO:tasks.workunit.client.0.smithi044.stdout: 180 1549238 622 1079917 487 898233 1612 2024-08-20T03:07:28.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:27 smithi044 ceph-mon[18790]: pgmap v469: 97 pgs: 97 active+clean; 648 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.7 MiB/s wr, 723 op/s 2024-08-20T03:07:28.385 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:27 smithi165 ceph-mon[22357]: pgmap v469: 97 pgs: 97 active+clean; 648 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.7 MiB/s wr, 723 op/s 2024-08-20T03:07:28.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:27 smithi099 ceph-mon[21984]: pgmap v469: 97 pgs: 97 active+clean; 648 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.7 MiB/s wr, 723 op/s 2024-08-20T03:07:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.139+0000 7fe8a4942640 1 -- 172.21.15.44:0/1734367810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8a010ceb0 msgr2=0x7fe8a010d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.139+0000 7fe8a4942640 1 --2- 172.21.15.44:0/1734367810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8a010ceb0 0x7fe8a010d310 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe898009ca0 tx=0x7fe89802a8f0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.139+0000 7fe8a4942640 1 -- 172.21.15.44:0/1734367810 shutdown_connections 2024-08-20T03:07:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.139+0000 7fe8a4942640 1 --2- 172.21.15.44:0/1734367810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a010d850 0x7fe8a0116200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.139+0000 7fe8a4942640 1 --2- 172.21.15.44:0/1734367810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8a010ceb0 0x7fe8a010d310 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.139+0000 7fe8a4942640 1 --2- 172.21.15.44:0/1734367810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a010e150 0x7fe8a010e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.139+0000 7fe8a4942640 1 -- 172.21.15.44:0/1734367810 >> 172.21.15.44:0/1734367810 conn(0x7fe8a006d8b0 msgr2=0x7fe8a006dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.142+0000 7fe8a4942640 1 -- 172.21.15.44:0/1734367810 shutdown_connections 2024-08-20T03:07:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.142+0000 7fe8a4942640 1 -- 172.21.15.44:0/1734367810 wait complete. 2024-08-20T03:07:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.143+0000 7fe8a4942640 1 Processor -- start 2024-08-20T03:07:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.143+0000 7fe8a4942640 1 -- start start 2024-08-20T03:07:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe8a4942640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a010ceb0 0x7fe8a01a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe8a4942640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a010d850 0x7fe8a01a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe8a4942640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8a010e150 0x7fe8a01a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe8a4942640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0119fa0 con 0x7fe8a010ceb0 2024-08-20T03:07:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe8a4942640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0119e20 con 0x7fe8a010d850 2024-08-20T03:07:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe8a4942640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a011a120 con 0x7fe8a010e150 2024-08-20T03:07:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe89f59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8a010e150 0x7fe8a01a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe89e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a010d850 0x7fe8a01a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe89f59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8a010e150 0x7fe8a01a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35680/0 (socket says 172.21.15.44:35680) 2024-08-20T03:07:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe89f59e640 1 -- 172.21.15.44:0/3367438244 learned_addr learned my addr 172.21.15.44:0/3367438244 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a010ceb0 0x7fe8a01a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe89f59e640 1 -- 172.21.15.44:0/3367438244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a010d850 msgr2=0x7fe8a01a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe89f59e640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a010d850 0x7fe8a01a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.144+0000 7fe89f59e640 1 -- 172.21.15.44:0/3367438244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a010ceb0 msgr2=0x7fe8a01a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.145+0000 7fe89f59e640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a010ceb0 0x7fe8a01a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.145+0000 7fe89f59e640 1 -- 172.21.15.44:0/3367438244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a01afa80 con 0x7fe8a010e150 2024-08-20T03:07:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.145+0000 7fe89e59c640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a010d850 0x7fe8a01a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:07:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.145+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a010ceb0 0x7fe8a01a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:07:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.145+0000 7fe89f59e640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8a010e150 0x7fe8a01a0c40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe89400b3f0 tx=0x7fe89400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.163+0000 7fe887fff640 1 -- 172.21.15.44:0/3367438244 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894015020 con 0x7fe8a010e150 2024-08-20T03:07:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.163+0000 7fe887fff640 1 -- 172.21.15.44:0/3367438244 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe894011ce0 con 0x7fe8a010e150 2024-08-20T03:07:29.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.165+0000 7fe887fff640 1 -- 172.21.15.44:0/3367438244 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894010dd0 con 0x7fe8a010e150 2024-08-20T03:07:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.169+0000 7fe8a4942640 1 -- 172.21.15.44:0/3367438244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a01afd70 con 0x7fe8a010e150 2024-08-20T03:07:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.169+0000 7fe8a4942640 1 -- 172.21.15.44:0/3367438244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a01b0180 con 0x7fe8a010e150 2024-08-20T03:07:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.176+0000 7fe8a800f740 1 Processor -- start 2024-08-20T03:07:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.176+0000 7fe887fff640 1 -- 172.21.15.44:0/3367438244 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe894018020 con 0x7fe8a010e150 2024-08-20T03:07:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.177+0000 7fe887fff640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe8880b3e20 0x7fe8880b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.178+0000 7fe8a800f740 1 -- start start 2024-08-20T03:07:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.178+0000 7fe8a800f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2de120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.178+0000 7fe8a800f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2de660 0x55d1bc2ea0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.178+0000 7fe8a800f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2ec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.178+0000 7fe8a800f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d1bc253360 con 0x55d1bc2de660 2024-08-20T03:07:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.178+0000 7fe8a800f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d1bc242860 con 0x55d1bc2ea5f0 2024-08-20T03:07:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.178+0000 7fe8a800f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d1bc2468a0 con 0x55d1bc2ddd40 2024-08-20T03:07:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.178+0000 7fe89e59c640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe8880b3e20 0x7fe8880b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.178+0000 7fe89ed9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2de120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.179+0000 7fe89ed9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2de120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35684/0 (socket says 172.21.15.44:35684) 2024-08-20T03:07:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.179+0000 7fe89ed9d640 1 -- 172.21.15.44:0/1481070436 learned_addr learned my addr 172.21.15.44:0/1481070436 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.180+0000 7fe89ed9d640 1 -- 172.21.15.44:0/1481070436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2ea5f0 msgr2=0x55d1bc2ec9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:07:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.180+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/1481070436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2ec9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.180+0000 7fe89ed9d640 1 -- 172.21.15.44:0/1481070436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2de660 msgr2=0x55d1bc2ea0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:07:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.180+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/1481070436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2de660 0x55d1bc2ea0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.180+0000 7fe89ed9d640 1 -- 172.21.15.44:0/1481070436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1bc1e9e40 con 0x55d1bc2ddd40 2024-08-20T03:07:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.180+0000 7fe89e59c640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe8880b3e20 0x7fe8880b62e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe89802aee0 tx=0x7fe89802adf0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.180+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/1481070436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2de120 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe890010580 tx=0x7fe890010a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=570f049fd9b69626 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.181+0000 7fe887fff640 1 -- 172.21.15.44:0/3367438244 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe8940e3410 con 0x7fe8a010e150 2024-08-20T03:07:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.182+0000 7fe8857fa640 1 -- 172.21.15.44:0/1481070436 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890011760 con 0x55d1bc2ddd40 2024-08-20T03:07:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.183+0000 7fe8857fa640 1 -- 172.21.15.44:0/1481070436 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe890011db0 con 0x55d1bc2ddd40 2024-08-20T03:07:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.184+0000 7fe8a800f740 1 -- 172.21.15.44:0/1481070436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 msgr2=0x55d1bc2de120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.184+0000 7fe8a800f740 1 --2- 172.21.15.44:0/1481070436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2de120 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe890010580 tx=0x7fe890010a40 comp rx=0 tx=0).stop 2024-08-20T03:07:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.188+0000 7fe8a800f740 1 -- 172.21.15.44:0/1481070436 shutdown_connections 2024-08-20T03:07:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.188+0000 7fe8a800f740 1 --2- 172.21.15.44:0/1481070436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2ec9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.188+0000 7fe8a800f740 1 --2- 172.21.15.44:0/1481070436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2de660 0x55d1bc2ea0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.188+0000 7fe8a800f740 1 --2- 172.21.15.44:0/1481070436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2de120 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.188+0000 7fe8a800f740 1 -- 172.21.15.44:0/1481070436 >> 172.21.15.44:0/1481070436 conn(0x55d1bc2dbb70 msgr2=0x55d1bc2dcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.190+0000 7fe8a800f740 1 -- 172.21.15.44:0/1481070436 shutdown_connections 2024-08-20T03:07:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.190+0000 7fe8a800f740 1 -- 172.21.15.44:0/1481070436 wait complete. 2024-08-20T03:07:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.191+0000 7fe8a800f740 1 -- start start 2024-08-20T03:07:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.191+0000 7fe8a800f740 1 Processor -- start 2024-08-20T03:07:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.191+0000 7fe8a800f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.191+0000 7fe8a800f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2de660 0x55d1bc2e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.191+0000 7fe8a800f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.191+0000 7fe8a800f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d1bc17bab0 con 0x55d1bc2ddd40 2024-08-20T03:07:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.191+0000 7fe8a800f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d1bc1c9780 con 0x55d1bc2de660 2024-08-20T03:07:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.191+0000 7fe8a800f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d1bc2ed320 con 0x55d1bc2ea5f0 2024-08-20T03:07:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.191+0000 7fe8a800f740 10 client.0 _ll_register_callbacks cb 0x55d1bbf51a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:07:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.192+0000 7fe89e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2de660 0x55d1bc2e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.192+0000 7fe89e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2de660 0x55d1bc2e7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:41188/0 (socket says 172.21.15.44:41188) 2024-08-20T03:07:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.192+0000 7fe89e59c640 1 -- 172.21.15.44:0/1448655742 learned_addr learned my addr 172.21.15.44:0/1448655742 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.192+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.192+0000 7fe89f59e640 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.192+0000 7fe89e59c640 1 -- 172.21.15.44:0/1448655742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ea5f0 msgr2=0x55d1bc2e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.192+0000 7fe89e59c640 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2e7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.192+0000 7fe89e59c640 1 -- 172.21.15.44:0/1448655742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2ddd40 msgr2=0x55d1bc2e9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.192+0000 7fe89e59c640 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2e9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.192+0000 7fe89e59c640 1 -- 172.21.15.44:0/1448655742 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1bbfa5790 con 0x55d1bc2de660 2024-08-20T03:07:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.192+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:07:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.193+0000 7fe89e59c640 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2de660 0x55d1bc2e7340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe898009ca0 tx=0x7fe8980095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=354e64790613aff4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.211+0000 7fe882ff5640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:07:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.211+0000 7fe882ff5640 1 -- 172.21.15.44:0/1448655742 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898004830 con 0x55d1bc2de660 2024-08-20T03:07:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.211+0000 7fe8a800f740 1 -- 172.21.15.44:0/1448655742 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1bc1f1e00 con 0x55d1bc2de660 2024-08-20T03:07:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.211+0000 7fe882ff5640 1 -- 172.21.15.44:0/1448655742 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe89802fd20 con 0x55d1bc2de660 2024-08-20T03:07:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.212+0000 7fe882ff5640 1 -- 172.21.15.44:0/1448655742 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe8980305f0 con 0x55d1bc2de660 2024-08-20T03:07:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.212+0000 7fe8a800f740 10 client.26042 fetch_fsmap learned FSMap version 30 2024-08-20T03:07:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.212+0000 7fe8a800f740 1 -- 172.21.15.44:0/1448655742 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d1bc01e620 con 0x55d1bc2de660 2024-08-20T03:07:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.213+0000 7fe882ff5640 1 -- 172.21.15.44:0/1448655742 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d1bc1f1e00 con 0x55d1bc2de660 2024-08-20T03:07:29.216 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.213+0000 7fe882ff5640 1 -- 172.21.15.44:0/1448655742 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe89805a050 con 0x55d1bc2de660 2024-08-20T03:07:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.214+0000 7fe8a800f740 10 client.26042 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:07:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.214+0000 7fe8a800f740 10 client.26042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:07:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.214+0000 7fe8a800f740 20 client.26042 populate_metadata read hostname 'smithi044' 2024-08-20T03:07:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.214+0000 7fe8a800f740 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d1bc3027d0 0x55d1bc304bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.214+0000 7fe8a800f740 4 client.26042 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:07:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.214+0000 7fe8a800f740 1 -- 172.21.15.44:0/1448655742 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1bc2552b0 con 0x55d1bc3027d0 2024-08-20T03:07:29.217 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.214+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d1bc3027d0 0x55d1bc304bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.215+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d1bc3027d0 0x55d1bc304bf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f46df5ccec94c11e server_cookie=3219d07e3127751c in_seq=0 out_seq=0 2024-08-20T03:07:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.218+0000 7fe882ff5640 10 client.26042 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe882ff5640 1 -- 172.21.15.44:0/1448655742 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1bc2552b0 con 0x55d1bc3027d0 2024-08-20T03:07:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe882ff5640 10 client.26042 handle_command_reply: tid=0 2024-08-20T03:07:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe8a800f740 1 client.26042 shutdown 2024-08-20T03:07:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe8a800f740 1 -- 172.21.15.44:0/1448655742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2de660 msgr2=0x55d1bc2e7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe8a800f740 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2de660 0x55d1bc2e7340 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe898009ca0 tx=0x7fe8980095c0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe8a800f740 1 -- 172.21.15.44:0/1448655742 shutdown_connections 2024-08-20T03:07:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe8a800f740 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d1bc3027d0 0x55d1bc304bf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe8a800f740 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe8a800f740 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2de660 0x55d1bc2e7340 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe882ff5640 0 client.26042 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe8a800f740 1 --2- 172.21.15.44:0/1448655742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe8a800f740 1 -- 172.21.15.44:0/1448655742 >> 172.21.15.44:0/1448655742 conn(0x55d1bc2dbb70 msgr2=0x55d1bc2e1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe8a800f740 1 -- 172.21.15.44:0/1448655742 shutdown_connections 2024-08-20T03:07:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.219+0000 7fe8a800f740 1 -- 172.21.15.44:0/1448655742 wait complete. 2024-08-20T03:07:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.220+0000 7fe8a800f740 20 client.26042 trim_cache size 0 max 16384 2024-08-20T03:07:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.254+0000 7fe8a800f740 1 Processor -- start 2024-08-20T03:07:29.257 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.254+0000 7fe8a800f740 1 -- start start 2024-08-20T03:07:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.254+0000 7fe8a800f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.254+0000 7fe8a800f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2de660 0x55d1bc2d69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.254+0000 7fe8a800f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.254+0000 7fe8a800f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b44c0 con 0x55d1bc2ea5f0 2024-08-20T03:07:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.254+0000 7fe89ed9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.255+0000 7fe89ed9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2fed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35706/0 (socket says 172.21.15.44:35706) 2024-08-20T03:07:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.255+0000 7fe89ed9d640 1 -- 172.21.15.44:0/798727306 learned_addr learned my addr 172.21.15.44:0/798727306 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.255+0000 7fe89e59c640 1 --2- 172.21.15.44:0/798727306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2de660 0x55d1bc2d69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.255+0000 7fe8a800f740 1 -- 172.21.15.44:0/798727306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b60c0 con 0x55d1bc2de660 2024-08-20T03:07:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.255+0000 7fe8a800f740 1 -- 172.21.15.44:0/798727306 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b4b90 con 0x55d1bc2ddd40 2024-08-20T03:07:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.255+0000 7fe89f59e640 1 --2- 172.21.15.44:0/798727306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.255+0000 7fe89ed9d640 1 -- 172.21.15.44:0/798727306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2de660 msgr2=0x55d1bc2d69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.255+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/798727306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2de660 0x55d1bc2d69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.255+0000 7fe89ed9d640 1 -- 172.21.15.44:0/798727306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2ea5f0 msgr2=0x55d1bc2eb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.255+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/798727306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2eb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.255+0000 7fe89ed9d640 1 -- 172.21.15.44:0/798727306 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1bc2d71a0 con 0x55d1bc2ddd40 2024-08-20T03:07:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.255+0000 7fe89f59e640 1 --2- 172.21.15.44:0/798727306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:07:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.256+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/798727306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2fed20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe890010f40 tx=0x7fe89000b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24fa0e53259d4638 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.256+0000 7fe884ff9640 1 -- 172.21.15.44:0/798727306 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890005040 con 0x55d1bc2ddd40 2024-08-20T03:07:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.256+0000 7fe884ff9640 1 -- 172.21.15.44:0/798727306 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe890004660 con 0x55d1bc2ddd40 2024-08-20T03:07:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.256+0000 7fe884ff9640 1 -- 172.21.15.44:0/798727306 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89000e830 con 0x55d1bc2ddd40 2024-08-20T03:07:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.257+0000 7fe8a800f740 1 -- 172.21.15.44:0/798727306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 msgr2=0x55d1bc2fed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.257+0000 7fe8a800f740 1 --2- 172.21.15.44:0/798727306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2fed20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe890010f40 tx=0x7fe89000b410 comp rx=0 tx=0).stop 2024-08-20T03:07:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.257+0000 7fe8a800f740 1 -- 172.21.15.44:0/798727306 shutdown_connections 2024-08-20T03:07:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.257+0000 7fe8a800f740 1 --2- 172.21.15.44:0/798727306 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.257+0000 7fe8a800f740 1 --2- 172.21.15.44:0/798727306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2de660 0x55d1bc2d69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.257+0000 7fe8a800f740 1 --2- 172.21.15.44:0/798727306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2fed20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.257+0000 7fe8a800f740 1 -- 172.21.15.44:0/798727306 >> 172.21.15.44:0/798727306 conn(0x55d1bc2dbb70 msgr2=0x55d1bc2e5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.257+0000 7fe8a800f740 1 -- 172.21.15.44:0/798727306 shutdown_connections 2024-08-20T03:07:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.257+0000 7fe8a800f740 1 -- 172.21.15.44:0/798727306 wait complete. 2024-08-20T03:07:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.258+0000 7fe8a800f740 1 -- start start 2024-08-20T03:07:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.258+0000 7fe8a800f740 1 Processor -- start 2024-08-20T03:07:29.262 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.259+0000 7fe8a800f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.259+0000 7fe8a800f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2de660 0x55d1bc2eb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.259+0000 7fe89ed9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.259+0000 7fe8a800f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2ebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.259+0000 7fe89ed9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2fb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:35708/0 (socket says 172.21.15.44:35708) 2024-08-20T03:07:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.259+0000 7fe89ed9d640 1 -- 172.21.15.44:0/2425120925 learned_addr learned my addr 172.21.15.44:0/2425120925 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.259+0000 7fe89e59c640 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2de660 0x55d1bc2eb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.259+0000 7fe89f59e640 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2ebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.259+0000 7fe8a800f740 1 -- 172.21.15.44:0/2425120925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b5930 con 0x55d1bc2de660 2024-08-20T03:07:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.259+0000 7fe8a800f740 1 -- 172.21.15.44:0/2425120925 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b5260 con 0x55d1bc2ea5f0 2024-08-20T03:07:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.260+0000 7fe8a800f740 1 -- 172.21.15.44:0/2425120925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a01b24c0 con 0x55d1bc2ddd40 2024-08-20T03:07:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.260+0000 7fe8a800f740 10 client.0 _ll_register_callbacks cb 0x7fe898005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:07:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.260+0000 7fe89ed9d640 1 -- 172.21.15.44:0/2425120925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2ea5f0 msgr2=0x55d1bc2ebc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:29.266 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.260+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2ebc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.260+0000 7fe89ed9d640 1 -- 172.21.15.44:0/2425120925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2de660 msgr2=0x55d1bc2eb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.260+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2de660 0x55d1bc2eb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.260+0000 7fe89ed9d640 1 -- 172.21.15.44:0/2425120925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1bc2f70f0 con 0x55d1bc2ddd40 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.260+0000 7fe89f59e640 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2ebc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.260+0000 7fe89ed9d640 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2fb6b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe89000e400 tx=0x7fe89001c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcd957320e757f20 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.260+0000 7fe882ff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe882ff5640 1 -- 172.21.15.44:0/2425120925 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890004220 con 0x55d1bc2ddd40 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe882ff5640 1 -- 172.21.15.44:0/2425120925 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe8900043c0 con 0x55d1bc2ddd40 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe8a800f740 1 -- 172.21.15.44:0/2425120925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe890004660 con 0x55d1bc2ddd40 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe882ff5640 1 -- 172.21.15.44:0/2425120925 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe890004810 con 0x55d1bc2ddd40 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe8a800f740 10 client.26017 fetch_fsmap learned FSMap version 30 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe8a800f740 1 -- 172.21.15.44:0/2425120925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d1bc36d650 con 0x55d1bc2ddd40 2024-08-20T03:07:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe882ff5640 1 -- 172.21.15.44:0/2425120925 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe890004660 con 0x55d1bc2ddd40 2024-08-20T03:07:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe882ff5640 1 -- 172.21.15.44:0/2425120925 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe8900035f0 con 0x55d1bc2ddd40 2024-08-20T03:07:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe8a800f740 10 client.26017 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:07:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe8a800f740 10 client.26017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:07:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe8a800f740 20 client.26017 populate_metadata read hostname 'smithi044' 2024-08-20T03:07:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe8a800f740 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d1bc36db50 0x55d1bc36ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe8a800f740 4 client.26017 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:07:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.261+0000 7fe8a800f740 1 -- 172.21.15.44:0/2425120925 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1bc250d50 con 0x55d1bc36db50 2024-08-20T03:07:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.262+0000 7fe89e59c640 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d1bc36db50 0x55d1bc36ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.262+0000 7fe882ff5640 10 client.26017 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:29.262+0000 7fe89e59c640 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d1bc36db50 0x55d1bc36ff70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7faa3eab05810f6 server_cookie=209b509567222ed0 in_seq=0 out_seq=0 2024-08-20T03:07:30.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:29 smithi044 ceph-mon[18790]: pgmap v470: 97 pgs: 97 active+clean; 653 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 569 op/s 2024-08-20T03:07:30.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:29 smithi165 ceph-mon[22357]: pgmap v470: 97 pgs: 97 active+clean; 653 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 569 op/s 2024-08-20T03:07:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:29 smithi099 ceph-mon[21984]: pgmap v470: 97 pgs: 97 active+clean; 653 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 569 op/s 2024-08-20T03:07:32.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:31 smithi044 ceph-mon[18790]: pgmap v471: 97 pgs: 97 active+clean; 654 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 497 op/s 2024-08-20T03:07:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:31 smithi165 ceph-mon[22357]: pgmap v471: 97 pgs: 97 active+clean; 654 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 497 op/s 2024-08-20T03:07:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:31 smithi099 ceph-mon[21984]: pgmap v471: 97 pgs: 97 active+clean; 654 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 497 op/s 2024-08-20T03:07:34.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:33 smithi044 ceph-mon[18790]: pgmap v472: 97 pgs: 97 active+clean; 716 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.4 MiB/s wr, 966 op/s 2024-08-20T03:07:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:33 smithi165 ceph-mon[22357]: pgmap v472: 97 pgs: 97 active+clean; 716 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.4 MiB/s wr, 966 op/s 2024-08-20T03:07:34.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:33 smithi099 ceph-mon[21984]: pgmap v472: 97 pgs: 97 active+clean; 716 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.4 MiB/s wr, 966 op/s 2024-08-20T03:07:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.175+0000 7fe882ff5640 1 -- 172.21.15.44:0/2425120925 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1bc250d50 con 0x55d1bc36db50 2024-08-20T03:07:35.179 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.175+0000 7fe882ff5640 10 client.26017 handle_command_reply: tid=0 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.175+0000 7fe8a800f740 1 client.26017 shutdown 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.175+0000 7fe8a800f740 1 -- 172.21.15.44:0/2425120925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 msgr2=0x55d1bc2fb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.175+0000 7fe8a800f740 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2fb6b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe89000e400 tx=0x7fe89001c500 comp rx=0 tx=0).stop 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.175+0000 7fe8a800f740 1 -- 172.21.15.44:0/2425120925 shutdown_connections 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.175+0000 7fe8a800f740 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d1bc36db50 0x55d1bc36ff70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.175+0000 7fe8a800f740 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d1bc2ea5f0 0x55d1bc2ebc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.175+0000 7fe8a800f740 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1bc2de660 0x55d1bc2eb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.175+0000 7fe882ff5640 0 client.26017 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.175+0000 7fe8a800f740 1 --2- 172.21.15.44:0/2425120925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d1bc2ddd40 0x55d1bc2fb6b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a800f740 1 -- 172.21.15.44:0/2425120925 >> 172.21.15.44:0/2425120925 conn(0x55d1bc2dbb70 msgr2=0x55d1bc2e8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a800f740 1 -- 172.21.15.44:0/2425120925 shutdown_connections 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a800f740 1 -- 172.21.15.44:0/2425120925 wait complete. 2024-08-20T03:07:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a800f740 20 client.26017 trim_cache size 0 max 16384 2024-08-20T03:07:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:07:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:07:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:07:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:07:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a4942640 1 -- 172.21.15.44:0/3367438244 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe8880b3e20 msgr2=0x7fe8880b62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a4942640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe8880b3e20 0x7fe8880b62e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe89802aee0 tx=0x7fe89802adf0 comp rx=0 tx=0).stop 2024-08-20T03:07:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a4942640 1 -- 172.21.15.44:0/3367438244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8a010e150 msgr2=0x7fe8a01a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a4942640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8a010e150 0x7fe8a01a0c40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe89400b3f0 tx=0x7fe89400b8b0 comp rx=0 tx=0).stop 2024-08-20T03:07:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a4942640 1 -- 172.21.15.44:0/3367438244 shutdown_connections 2024-08-20T03:07:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a4942640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe8880b3e20 0x7fe8880b62e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a4942640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8a010e150 0x7fe8a01a0c40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a4942640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a010d850 0x7fe8a01a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.176+0000 7fe8a4942640 1 --2- 172.21.15.44:0/3367438244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8a010ceb0 0x7fe8a01a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.177+0000 7fe8a4942640 1 -- 172.21.15.44:0/3367438244 >> 172.21.15.44:0/3367438244 conn(0x7fe8a006d8b0 msgr2=0x7fe8a010aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.177+0000 7fe8a4942640 1 -- 172.21.15.44:0/3367438244 shutdown_connections 2024-08-20T03:07:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:35.177+0000 7fe8a4942640 1 -- 172.21.15.44:0/3367438244 wait complete. 2024-08-20T03:07:35.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:34 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:07:35.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:34 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:07:35.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:34 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:07:36.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:35 smithi044 ceph-mon[18790]: pgmap v473: 97 pgs: 97 active+clean; 745 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 1.17k op/s 2024-08-20T03:07:36.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:07:36.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:07:36.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:07:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:35 smithi165 ceph-mon[22357]: pgmap v473: 97 pgs: 97 active+clean; 745 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 1.17k op/s 2024-08-20T03:07:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:07:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:07:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:07:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:35 smithi099 ceph-mon[21984]: pgmap v473: 97 pgs: 97 active+clean; 745 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 1.17k op/s 2024-08-20T03:07:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:07:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:07:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:07:37.480 INFO:tasks.workunit.client.0.smithi044.stdout: 182 520404 134 361389 37 304372 339 2024-08-20T03:07:38.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:37 smithi044 ceph-mon[18790]: pgmap v474: 97 pgs: 97 active+clean; 740 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.8 MiB/s wr, 1.44k op/s 2024-08-20T03:07:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:37 smithi165 ceph-mon[22357]: pgmap v474: 97 pgs: 97 active+clean; 740 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.8 MiB/s wr, 1.44k op/s 2024-08-20T03:07:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:37 smithi099 ceph-mon[21984]: pgmap v474: 97 pgs: 97 active+clean; 740 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.8 MiB/s wr, 1.44k op/s 2024-08-20T03:07:40.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:39 smithi044 ceph-mon[18790]: pgmap v475: 97 pgs: 97 active+clean; 718 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 1.88k op/s 2024-08-20T03:07:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:39 smithi165 ceph-mon[22357]: pgmap v475: 97 pgs: 97 active+clean; 718 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 1.88k op/s 2024-08-20T03:07:40.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:39 smithi099 ceph-mon[21984]: pgmap v475: 97 pgs: 97 active+clean; 718 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 1.88k op/s 2024-08-20T03:07:42.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:41 smithi044 ceph-mon[18790]: pgmap v476: 97 pgs: 97 active+clean; 718 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.97k op/s 2024-08-20T03:07:42.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:41 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:07:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:41 smithi165 ceph-mon[22357]: pgmap v476: 97 pgs: 97 active+clean; 718 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.97k op/s 2024-08-20T03:07:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:41 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:07:42.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:41 smithi099 ceph-mon[21984]: pgmap v476: 97 pgs: 97 active+clean; 718 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.97k op/s 2024-08-20T03:07:42.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:41 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:07:42.981 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:07:44.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:43 smithi044 ceph-mon[18790]: pgmap v477: 97 pgs: 97 active+clean; 725 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 2.22k op/s 2024-08-20T03:07:44.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:43 smithi165 ceph-mon[22357]: pgmap v477: 97 pgs: 97 active+clean; 725 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 2.22k op/s 2024-08-20T03:07:44.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:43 smithi099 ceph-mon[21984]: pgmap v477: 97 pgs: 97 active+clean; 725 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 2.22k op/s 2024-08-20T03:07:44.791 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:07:45.196 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:44 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:07:45.197 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:44 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:07:45.385 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:44 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:07:45.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:44 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:07:45.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:44 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:07:45.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:44 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:07:45.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.587+0000 7f14b77c1640 1 -- 172.21.15.44:0/4244175056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14b01073a0 msgr2=0x7f14b01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:45.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.587+0000 7f14b77c1640 1 --2- 172.21.15.44:0/4244175056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14b01073a0 0x7f14b01077e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f14ac01ab70 tx=0x7f14ac03b6d0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.592+0000 7f14b77c1640 1 -- 172.21.15.44:0/4244175056 shutdown_connections 2024-08-20T03:07:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.592+0000 7f14b77c1640 1 --2- 172.21.15.44:0/4244175056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b0107d20 0x7f14b010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.592+0000 7f14b77c1640 1 --2- 172.21.15.44:0/4244175056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14b01073a0 0x7f14b01077e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.592+0000 7f14b77c1640 1 --2- 172.21.15.44:0/4244175056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b0101990 0x7f14b0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.592+0000 7f14b77c1640 1 -- 172.21.15.44:0/4244175056 >> 172.21.15.44:0/4244175056 conn(0x7f14b00fd440 msgr2=0x7f14b00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:45.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.594+0000 7f14b77c1640 1 -- 172.21.15.44:0/4244175056 shutdown_connections 2024-08-20T03:07:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.596+0000 7f14b77c1640 1 -- 172.21.15.44:0/4244175056 wait complete. 2024-08-20T03:07:45.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.597+0000 7f14b77c1640 1 Processor -- start 2024-08-20T03:07:45.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.599+0000 7f14b77c1640 1 -- start start 2024-08-20T03:07:45.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.599+0000 7f14b77c1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b0101990 0x7f14b0074600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.599+0000 7f14b77c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14b01073a0 0x7f14b0074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.599+0000 7f14b77c1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b0107d20 0x7f14b006ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.600+0000 7f14b77c1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14b006d1a0 con 0x7f14b01073a0 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.600+0000 7f14b77c1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14b006d020 con 0x7f14b0107d20 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.600+0000 7f14b77c1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14b006d320 con 0x7f14b0101990 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.600+0000 7f14b4d5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14b01073a0 0x7f14b0074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.600+0000 7f14b4d5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14b01073a0 0x7f14b0074b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43206/0 (socket says 172.21.15.44:43206) 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.600+0000 7f14b4d5c640 1 -- 172.21.15.44:0/2320263493 learned_addr learned my addr 172.21.15.44:0/2320263493 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.600+0000 7f14b5d5e640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b0107d20 0x7f14b006ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.600+0000 7f14b555d640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b0101990 0x7f14b0074600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.601+0000 7f14b4d5c640 1 -- 172.21.15.44:0/2320263493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b0101990 msgr2=0x7f14b0074600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.601+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b0101990 0x7f14b0074600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.601+0000 7f14b4d5c640 1 -- 172.21.15.44:0/2320263493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b0107d20 msgr2=0x7f14b006ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.601+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b0107d20 0x7f14b006ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.601+0000 7f14b4d5c640 1 -- 172.21.15.44:0/2320263493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b006f330 con 0x7f14b01073a0 2024-08-20T03:07:45.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.601+0000 7f14b5d5e640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b0107d20 0x7f14b006ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:07:45.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.601+0000 7f14b555d640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b0101990 0x7f14b0074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:07:45.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.601+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14b01073a0 0x7f14b0074b40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0058c0 tx=0x7f14ac0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:45.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.603+0000 7f14a67fc640 1 -- 172.21.15.44:0/2320263493 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac071020 con 0x7f14b01073a0 2024-08-20T03:07:45.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.603+0000 7f14a67fc640 1 -- 172.21.15.44:0/2320263493 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14ac0035f0 con 0x7f14b01073a0 2024-08-20T03:07:45.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.603+0000 7f14a67fc640 1 -- 172.21.15.44:0/2320263493 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac046070 con 0x7f14b01073a0 2024-08-20T03:07:45.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.604+0000 7f14b77c1640 1 -- 172.21.15.44:0/2320263493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b006f5c0 con 0x7f14b01073a0 2024-08-20T03:07:45.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.604+0000 7f14b77c1640 1 -- 172.21.15.44:0/2320263493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b0110350 con 0x7f14b01073a0 2024-08-20T03:07:45.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.612+0000 7f14b9bba740 1 Processor -- start 2024-08-20T03:07:45.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.614+0000 7f14b9bba740 1 -- start start 2024-08-20T03:07:45.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.614+0000 7f14b9bba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 0x5603ade69ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.614+0000 7f14b9bba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade6a000 0x5603ade75a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.614+0000 7f14b9bba740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 0x5603ade78380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.614+0000 7f14b9bba740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5603adde08e0 con 0x5603ade6a000 2024-08-20T03:07:45.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.614+0000 7f14b9bba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5603addcfde0 con 0x5603ade696e0 2024-08-20T03:07:45.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.614+0000 7f14b9bba740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5603addd3e20 con 0x5603ade75f90 2024-08-20T03:07:45.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.617+0000 7f14b4d5c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 0x5603ade78380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.617+0000 7f14a67fc640 1 -- 172.21.15.44:0/2320263493 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f14ac074020 con 0x7f14b01073a0 2024-08-20T03:07:45.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.617+0000 7f14b4d5c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 0x5603ade78380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59552/0 (socket says 172.21.15.44:59552) 2024-08-20T03:07:45.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.617+0000 7f14b4d5c640 1 -- 172.21.15.44:0/1116137102 learned_addr learned my addr 172.21.15.44:0/1116137102 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:45.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.617+0000 7f14b555d640 1 --2- 172.21.15.44:0/1116137102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade6a000 0x5603ade75a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.617+0000 7f14b4d5c640 1 -- 172.21.15.44:0/1116137102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 msgr2=0x5603ade69ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:45.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.617+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/1116137102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 0x5603ade69ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.617+0000 7f14b4d5c640 1 -- 172.21.15.44:0/1116137102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade6a000 msgr2=0x5603ade75a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:45.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.617+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/1116137102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade6a000 0x5603ade75a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.617+0000 7f14b4d5c640 1 -- 172.21.15.44:0/1116137102 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603adb2f650 con 0x5603ade75f90 2024-08-20T03:07:45.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.617+0000 7f14b555d640 1 --2- 172.21.15.44:0/1116137102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade6a000 0x5603ade75a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:07:45.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.618+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/1116137102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 0x5603ade78380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f14ac136660 tx=0x7f14ac13ad30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3436b9d398e9aa7e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:45.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.618+0000 7f149b7fe640 1 -- 172.21.15.44:0/1116137102 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac13fd60 con 0x5603ade75f90 2024-08-20T03:07:45.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.618+0000 7f149b7fe640 1 -- 172.21.15.44:0/1116137102 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14ac14a930 con 0x5603ade75f90 2024-08-20T03:07:45.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.618+0000 7f149b7fe640 1 -- 172.21.15.44:0/1116137102 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac1418e0 con 0x5603ade75f90 2024-08-20T03:07:45.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.620+0000 7f14a67fc640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f149c0b3f50 0x7f149c0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.620+0000 7f14a67fc640 1 -- 172.21.15.44:0/2320263493 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f14ac138ed0 con 0x7f14b01073a0 2024-08-20T03:07:45.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.620+0000 7f14b9bba740 1 -- 172.21.15.44:0/1116137102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 msgr2=0x5603ade78380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:45.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.620+0000 7f14b9bba740 1 --2- 172.21.15.44:0/1116137102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 0x5603ade78380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f14ac136660 tx=0x7f14ac13ad30 comp rx=0 tx=0).stop 2024-08-20T03:07:45.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.624+0000 7f14b5d5e640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f149c0b3f50 0x7f149c0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.625+0000 7f14b5d5e640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f149c0b3f50 0x7f149c0b6410 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f14a000b840 tx=0x7f14a0008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:45.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.626+0000 7f14b9bba740 1 -- 172.21.15.44:0/1116137102 shutdown_connections 2024-08-20T03:07:45.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.626+0000 7f14b9bba740 1 --2- 172.21.15.44:0/1116137102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 0x5603ade78380 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.626+0000 7f14b9bba740 1 --2- 172.21.15.44:0/1116137102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade6a000 0x5603ade75a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.626+0000 7f14b9bba740 1 --2- 172.21.15.44:0/1116137102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 0x5603ade69ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.626+0000 7f14b9bba740 1 -- 172.21.15.44:0/1116137102 >> 172.21.15.44:0/1116137102 conn(0x5603ade67510 msgr2=0x5603ade686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:45.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.626+0000 7f14b9bba740 1 -- 172.21.15.44:0/1116137102 shutdown_connections 2024-08-20T03:07:45.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.630+0000 7f14b9bba740 1 -- 172.21.15.44:0/1116137102 wait complete. 2024-08-20T03:07:45.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.630+0000 7f14b9bba740 1 -- start start 2024-08-20T03:07:45.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.630+0000 7f14b9bba740 1 Processor -- start 2024-08-20T03:07:45.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.632+0000 7f14b9bba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade696e0 0x5603ade72fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.632+0000 7f14b9bba740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade6a000 0x5603ade734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.632+0000 7f14b9bba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade75f90 0x5603ade73a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.632+0000 7f14b9bba740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5603adc32700 con 0x5603ade696e0 2024-08-20T03:07:45.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.632+0000 7f14b9bba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5603add567c0 con 0x5603ade75f90 2024-08-20T03:07:45.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.632+0000 7f14b9bba740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5603ade78a00 con 0x5603ade6a000 2024-08-20T03:07:45.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.632+0000 7f14b9bba740 10 client.0 _ll_register_callbacks cb 0x5603addc96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:07:45.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.633+0000 7f14b4d5c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade6a000 0x5603ade734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.633+0000 7f14b5d5e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade75f90 0x5603ade73a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.633+0000 7f14b555d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade696e0 0x5603ade72fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.633+0000 7f14b5d5e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade75f90 0x5603ade73a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:36044/0 (socket says 172.21.15.44:36044) 2024-08-20T03:07:45.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.633+0000 7f14b5d5e640 1 -- 172.21.15.44:0/2703428778 learned_addr learned my addr 172.21.15.44:0/2703428778 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:45.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.634+0000 7f14b5d5e640 1 -- 172.21.15.44:0/2703428778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade6a000 msgr2=0x5603ade734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:45.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.634+0000 7f14b5d5e640 1 --2- 172.21.15.44:0/2703428778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade6a000 0x5603ade734e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.634+0000 7f14b5d5e640 1 -- 172.21.15.44:0/2703428778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade696e0 msgr2=0x5603ade72fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:45.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.634+0000 7f14b5d5e640 1 --2- 172.21.15.44:0/2703428778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade696e0 0x5603ade72fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.634+0000 7f14b5d5e640 1 -- 172.21.15.44:0/2703428778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ade81d70 con 0x5603ade75f90 2024-08-20T03:07:45.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.634+0000 7f14b555d640 1 --2- 172.21.15.44:0/2703428778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade696e0 0x5603ade72fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:07:45.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.636+0000 7f14b5d5e640 1 --2- 172.21.15.44:0/2703428778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade75f90 0x5603ade73a20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f14a000dd40 tx=0x7f14a0004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d64e98022a84a48a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:45.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.637+0000 7f1498ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:07:45.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.637+0000 7f1498ff9640 1 -- 172.21.15.44:0/2703428778 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0024060 con 0x5603ade75f90 2024-08-20T03:07:45.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.637+0000 7f14b9bba740 1 -- 172.21.15.44:0/2703428778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603add7f100 con 0x5603ade75f90 2024-08-20T03:07:45.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.637+0000 7f1498ff9640 1 -- 172.21.15.44:0/2703428778 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f14a001f400 con 0x5603ade75f90 2024-08-20T03:07:45.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.641+0000 7f1498ff9640 1 -- 172.21.15.44:0/2703428778 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f14a001f600 con 0x5603ade75f90 2024-08-20T03:07:45.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.642+0000 7f14b9bba740 10 client.26054 fetch_fsmap learned FSMap version 30 2024-08-20T03:07:45.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.642+0000 7f14b9bba740 1 -- 172.21.15.44:0/2703428778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5603ade84740 con 0x5603ade75f90 2024-08-20T03:07:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.644+0000 7f1498ff9640 1 -- 172.21.15.44:0/2703428778 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5603add7f100 con 0x5603ade75f90 2024-08-20T03:07:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.644+0000 7f1498ff9640 1 -- 172.21.15.44:0/2703428778 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f14a0027020 con 0x5603ade75f90 2024-08-20T03:07:45.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.644+0000 7f14b9bba740 10 client.26054 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:07:45.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.644+0000 7f14b9bba740 10 client.26054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:07:45.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.644+0000 7f14b9bba740 20 client.26054 populate_metadata read hostname 'smithi044' 2024-08-20T03:07:45.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.644+0000 7f14b9bba740 1 --2- 172.21.15.44:0/2703428778 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5603ade860d0 0x5603ade88530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.644+0000 7f14b9bba740 4 client.26054 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:07:45.648 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.644+0000 7f14b9bba740 1 -- 172.21.15.44:0/2703428778 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603adde2830 con 0x5603ade860d0 2024-08-20T03:07:45.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.647+0000 7f14b555d640 1 --2- 172.21.15.44:0/2703428778 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5603ade860d0 0x5603ade88530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.662+0000 7f14b555d640 1 --2- 172.21.15.44:0/2703428778 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5603ade860d0 0x5603ade88530 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e9f84e39b2ffe2 server_cookie=b1a95da685e4090e in_seq=0 out_seq=0 2024-08-20T03:07:45.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.663+0000 7f1498ff9640 10 client.26054 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:45.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.666+0000 7f1498ff9640 1 -- 172.21.15.44:0/2703428778 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603adde2830 con 0x5603ade860d0 2024-08-20T03:07:45.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.666+0000 7f1498ff9640 10 client.26054 handle_command_reply: tid=0 2024-08-20T03:07:45.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.666+0000 7f14b9bba740 1 client.26054 shutdown 2024-08-20T03:07:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.677+0000 7f14b9bba740 1 -- 172.21.15.44:0/2703428778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade75f90 msgr2=0x5603ade73a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.677+0000 7f14b9bba740 1 --2- 172.21.15.44:0/2703428778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade75f90 0x5603ade73a20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f14a000dd40 tx=0x7f14a0004700 comp rx=0 tx=0).stop 2024-08-20T03:07:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.677+0000 7f14b9bba740 1 -- 172.21.15.44:0/2703428778 shutdown_connections 2024-08-20T03:07:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.677+0000 7f14b9bba740 1 --2- 172.21.15.44:0/2703428778 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5603ade860d0 0x5603ade88530 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.677+0000 7f14b9bba740 1 --2- 172.21.15.44:0/2703428778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade75f90 0x5603ade73a20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.677+0000 7f14b9bba740 1 --2- 172.21.15.44:0/2703428778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade6a000 0x5603ade734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.677+0000 7f14b9bba740 1 --2- 172.21.15.44:0/2703428778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade696e0 0x5603ade72fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.677+0000 7f14b9bba740 1 -- 172.21.15.44:0/2703428778 >> 172.21.15.44:0/2703428778 conn(0x5603ade67510 msgr2=0x5603ade6d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:45.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.677+0000 7f1498ff9640 0 client.26054 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:45.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.679+0000 7f14b9bba740 1 -- 172.21.15.44:0/2703428778 shutdown_connections 2024-08-20T03:07:45.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.679+0000 7f14b9bba740 1 -- 172.21.15.44:0/2703428778 wait complete. 2024-08-20T03:07:45.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.679+0000 7f14b9bba740 20 client.26054 trim_cache size 0 max 16384 2024-08-20T03:07:45.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.717+0000 7f14b9bba740 1 Processor -- start 2024-08-20T03:07:45.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.720+0000 7f14b9bba740 1 -- start start 2024-08-20T03:07:45.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.721+0000 7f14b9bba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 0x5603ade6d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.721+0000 7f14b9bba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade6a000 0x5603ade8de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.721+0000 7f14b9bba740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 0x5603ade8e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.721+0000 7f14b9bba740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0118c90 con 0x5603ade6a000 2024-08-20T03:07:45.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.721+0000 7f14b9bba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14b011a890 con 0x5603ade696e0 2024-08-20T03:07:45.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.721+0000 7f14b9bba740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0119360 con 0x5603ade75f90 2024-08-20T03:07:45.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.723+0000 7f14b5d5e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 0x5603ade8e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.723+0000 7f14b5d5e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 0x5603ade8e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:59580/0 (socket says 172.21.15.44:59580) 2024-08-20T03:07:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.723+0000 7f14b5d5e640 1 -- 172.21.15.44:0/592623314 learned_addr learned my addr 172.21.15.44:0/592623314 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.723+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/592623314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade6a000 0x5603ade8de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.724+0000 7f14b555d640 1 --2- 172.21.15.44:0/592623314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 0x5603ade6d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.724+0000 7f14b4d5c640 1 -- 172.21.15.44:0/592623314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 msgr2=0x5603ade8e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.724+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/592623314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 0x5603ade8e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.724+0000 7f14b4d5c640 1 -- 172.21.15.44:0/592623314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 msgr2=0x5603ade6d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.724+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/592623314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 0x5603ade6d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.724+0000 7f14b4d5c640 1 -- 172.21.15.44:0/592623314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ade67f90 con 0x5603ade6a000 2024-08-20T03:07:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.724+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/592623314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade6a000 0x5603ade8de00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f14ac03b710 tx=0x7f14ac003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5d8c2c51c60b95 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.725+0000 7f149affd640 1 -- 172.21.15.44:0/592623314 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0fbaa0 con 0x5603ade6a000 2024-08-20T03:07:45.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.725+0000 7f149affd640 1 -- 172.21.15.44:0/592623314 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14ac0ff6f0 con 0x5603ade6a000 2024-08-20T03:07:45.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.725+0000 7f149affd640 1 -- 172.21.15.44:0/592623314 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0fb370 con 0x5603ade6a000 2024-08-20T03:07:45.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.726+0000 7f14b9bba740 1 -- 172.21.15.44:0/592623314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade6a000 msgr2=0x5603ade8de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:45.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.726+0000 7f14b9bba740 1 --2- 172.21.15.44:0/592623314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade6a000 0x5603ade8de00 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f14ac03b710 tx=0x7f14ac003e10 comp rx=0 tx=0).stop 2024-08-20T03:07:45.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.726+0000 7f14b9bba740 1 -- 172.21.15.44:0/592623314 shutdown_connections 2024-08-20T03:07:45.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.726+0000 7f14b9bba740 1 --2- 172.21.15.44:0/592623314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade75f90 0x5603ade8e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.726+0000 7f14b9bba740 1 --2- 172.21.15.44:0/592623314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade6a000 0x5603ade8de00 secure :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f14ac03b710 tx=0x7f14ac003e10 comp rx=0 tx=0).stop 2024-08-20T03:07:45.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.726+0000 7f14b9bba740 1 --2- 172.21.15.44:0/592623314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 0x5603ade6d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.726+0000 7f14b9bba740 1 -- 172.21.15.44:0/592623314 >> 172.21.15.44:0/592623314 conn(0x5603ade67510 msgr2=0x5603ade71820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:45.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.732+0000 7f14b9bba740 1 -- 172.21.15.44:0/592623314 shutdown_connections 2024-08-20T03:07:45.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.732+0000 7f14b9bba740 1 -- 172.21.15.44:0/592623314 wait complete. 2024-08-20T03:07:45.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.733+0000 7f14b9bba740 1 -- start start 2024-08-20T03:07:45.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.733+0000 7f14b9bba740 1 Processor -- start 2024-08-20T03:07:45.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.734+0000 7f14b9bba740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 0x5603ade7dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.734+0000 7f14b9bba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade75f90 0x5603ade7e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.734+0000 7f14b9bba740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade795a0 0x5603ade79a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.734+0000 7f14b9bba740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14b011a100 con 0x5603ade75f90 2024-08-20T03:07:45.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.734+0000 7f14b9bba740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0119a30 con 0x5603ade696e0 2024-08-20T03:07:45.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.734+0000 7f14b9bba740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f14b0116c70 con 0x5603ade795a0 2024-08-20T03:07:45.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.734+0000 7f14b9bba740 10 client.0 _ll_register_callbacks cb 0x7f14a000b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:07:45.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.734+0000 7f14b5d5e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade795a0 0x5603ade79a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.734+0000 7f14b4d5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade75f90 0x5603ade7e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.734+0000 7f14b4d5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade75f90 0x5603ade7e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:43236/0 (socket says 172.21.15.44:43236) 2024-08-20T03:07:45.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.734+0000 7f14b4d5c640 1 -- 172.21.15.44:0/724199644 learned_addr learned my addr 172.21.15.44:0/724199644 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:45.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.735+0000 7f14b555d640 1 --2- 172.21.15.44:0/724199644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 0x5603ade7dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.735+0000 7f14b4d5c640 1 -- 172.21.15.44:0/724199644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade795a0 msgr2=0x5603ade79a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.735+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/724199644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade795a0 0x5603ade79a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.735+0000 7f14b4d5c640 1 -- 172.21.15.44:0/724199644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 msgr2=0x5603ade7dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.735+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/724199644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 0x5603ade7dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.735+0000 7f14b4d5c640 1 -- 172.21.15.44:0/724199644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ade7a250 con 0x5603ade75f90 2024-08-20T03:07:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.736+0000 7f14b4d5c640 1 --2- 172.21.15.44:0/724199644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade75f90 0x5603ade7e560 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0fbc60 tx=0x7f14ac0fc0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bcf4c33be036e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.736+0000 7f1498ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:07:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.736+0000 7f1498ff9640 1 -- 172.21.15.44:0/724199644 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0ff350 con 0x5603ade75f90 2024-08-20T03:07:45.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.736+0000 7f1498ff9640 1 -- 172.21.15.44:0/724199644 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f14ac136690 con 0x5603ade75f90 2024-08-20T03:07:45.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.740+0000 7f14b9bba740 1 -- 172.21.15.44:0/724199644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ac0ff6f0 con 0x5603ade75f90 2024-08-20T03:07:45.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.740+0000 7f1498ff9640 1 -- 172.21.15.44:0/724199644 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f14ac0ff530 con 0x5603ade75f90 2024-08-20T03:07:45.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.741+0000 7f14b9bba740 10 client.16581 fetch_fsmap learned FSMap version 30 2024-08-20T03:07:45.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.741+0000 7f14b9bba740 1 -- 172.21.15.44:0/724199644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5603adef0f10 con 0x5603ade75f90 2024-08-20T03:07:45.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.741+0000 7f1498ff9640 1 -- 172.21.15.44:0/724199644 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14ac0ff6f0 con 0x5603ade75f90 2024-08-20T03:07:45.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.741+0000 7f1498ff9640 1 -- 172.21.15.44:0/724199644 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f14ac13e040 con 0x5603ade75f90 2024-08-20T03:07:45.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.742+0000 7f14b9bba740 10 client.16581 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:07:45.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.742+0000 7f14b9bba740 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:07:45.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.742+0000 7f14b9bba740 20 client.16581 populate_metadata read hostname 'smithi044' 2024-08-20T03:07:45.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.742+0000 7f14b9bba740 1 --2- 172.21.15.44:0/724199644 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5603adef11f0 0x5603adef35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.742+0000 7f14b9bba740 4 client.16581 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:07:45.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.742+0000 7f14b9bba740 1 -- 172.21.15.44:0/724199644 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5603addde2d0 con 0x5603adef11f0 2024-08-20T03:07:45.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.746+0000 7f14b555d640 1 --2- 172.21.15.44:0/724199644 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5603adef11f0 0x5603adef35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.753+0000 7f14b555d640 1 --2- 172.21.15.44:0/724199644 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5603adef11f0 0x5603adef35e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85ee784bbe7fbd3f server_cookie=ce9e69cd49f22255 in_seq=0 out_seq=0 2024-08-20T03:07:45.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.753+0000 7f1498ff9640 10 client.16581 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:45.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.758+0000 7f1498ff9640 1 -- 172.21.15.44:0/724199644 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5603addde2d0 con 0x5603adef11f0 2024-08-20T03:07:45.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.758+0000 7f1498ff9640 10 client.16581 handle_command_reply: tid=0 2024-08-20T03:07:45.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.760+0000 7f14b9bba740 1 client.16581 shutdown 2024-08-20T03:07:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.764+0000 7f14b9bba740 1 -- 172.21.15.44:0/724199644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade75f90 msgr2=0x5603ade7e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.764+0000 7f14b9bba740 1 --2- 172.21.15.44:0/724199644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade75f90 0x5603ade7e560 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0fbc60 tx=0x7f14ac0fc0a0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.770+0000 7f14b9bba740 1 -- 172.21.15.44:0/724199644 shutdown_connections 2024-08-20T03:07:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.770+0000 7f14b9bba740 1 --2- 172.21.15.44:0/724199644 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5603adef11f0 0x5603adef35e0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.770+0000 7f14b9bba740 1 --2- 172.21.15.44:0/724199644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603ade795a0 0x5603ade79a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.770+0000 7f14b9bba740 1 --2- 172.21.15.44:0/724199644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5603ade75f90 0x5603ade7e560 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.770+0000 7f14b9bba740 1 --2- 172.21.15.44:0/724199644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5603ade696e0 0x5603ade7dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.770+0000 7f14b9bba740 1 -- 172.21.15.44:0/724199644 >> 172.21.15.44:0/724199644 conn(0x5603ade67510 msgr2=0x5603ade664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:45.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.770+0000 7f1498ff9640 0 client.16581 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.774+0000 7f14b9bba740 1 -- 172.21.15.44:0/724199644 shutdown_connections 2024-08-20T03:07:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.775+0000 7f14b9bba740 1 -- 172.21.15.44:0/724199644 wait complete. 2024-08-20T03:07:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.775+0000 7f14b9bba740 20 client.16581 trim_cache size 0 max 16384 2024-08-20T03:07:45.779 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:07:45.779 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-08-20T03:07:45.779 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-08-20T03:07:45.779 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:07:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.777+0000 7f14b77c1640 1 -- 172.21.15.44:0/2320263493 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f149c0b3f50 msgr2=0x7f149c0b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.777+0000 7f14b77c1640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f149c0b3f50 0x7f149c0b6410 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f14a000b840 tx=0x7f14a0008040 comp rx=0 tx=0).stop 2024-08-20T03:07:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.777+0000 7f14b77c1640 1 -- 172.21.15.44:0/2320263493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14b01073a0 msgr2=0x7f14b0074b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:45.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.777+0000 7f14b77c1640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14b01073a0 0x7f14b0074b40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0058c0 tx=0x7f14ac0043b0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.779+0000 7f14b77c1640 1 -- 172.21.15.44:0/2320263493 shutdown_connections 2024-08-20T03:07:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.779+0000 7f14b77c1640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f149c0b3f50 0x7f149c0b6410 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.779+0000 7f14b77c1640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b0107d20 0x7f14b006ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.779+0000 7f14b77c1640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14b01073a0 0x7f14b0074b40 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.779+0000 7f14b77c1640 1 --2- 172.21.15.44:0/2320263493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14b0101990 0x7f14b0074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.779+0000 7f14b77c1640 1 -- 172.21.15.44:0/2320263493 >> 172.21.15.44:0/2320263493 conn(0x7f14b00fd440 msgr2=0x7f14b00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.782+0000 7f14b77c1640 1 -- 172.21.15.44:0/2320263493 shutdown_connections 2024-08-20T03:07:45.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:45.786+0000 7f14b77c1640 1 -- 172.21.15.44:0/2320263493 wait complete. 2024-08-20T03:07:46.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:45 smithi044 ceph-mon[18790]: pgmap v478: 97 pgs: 97 active+clean; 730 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 1.76k op/s 2024-08-20T03:07:46.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:45 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:07:46.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:45 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:07:46.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:45 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:07:46.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:45 smithi165 ceph-mon[22357]: pgmap v478: 97 pgs: 97 active+clean; 730 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 1.76k op/s 2024-08-20T03:07:46.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:45 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:07:46.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:45 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:07:46.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:45 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:07:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:45 smithi099 ceph-mon[21984]: pgmap v478: 97 pgs: 97 active+clean; 730 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 1.76k op/s 2024-08-20T03:07:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:45 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:07:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:45 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:07:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:45 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:07:46.926 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T03:07:46.926 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:07:47.480 INFO:tasks.workunit.client.0.smithi044.stdout: 187 1913237 521 1302478 358 1106384 1474 2024-08-20T03:07:48.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:47 smithi044 ceph-mon[18790]: pgmap v479: 97 pgs: 97 active+clean; 735 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 1.31k op/s 2024-08-20T03:07:48.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:47 smithi165 ceph-mon[22357]: pgmap v479: 97 pgs: 97 active+clean; 735 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 1.31k op/s 2024-08-20T03:07:48.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:47 smithi099 ceph-mon[21984]: pgmap v479: 97 pgs: 97 active+clean; 735 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.8 MiB/s wr, 1.31k op/s 2024-08-20T03:07:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.022+0000 7eff210cc640 1 -- 172.21.15.44:0/2815820893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff1c074cd0 msgr2=0x7eff1c109e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:50.024 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.022+0000 7eff210cc640 1 --2- 172.21.15.44:0/2815820893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff1c074cd0 0x7eff1c109e80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009950 tx=0x7eff0c02a460 comp rx=0 tx=0).stop 2024-08-20T03:07:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.025+0000 7eff210cc640 1 -- 172.21.15.44:0/2815820893 shutdown_connections 2024-08-20T03:07:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.025+0000 7eff210cc640 1 --2- 172.21.15.44:0/2815820893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff1c115380 0x7eff1c117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.025+0000 7eff210cc640 1 --2- 172.21.15.44:0/2815820893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff1c10a4a0 0x7eff1c10a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.025+0000 7eff210cc640 1 --2- 172.21.15.44:0/2815820893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff1c074cd0 0x7eff1c109e80 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.025+0000 7eff210cc640 1 -- 172.21.15.44:0/2815820893 >> 172.21.15.44:0/2815820893 conn(0x7eff1c06d8e0 msgr2=0x7eff1c06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.026+0000 7eff210cc640 1 -- 172.21.15.44:0/2815820893 shutdown_connections 2024-08-20T03:07:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.030+0000 7eff210cc640 1 -- 172.21.15.44:0/2815820893 wait complete. 2024-08-20T03:07:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.031+0000 7eff210cc640 1 Processor -- start 2024-08-20T03:07:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.031+0000 7eff210cc640 1 -- start start 2024-08-20T03:07:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff210cc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff1c10a4a0 0x7eff1c1148c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff210cc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff1c115380 0x7eff1c111b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff210cc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff1c112040 0x7eff1c1124f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff210cc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c10caa0 con 0x7eff1c115380 2024-08-20T03:07:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff210cc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c10c920 con 0x7eff1c10a4a0 2024-08-20T03:07:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff210cc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c10cc20 con 0x7eff1c112040 2024-08-20T03:07:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff1a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff1c115380 0x7eff1c111b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff1a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff1c115380 0x7eff1c111b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35892/0 (socket says 172.21.15.44:35892) 2024-08-20T03:07:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff1a59c640 1 -- 172.21.15.44:0/3264812243 learned_addr learned my addr 172.21.15.44:0/3264812243 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff1a59c640 1 -- 172.21.15.44:0/3264812243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff1c112040 msgr2=0x7eff1c1124f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T03:07:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff1a59c640 1 --2- 172.21.15.44:0/3264812243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff1c112040 0x7eff1c1124f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff1a59c640 1 -- 172.21.15.44:0/3264812243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff1c10a4a0 msgr2=0x7eff1c1148c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T03:07:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.032+0000 7eff1a59c640 1 --2- 172.21.15.44:0/3264812243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff1c10a4a0 0x7eff1c1148c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.033+0000 7eff1a59c640 1 -- 172.21.15.44:0/3264812243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c11dad0 con 0x7eff1c115380 2024-08-20T03:07:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.033+0000 7eff1a59c640 1 --2- 172.21.15.44:0/3264812243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff1c115380 0x7eff1c111b00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7eff1000d770 tx=0x7eff1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.055+0000 7eff03fff640 1 -- 172.21.15.44:0/3264812243 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10017070 con 0x7eff1c115380 2024-08-20T03:07:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.055+0000 7eff210cc640 1 -- 172.21.15.44:0/3264812243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c11ddc0 con 0x7eff1c115380 2024-08-20T03:07:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.055+0000 7eff210cc640 1 -- 172.21.15.44:0/3264812243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c11e310 con 0x7eff1c115380 2024-08-20T03:07:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.055+0000 7eff03fff640 1 -- 172.21.15.44:0/3264812243 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff10009c30 con 0x7eff1c115380 2024-08-20T03:07:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.055+0000 7eff03fff640 1 -- 172.21.15.44:0/3264812243 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10003cc0 con 0x7eff1c115380 2024-08-20T03:07:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.066+0000 7eff03fff640 1 -- 172.21.15.44:0/3264812243 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7eff10010460 con 0x7eff1c115380 2024-08-20T03:07:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.067+0000 7eff03fff640 1 --2- 172.21.15.44:0/3264812243 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7eff040b3f50 0x7eff040b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.068+0000 7eff1ad9d640 1 --2- 172.21.15.44:0/3264812243 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7eff040b3f50 0x7eff040b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.069+0000 7eff1ad9d640 1 --2- 172.21.15.44:0/3264812243 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7eff040b3f50 0x7eff040b6410 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0096c0 tx=0x7eff0c005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.069+0000 7eff23ec5740 1 Processor -- start 2024-08-20T03:07:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.069+0000 7eff23ec5740 1 -- start start 2024-08-20T03:07:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.069+0000 7eff23ec5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb795d40 0x5608eb796120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.069+0000 7eff23ec5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb796660 0x5608eb7a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.071 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.069+0000 7eff23ec5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb7a25f0 0x5608eb7a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.069+0000 7eff23ec5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5608eb70b360 con 0x5608eb796660 2024-08-20T03:07:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.069+0000 7eff23ec5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5608eb6fa860 con 0x5608eb7a25f0 2024-08-20T03:07:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.069+0000 7eff23ec5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5608eb6fe8a0 con 0x5608eb795d40 2024-08-20T03:07:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.070+0000 7eff1a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb796660 0x5608eb7a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.070+0000 7eff1a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb796660 0x5608eb7a20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35904/0 (socket says 172.21.15.44:35904) 2024-08-20T03:07:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.070+0000 7eff1a59c640 1 -- 172.21.15.44:0/702188000 learned_addr learned my addr 172.21.15.44:0/702188000 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.071+0000 7eff1a59c640 1 -- 172.21.15.44:0/702188000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb795d40 msgr2=0x5608eb796120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:07:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.071+0000 7eff1a59c640 1 --2- 172.21.15.44:0/702188000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb795d40 0x5608eb796120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.071+0000 7eff1a59c640 1 -- 172.21.15.44:0/702188000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb7a25f0 msgr2=0x5608eb7a49e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:07:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.071+0000 7eff1a59c640 1 --2- 172.21.15.44:0/702188000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb7a25f0 0x5608eb7a49e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.071+0000 7eff1a59c640 1 -- 172.21.15.44:0/702188000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608eb6a1e40 con 0x5608eb796660 2024-08-20T03:07:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.071+0000 7eff1a59c640 1 --2- 172.21.15.44:0/702188000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb796660 0x5608eb7a20b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7eff100e1f80 tx=0x7eff10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db339244c0e9f5b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.074+0000 7eff03fff640 1 -- 172.21.15.44:0/3264812243 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7eff100e48e0 con 0x7eff1c115380 2024-08-20T03:07:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.116+0000 7eff017fa640 1 -- 172.21.15.44:0/702188000 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff100f3070 con 0x5608eb796660 2024-08-20T03:07:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.116+0000 7eff017fa640 1 -- 172.21.15.44:0/702188000 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff100035f0 con 0x5608eb796660 2024-08-20T03:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.118+0000 7eff23ec5740 1 -- 172.21.15.44:0/702188000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb796660 msgr2=0x5608eb7a20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.119+0000 7eff23ec5740 1 --2- 172.21.15.44:0/702188000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb796660 0x5608eb7a20b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7eff100e1f80 tx=0x7eff10017000 comp rx=0 tx=0).stop 2024-08-20T03:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.119+0000 7eff23ec5740 1 -- 172.21.15.44:0/702188000 shutdown_connections 2024-08-20T03:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.119+0000 7eff23ec5740 1 --2- 172.21.15.44:0/702188000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb7a25f0 0x5608eb7a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.119+0000 7eff23ec5740 1 --2- 172.21.15.44:0/702188000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb796660 0x5608eb7a20b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.119+0000 7eff23ec5740 1 --2- 172.21.15.44:0/702188000 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb795d40 0x5608eb796120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.119+0000 7eff23ec5740 1 -- 172.21.15.44:0/702188000 >> 172.21.15.44:0/702188000 conn(0x5608eb793b70 msgr2=0x5608eb794d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.119+0000 7eff23ec5740 1 -- 172.21.15.44:0/702188000 shutdown_connections 2024-08-20T03:07:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.120+0000 7eff23ec5740 1 -- 172.21.15.44:0/702188000 wait complete. 2024-08-20T03:07:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.120+0000 7eff23ec5740 1 -- start start 2024-08-20T03:07:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.120+0000 7eff23ec5740 1 Processor -- start 2024-08-20T03:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.121+0000 7eff23ec5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb795d40 0x5608eb7a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.121+0000 7eff23ec5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb796660 0x5608eb79f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.121+0000 7eff23ec5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a25f0 0x5608eb79f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.121+0000 7eff23ec5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5608eb633ab0 con 0x5608eb7a25f0 2024-08-20T03:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.121+0000 7eff23ec5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5608eb681780 con 0x5608eb795d40 2024-08-20T03:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.121+0000 7eff23ec5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5608eb7a5320 con 0x5608eb796660 2024-08-20T03:07:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.121+0000 7eff23ec5740 10 client.0 _ll_register_callbacks cb 0x5608eb409a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:07:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.122+0000 7eff1b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a25f0 0x5608eb79f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.122+0000 7eff1b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a25f0 0x5608eb79f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35916/0 (socket says 172.21.15.44:35916) 2024-08-20T03:07:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.122+0000 7eff1b59e640 1 -- 172.21.15.44:0/913609923 learned_addr learned my addr 172.21.15.44:0/913609923 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.123+0000 7eff1b59e640 1 -- 172.21.15.44:0/913609923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb796660 msgr2=0x5608eb79f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.123+0000 7eff1b59e640 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb796660 0x5608eb79f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.123+0000 7eff1b59e640 1 -- 172.21.15.44:0/913609923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb795d40 msgr2=0x5608eb7a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.123+0000 7eff1ad9d640 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb795d40 0x5608eb7a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.124+0000 7eff1b59e640 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb795d40 0x5608eb7a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.124+0000 7eff1b59e640 1 -- 172.21.15.44:0/913609923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608eb45d790 con 0x5608eb7a25f0 2024-08-20T03:07:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.124+0000 7eff1ad9d640 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb795d40 0x5608eb7a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:07:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.125+0000 7eff1b59e640 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a25f0 0x5608eb79f940 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7eff1400b440 tx=0x7eff1400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a448aadaac8837d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.148+0000 7efefeff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:07:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.148+0000 7efefeff5640 1 -- 172.21.15.44:0/913609923 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14015020 con 0x5608eb7a25f0 2024-08-20T03:07:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.148+0000 7eff23ec5740 1 -- 172.21.15.44:0/913609923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5608eb6a9e00 con 0x5608eb7a25f0 2024-08-20T03:07:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.149+0000 7efefeff5640 1 -- 172.21.15.44:0/913609923 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7eff14011800 con 0x5608eb7a25f0 2024-08-20T03:07:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.149+0000 7efefeff5640 1 -- 172.21.15.44:0/913609923 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff14020050 con 0x5608eb7a25f0 2024-08-20T03:07:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.149+0000 7eff23ec5740 10 client.16593 fetch_fsmap learned FSMap version 30 2024-08-20T03:07:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.158+0000 7eff23ec5740 1 -- 172.21.15.44:0/913609923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5608eb4d6620 con 0x5608eb7a25f0 2024-08-20T03:07:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.165+0000 7efefeff5640 1 -- 172.21.15.44:0/913609923 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5608eb6a9e00 con 0x5608eb7a25f0 2024-08-20T03:07:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.165+0000 7efefeff5640 1 -- 172.21.15.44:0/913609923 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7eff14019ca0 con 0x5608eb7a25f0 2024-08-20T03:07:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.165+0000 7eff23ec5740 10 client.16593 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:07:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.165+0000 7eff23ec5740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:07:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.165+0000 7eff23ec5740 20 client.16593 populate_metadata read hostname 'smithi044' 2024-08-20T03:07:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.165+0000 7eff23ec5740 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5608eb7b2680 0x5608eb7b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.167 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.165+0000 7eff23ec5740 4 client.16593 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.165+0000 7eff23ec5740 1 -- 172.21.15.44:0/913609923 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608eb70d2b0 con 0x5608eb7b2680 2024-08-20T03:07:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.165+0000 7eff1ad9d640 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5608eb7b2680 0x5608eb7b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.167+0000 7eff1ad9d640 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5608eb7b2680 0x5608eb7b4b40 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=153c6b8a8e7f4cf8 server_cookie=84f9ee47d52fcb1c in_seq=0 out_seq=0 2024-08-20T03:07:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.170+0000 7efefeff5640 10 client.16593 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.172+0000 7efefeff5640 1 -- 172.21.15.44:0/913609923 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5608eb70d2b0 con 0x5608eb7b2680 2024-08-20T03:07:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.172+0000 7efefeff5640 10 client.16593 handle_command_reply: tid=0 2024-08-20T03:07:50.174 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.172+0000 7eff23ec5740 1 client.16593 shutdown 2024-08-20T03:07:50.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.173+0000 7eff23ec5740 1 -- 172.21.15.44:0/913609923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a25f0 msgr2=0x5608eb79f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:50.175 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.173+0000 7eff23ec5740 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a25f0 0x5608eb79f940 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7eff1400b440 tx=0x7eff1400b900 comp rx=0 tx=0).stop 2024-08-20T03:07:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.174+0000 7eff23ec5740 1 -- 172.21.15.44:0/913609923 shutdown_connections 2024-08-20T03:07:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.174+0000 7eff23ec5740 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5608eb7b2680 0x5608eb7b4b40 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.174+0000 7eff23ec5740 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a25f0 0x5608eb79f940 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.174+0000 7eff23ec5740 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb796660 0x5608eb79f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.174+0000 7eff23ec5740 1 --2- 172.21.15.44:0/913609923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb795d40 0x5608eb7a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.174+0000 7eff23ec5740 1 -- 172.21.15.44:0/913609923 >> 172.21.15.44:0/913609923 conn(0x5608eb793b70 msgr2=0x5608eb799710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:50.176 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.174+0000 7efefeff5640 0 client.16593 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.175+0000 7eff23ec5740 1 -- 172.21.15.44:0/913609923 shutdown_connections 2024-08-20T03:07:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.175+0000 7eff23ec5740 1 -- 172.21.15.44:0/913609923 wait complete. 2024-08-20T03:07:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.175+0000 7eff23ec5740 20 client.16593 trim_cache size 0 max 16384 2024-08-20T03:07:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.276+0000 7eff23ec5740 1 Processor -- start 2024-08-20T03:07:50.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:49 smithi044 ceph-mon[18790]: pgmap v480: 97 pgs: 97 active+clean; 734 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 1.04k op/s 2024-08-20T03:07:50.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.284+0000 7eff23ec5740 1 -- start start 2024-08-20T03:07:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.285+0000 7eff23ec5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb795d40 0x5608eb7ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.285+0000 7eff1ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb795d40 0x5608eb7ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.285+0000 7eff1ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb795d40 0x5608eb7ae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35928/0 (socket says 172.21.15.44:35928) 2024-08-20T03:07:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.286+0000 7eff1ad9d640 1 -- 172.21.15.44:0/2018113320 learned_addr learned my addr 172.21.15.44:0/2018113320 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.286+0000 7eff23ec5740 1 --2- 172.21.15.44:0/2018113320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb796660 0x5608eb79db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.286+0000 7eff23ec5740 1 --2- 172.21.15.44:0/2018113320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb7a25f0 0x5608eb79e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.286+0000 7eff23ec5740 1 -- 172.21.15.44:0/2018113320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff0800e800 con 0x5608eb795d40 2024-08-20T03:07:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.286+0000 7eff23ec5740 1 -- 172.21.15.44:0/2018113320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7eff08010400 con 0x5608eb796660 2024-08-20T03:07:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.286+0000 7eff1a59c640 1 --2- 172.21.15.44:0/2018113320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb796660 0x5608eb79db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.286+0000 7eff23ec5740 1 -- 172.21.15.44:0/2018113320 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff0800eed0 con 0x5608eb7a25f0 2024-08-20T03:07:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.286+0000 7eff1b59e640 1 --2- 172.21.15.44:0/2018113320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb7a25f0 0x5608eb79e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.287+0000 7eff1a59c640 1 -- 172.21.15.44:0/2018113320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb7a25f0 msgr2=0x5608eb79e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.287+0000 7eff1a59c640 1 --2- 172.21.15.44:0/2018113320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb7a25f0 0x5608eb79e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.287+0000 7eff1a59c640 1 -- 172.21.15.44:0/2018113320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb795d40 msgr2=0x5608eb7ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.287+0000 7eff1a59c640 1 --2- 172.21.15.44:0/2018113320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb795d40 0x5608eb7ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.287+0000 7eff1a59c640 1 -- 172.21.15.44:0/2018113320 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608eb7a0440 con 0x5608eb796660 2024-08-20T03:07:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.287+0000 7eff1ad9d640 1 --2- 172.21.15.44:0/2018113320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb795d40 0x5608eb7ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:07:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.287+0000 7eff1a59c640 1 --2- 172.21.15.44:0/2018113320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb796660 0x5608eb79db70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5608eb798ec0 tx=0x7eff1000d6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c285dafeb9ddc10 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.297+0000 7eff00ff9640 1 -- 172.21.15.44:0/2018113320 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10020070 con 0x5608eb796660 2024-08-20T03:07:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.297+0000 7eff00ff9640 1 -- 172.21.15.44:0/2018113320 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff100035f0 con 0x5608eb796660 2024-08-20T03:07:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.297+0000 7eff00ff9640 1 -- 172.21.15.44:0/2018113320 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff100aac70 con 0x5608eb796660 2024-08-20T03:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.298+0000 7eff23ec5740 1 -- 172.21.15.44:0/2018113320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb796660 msgr2=0x5608eb79db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.298+0000 7eff23ec5740 1 --2- 172.21.15.44:0/2018113320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb796660 0x5608eb79db70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5608eb798ec0 tx=0x7eff1000d6c0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.299+0000 7eff23ec5740 1 -- 172.21.15.44:0/2018113320 shutdown_connections 2024-08-20T03:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.299+0000 7eff23ec5740 1 --2- 172.21.15.44:0/2018113320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb7a25f0 0x5608eb79e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.299+0000 7eff23ec5740 1 --2- 172.21.15.44:0/2018113320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb796660 0x5608eb79db70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.299+0000 7eff23ec5740 1 --2- 172.21.15.44:0/2018113320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb795d40 0x5608eb7ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.299+0000 7eff23ec5740 1 -- 172.21.15.44:0/2018113320 >> 172.21.15.44:0/2018113320 conn(0x5608eb793b70 msgr2=0x5608eb79f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.299+0000 7eff23ec5740 1 -- 172.21.15.44:0/2018113320 shutdown_connections 2024-08-20T03:07:50.301 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.299+0000 7eff23ec5740 1 -- 172.21.15.44:0/2018113320 wait complete. 2024-08-20T03:07:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.300+0000 7eff23ec5740 1 -- start start 2024-08-20T03:07:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.300+0000 7eff23ec5740 1 Processor -- start 2024-08-20T03:07:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff23ec5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb795d40 0x5608eb7a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff23ec5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb7a25f0 0x5608eb7a4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff1ad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb795d40 0x5608eb7a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.303 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff1ad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb795d40 0x5608eb7a3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:60634/0 (socket says 172.21.15.44:60634) 2024-08-20T03:07:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff23ec5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a2ac0 0x5608eb7a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff1a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb7a25f0 0x5608eb7a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff1a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb7a25f0 0x5608eb7a4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32768/0 (socket says 172.21.15.44:32768) 2024-08-20T03:07:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff1b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a2ac0 0x5608eb7a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff1b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a2ac0 0x5608eb7a2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35940/0 (socket says 172.21.15.44:35940) 2024-08-20T03:07:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff1ad9d640 1 -- 172.21.15.44:0/3926963701 learned_addr learned my addr 172.21.15.44:0/3926963701 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff23ec5740 1 -- 172.21.15.44:0/3926963701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7eff0800fc70 con 0x5608eb7a2ac0 2024-08-20T03:07:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff23ec5740 1 -- 172.21.15.44:0/3926963701 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7eff0800f5a0 con 0x5608eb7a25f0 2024-08-20T03:07:50.304 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.301+0000 7eff23ec5740 1 -- 172.21.15.44:0/3926963701 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7eff0800c7e0 con 0x5608eb795d40 2024-08-20T03:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.302+0000 7eff23ec5740 10 client.0 _ll_register_callbacks cb 0x7eff14009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.302+0000 7eff1a59c640 1 -- 172.21.15.44:0/3926963701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb795d40 msgr2=0x5608eb7a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.302+0000 7eff1a59c640 1 --2- 172.21.15.44:0/3926963701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb795d40 0x5608eb7a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.302+0000 7eff1a59c640 1 -- 172.21.15.44:0/3926963701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a2ac0 msgr2=0x5608eb7a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.302+0000 7eff1a59c640 1 --2- 172.21.15.44:0/3926963701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a2ac0 0x5608eb7a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.302+0000 7eff1a59c640 1 -- 172.21.15.44:0/3926963701 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608eb7a3750 con 0x5608eb7a25f0 2024-08-20T03:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.302+0000 7eff1b59e640 1 --2- 172.21.15.44:0/3926963701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a2ac0 0x5608eb7a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:07:50.305 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.303+0000 7eff1a59c640 1 --2- 172.21.15.44:0/3926963701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb7a25f0 0x5608eb7a4250 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5608eb796630 tx=0x7eff100ea5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af30d3b0207301c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.310+0000 7efefeff5640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:07:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.310+0000 7efefeff5640 1 -- 172.21.15.44:0/3926963701 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10020070 con 0x5608eb7a25f0 2024-08-20T03:07:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.310+0000 7efefeff5640 1 -- 172.21.15.44:0/3926963701 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7eff100e60b0 con 0x5608eb7a25f0 2024-08-20T03:07:50.313 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.311+0000 7eff23ec5740 1 -- 172.21.15.44:0/3926963701 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff100035f0 con 0x5608eb7a25f0 2024-08-20T03:07:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.313+0000 7efefeff5640 1 -- 172.21.15.44:0/3926963701 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff100aac70 con 0x5608eb7a25f0 2024-08-20T03:07:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.313+0000 7eff23ec5740 10 client.26063 fetch_fsmap learned FSMap version 30 2024-08-20T03:07:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.313+0000 7eff23ec5740 1 -- 172.21.15.44:0/3926963701 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5608eb7aa6d0 con 0x5608eb7a25f0 2024-08-20T03:07:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.317+0000 7efefeff5640 1 -- 172.21.15.44:0/3926963701 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff10003e60 con 0x5608eb7a25f0 2024-08-20T03:07:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.317+0000 7efefeff5640 1 -- 172.21.15.44:0/3926963701 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7eff100e22c0 con 0x5608eb7a25f0 2024-08-20T03:07:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.317+0000 7eff23ec5740 10 client.26063 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:07:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.317+0000 7eff23ec5740 10 client.26063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:07:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.317+0000 7eff23ec5740 20 client.26063 populate_metadata read hostname 'smithi044' 2024-08-20T03:07:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.317+0000 7eff23ec5740 1 --2- 172.21.15.44:0/3926963701 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5608eb7aa960 0x5608eb7aad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.317+0000 7eff23ec5740 4 client.26063 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:07:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.317+0000 7eff23ec5740 1 -- 172.21.15.44:0/3926963701 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7eff100035f0 con 0x5608eb7aa960 2024-08-20T03:07:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.317+0000 7eff1ad9d640 1 --2- 172.21.15.44:0/3926963701 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5608eb7aa960 0x5608eb7aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.318+0000 7eff1ad9d640 1 --2- 172.21.15.44:0/3926963701 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5608eb7aa960 0x5608eb7aad70 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b0bd3ef3ee3bc7 server_cookie=ca52ffc2f4fee in_seq=0 out_seq=0 2024-08-20T03:07:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:50.318+0000 7efefeff5640 10 client.26063 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:49 smithi165 ceph-mon[22357]: pgmap v480: 97 pgs: 97 active+clean; 734 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 1.04k op/s 2024-08-20T03:07:50.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:49 smithi099 ceph-mon[21984]: pgmap v480: 97 pgs: 97 active+clean; 734 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 1.04k op/s 2024-08-20T03:07:51.482 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:07:52.301 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:51 smithi044 ceph-mon[18790]: pgmap v481: 97 pgs: 97 active+clean; 746 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 612 op/s 2024-08-20T03:07:52.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.377+0000 7f24c2e3b640 1 -- 172.21.15.44:0/3196245316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24bc1020d0 msgr2=0x7f24bc1065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:52.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.377+0000 7f24c2e3b640 1 --2- 172.21.15.44:0/3196245316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24bc1020d0 0x7f24bc1065a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009ca0 tx=0x7f24b402a8f0 comp rx=0 tx=0).stop 2024-08-20T03:07:52.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.378+0000 7f24c2e3b640 1 -- 172.21.15.44:0/3196245316 shutdown_connections 2024-08-20T03:07:52.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.378+0000 7f24c2e3b640 1 --2- 172.21.15.44:0/3196245316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24bc106bf0 0x7f24bc107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:52.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.378+0000 7f24c2e3b640 1 --2- 172.21.15.44:0/3196245316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24bc1020d0 0x7f24bc1065a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:52.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.378+0000 7f24c2e3b640 1 --2- 172.21.15.44:0/3196245316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24bc1017b0 0x7f24bc101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:52.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.378+0000 7f24c2e3b640 1 -- 172.21.15.44:0/3196245316 >> 172.21.15.44:0/3196245316 conn(0x7f24bc0fd420 msgr2=0x7f24bc0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:52.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.380+0000 7f24c2e3b640 1 -- 172.21.15.44:0/3196245316 shutdown_connections 2024-08-20T03:07:52.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.380+0000 7f24c2e3b640 1 -- 172.21.15.44:0/3196245316 wait complete. 2024-08-20T03:07:52.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.381+0000 7f24c2e3b640 1 Processor -- start 2024-08-20T03:07:52.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.381+0000 7f24c2e3b640 1 -- start start 2024-08-20T03:07:52.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.381+0000 7f24c2e3b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24bc1020d0 0x7f24bc107d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:52.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.381+0000 7f24c2e3b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24bc106bf0 0x7f24bc108280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:52.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.381+0000 7f24c2e3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24bc10cf70 0x7f24bc10d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:52.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.381+0000 7f24c2e3b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc06d1a0 con 0x7f24bc1020d0 2024-08-20T03:07:52.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.381+0000 7f24c2e3b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc06d020 con 0x7f24bc10cf70 2024-08-20T03:07:52.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.381+0000 7f24c2e3b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f24bc06d320 con 0x7f24bc106bf0 2024-08-20T03:07:52.385 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:51 smithi165 ceph-mon[22357]: pgmap v481: 97 pgs: 97 active+clean; 746 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 612 op/s 2024-08-20T03:07:52.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.382+0000 7f24bbfff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24bc106bf0 0x7f24bc108280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:52.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.382+0000 7f24c0bd7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24bc1020d0 0x7f24bc107d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:52.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.382+0000 7f24c0bd7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24bc1020d0 0x7f24bc107d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:35954/0 (socket says 172.21.15.44:35954) 2024-08-20T03:07:52.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.382+0000 7f24c0bd7640 1 -- 172.21.15.44:0/2936759427 learned_addr learned my addr 172.21.15.44:0/2936759427 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:07:52.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.383+0000 7f24c0bd7640 1 -- 172.21.15.44:0/2936759427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24bc106bf0 msgr2=0x7f24bc108280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:52.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.383+0000 7f24c0bd7640 1 --2- 172.21.15.44:0/2936759427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24bc106bf0 0x7f24bc108280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:52.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.383+0000 7f24c0bd7640 1 -- 172.21.15.44:0/2936759427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24bc10cf70 msgr2=0x7f24bc10d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:52.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.383+0000 7f24c0bd7640 1 --2- 172.21.15.44:0/2936759427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24bc10cf70 0x7f24bc10d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:52.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.383+0000 7f24c0bd7640 1 -- 172.21.15.44:0/2936759427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24bc10db50 con 0x7f24bc1020d0 2024-08-20T03:07:52.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.383+0000 7f24c0bd7640 1 --2- 172.21.15.44:0/2936759427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24bc1020d0 0x7f24bc107d40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f24b000b430 tx=0x7f24b000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:52.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.384+0000 7f24b9ffb640 1 -- 172.21.15.44:0/2936759427 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0015020 con 0x7f24bc1020d0 2024-08-20T03:07:52.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.384+0000 7f24c2e3b640 1 -- 172.21.15.44:0/2936759427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24bc114110 con 0x7f24bc1020d0 2024-08-20T03:07:52.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.384+0000 7f24c2e3b640 1 -- 172.21.15.44:0/2936759427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24bc114650 con 0x7f24bc1020d0 2024-08-20T03:07:52.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.385+0000 7f24b9ffb640 1 -- 172.21.15.44:0/2936759427 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24b0011ce0 con 0x7f24bc1020d0 2024-08-20T03:07:52.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.385+0000 7f24b9ffb640 1 -- 172.21.15.44:0/2936759427 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0010dc0 con 0x7f24bc1020d0 2024-08-20T03:07:52.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:51 smithi099 ceph-mon[21984]: pgmap v481: 97 pgs: 97 active+clean; 746 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 612 op/s 2024-08-20T03:07:52.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.389+0000 7f24b9ffb640 1 -- 172.21.15.44:0/2936759427 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f24b0018020 con 0x7f24bc1020d0 2024-08-20T03:07:52.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.390+0000 7f24b9ffb640 1 --2- 172.21.15.44:0/2936759427 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f24a40b3ed0 0x7f24a40b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:07:52.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.391+0000 7f24b9ffb640 1 -- 172.21.15.44:0/2936759427 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f24b00a6930 con 0x7f24bc1020d0 2024-08-20T03:07:52.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.393+0000 7f24c2e3b640 1 -- 172.21.15.44:0/2936759427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a80052c0 con 0x7f24bc1020d0 2024-08-20T03:07:52.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.394+0000 7f24bbfff640 1 --2- 172.21.15.44:0/2936759427 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f24a40b3ed0 0x7f24a40b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:07:52.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.402+0000 7f24b9ffb640 1 -- 172.21.15.44:0/2936759427 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f24b001f020 con 0x7f24bc1020d0 2024-08-20T03:07:52.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.402+0000 7f24bbfff640 1 --2- 172.21.15.44:0/2936759427 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f24a40b3ed0 0x7f24a40b6390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f24bc10e000 tx=0x7f24b4000f40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:07:52.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.853+0000 7f24c2e3b640 1 -- 172.21.15.44:0/2936759427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24a8005880 con 0x7f24bc1020d0 2024-08-20T03:07:52.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.857+0000 7f24b9ffb640 1 -- 172.21.15.44:0/2936759427 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11972 (secure 0 0 0) 0x7f24b00aaa40 con 0x7f24bc1020d0 2024-08-20T03:07:52.870 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:07:52.870 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":30,"btime":"2024-08-20T03:03:44:997954+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:03:44.335961+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24569,24583,24596,24589,24604]},"id":1}]} 2024-08-20T03:07:52.871 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 30 2024-08-20T03:07:52.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.868+0000 7f24c2e3b640 1 -- 172.21.15.44:0/2936759427 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f24a40b3ed0 msgr2=0x7f24a40b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:52.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.869+0000 7f24c2e3b640 1 --2- 172.21.15.44:0/2936759427 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f24a40b3ed0 0x7f24a40b6390 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f24bc10e000 tx=0x7f24b4000f40 comp rx=0 tx=0).stop 2024-08-20T03:07:52.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.869+0000 7f24c2e3b640 1 -- 172.21.15.44:0/2936759427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24bc1020d0 msgr2=0x7f24bc107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:52.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.869+0000 7f24c2e3b640 1 --2- 172.21.15.44:0/2936759427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24bc1020d0 0x7f24bc107d40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f24b000b430 tx=0x7f24b000b8f0 comp rx=0 tx=0).stop 2024-08-20T03:07:52.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.870+0000 7f24c2e3b640 1 -- 172.21.15.44:0/2936759427 shutdown_connections 2024-08-20T03:07:52.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.870+0000 7f24c2e3b640 1 --2- 172.21.15.44:0/2936759427 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f24a40b3ed0 0x7f24a40b6390 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f24bc10e000 tx=0x7f24b4000f40 comp rx=0 tx=0).stop 2024-08-20T03:07:52.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.870+0000 7f24c2e3b640 1 --2- 172.21.15.44:0/2936759427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24bc10cf70 0x7f24bc10d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:52.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.870+0000 7f24c2e3b640 1 --2- 172.21.15.44:0/2936759427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24bc106bf0 0x7f24bc108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:52.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.870+0000 7f24c2e3b640 1 --2- 172.21.15.44:0/2936759427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24bc1020d0 0x7f24bc107d40 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:52.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.870+0000 7f24c2e3b640 1 -- 172.21.15.44:0/2936759427 >> 172.21.15.44:0/2936759427 conn(0x7f24bc0fd420 msgr2=0x7f24bc1044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:52.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.876+0000 7f24c2e3b640 1 -- 172.21.15.44:0/2936759427 shutdown_connections 2024-08-20T03:07:52.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:07:52.876+0000 7f24c2e3b640 1 -- 172.21.15.44:0/2936759427 wait complete. 2024-08-20T03:07:53.290 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:53 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2936759427' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:07:53.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:53 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2936759427' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:07:53.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:53 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2936759427' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:07:54.062 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:54 smithi044 ceph-mon[18790]: pgmap v482: 97 pgs: 97 active+clean; 746 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 505 op/s 2024-08-20T03:07:54.242 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-20T03:07:54.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:54 smithi165 ceph-mon[22357]: pgmap v482: 97 pgs: 97 active+clean; 746 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 505 op/s 2024-08-20T03:07:54.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:54 smithi099 ceph-mon[21984]: pgmap v482: 97 pgs: 97 active+clean; 746 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 505 op/s 2024-08-20T03:07:56.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:56 smithi044 ceph-mon[18790]: pgmap v483: 97 pgs: 97 active+clean; 756 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 559 op/s 2024-08-20T03:07:56.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:56 smithi165 ceph-mon[22357]: pgmap v483: 97 pgs: 97 active+clean; 756 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 559 op/s 2024-08-20T03:07:56.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:56 smithi099 ceph-mon[21984]: pgmap v483: 97 pgs: 97 active+clean; 756 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 559 op/s 2024-08-20T03:07:57.480 INFO:tasks.workunit.client.0.smithi044.stdout: 193 1859270 337 1273560 364 1086543 1305 2024-08-20T03:07:58.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:07:58 smithi165 ceph-mon[22357]: pgmap v484: 97 pgs: 97 active+clean; 767 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 708 op/s 2024-08-20T03:07:58.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:07:58 smithi099 ceph-mon[21984]: pgmap v484: 97 pgs: 97 active+clean; 767 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 708 op/s 2024-08-20T03:07:58.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:07:58 smithi044 ceph-mon[18790]: pgmap v484: 97 pgs: 97 active+clean; 767 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 708 op/s 2024-08-20T03:07:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.005+0000 7efefeff5640 1 -- 172.21.15.44:0/3926963701 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7eff100035f0 con 0x5608eb7aa960 2024-08-20T03:07:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.005+0000 7efefeff5640 10 client.26063 handle_command_reply: tid=0 2024-08-20T03:07:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.005+0000 7eff23ec5740 1 client.26063 shutdown 2024-08-20T03:07:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.005+0000 7eff23ec5740 1 -- 172.21.15.44:0/3926963701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb7a25f0 msgr2=0x5608eb7a4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.005+0000 7eff23ec5740 1 --2- 172.21.15.44:0/3926963701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb7a25f0 0x5608eb7a4250 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5608eb796630 tx=0x7eff100ea5b0 comp rx=0 tx=0).stop 2024-08-20T03:07:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.005+0000 7eff23ec5740 1 -- 172.21.15.44:0/3926963701 shutdown_connections 2024-08-20T03:07:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.005+0000 7eff23ec5740 1 --2- 172.21.15.44:0/3926963701 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5608eb7aa960 0x5608eb7aad70 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.005+0000 7eff23ec5740 1 --2- 172.21.15.44:0/3926963701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5608eb7a2ac0 0x5608eb7a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.005+0000 7eff23ec5740 1 --2- 172.21.15.44:0/3926963701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5608eb7a25f0 0x5608eb7a4250 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.005+0000 7eff23ec5740 1 --2- 172.21.15.44:0/3926963701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5608eb795d40 0x5608eb7a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.006+0000 7efefeff5640 0 client.26063 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:07:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.006+0000 7eff23ec5740 1 -- 172.21.15.44:0/3926963701 >> 172.21.15.44:0/3926963701 conn(0x5608eb793b70 msgr2=0x5608eb79da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.006+0000 7eff23ec5740 1 -- 172.21.15.44:0/3926963701 shutdown_connections 2024-08-20T03:07:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.006+0000 7eff23ec5740 1 -- 172.21.15.44:0/3926963701 wait complete. 2024-08-20T03:07:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.006+0000 7eff23ec5740 20 client.26063 trim_cache size 0 max 16384 2024-08-20T03:07:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:07:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:07:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:07:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.116+0000 7eff210cc640 1 -- 172.21.15.44:0/3264812243 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7eff040b3f50 msgr2=0x7eff040b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.116+0000 7eff210cc640 1 --2- 172.21.15.44:0/3264812243 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7eff040b3f50 0x7eff040b6410 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0096c0 tx=0x7eff0c005fd0 comp rx=0 tx=0).stop 2024-08-20T03:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.116+0000 7eff210cc640 1 -- 172.21.15.44:0/3264812243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff1c115380 msgr2=0x7eff1c111b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:07:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.116+0000 7eff210cc640 1 --2- 172.21.15.44:0/3264812243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff1c115380 0x7eff1c111b00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7eff1000d770 tx=0x7eff1000dc30 comp rx=0 tx=0).stop 2024-08-20T03:07:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.120+0000 7eff210cc640 1 -- 172.21.15.44:0/3264812243 shutdown_connections 2024-08-20T03:07:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.120+0000 7eff210cc640 1 --2- 172.21.15.44:0/3264812243 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7eff040b3f50 0x7eff040b6410 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.120+0000 7eff210cc640 1 --2- 172.21.15.44:0/3264812243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff1c112040 0x7eff1c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.120+0000 7eff210cc640 1 --2- 172.21.15.44:0/3264812243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7eff1c115380 0x7eff1c111b00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.121+0000 7eff210cc640 1 --2- 172.21.15.44:0/3264812243 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff1c10a4a0 0x7eff1c1148c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:07:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.121+0000 7eff210cc640 1 -- 172.21.15.44:0/3264812243 >> 172.21.15.44:0/3264812243 conn(0x7eff1c06d8e0 msgr2=0x7eff1c0739f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.121+0000 7eff210cc640 1 -- 172.21.15.44:0/3264812243 shutdown_connections 2024-08-20T03:07:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:07:59.123+0000 7eff210cc640 1 -- 172.21.15.44:0/3264812243 wait complete. 2024-08-20T03:08:00.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:00 smithi165 ceph-mon[22357]: pgmap v485: 97 pgs: 97 active+clean; 792 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 1.12k op/s 2024-08-20T03:08:00.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:00 smithi099 ceph-mon[21984]: pgmap v485: 97 pgs: 97 active+clean; 792 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 1.12k op/s 2024-08-20T03:08:00.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:00 smithi044 ceph-mon[18790]: pgmap v485: 97 pgs: 97 active+clean; 792 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 1.12k op/s 2024-08-20T03:08:00.639 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:08:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.714+0000 7efd3bbef640 1 -- 172.21.15.44:0/3834143713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd3410a260 msgr2=0x7efd3410a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.714+0000 7efd3bbef640 1 --2- 172.21.15.44:0/3834143713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd3410a260 0x7efd3410a6a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7efd30009900 tx=0x7efd3002a4d0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.718+0000 7efd3bbef640 1 -- 172.21.15.44:0/3834143713 shutdown_connections 2024-08-20T03:08:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.718+0000 7efd3bbef640 1 --2- 172.21.15.44:0/3834143713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3411bdd0 0x7efd3411e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.718+0000 7efd3bbef640 1 --2- 172.21.15.44:0/3834143713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd3410a260 0x7efd3410a6a0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.718+0000 7efd3bbef640 1 --2- 172.21.15.44:0/3834143713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd34109e80 0x7efd34110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.718+0000 7efd3bbef640 1 -- 172.21.15.44:0/3834143713 >> 172.21.15.44:0/3834143713 conn(0x7efd3406dcc0 msgr2=0x7efd3406e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:01.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.720+0000 7efd3bbef640 1 -- 172.21.15.44:0/3834143713 shutdown_connections 2024-08-20T03:08:01.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.723+0000 7efd3bbef640 1 -- 172.21.15.44:0/3834143713 wait complete. 2024-08-20T03:08:01.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.724+0000 7efd3bbef640 1 Processor -- start 2024-08-20T03:08:01.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.724+0000 7efd3bbef640 1 -- start start 2024-08-20T03:08:01.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.725+0000 7efd3bbef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd34109e80 0x7efd341a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.725+0000 7efd3bbef640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3410a260 0x7efd341a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.725+0000 7efd3bbef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd3411bdd0 0x7efd341ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.725+0000 7efd3998b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd34109e80 0x7efd341a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.725+0000 7efd3998b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd34109e80 0x7efd341a7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53686/0 (socket says 172.21.15.44:53686) 2024-08-20T03:08:01.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.726+0000 7efd3918a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3410a260 0x7efd341a82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.726+0000 7efd3918a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3410a260 0x7efd341a82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58060/0 (socket says 172.21.15.44:58060) 2024-08-20T03:08:01.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.726+0000 7efd3998b640 1 -- 172.21.15.44:0/3347032432 learned_addr learned my addr 172.21.15.44:0/3347032432 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:01.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.726+0000 7efd3bbef640 1 -- 172.21.15.44:0/3347032432 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd341218b0 con 0x7efd3411bdd0 2024-08-20T03:08:01.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.726+0000 7efd3bbef640 1 -- 172.21.15.44:0/3347032432 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efd34121730 con 0x7efd34109e80 2024-08-20T03:08:01.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.726+0000 7efd3bbef640 1 -- 172.21.15.44:0/3347032432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efd34121a30 con 0x7efd3410a260 2024-08-20T03:08:01.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.726+0000 7efd3a18c640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd3411bdd0 0x7efd341ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.730+0000 7efd3918a640 1 -- 172.21.15.44:0/3347032432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd34109e80 msgr2=0x7efd341a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:01.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.730+0000 7efd3918a640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd34109e80 0x7efd341a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.730+0000 7efd3918a640 1 -- 172.21.15.44:0/3347032432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd3411bdd0 msgr2=0x7efd341ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:01.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.730+0000 7efd3918a640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd3411bdd0 0x7efd341ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.731+0000 7efd3918a640 1 -- 172.21.15.44:0/3347032432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd341ade50 con 0x7efd3410a260 2024-08-20T03:08:01.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.731+0000 7efd3998b640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd34109e80 0x7efd341a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:01.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.731+0000 7efd3918a640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3410a260 0x7efd341a82e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd30009850 tx=0x7efd30031c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:01.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.732+0000 7efd2affd640 1 -- 172.21.15.44:0/3347032432 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30040030 con 0x7efd3410a260 2024-08-20T03:08:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.732+0000 7efd3bbef640 1 -- 172.21.15.44:0/3347032432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd341ae0e0 con 0x7efd3410a260 2024-08-20T03:08:01.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.733+0000 7efd3bbef640 1 -- 172.21.15.44:0/3347032432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd341b4800 con 0x7efd3410a260 2024-08-20T03:08:01.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.736+0000 7efd2affd640 1 -- 172.21.15.44:0/3347032432 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd3003c8f0 con 0x7efd3410a260 2024-08-20T03:08:01.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.736+0000 7efd2affd640 1 -- 172.21.15.44:0/3347032432 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30044990 con 0x7efd3410a260 2024-08-20T03:08:01.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.743+0000 7efd2affd640 1 -- 172.21.15.44:0/3347032432 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7efd30043030 con 0x7efd3410a260 2024-08-20T03:08:01.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.749+0000 7efd3dfe8740 1 Processor -- start 2024-08-20T03:08:01.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.751+0000 7efd3dfe8740 1 -- start start 2024-08-20T03:08:01.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.751+0000 7efd3dfe8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba6c730 0x55798ba6cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.751+0000 7efd2affd640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efd200b3ed0 0x7efd200b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.752+0000 7efd3dfe8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6d050 0x55798ba78aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.752+0000 7efd3dfe8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba78fe0 0x55798ba7b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.752+0000 7efd3dfe8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55798b9e3930 con 0x55798ba6d050 2024-08-20T03:08:01.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.752+0000 7efd3dfe8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55798b9d2e30 con 0x55798ba6c730 2024-08-20T03:08:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.752+0000 7efd3998b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba6c730 0x55798ba6cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.752+0000 7efd3998b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba6c730 0x55798ba6cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53700/0 (socket says 172.21.15.44:53700) 2024-08-20T03:08:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.752+0000 7efd3998b640 1 -- 172.21.15.44:0/3561746389 learned_addr learned my addr 172.21.15.44:0/3561746389 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.752+0000 7efd3a18c640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efd200b3ed0 0x7efd200b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.752+0000 7efd3918a640 1 --2- 172.21.15.44:0/3561746389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6d050 0x55798ba78aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.752+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3561746389 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55798b9d6e70 con 0x55798ba78fe0 2024-08-20T03:08:01.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.753+0000 7efd3a18c640 1 --2- 172.21.15.44:0/3561746389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba78fe0 0x55798ba7b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.753+0000 7efd2affd640 1 -- 172.21.15.44:0/3347032432 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efd30104640 con 0x7efd3410a260 2024-08-20T03:08:01.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.753+0000 7efd3918a640 1 -- 172.21.15.44:0/3561746389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba78fe0 msgr2=0x55798ba7b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:01.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.753+0000 7efd3918a640 1 --2- 172.21.15.44:0/3561746389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba78fe0 0x55798ba7b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.753+0000 7efd3918a640 1 -- 172.21.15.44:0/3561746389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba6c730 msgr2=0x55798ba6cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:01.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.753+0000 7efd3918a640 1 --2- 172.21.15.44:0/3561746389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba6c730 0x55798ba6cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.753+0000 7efd3918a640 1 -- 172.21.15.44:0/3561746389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798b732650 con 0x55798ba6d050 2024-08-20T03:08:01.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.754+0000 7efd3918a640 1 --2- 172.21.15.44:0/3561746389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6d050 0x55798ba78aa0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efd30101dd0 tx=0x7efd30037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2011ce8161d2587b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:01.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.755+0000 7efd3a18c640 1 --2- 172.21.15.44:0/3561746389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba78fe0 0x55798ba7b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:08:01.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.755+0000 7efd3a18c640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efd200b3ed0 0x7efd200b6390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efd2c005fd0 tx=0x7efd2c009f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:01.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.757+0000 7efd1ffff640 1 -- 172.21.15.44:0/3561746389 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3004a040 con 0x55798ba6d050 2024-08-20T03:08:01.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.757+0000 7efd1ffff640 1 -- 172.21.15.44:0/3561746389 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd300cae80 con 0x55798ba6d050 2024-08-20T03:08:01.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.757+0000 7efd1ffff640 1 -- 172.21.15.44:0/3561746389 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300d4940 con 0x55798ba6d050 2024-08-20T03:08:01.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.761+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3561746389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6d050 msgr2=0x55798ba78aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:01.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.761+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3561746389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6d050 0x55798ba78aa0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efd30101dd0 tx=0x7efd30037000 comp rx=0 tx=0).stop 2024-08-20T03:08:01.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.769+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3561746389 shutdown_connections 2024-08-20T03:08:01.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.769+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3561746389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba78fe0 0x55798ba7b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.769+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3561746389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6d050 0x55798ba78aa0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.769+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3561746389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba6c730 0x55798ba6cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.769+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3561746389 >> 172.21.15.44:0/3561746389 conn(0x55798ba6a560 msgr2=0x55798ba6b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:01.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.780+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3561746389 shutdown_connections 2024-08-20T03:08:01.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.781+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3561746389 wait complete. 2024-08-20T03:08:01.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.781+0000 7efd3dfe8740 1 -- start start 2024-08-20T03:08:01.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.781+0000 7efd3dfe8740 1 Processor -- start 2024-08-20T03:08:01.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.785+0000 7efd3dfe8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba75f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.785+0000 7efd3dfe8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba6d050 0x55798ba76490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.785+0000 7efd3dfe8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba78fe0 0x55798ba769d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.785+0000 7efd3998b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba75f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.785+0000 7efd3998b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba75f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47818/0 (socket says 172.21.15.44:47818) 2024-08-20T03:08:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.785+0000 7efd3998b640 1 -- 172.21.15.44:0/3372677166 learned_addr learned my addr 172.21.15.44:0/3372677166 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.785+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3372677166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55798b835700 con 0x55798ba6c730 2024-08-20T03:08:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.785+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3372677166 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55798b959810 con 0x55798ba78fe0 2024-08-20T03:08:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.785+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3372677166 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55798ba7ba50 con 0x55798ba6d050 2024-08-20T03:08:01.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.785+0000 7efd3dfe8740 10 client.0 _ll_register_callbacks cb 0x55798b9cc700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:01.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.787+0000 7efd3918a640 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba6d050 0x55798ba76490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.787+0000 7efd3a18c640 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba78fe0 0x55798ba769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.787+0000 7efd3998b640 1 -- 172.21.15.44:0/3372677166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba6d050 msgr2=0x55798ba76490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.787+0000 7efd3998b640 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba6d050 0x55798ba76490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.787+0000 7efd3998b640 1 -- 172.21.15.44:0/3372677166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba78fe0 msgr2=0x55798ba769d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.787+0000 7efd3998b640 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba78fe0 0x55798ba769d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.787+0000 7efd3998b640 1 -- 172.21.15.44:0/3372677166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798ba84d50 con 0x55798ba6c730 2024-08-20T03:08:01.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.788+0000 7efd3998b640 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba75f50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efd24004490 tx=0x7efd240039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b7494c1f7f73f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.788+0000 7efd1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:08:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.789+0000 7efd1d7fa640 1 -- 172.21.15.44:0/3372677166 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24012980 con 0x55798ba6c730 2024-08-20T03:08:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.789+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3372677166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55798b982150 con 0x55798ba6c730 2024-08-20T03:08:01.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.789+0000 7efd1d7fa640 1 -- 172.21.15.44:0/3372677166 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efd24012b20 con 0x55798ba6c730 2024-08-20T03:08:01.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.791+0000 7efd1d7fa640 1 -- 172.21.15.44:0/3372677166 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd24012d20 con 0x55798ba6c730 2024-08-20T03:08:01.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.796+0000 7efd3dfe8740 10 client.16617 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.796+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3372677166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55798ba87700 con 0x55798ba6c730 2024-08-20T03:08:01.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.799+0000 7efd1d7fa640 1 -- 172.21.15.44:0/3372677166 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55798b982150 con 0x55798ba6c730 2024-08-20T03:08:01.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.800+0000 7efd1d7fa640 1 -- 172.21.15.44:0/3372677166 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7efd24019070 con 0x55798ba6c730 2024-08-20T03:08:01.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.800+0000 7efd3dfe8740 10 client.16617 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:01.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.800+0000 7efd3dfe8740 10 client.16617 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:08:01.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.800+0000 7efd3dfe8740 20 client.16617 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:01.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.800+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55798ba88f60 0x55798ba8b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.800+0000 7efd3dfe8740 4 client.16617 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:08:01.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.800+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3372677166 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55798b9e5880 con 0x55798ba88f60 2024-08-20T03:08:01.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.806+0000 7efd3918a640 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55798ba88f60 0x55798ba8b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.820+0000 7efd3918a640 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55798ba88f60 0x55798ba8b3c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3a1b67f0d2472bf server_cookie=2acdb884a4e3d301 in_seq=0 out_seq=0 2024-08-20T03:08:01.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.820+0000 7efd1d7fa640 10 client.16617 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:08:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.822+0000 7efd1d7fa640 1 -- 172.21.15.44:0/3372677166 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55798b9e5880 con 0x55798ba88f60 2024-08-20T03:08:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.822+0000 7efd1d7fa640 10 client.16617 handle_command_reply: tid=0 2024-08-20T03:08:01.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.822+0000 7efd3dfe8740 1 client.16617 shutdown 2024-08-20T03:08:01.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.831+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3372677166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 msgr2=0x55798ba75f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:01.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.831+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba75f50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efd24004490 tx=0x7efd240039c0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.836+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3372677166 shutdown_connections 2024-08-20T03:08:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.836+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55798ba88f60 0x55798ba8b3c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.838 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.836+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba78fe0 0x55798ba769d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.836+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba6d050 0x55798ba76490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.836+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3372677166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba75f50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.836+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3372677166 >> 172.21.15.44:0/3372677166 conn(0x55798ba6a560 msgr2=0x55798ba70100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:01.839 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.836+0000 7efd1d7fa640 0 client.16617 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:08:01.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.841+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3372677166 shutdown_connections 2024-08-20T03:08:01.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.843+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3372677166 wait complete. 2024-08-20T03:08:01.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.843+0000 7efd3dfe8740 20 client.16617 trim_cache size 0 max 16384 2024-08-20T03:08:01.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.871+0000 7efd3dfe8740 1 Processor -- start 2024-08-20T03:08:01.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.875+0000 7efd3dfe8740 1 -- start start 2024-08-20T03:08:01.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.875+0000 7efd3dfe8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.875+0000 7efd3dfe8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba6d050 0x55798ba76450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.875+0000 7efd3dfe8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba78fe0 0x55798ba76990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.875+0000 7efd3dfe8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd1800e700 con 0x55798ba6c730 2024-08-20T03:08:01.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.875+0000 7efd3dfe8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efd18010300 con 0x55798ba78fe0 2024-08-20T03:08:01.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.875+0000 7efd3dfe8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efd1800edd0 con 0x55798ba6d050 2024-08-20T03:08:01.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.876+0000 7efd3918a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba6d050 0x55798ba76450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.876+0000 7efd3918a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba6d050 0x55798ba76450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:58100/0 (socket says 172.21.15.44:58100) 2024-08-20T03:08:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.876+0000 7efd3918a640 1 -- 172.21.15.44:0/4055841270 learned_addr learned my addr 172.21.15.44:0/4055841270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.878+0000 7efd3a18c640 1 --2- 172.21.15.44:0/4055841270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba78fe0 0x55798ba76990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.878+0000 7efd3998b640 1 --2- 172.21.15.44:0/4055841270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.878+0000 7efd3918a640 1 -- 172.21.15.44:0/4055841270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba78fe0 msgr2=0x55798ba76990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.878+0000 7efd3918a640 1 --2- 172.21.15.44:0/4055841270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba78fe0 0x55798ba76990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.878+0000 7efd3918a640 1 -- 172.21.15.44:0/4055841270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 msgr2=0x55798ba707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.878+0000 7efd3918a640 1 --2- 172.21.15.44:0/4055841270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.878+0000 7efd3918a640 1 -- 172.21.15.44:0/4055841270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798ba6ce80 con 0x55798ba6d050 2024-08-20T03:08:01.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.879+0000 7efd3998b640 1 --2- 172.21.15.44:0/4055841270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:08:01.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.881+0000 7efd3918a640 1 --2- 172.21.15.44:0/4055841270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba6d050 0x55798ba76450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efd300cb070 tx=0x7efd3002a9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa84788313f91e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:01.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.887+0000 7efd1f7fe640 1 -- 172.21.15.44:0/4055841270 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3004a040 con 0x55798ba6d050 2024-08-20T03:08:01.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.887+0000 7efd1f7fe640 1 -- 172.21.15.44:0/4055841270 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd300cae80 con 0x55798ba6d050 2024-08-20T03:08:01.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.887+0000 7efd1f7fe640 1 -- 172.21.15.44:0/4055841270 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300c7600 con 0x55798ba6d050 2024-08-20T03:08:01.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.890+0000 7efd3dfe8740 1 -- 172.21.15.44:0/4055841270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba6d050 msgr2=0x55798ba76450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:01.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.890+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/4055841270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba6d050 0x55798ba76450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efd300cb070 tx=0x7efd3002a9d0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.894+0000 7efd3dfe8740 1 -- 172.21.15.44:0/4055841270 shutdown_connections 2024-08-20T03:08:01.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.894+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/4055841270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba78fe0 0x55798ba76990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.894+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/4055841270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba6d050 0x55798ba76450 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.894+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/4055841270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.894+0000 7efd3dfe8740 1 -- 172.21.15.44:0/4055841270 >> 172.21.15.44:0/4055841270 conn(0x55798ba6a560 msgr2=0x55798ba694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:01.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.898+0000 7efd3dfe8740 1 -- 172.21.15.44:0/4055841270 shutdown_connections 2024-08-20T03:08:01.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.899+0000 7efd3dfe8740 1 -- 172.21.15.44:0/4055841270 wait complete. 2024-08-20T03:08:01.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.899+0000 7efd3dfe8740 1 -- start start 2024-08-20T03:08:01.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.899+0000 7efd3dfe8740 1 Processor -- start 2024-08-20T03:08:01.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.900+0000 7efd3dfe8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba7bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.901+0000 7efd3998b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba7bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.901+0000 7efd3998b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba7bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47844/0 (socket says 172.21.15.44:47844) 2024-08-20T03:08:01.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.901+0000 7efd3998b640 1 -- 172.21.15.44:0/3270615216 learned_addr learned my addr 172.21.15.44:0/3270615216 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:01.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.901+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba6d050 0x55798ba7c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.901+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba78fe0 0x55798ba7c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.901+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3270615216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efd1800fb70 con 0x55798ba6c730 2024-08-20T03:08:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.901+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3270615216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efd1800f4a0 con 0x55798ba6d050 2024-08-20T03:08:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.902+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3270615216 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7efd1800c6e0 con 0x55798ba78fe0 2024-08-20T03:08:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.902+0000 7efd3918a640 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba6d050 0x55798ba7c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.902+0000 7efd3a18c640 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba78fe0 0x55798ba7c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.902+0000 7efd3998b640 1 -- 172.21.15.44:0/3270615216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba78fe0 msgr2=0x55798ba7c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.902+0000 7efd3998b640 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba78fe0 0x55798ba7c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.902+0000 7efd3998b640 1 -- 172.21.15.44:0/3270615216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba6d050 msgr2=0x55798ba7c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.902+0000 7efd3998b640 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba6d050 0x55798ba7c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.902+0000 7efd3998b640 1 -- 172.21.15.44:0/3270615216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798baf1db0 con 0x55798ba6c730 2024-08-20T03:08:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.903+0000 7efd3918a640 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba6d050 0x55798ba7c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:01.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.903+0000 7efd3a18c640 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba78fe0 0x55798ba7c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.903+0000 7efd3998b640 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba7bc60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7efd24003ec0 tx=0x7efd24010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ca20d51d3c3a8f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.903+0000 7efd3dfe8740 10 client.0 _ll_register_callbacks cb 0x7efd2400bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:01.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.903+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3270615216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd300cae80 con 0x55798ba6c730 2024-08-20T03:08:01.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.904+0000 7efd1d7fa640 10 client.16623 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:08:01.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.904+0000 7efd1d7fa640 1 -- 172.21.15.44:0/3270615216 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24005e50 con 0x55798ba6c730 2024-08-20T03:08:01.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.904+0000 7efd1d7fa640 1 -- 172.21.15.44:0/3270615216 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7efd24012990 con 0x55798ba6c730 2024-08-20T03:08:01.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.904+0000 7efd1d7fa640 1 -- 172.21.15.44:0/3270615216 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efd24033050 con 0x55798ba6c730 2024-08-20T03:08:01.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.909+0000 7efd3dfe8740 10 client.16623 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:01.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.909+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3270615216 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55798baf2df0 con 0x55798ba6c730 2024-08-20T03:08:01.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.909+0000 7efd1d7fa640 1 -- 172.21.15.44:0/3270615216 <== mon.0 v2:172.21.15.44:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7efd2402c060 con 0x55798ba6c730 2024-08-20T03:08:01.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.909+0000 7efd3dfe8740 10 client.16623 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:01.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.909+0000 7efd3dfe8740 10 client.16623 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:08:01.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.909+0000 7efd3dfe8740 20 client.16623 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:01.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.909+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55798baf3480 0x55798baf5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:01.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.909+0000 7efd3dfe8740 4 client.16623 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:08:01.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.909+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3270615216 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7efd300cae80 con 0x55798baf3480 2024-08-20T03:08:01.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.912+0000 7efd3918a640 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55798baf3480 0x55798baf5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:01.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.924+0000 7efd3918a640 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55798baf3480 0x55798baf5940 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dcd167dcfcb18fe0 server_cookie=1821517806524f17 in_seq=0 out_seq=0 2024-08-20T03:08:01.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.925+0000 7efd1d7fa640 10 client.16623 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:08:01.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.929+0000 7efd1d7fa640 1 -- 172.21.15.44:0/3270615216 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7efd300cae80 con 0x55798baf3480 2024-08-20T03:08:01.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.929+0000 7efd1d7fa640 10 client.16623 handle_command_reply: tid=0 2024-08-20T03:08:01.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.929+0000 7efd3dfe8740 1 client.16623 shutdown 2024-08-20T03:08:01.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.930+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3270615216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 msgr2=0x55798ba7bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:01.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.930+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba7bc60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7efd24003ec0 tx=0x7efd24010bd0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.936+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3270615216 shutdown_connections 2024-08-20T03:08:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.936+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55798ba78fe0 0x55798ba7c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.936+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55798baf3480 0x55798baf5940 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.936+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55798ba6d050 0x55798ba7c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.936+0000 7efd3dfe8740 1 --2- 172.21.15.44:0/3270615216 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798ba6c730 0x55798ba7bc60 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.936+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3270615216 >> 172.21.15.44:0/3270615216 conn(0x55798ba6a560 msgr2=0x55798ba77580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.936+0000 7efd1d7fa640 0 client.16623 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:08:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.936+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3270615216 shutdown_connections 2024-08-20T03:08:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.936+0000 7efd3dfe8740 1 -- 172.21.15.44:0/3270615216 wait complete. 2024-08-20T03:08:01.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.937+0000 7efd3dfe8740 20 client.16623 trim_cache size 0 max 16384 2024-08-20T03:08:01.943 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:08:01.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.941+0000 7efd3bbef640 1 -- 172.21.15.44:0/3347032432 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efd200b3ed0 msgr2=0x7efd200b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:01.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.941+0000 7efd3bbef640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efd200b3ed0 0x7efd200b6390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efd2c005fd0 tx=0x7efd2c009f90 comp rx=0 tx=0).stop 2024-08-20T03:08:01.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.941+0000 7efd3bbef640 1 -- 172.21.15.44:0/3347032432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3410a260 msgr2=0x7efd341a82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:01.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.941+0000 7efd3bbef640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3410a260 0x7efd341a82e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd30009850 tx=0x7efd30031c40 comp rx=0 tx=0).stop 2024-08-20T03:08:01.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.942+0000 7efd3bbef640 1 -- 172.21.15.44:0/3347032432 shutdown_connections 2024-08-20T03:08:01.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.942+0000 7efd3bbef640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7efd200b3ed0 0x7efd200b6390 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.942+0000 7efd3bbef640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efd3411bdd0 0x7efd341ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.942+0000 7efd3bbef640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3410a260 0x7efd341a82e0 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd30009850 tx=0x7efd30031c40 comp rx=0 tx=0).stop 2024-08-20T03:08:01.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.942+0000 7efd3bbef640 1 --2- 172.21.15.44:0/3347032432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd34109e80 0x7efd341a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:01.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.942+0000 7efd3bbef640 1 -- 172.21.15.44:0/3347032432 >> 172.21.15.44:0/3347032432 conn(0x7efd3406dcc0 msgr2=0x7efd3411c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.942+0000 7efd3bbef640 1 -- 172.21.15.44:0/3347032432 shutdown_connections 2024-08-20T03:08:01.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:01.943+0000 7efd3bbef640 1 -- 172.21.15.44:0/3347032432 wait complete. 2024-08-20T03:08:02.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:02 smithi165 ceph-mon[22357]: pgmap v486: 97 pgs: 97 active+clean; 800 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.7 MiB/s wr, 1.18k op/s 2024-08-20T03:08:02.408 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:02 smithi044 ceph-mon[18790]: pgmap v486: 97 pgs: 97 active+clean; 800 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.7 MiB/s wr, 1.18k op/s 2024-08-20T03:08:02.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:02 smithi099 ceph-mon[21984]: pgmap v486: 97 pgs: 97 active+clean; 800 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.7 MiB/s wr, 1.18k op/s 2024-08-20T03:08:03.308 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-20T03:08:04.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:04 smithi165 ceph-mon[22357]: pgmap v487: 97 pgs: 97 active+clean; 782 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 1.23k op/s 2024-08-20T03:08:04.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:04 smithi099 ceph-mon[21984]: pgmap v487: 97 pgs: 97 active+clean; 782 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 1.23k op/s 2024-08-20T03:08:04.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:04 smithi044 ceph-mon[18790]: pgmap v487: 97 pgs: 97 active+clean; 782 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 1.23k op/s 2024-08-20T03:08:04.696 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:08:06.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:06 smithi044 ceph-mon[18790]: pgmap v488: 97 pgs: 97 active+clean; 784 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 1.31k op/s 2024-08-20T03:08:06.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:06 smithi165 ceph-mon[22357]: pgmap v488: 97 pgs: 97 active+clean; 784 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 1.31k op/s 2024-08-20T03:08:06.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:06 smithi099 ceph-mon[21984]: pgmap v488: 97 pgs: 97 active+clean; 784 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.6 MiB/s wr, 1.31k op/s 2024-08-20T03:08:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:05.994+0000 7f428f59e640 1 -- 172.21.15.44:0/3818559879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f429010a260 msgr2=0x7f429010a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:05.994+0000 7f428f59e640 1 --2- 172.21.15.44:0/3818559879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f429010a260 0x7f429010a6a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4288009cd0 tx=0x7f428802a930 comp rx=0 tx=0).stop 2024-08-20T03:08:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:05.997+0000 7f428f59e640 1 -- 172.21.15.44:0/3818559879 shutdown_connections 2024-08-20T03:08:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:05.997+0000 7f428f59e640 1 --2- 172.21.15.44:0/3818559879 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f429011bdd0 0x7f429011e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:05.997+0000 7f428f59e640 1 --2- 172.21.15.44:0/3818559879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f429010a260 0x7f429010a6a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:05.997+0000 7f428f59e640 1 --2- 172.21.15.44:0/3818559879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4290109e80 0x7f4290110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:05.997+0000 7f428f59e640 1 -- 172.21.15.44:0/3818559879 >> 172.21.15.44:0/3818559879 conn(0x7f429006c830 msgr2=0x7f429006cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.002+0000 7f428f59e640 1 -- 172.21.15.44:0/3818559879 shutdown_connections 2024-08-20T03:08:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.002+0000 7f428f59e640 1 -- 172.21.15.44:0/3818559879 wait complete. 2024-08-20T03:08:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.004+0000 7f428f59e640 1 Processor -- start 2024-08-20T03:08:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.004+0000 7f428f59e640 1 -- start start 2024-08-20T03:08:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.005+0000 7f428f59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4290109e80 0x7f429011b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.005+0000 7f428f59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f429010a260 0x7f429011b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.005+0000 7f428f59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f429011bdd0 0x7f42901154f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.005+0000 7f428e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4290109e80 0x7f429011b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.006+0000 7f428e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4290109e80 0x7f429011b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53760/0 (socket says 172.21.15.44:53760) 2024-08-20T03:08:07.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.006+0000 7f428dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f429010a260 0x7f429011b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.006+0000 7f428ed9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f429011bdd0 0x7f42901154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.006+0000 7f428dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f429010a260 0x7f429011b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47868/0 (socket says 172.21.15.44:47868) 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.006+0000 7f428dd9b640 1 -- 172.21.15.44:0/1929313296 learned_addr learned my addr 172.21.15.44:0/1929313296 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.006+0000 7f428f59e640 1 -- 172.21.15.44:0/1929313296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4290111eb0 con 0x7f429010a260 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.006+0000 7f428f59e640 1 -- 172.21.15.44:0/1929313296 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4290111d30 con 0x7f4290109e80 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.006+0000 7f428f59e640 1 -- 172.21.15.44:0/1929313296 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4290112030 con 0x7f429011bdd0 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.010+0000 7f428dd9b640 1 -- 172.21.15.44:0/1929313296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f429011bdd0 msgr2=0x7f42901154f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.010+0000 7f428dd9b640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f429011bdd0 0x7f42901154f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.010+0000 7f428dd9b640 1 -- 172.21.15.44:0/1929313296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4290109e80 msgr2=0x7f429011b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.010+0000 7f428dd9b640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4290109e80 0x7f429011b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.010+0000 7f428dd9b640 1 -- 172.21.15.44:0/1929313296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4290115db0 con 0x7f429010a260 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.011+0000 7f428e59c640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4290109e80 0x7f429011b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:08:07.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.011+0000 7f428dd9b640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f429010a260 0x7f429011b710 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4288000f80 tx=0x7f42880305f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.012+0000 7f427f7fe640 1 -- 172.21.15.44:0/1929313296 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288030960 con 0x7f429010a260 2024-08-20T03:08:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.012+0000 7f427f7fe640 1 -- 172.21.15.44:0/1929313296 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428802d070 con 0x7f429010a260 2024-08-20T03:08:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.012+0000 7f427f7fe640 1 -- 172.21.15.44:0/1929313296 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428803a420 con 0x7f429010a260 2024-08-20T03:08:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.012+0000 7f428f59e640 1 -- 172.21.15.44:0/1929313296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4290116040 con 0x7f429010a260 2024-08-20T03:08:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.012+0000 7f428f59e640 1 -- 172.21.15.44:0/1929313296 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42901bcd90 con 0x7f429010a260 2024-08-20T03:08:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.017+0000 7f4297069740 1 Processor -- start 2024-08-20T03:08:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.017+0000 7f4297069740 1 -- start start 2024-08-20T03:08:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.023+0000 7f427f7fe640 1 -- 172.21.15.44:0/1929313296 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f428803a5c0 con 0x7f429010a260 2024-08-20T03:08:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.024+0000 7f4297069740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a881fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.024+0000 7f4297069740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a88205d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.024+0000 7f4297069740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a882e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.024+0000 7f4297069740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5642a8796930 con 0x5642a881f820 2024-08-20T03:08:07.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.024+0000 7f4297069740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5642a8785e30 con 0x5642a8820190 2024-08-20T03:08:07.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.024+0000 7f4297069740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5642a8789e70 con 0x5642a882c010 2024-08-20T03:08:07.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.025+0000 7f427f7fe640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42780b4010 0x7f42780b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.025+0000 7f427f7fe640 1 -- 172.21.15.44:0/1929313296 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f4288105da0 con 0x7f429010a260 2024-08-20T03:08:07.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.026+0000 7f428ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a881fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.026+0000 7f428ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a881fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47876/0 (socket says 172.21.15.44:47876) 2024-08-20T03:08:07.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.026+0000 7f428ed9d640 1 -- 172.21.15.44:0/1760360640 learned_addr learned my addr 172.21.15.44:0/1760360640 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:07.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.027+0000 7f428ed9d640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42780b4010 0x7f42780b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.028+0000 7f428ed9d640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42780b4010 0x7f42780b64d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f428000cfd0 tx=0x7f428000cb40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:07.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.028+0000 7f428ed9d640 1 -- 172.21.15.44:0/1760360640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 msgr2=0x5642a882e4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:08:07.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.028+0000 7f428ed9d640 1 --2- 172.21.15.44:0/1760360640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a882e4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.028+0000 7f428ed9d640 1 -- 172.21.15.44:0/1760360640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 msgr2=0x5642a88205d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:08:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.028+0000 7f428ed9d640 1 --2- 172.21.15.44:0/1760360640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a88205d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.028+0000 7f428ed9d640 1 -- 172.21.15.44:0/1760360640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642a84e5650 con 0x5642a881f820 2024-08-20T03:08:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.032+0000 7f428ed9d640 1 --2- 172.21.15.44:0/1760360640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a881fc50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4280011b10 tx=0x7f428001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91985e570e7df7c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.032+0000 7f4277fff640 1 -- 172.21.15.44:0/1760360640 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280005220 con 0x5642a881f820 2024-08-20T03:08:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.033+0000 7f4277fff640 1 -- 172.21.15.44:0/1760360640 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4280022070 con 0x5642a881f820 2024-08-20T03:08:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.034+0000 7f4297069740 1 -- 172.21.15.44:0/1760360640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 msgr2=0x5642a881fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.034+0000 7f4297069740 1 --2- 172.21.15.44:0/1760360640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a881fc50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4280011b10 tx=0x7f428001d000 comp rx=0 tx=0).stop 2024-08-20T03:08:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.038+0000 7f4297069740 1 -- 172.21.15.44:0/1760360640 shutdown_connections 2024-08-20T03:08:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.038+0000 7f4297069740 1 --2- 172.21.15.44:0/1760360640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a882e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.038+0000 7f4297069740 1 --2- 172.21.15.44:0/1760360640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a88205d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.038+0000 7f4297069740 1 --2- 172.21.15.44:0/1760360640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a881fc50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.038+0000 7f4297069740 1 -- 172.21.15.44:0/1760360640 >> 172.21.15.44:0/1760360640 conn(0x5642a881da80 msgr2=0x5642a881ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.038+0000 7f4297069740 1 -- 172.21.15.44:0/1760360640 shutdown_connections 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.038+0000 7f4297069740 1 -- 172.21.15.44:0/1760360640 wait complete. 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.039+0000 7f4297069740 1 -- start start 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.040+0000 7f4297069740 1 Processor -- start 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.040+0000 7f4297069740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a8828f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.040+0000 7f4297069740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a88294a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.041+0000 7f4297069740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a88299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.041+0000 7f4297069740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5642a85e8700 con 0x5642a881f820 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.041+0000 7f428dd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a88294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.041+0000 7f428dd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a88294a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53782/0 (socket says 172.21.15.44:53782) 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.041+0000 7f428dd9b640 1 -- 172.21.15.44:0/2170250576 learned_addr learned my addr 172.21.15.44:0/2170250576 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:07.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.041+0000 7f4297069740 1 -- 172.21.15.44:0/2170250576 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5642a870c810 con 0x5642a8820190 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.041+0000 7f4297069740 1 -- 172.21.15.44:0/2170250576 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5642a882efb0 con 0x5642a882c010 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.041+0000 7f4297069740 10 client.0 _ll_register_callbacks cb 0x5642a877f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.042+0000 7f428ed9d640 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a88299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.042+0000 7f428dd9b640 1 -- 172.21.15.44:0/2170250576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 msgr2=0x5642a88299e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.042+0000 7f428dd9b640 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a88299e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.042+0000 7f428dd9b640 1 -- 172.21.15.44:0/2170250576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 msgr2=0x5642a8828f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.046+0000 7f428e59c640 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a8828f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.046+0000 7f428dd9b640 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a8828f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.046+0000 7f428dd9b640 1 -- 172.21.15.44:0/2170250576 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642a88383e0 con 0x5642a8820190 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.047+0000 7f428dd9b640 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a88294a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4288031040 tx=0x7f42880cc420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b6cda5da1369d8d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.047+0000 7f428ed9d640 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a88299e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.047+0000 7f428e59c640 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a8828f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.048+0000 7f4275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:08:07.136 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.048+0000 7f4275ffb640 1 -- 172.21.15.44:0/2170250576 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42880e3070 con 0x5642a8820190 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.048+0000 7f4297069740 1 -- 172.21.15.44:0/2170250576 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5642a8735150 con 0x5642a8820190 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.048+0000 7f4275ffb640 1 -- 172.21.15.44:0/2170250576 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f42880c90c0 con 0x5642a8820190 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.055+0000 7f4275ffb640 1 -- 172.21.15.44:0/2170250576 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42880e73e0 con 0x5642a8820190 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.056+0000 7f4297069740 10 client.26081 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.057+0000 7f4297069740 1 -- 172.21.15.44:0/2170250576 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5642a883ad90 con 0x5642a8820190 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.059+0000 7f4275ffb640 1 -- 172.21.15.44:0/2170250576 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4288036040 con 0x5642a8820190 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.059+0000 7f4275ffb640 1 -- 172.21.15.44:0/2170250576 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4288103780 con 0x5642a8820190 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.059+0000 7f4297069740 10 client.26081 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.059+0000 7f4297069740 10 client.26081 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.059+0000 7f4297069740 20 client.26081 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.059+0000 7f4297069740 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5642a883c240 0x5642a883e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.059+0000 7f4297069740 4 client.26081 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.059+0000 7f4297069740 1 -- 172.21.15.44:0/2170250576 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642a8735150 con 0x5642a883c240 2024-08-20T03:08:07.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.059+0000 7f428e59c640 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5642a883c240 0x5642a883e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.061+0000 7f4275ffb640 10 client.26081 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.061+0000 7f428e59c640 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5642a883c240 0x5642a883e670 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62735df67e6ff9fc server_cookie=179fee172b11332e in_seq=0 out_seq=0 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.062+0000 7f4275ffb640 1 -- 172.21.15.44:0/2170250576 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642a8735150 con 0x5642a883c240 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.062+0000 7f4275ffb640 10 client.26081 handle_command_reply: tid=0 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.062+0000 7f4297069740 1 client.26081 shutdown 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.062+0000 7f4297069740 1 -- 172.21.15.44:0/2170250576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 msgr2=0x5642a88294a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.062+0000 7f4297069740 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a88294a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4288031040 tx=0x7f42880cc420 comp rx=0 tx=0).stop 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.065+0000 7f4297069740 1 -- 172.21.15.44:0/2170250576 shutdown_connections 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.065+0000 7f4297069740 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5642a883c240 0x5642a883e670 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.065+0000 7f4297069740 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a88299e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.065+0000 7f4297069740 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a88294a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.065+0000 7f4297069740 1 --2- 172.21.15.44:0/2170250576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a8828f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.065+0000 7f4297069740 1 -- 172.21.15.44:0/2170250576 >> 172.21.15.44:0/2170250576 conn(0x5642a881da80 msgr2=0x5642a881ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:07.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.065+0000 7f4275ffb640 0 client.26081 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.066+0000 7f4297069740 1 -- 172.21.15.44:0/2170250576 shutdown_connections 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.067+0000 7f4297069740 1 -- 172.21.15.44:0/2170250576 wait complete. 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.067+0000 7f4297069740 20 client.26081 trim_cache size 0 max 16384 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.102+0000 7f4297069740 1 Processor -- start 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.106+0000 7f4297069740 1 -- start start 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.106+0000 7f4297069740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a882d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.106+0000 7f4297069740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a882d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.106+0000 7f4297069740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a8822e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.107+0000 7f4297069740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42901c5670 con 0x5642a881f820 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.107+0000 7f4297069740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42901c7270 con 0x5642a8820190 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.107+0000 7f4297069740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42901c5d40 con 0x5642a882c010 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.107+0000 7f428dd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a882d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.107+0000 7f428dd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a882d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53792/0 (socket says 172.21.15.44:53792) 2024-08-20T03:08:07.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.107+0000 7f428dd9b640 1 -- 172.21.15.44:0/1441145302 learned_addr learned my addr 172.21.15.44:0/1441145302 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.107+0000 7f428ed9d640 1 --2- 172.21.15.44:0/1441145302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a8822e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.107+0000 7f428dd9b640 1 -- 172.21.15.44:0/1441145302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 msgr2=0x5642a8822e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.107+0000 7f428dd9b640 1 --2- 172.21.15.44:0/1441145302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a8822e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.107+0000 7f428dd9b640 1 -- 172.21.15.44:0/1441145302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 msgr2=0x5642a882d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.107+0000 7f428dd9b640 1 --2- 172.21.15.44:0/1441145302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a882d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.108+0000 7f428dd9b640 1 -- 172.21.15.44:0/1441145302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642a87effa0 con 0x5642a8820190 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.108+0000 7f428dd9b640 1 --2- 172.21.15.44:0/1441145302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a882d9a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4288005e30 tx=0x7f42880cc550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f9ec5d7e324211e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.108+0000 7f42777fe640 1 -- 172.21.15.44:0/1441145302 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42880e3070 con 0x5642a8820190 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.108+0000 7f42777fe640 1 -- 172.21.15.44:0/1441145302 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4288036070 con 0x5642a8820190 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.109+0000 7f4297069740 1 -- 172.21.15.44:0/1441145302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 msgr2=0x5642a882d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.109+0000 7f4297069740 1 --2- 172.21.15.44:0/1441145302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a882d9a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4288005e30 tx=0x7f42880cc550 comp rx=0 tx=0).stop 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.111+0000 7f4297069740 1 -- 172.21.15.44:0/1441145302 shutdown_connections 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.111+0000 7f4297069740 1 --2- 172.21.15.44:0/1441145302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a8822e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.111+0000 7f4297069740 1 --2- 172.21.15.44:0/1441145302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a8820190 0x5642a882d9a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.111+0000 7f4297069740 1 --2- 172.21.15.44:0/1441145302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a881f820 0x5642a882d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.111+0000 7f4297069740 1 -- 172.21.15.44:0/1441145302 >> 172.21.15.44:0/1441145302 conn(0x5642a881da80 msgr2=0x5642a88277e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.118+0000 7f4297069740 1 -- 172.21.15.44:0/1441145302 shutdown_connections 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.122+0000 7f4297069740 1 -- 172.21.15.44:0/1441145302 wait complete. 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.122+0000 7f4297069740 1 -- start start 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.122+0000 7f4297069740 1 Processor -- start 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.123+0000 7f4297069740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a881f820 0x5642a88430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.123+0000 7f4297069740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a8820190 0x5642a8843600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.123+0000 7f4297069740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a88332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.123+0000 7f4297069740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f42901c6ae0 con 0x5642a8820190 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.123+0000 7f4297069740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42901c6410 con 0x5642a881f820 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.123+0000 7f4297069740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f42901c3650 con 0x5642a882c010 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.123+0000 7f4297069740 10 client.0 _ll_register_callbacks cb 0x7f4288005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.123+0000 7f428dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a8820190 0x5642a8843600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.124+0000 7f428dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a8820190 0x5642a8843600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47894/0 (socket says 172.21.15.44:47894) 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.124+0000 7f428dd9b640 1 -- 172.21.15.44:0/1581316388 learned_addr learned my addr 172.21.15.44:0/1581316388 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.124+0000 7f428dd9b640 1 -- 172.21.15.44:0/1581316388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 msgr2=0x5642a88332c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.124+0000 7f428dd9b640 1 --2- 172.21.15.44:0/1581316388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a88332c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.124+0000 7f428dd9b640 1 -- 172.21.15.44:0/1581316388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a881f820 msgr2=0x5642a88430c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.124+0000 7f428dd9b640 1 --2- 172.21.15.44:0/1581316388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a881f820 0x5642a88430c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.124+0000 7f428dd9b640 1 -- 172.21.15.44:0/1581316388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642a8843eb0 con 0x5642a8820190 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.125+0000 7f428dd9b640 1 --2- 172.21.15.44:0/1581316388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a8820190 0x5642a8843600 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f42880cc7d0 tx=0x7f428803b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3892107798a6a65e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.125+0000 7f4275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.125+0000 7f4275ffb640 1 -- 172.21.15.44:0/1581316388 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42880e3070 con 0x5642a8820190 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.126+0000 7f4297069740 1 -- 172.21.15.44:0/1581316388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5642a8798880 con 0x5642a8820190 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.126+0000 7f4275ffb640 1 -- 172.21.15.44:0/1581316388 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f42880c86f0 con 0x5642a8820190 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.126+0000 7f4275ffb640 1 -- 172.21.15.44:0/1581316388 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f42880cc8a0 con 0x5642a8820190 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.127+0000 7f4297069740 10 client.16635 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:07.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.128+0000 7f4297069740 1 -- 172.21.15.44:0/1581316388 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5642a8843eb0 con 0x5642a8820190 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.128+0000 7f4275ffb640 1 -- 172.21.15.44:0/1581316388 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5642a8798880 con 0x5642a8820190 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.129+0000 7f4275ffb640 1 -- 172.21.15.44:0/1581316388 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f42880dc050 con 0x5642a8820190 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.130+0000 7f4297069740 10 client.16635 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.130+0000 7f4297069740 10 client.16635 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.130+0000 7f4297069740 20 client.16635 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.130+0000 7f4297069740 1 --2- 172.21.15.44:0/1581316388 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5642a8834bd0 0x5642a88a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.130+0000 7f4297069740 4 client.16635 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.130+0000 7f4297069740 1 -- 172.21.15.44:0/1581316388 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642a8794320 con 0x5642a8834bd0 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.131+0000 7f428e59c640 1 --2- 172.21.15.44:0/1581316388 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5642a8834bd0 0x5642a88a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.141+0000 7f428e59c640 1 --2- 172.21.15.44:0/1581316388 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5642a8834bd0 0x5642a88a8f60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4bdf3eab2b9af2da server_cookie=3f8acb75e43cf696 in_seq=0 out_seq=0 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.141+0000 7f4275ffb640 10 client.16635 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.141+0000 7f4275ffb640 1 -- 172.21.15.44:0/1581316388 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642a8794320 con 0x5642a8834bd0 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.141+0000 7f4275ffb640 10 client.16635 handle_command_reply: tid=0 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.141+0000 7f4297069740 1 client.16635 shutdown 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.145+0000 7f4297069740 1 -- 172.21.15.44:0/1581316388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a8820190 msgr2=0x5642a8843600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:07.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.145+0000 7f4297069740 1 --2- 172.21.15.44:0/1581316388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a8820190 0x5642a8843600 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f42880cc7d0 tx=0x7f428803b8c0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.146+0000 7f4297069740 1 -- 172.21.15.44:0/1581316388 shutdown_connections 2024-08-20T03:08:07.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.146+0000 7f4297069740 1 --2- 172.21.15.44:0/1581316388 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5642a8834bd0 0x5642a88a8f60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.146+0000 7f4297069740 1 --2- 172.21.15.44:0/1581316388 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5642a882c010 0x5642a88332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.146+0000 7f4297069740 1 --2- 172.21.15.44:0/1581316388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642a8820190 0x5642a8843600 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.146+0000 7f4297069740 1 --2- 172.21.15.44:0/1581316388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642a881f820 0x5642a88430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.146+0000 7f4275ffb640 0 client.16635 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:08:07.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.146+0000 7f4297069740 1 -- 172.21.15.44:0/1581316388 >> 172.21.15.44:0/1581316388 conn(0x5642a881da80 msgr2=0x5642a8822e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:07.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.146+0000 7f4297069740 1 -- 172.21.15.44:0/1581316388 shutdown_connections 2024-08-20T03:08:07.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.146+0000 7f4297069740 1 -- 172.21.15.44:0/1581316388 wait complete. 2024-08-20T03:08:07.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.146+0000 7f4297069740 20 client.16635 trim_cache size 0 max 16384 2024-08-20T03:08:07.144 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:08:07.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.151+0000 7f427d7fa640 1 -- 172.21.15.44:0/1929313296 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42780b4010 msgr2=0x7f42780b64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:07.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.151+0000 7f427d7fa640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42780b4010 0x7f42780b64d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f428000cfd0 tx=0x7f428000cb40 comp rx=0 tx=0).stop 2024-08-20T03:08:07.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.151+0000 7f427d7fa640 1 -- 172.21.15.44:0/1929313296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f429010a260 msgr2=0x7f429011b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:07.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.151+0000 7f427d7fa640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f429010a260 0x7f429011b710 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4288000f80 tx=0x7f42880305f0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.151+0000 7f427d7fa640 1 -- 172.21.15.44:0/1929313296 shutdown_connections 2024-08-20T03:08:07.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.151+0000 7f427d7fa640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f42780b4010 0x7f42780b64d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.151+0000 7f427d7fa640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f429011bdd0 0x7f42901154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.151+0000 7f427d7fa640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f429010a260 0x7f429011b710 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.151+0000 7f427d7fa640 1 --2- 172.21.15.44:0/1929313296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4290109e80 0x7f429011b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:07.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.151+0000 7f427d7fa640 1 -- 172.21.15.44:0/1929313296 >> 172.21.15.44:0/1929313296 conn(0x7f429006c830 msgr2=0x7f4290071260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:07.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.155+0000 7f427d7fa640 1 -- 172.21.15.44:0/1929313296 shutdown_connections 2024-08-20T03:08:07.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:06.155+0000 7f427d7fa640 1 -- 172.21.15.44:0/1929313296 wait complete. 2024-08-20T03:08:07.480 INFO:tasks.workunit.client.0.smithi044.stdout: 197 2031531 522 1398509 398 1191672 1367 2024-08-20T03:08:08.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:08 smithi165 ceph-mon[22357]: pgmap v489: 97 pgs: 97 active+clean; 786 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.6 MiB/s wr, 982 op/s 2024-08-20T03:08:08.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:08 smithi099 ceph-mon[21984]: pgmap v489: 97 pgs: 97 active+clean; 786 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.6 MiB/s wr, 982 op/s 2024-08-20T03:08:08.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:08 smithi044 ceph-mon[18790]: pgmap v489: 97 pgs: 97 active+clean; 786 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.6 MiB/s wr, 982 op/s 2024-08-20T03:08:09.100 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T03:08:10.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:10 smithi165 ceph-mon[22357]: pgmap v490: 97 pgs: 97 active+clean; 790 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.9 MiB/s wr, 780 op/s 2024-08-20T03:08:10.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:10 smithi099 ceph-mon[21984]: pgmap v490: 97 pgs: 97 active+clean; 790 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.9 MiB/s wr, 780 op/s 2024-08-20T03:08:10.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:10 smithi044 ceph-mon[18790]: pgmap v490: 97 pgs: 97 active+clean; 790 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.9 MiB/s wr, 780 op/s 2024-08-20T03:08:12.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:12 smithi165 ceph-mon[22357]: pgmap v491: 97 pgs: 97 active+clean; 798 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 365 op/s 2024-08-20T03:08:12.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:12 smithi099 ceph-mon[21984]: pgmap v491: 97 pgs: 97 active+clean; 798 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 365 op/s 2024-08-20T03:08:12.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:12 smithi044 ceph-mon[18790]: pgmap v491: 97 pgs: 97 active+clean; 798 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.3 MiB/s wr, 365 op/s 2024-08-20T03:08:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.322+0000 7fe3bad2d640 1 -- 172.21.15.44:0/704710553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3b410a4a0 msgr2=0x7fe3b410a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.322+0000 7fe3bad2d640 1 --2- 172.21.15.44:0/704710553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3b410a4a0 0x7fe3b410a8e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009ca0 tx=0x7fe3ac02a8b0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.323+0000 7fe3bad2d640 1 -- 172.21.15.44:0/704710553 shutdown_connections 2024-08-20T03:08:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.323+0000 7fe3bad2d640 1 --2- 172.21.15.44:0/704710553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b4115380 0x7fe3b41177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.323+0000 7fe3bad2d640 1 --2- 172.21.15.44:0/704710553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3b410a4a0 0x7fe3b410a8e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.323+0000 7fe3bad2d640 1 --2- 172.21.15.44:0/704710553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b4074ca0 0x7fe3b4109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.323+0000 7fe3bad2d640 1 -- 172.21.15.44:0/704710553 >> 172.21.15.44:0/704710553 conn(0x7fe3b406d8b0 msgr2=0x7fe3b406dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.357+0000 7fe3bad2d640 1 -- 172.21.15.44:0/704710553 shutdown_connections 2024-08-20T03:08:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.357+0000 7fe3bad2d640 1 -- 172.21.15.44:0/704710553 wait complete. 2024-08-20T03:08:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.358+0000 7fe3bad2d640 1 Processor -- start 2024-08-20T03:08:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.359+0000 7fe3bad2d640 1 -- start start 2024-08-20T03:08:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.359+0000 7fe3bad2d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b4074ca0 0x7fe3b41a78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.359+0000 7fe3bad2d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3b4115380 0x7fe3b41a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.359+0000 7fe3bad2d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b41ad1c0 0x7fe3b41ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.359+0000 7fe3bad2d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b4121630 con 0x7fe3b4115380 2024-08-20T03:08:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.359+0000 7fe3bad2d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41214b0 con 0x7fe3b41ad1c0 2024-08-20T03:08:13.362 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.359+0000 7fe3bad2d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41217b0 con 0x7fe3b4074ca0 2024-08-20T03:08:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.359+0000 7fe3b9d2b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b4074ca0 0x7fe3b41a78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.360+0000 7fe3ba52c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b41ad1c0 0x7fe3b41ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.360+0000 7fe3ba52c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b41ad1c0 0x7fe3b41ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:38188/0 (socket says 172.21.15.44:38188) 2024-08-20T03:08:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.360+0000 7fe3ba52c640 1 -- 172.21.15.44:0/2500890283 learned_addr learned my addr 172.21.15.44:0/2500890283 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.359+0000 7fe3b9d2b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b4074ca0 0x7fe3b41a78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:55908/0 (socket says 172.21.15.44:55908) 2024-08-20T03:08:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.360+0000 7fe3b952a640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3b4115380 0x7fe3b41a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.360+0000 7fe3ba52c640 1 -- 172.21.15.44:0/2500890283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b4074ca0 msgr2=0x7fe3b41a78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.360+0000 7fe3ba52c640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b4074ca0 0x7fe3b41a78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.360+0000 7fe3ba52c640 1 -- 172.21.15.44:0/2500890283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3b4115380 msgr2=0x7fe3b41a7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.360+0000 7fe3ba52c640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3b4115380 0x7fe3b41a7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.363 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.360+0000 7fe3ba52c640 1 -- 172.21.15.44:0/2500890283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b41addf0 con 0x7fe3b41ad1c0 2024-08-20T03:08:13.364 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.360+0000 7fe3ba52c640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b41ad1c0 0x7fe3b41ad670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000d9e0 tx=0x7fe3b000dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.366+0000 7fe3aaffd640 1 -- 172.21.15.44:0/2500890283 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000cb50 con 0x7fe3b41ad1c0 2024-08-20T03:08:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.366+0000 7fe3bad2d640 1 -- 172.21.15.44:0/2500890283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b41187a0 con 0x7fe3b41ad1c0 2024-08-20T03:08:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.366+0000 7fe3bad2d640 1 -- 172.21.15.44:0/2500890283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b4118cc0 con 0x7fe3b41ad1c0 2024-08-20T03:08:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.370+0000 7fe3bee5a740 1 Processor -- start 2024-08-20T03:08:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.370+0000 7fe3aaffd640 1 -- 172.21.15.44:0/2500890283 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b000ccf0 con 0x7fe3b41ad1c0 2024-08-20T03:08:13.372 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.370+0000 7fe3aaffd640 1 -- 172.21.15.44:0/2500890283 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0015410 con 0x7fe3b41ad1c0 2024-08-20T03:08:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.373+0000 7fe3aaffd640 1 -- 172.21.15.44:0/2500890283 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe3b0015630 con 0x7fe3b41ad1c0 2024-08-20T03:08:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.373+0000 7fe3bee5a740 1 -- start start 2024-08-20T03:08:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.373+0000 7fe3bee5a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820bd40 0x55fc0820c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.374+0000 7fe3bee5a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820c660 0x55fc082180b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.374+0000 7fe3bee5a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0821a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.374+0000 7fe3bee5a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fc08181360 con 0x55fc0820bd40 2024-08-20T03:08:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.374+0000 7fe3bee5a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fc08170860 con 0x55fc0820c660 2024-08-20T03:08:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.374+0000 7fe3bee5a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fc081748a0 con 0x55fc082185f0 2024-08-20T03:08:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.374+0000 7fe3b952a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0821a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.375+0000 7fe3b9d2b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820c660 0x55fc082180b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.375+0000 7fe3b952a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0821a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:55918/0 (socket says 172.21.15.44:55918) 2024-08-20T03:08:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.375+0000 7fe3b952a640 1 -- 172.21.15.44:0/3893699261 learned_addr learned my addr 172.21.15.44:0/3893699261 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.375+0000 7fe3b952a640 1 -- 172.21.15.44:0/3893699261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820c660 msgr2=0x55fc082180b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.375+0000 7fe3b952a640 1 --2- 172.21.15.44:0/3893699261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820c660 0x55fc082180b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.375+0000 7fe3b952a640 1 -- 172.21.15.44:0/3893699261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820bd40 msgr2=0x55fc0820c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.375+0000 7fe3b952a640 1 --2- 172.21.15.44:0/3893699261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820bd40 0x55fc0820c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.375+0000 7fe3b952a640 1 -- 172.21.15.44:0/3893699261 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc08117e40 con 0x55fc082185f0 2024-08-20T03:08:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.375+0000 7fe3b952a640 1 --2- 172.21.15.44:0/3893699261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0821a9e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009ca0 tx=0x7fe3ac005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bde843d2bff7995 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.376+0000 7fe3aaffd640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe3a00b3f50 0x7fe3a00b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.377+0000 7fe3aaffd640 1 -- 172.21.15.44:0/2500890283 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe3b00e3480 con 0x7fe3b41ad1c0 2024-08-20T03:08:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.380+0000 7fe3b9d2b640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe3a00b3f50 0x7fe3a00b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.386+0000 7fe3b9d2b640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe3a00b3f50 0x7fe3a00b6410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55fc0820eec0 tx=0x7fe3a400b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.394+0000 7fe39ffff640 1 -- 172.21.15.44:0/3893699261 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac009800 con 0x55fc082185f0 2024-08-20T03:08:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.394+0000 7fe39ffff640 1 -- 172.21.15.44:0/3893699261 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3ac036070 con 0x55fc082185f0 2024-08-20T03:08:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.395+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3893699261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 msgr2=0x55fc0821a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.395+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/3893699261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0821a9e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009ca0 tx=0x7fe3ac005f80 comp rx=0 tx=0).stop 2024-08-20T03:08:13.419 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:08:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.424+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3893699261 shutdown_connections 2024-08-20T03:08:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.424+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/3893699261 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0821a9e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.424+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/3893699261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820c660 0x55fc082180b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.424+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/3893699261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820bd40 0x55fc0820c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.424+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3893699261 >> 172.21.15.44:0/3893699261 conn(0x55fc08209b70 msgr2=0x55fc0820ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.425+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3893699261 shutdown_connections 2024-08-20T03:08:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.425+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3893699261 wait complete. 2024-08-20T03:08:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.425+0000 7fe3bee5a740 1 -- start start 2024-08-20T03:08:13.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.425+0000 7fe3bee5a740 1 Processor -- start 2024-08-20T03:08:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.426+0000 7fe3bee5a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc08217e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.426+0000 7fe3bee5a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc0820c660 0x55fc08215270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.426+0000 7fe3bee5a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc082185f0 0x55fc082157d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.426+0000 7fe3bee5a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55fc080a9ab0 con 0x55fc082185f0 2024-08-20T03:08:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.426+0000 7fe3bee5a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fc080f7780 con 0x55fc0820bd40 2024-08-20T03:08:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.426+0000 7fe3bee5a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55fc0821b2c0 con 0x55fc0820c660 2024-08-20T03:08:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.426+0000 7fe3bee5a740 10 client.0 _ll_register_callbacks cb 0x55fc07e7fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.427+0000 7fe3b952a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc0820c660 0x55fc08215270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.427+0000 7fe3b952a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc0820c660 0x55fc08215270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:55926/0 (socket says 172.21.15.44:55926) 2024-08-20T03:08:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.427+0000 7fe3b952a640 1 -- 172.21.15.44:0/3728751527 learned_addr learned my addr 172.21.15.44:0/3728751527 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.427+0000 7fe3b9d2b640 1 --2- 172.21.15.44:0/3728751527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc08217e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.427+0000 7fe3b952a640 1 -- 172.21.15.44:0/3728751527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 msgr2=0x55fc08217e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.427+0000 7fe3b952a640 1 --2- 172.21.15.44:0/3728751527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc08217e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.427+0000 7fe3b952a640 1 -- 172.21.15.44:0/3728751527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc082185f0 msgr2=0x55fc082157d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.427+0000 7fe3b952a640 1 --2- 172.21.15.44:0/3728751527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc082185f0 0x55fc082157d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.427+0000 7fe3b952a640 1 -- 172.21.15.44:0/3728751527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc07ed3790 con 0x55fc0820c660 2024-08-20T03:08:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.428+0000 7fe3b952a640 1 --2- 172.21.15.44:0/3728751527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc0820c660 0x55fc08215270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac002230 tx=0x7fe3ac003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=371521133c0945f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.439+0000 7fe39d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:08:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.440+0000 7fe39d7fa640 1 -- 172.21.15.44:0/3728751527 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0309d0 con 0x55fc0820c660 2024-08-20T03:08:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.440+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3728751527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc0811fe00 con 0x55fc0820c660 2024-08-20T03:08:13.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.440+0000 7fe39d7fa640 1 -- 172.21.15.44:0/3728751527 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe3ac004260 con 0x55fc0820c660 2024-08-20T03:08:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.440+0000 7fe39d7fa640 1 -- 172.21.15.44:0/3728751527 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3ac004480 con 0x55fc0820c660 2024-08-20T03:08:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.441+0000 7fe3bee5a740 10 client.26074 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.441+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3728751527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fc07f4c620 con 0x55fc0820c660 2024-08-20T03:08:13.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.441+0000 7fe39d7fa640 1 -- 172.21.15.44:0/3728751527 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fc0811fe00 con 0x55fc0820c660 2024-08-20T03:08:13.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.442+0000 7fe39d7fa640 1 -- 172.21.15.44:0/3728751527 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe3ac057020 con 0x55fc0820c660 2024-08-20T03:08:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.442+0000 7fe3bee5a740 10 client.26074 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.442+0000 7fe3bee5a740 10 client.26074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:08:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.442+0000 7fe3bee5a740 20 client.26074 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.442+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/3728751527 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55fc08228040 0x55fc0822a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.443+0000 7fe3bee5a740 4 client.26074 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:08:13.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.443+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3728751527 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc081832b0 con 0x55fc08228040 2024-08-20T03:08:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.443+0000 7fe3b9d2b640 1 --2- 172.21.15.44:0/3728751527 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55fc08228040 0x55fc0822a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.443+0000 7fe3b9d2b640 1 --2- 172.21.15.44:0/3728751527 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55fc08228040 0x55fc0822a460 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=459c13cafd3fe7ae server_cookie=2eb7ef09a07da877 in_seq=0 out_seq=0 2024-08-20T03:08:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.444+0000 7fe39d7fa640 10 client.26074 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.449+0000 7fe39d7fa640 1 -- 172.21.15.44:0/3728751527 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc081832b0 con 0x55fc08228040 2024-08-20T03:08:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.449+0000 7fe39d7fa640 10 client.26074 handle_command_reply: tid=0 2024-08-20T03:08:13.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.449+0000 7fe3bee5a740 1 client.26074 shutdown 2024-08-20T03:08:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.543+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3728751527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc0820c660 msgr2=0x55fc08215270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.543+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/3728751527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc0820c660 0x55fc08215270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac002230 tx=0x7fe3ac003ba0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.544+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3728751527 shutdown_connections 2024-08-20T03:08:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.544+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/3728751527 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55fc08228040 0x55fc0822a460 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.544+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/3728751527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc082185f0 0x55fc082157d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.544+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/3728751527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc0820c660 0x55fc08215270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.544+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/3728751527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc08217e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.544+0000 7fe39d7fa640 0 client.26074 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.544+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3728751527 >> 172.21.15.44:0/3728751527 conn(0x55fc08209b70 msgr2=0x55fc0820f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.544+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3728751527 shutdown_connections 2024-08-20T03:08:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.545+0000 7fe3bee5a740 1 -- 172.21.15.44:0/3728751527 wait complete. 2024-08-20T03:08:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.545+0000 7fe3bee5a740 20 client.26074 trim_cache size 0 max 16384 2024-08-20T03:08:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.803+0000 7fe3bee5a740 1 Processor -- start 2024-08-20T03:08:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.803+0000 7fe3bee5a740 1 -- start start 2024-08-20T03:08:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.804+0000 7fe3bee5a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc082244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.804+0000 7fe3bee5a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 0x55fc08209420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.804+0000 7fe3bee5a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0822fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.804+0000 7fe3bee5a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe39800e5a0 con 0x55fc0820c660 2024-08-20T03:08:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.804+0000 7fe3bee5a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3980101a0 con 0x55fc0820bd40 2024-08-20T03:08:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.804+0000 7fe3bee5a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe39800ec70 con 0x55fc082185f0 2024-08-20T03:08:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.804+0000 7fe3b952a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 0x55fc08209420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.805+0000 7fe3ba52c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0822fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.805+0000 7fe3ba52c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0822fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:55938/0 (socket says 172.21.15.44:55938) 2024-08-20T03:08:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.805+0000 7fe3ba52c640 1 -- 172.21.15.44:0/1979197928 learned_addr learned my addr 172.21.15.44:0/1979197928 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.805+0000 7fe3b9d2b640 1 --2- 172.21.15.44:0/1979197928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc082244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.805+0000 7fe3ba52c640 1 -- 172.21.15.44:0/1979197928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 msgr2=0x55fc082244c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.805+0000 7fe3ba52c640 1 --2- 172.21.15.44:0/1979197928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc082244c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.805+0000 7fe3ba52c640 1 -- 172.21.15.44:0/1979197928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 msgr2=0x55fc08209420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.805+0000 7fe3ba52c640 1 --2- 172.21.15.44:0/1979197928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 0x55fc08209420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.805+0000 7fe3ba52c640 1 -- 172.21.15.44:0/1979197928 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc082051a0 con 0x55fc082185f0 2024-08-20T03:08:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.805+0000 7fe3b952a640 1 --2- 172.21.15.44:0/1979197928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 0x55fc08209420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:08:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.806+0000 7fe3ba52c640 1 --2- 172.21.15.44:0/1979197928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0822fd30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00e0c40 tx=0x7fe3b00aa060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3250155f8616c303 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.832+0000 7fe39f7fe640 1 -- 172.21.15.44:0/1979197928 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00b3150 con 0x55fc082185f0 2024-08-20T03:08:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.832+0000 7fe39f7fe640 1 -- 172.21.15.44:0/1979197928 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b00a64b0 con 0x55fc082185f0 2024-08-20T03:08:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.832+0000 7fe39f7fe640 1 -- 172.21.15.44:0/1979197928 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00046c0 con 0x55fc082185f0 2024-08-20T03:08:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.834+0000 7fe3bee5a740 1 -- 172.21.15.44:0/1979197928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 msgr2=0x55fc0822fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.834+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/1979197928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0822fd30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00e0c40 tx=0x7fe3b00aa060 comp rx=0 tx=0).stop 2024-08-20T03:08:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.836+0000 7fe3bee5a740 1 -- 172.21.15.44:0/1979197928 shutdown_connections 2024-08-20T03:08:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.836+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/1979197928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0822fd30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.836+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/1979197928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 0x55fc08209420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.836+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/1979197928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc082244c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.836+0000 7fe3bee5a740 1 -- 172.21.15.44:0/1979197928 >> 172.21.15.44:0/1979197928 conn(0x55fc08209b70 msgr2=0x55fc08213af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.839+0000 7fe3bee5a740 1 -- 172.21.15.44:0/1979197928 shutdown_connections 2024-08-20T03:08:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.839+0000 7fe3bee5a740 1 -- 172.21.15.44:0/1979197928 wait complete. 2024-08-20T03:08:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.840+0000 7fe3bee5a740 1 -- start start 2024-08-20T03:08:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.840+0000 7fe3bee5a740 1 Processor -- start 2024-08-20T03:08:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.841+0000 7fe3bee5a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc082209a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.841+0000 7fe3bee5a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 0x55fc08220f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3b9d2b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc082209a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3b952a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 0x55fc08220f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3b9d2b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc082209a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:38228/0 (socket says 172.21.15.44:38228) 2024-08-20T03:08:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3b952a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 0x55fc08220f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38788/0 (socket says 172.21.15.44:38788) 2024-08-20T03:08:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3bee5a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0821c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3b952a640 1 -- 172.21.15.44:0/2005122847 learned_addr learned my addr 172.21.15.44:0/2005122847 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3bee5a740 1 -- 172.21.15.44:0/2005122847 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe39800fa10 con 0x55fc0820c660 2024-08-20T03:08:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3bee5a740 1 -- 172.21.15.44:0/2005122847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe39800f340 con 0x55fc0820bd40 2024-08-20T03:08:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3bee5a740 1 -- 172.21.15.44:0/2005122847 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe39800c630 con 0x55fc082185f0 2024-08-20T03:08:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3bee5a740 10 client.0 _ll_register_callbacks cb 0x7fe3a400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3ba52c640 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0821c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3b9d2b640 1 -- 172.21.15.44:0/2005122847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 msgr2=0x55fc0821c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3b9d2b640 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0821c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3b9d2b640 1 -- 172.21.15.44:0/2005122847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 msgr2=0x55fc08220f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3b9d2b640 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 0x55fc08220f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3b9d2b640 1 -- 172.21.15.44:0/2005122847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc0821c8e0 con 0x55fc0820bd40 2024-08-20T03:08:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.842+0000 7fe3b952a640 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 0x55fc08220f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.843+0000 7fe3b9d2b640 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc082209a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55fc0822ac70 tx=0x7fe3a40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd7e8c49287e040 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.855+0000 7fe39d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:08:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.855+0000 7fe39d7fa640 1 -- 172.21.15.44:0/2005122847 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a401ea10 con 0x55fc0820bd40 2024-08-20T03:08:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.855+0000 7fe39d7fa640 1 -- 172.21.15.44:0/2005122847 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe3a401ebb0 con 0x55fc0820bd40 2024-08-20T03:08:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.861+0000 7fe3bee5a740 1 -- 172.21.15.44:0/2005122847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3b00a64b0 con 0x55fc0820bd40 2024-08-20T03:08:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.863+0000 7fe39d7fa640 1 -- 172.21.15.44:0/2005122847 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3a401edf0 con 0x55fc0820bd40 2024-08-20T03:08:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.865+0000 7fe3bee5a740 10 client.26102 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.865+0000 7fe3bee5a740 1 -- 172.21.15.44:0/2005122847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fc08292a40 con 0x55fc0820bd40 2024-08-20T03:08:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.870+0000 7fe39d7fa640 1 -- 172.21.15.44:0/2005122847 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a402a830 con 0x55fc0820bd40 2024-08-20T03:08:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.870+0000 7fe39d7fa640 1 -- 172.21.15.44:0/2005122847 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe3a4033730 con 0x55fc0820bd40 2024-08-20T03:08:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.871+0000 7fe3bee5a740 10 client.26102 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.871+0000 7fe3bee5a740 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:08:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.871+0000 7fe3bee5a740 20 client.26102 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.871+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55fc08293150 0x55fc08295570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.871+0000 7fe3bee5a740 4 client.26102 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:08:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.872+0000 7fe3b952a640 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55fc08293150 0x55fc08295570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.872+0000 7fe3bee5a740 1 -- 172.21.15.44:0/2005122847 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe3b00a64b0 con 0x55fc08293150 2024-08-20T03:08:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.872+0000 7fe3b952a640 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55fc08293150 0x55fc08295570 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44dbcd28e186546f server_cookie=e2532b357b2a9841 in_seq=0 out_seq=0 2024-08-20T03:08:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:13.888+0000 7fe39d7fa640 10 client.26102 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:14.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:14 smithi044 ceph-mon[18790]: pgmap v492: 97 pgs: 97 active+clean; 804 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 300 op/s 2024-08-20T03:08:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.378+0000 7fe1335ab640 1 -- 172.21.15.44:0/885632192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe12c113820 msgr2=0x7fe12c115c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:14.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.379+0000 7fe1335ab640 1 --2- 172.21.15.44:0/885632192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe12c113820 0x7fe12c115c10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe124009ca0 tx=0x7fe12402a8d0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.385 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:14 smithi165 ceph-mon[22357]: pgmap v492: 97 pgs: 97 active+clean; 804 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 300 op/s 2024-08-20T03:08:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.384+0000 7fe1335ab640 1 -- 172.21.15.44:0/885632192 shutdown_connections 2024-08-20T03:08:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.384+0000 7fe1335ab640 1 --2- 172.21.15.44:0/885632192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe12c113820 0x7fe12c115c10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.384+0000 7fe1335ab640 1 --2- 172.21.15.44:0/885632192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe12c075b90 0x7fe12c1132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.384+0000 7fe1335ab640 1 --2- 172.21.15.44:0/885632192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe12c075270 0x7fe12c075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.384+0000 7fe1335ab640 1 -- 172.21.15.44:0/885632192 >> 172.21.15.44:0/885632192 conn(0x7fe12c0fe590 msgr2=0x7fe12c1009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:14.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.393+0000 7fe1335ab640 1 -- 172.21.15.44:0/885632192 shutdown_connections 2024-08-20T03:08:14.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.396+0000 7fe1335ab640 1 -- 172.21.15.44:0/885632192 wait complete. 2024-08-20T03:08:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.397+0000 7fe1335ab640 1 Processor -- start 2024-08-20T03:08:14.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.397+0000 7fe1335ab640 1 -- start start 2024-08-20T03:08:14.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.398+0000 7fe1335ab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe12c075270 0x7fe12c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.398+0000 7fe1335ab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe12c075b90 0x7fe12c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.398+0000 7fe1335ab640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe12c113820 0x7fe12c1a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.398+0000 7fe131347640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe12c075270 0x7fe12c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.399+0000 7fe131347640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe12c075270 0x7fe12c19f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:38242/0 (socket says 172.21.15.44:38242) 2024-08-20T03:08:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.399+0000 7fe131347640 1 -- 172.21.15.44:0/2729973312 learned_addr learned my addr 172.21.15.44:0/2729973312 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:14.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.399+0000 7fe131b48640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe12c113820 0x7fe12c1a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.399+0000 7fe1335ab640 1 -- 172.21.15.44:0/2729973312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c119160 con 0x7fe12c075b90 2024-08-20T03:08:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.399+0000 7fe1335ab640 1 -- 172.21.15.44:0/2729973312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c118fe0 con 0x7fe12c075270 2024-08-20T03:08:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.399+0000 7fe1335ab640 1 -- 172.21.15.44:0/2729973312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c1192e0 con 0x7fe12c113820 2024-08-20T03:08:14.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.399+0000 7fe130b46640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe12c075b90 0x7fe12c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.400+0000 7fe130b46640 1 -- 172.21.15.44:0/2729973312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe12c113820 msgr2=0x7fe12c1a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.400+0000 7fe130b46640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe12c113820 0x7fe12c1a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.400+0000 7fe130b46640 1 -- 172.21.15.44:0/2729973312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe12c075270 msgr2=0x7fe12c19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.400+0000 7fe130b46640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe12c075270 0x7fe12c19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.400+0000 7fe130b46640 1 -- 172.21.15.44:0/2729973312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12c1a56c0 con 0x7fe12c075b90 2024-08-20T03:08:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.400+0000 7fe130b46640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe12c075b90 0x7fe12c19fb50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe12800c970 tx=0x7fe12800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.401+0000 7fe131347640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe12c075270 0x7fe12c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:08:14.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.401+0000 7fe131b48640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe12c113820 0x7fe12c1a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:08:14.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:14 smithi099 ceph-mon[21984]: pgmap v492: 97 pgs: 97 active+clean; 804 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 300 op/s 2024-08-20T03:08:14.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.406+0000 7fe1227fc640 1 -- 172.21.15.44:0/2729973312 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe128007bb0 con 0x7fe12c075b90 2024-08-20T03:08:14.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.406+0000 7fe1335ab640 1 -- 172.21.15.44:0/2729973312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12c1a59b0 con 0x7fe12c075b90 2024-08-20T03:08:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.406+0000 7fe1335ab640 1 -- 172.21.15.44:0/2729973312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12c1ac0f0 con 0x7fe12c075b90 2024-08-20T03:08:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.417+0000 7fe1359a4740 1 Processor -- start 2024-08-20T03:08:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.417+0000 7fe1359a4740 1 -- start start 2024-08-20T03:08:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.418+0000 7fe1359a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 0x556368d99c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.418+0000 7fe1359a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368da5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.418+0000 7fe1359a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368da6130 0x556368da8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.419+0000 7fe131347640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368da5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.419+0000 7fe131347640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368da5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:38254/0 (socket says 172.21.15.44:38254) 2024-08-20T03:08:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.419+0000 7fe131347640 1 -- 172.21.15.44:0/3501587713 learned_addr learned my addr 172.21.15.44:0/3501587713 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.419+0000 7fe130b46640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368da6130 0x556368da8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.419+0000 7fe1359a4740 1 -- 172.21.15.44:0/3501587713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556368d10930 con 0x556368da6130 2024-08-20T03:08:14.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.419+0000 7fe1359a4740 1 -- 172.21.15.44:0/3501587713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556368cffe30 con 0x556368d9a1a0 2024-08-20T03:08:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.419+0000 7fe1359a4740 1 -- 172.21.15.44:0/3501587713 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556368d03e70 con 0x556368d99880 2024-08-20T03:08:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.419+0000 7fe131b48640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 0x556368d99c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.419+0000 7fe1227fc640 1 -- 172.21.15.44:0/2729973312 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe128007d50 con 0x7fe12c075b90 2024-08-20T03:08:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.420+0000 7fe1227fc640 1 -- 172.21.15.44:0/2729973312 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe128004df0 con 0x7fe12c075b90 2024-08-20T03:08:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.420+0000 7fe1227fc640 1 -- 172.21.15.44:0/2729973312 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe12800f510 con 0x7fe12c075b90 2024-08-20T03:08:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.420+0000 7fe131347640 1 -- 172.21.15.44:0/3501587713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 msgr2=0x556368d99c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.420+0000 7fe131347640 1 --2- 172.21.15.44:0/3501587713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 0x556368d99c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.420+0000 7fe131347640 1 -- 172.21.15.44:0/3501587713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368da6130 msgr2=0x556368da8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:14.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.420+0000 7fe131347640 1 --2- 172.21.15.44:0/3501587713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368da6130 0x556368da8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.421+0000 7fe131347640 1 -- 172.21.15.44:0/3501587713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556368a5f650 con 0x556368d9a1a0 2024-08-20T03:08:14.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.421+0000 7fe131b48640 1 --2- 172.21.15.44:0/3501587713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 0x556368d99c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:14.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.422+0000 7fe131347640 1 --2- 172.21.15.44:0/3501587713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368da5bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00aa70 tx=0x7fe11c00af30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0910e1f242baa72 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:14.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.422+0000 7fe1227fc640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe1180b3f40 0x7fe1180b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.423+0000 7fe1227fc640 1 -- 172.21.15.44:0/2729973312 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe1280e4c10 con 0x7fe12c075b90 2024-08-20T03:08:14.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.423+0000 7fe1177fe640 1 -- 172.21.15.44:0/3501587713 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c015070 con 0x556368d9a1a0 2024-08-20T03:08:14.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.423+0000 7fe131b48640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe1180b3f40 0x7fe1180b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.423+0000 7fe1177fe640 1 -- 172.21.15.44:0/3501587713 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe11c00ccc0 con 0x556368d9a1a0 2024-08-20T03:08:14.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.424+0000 7fe1359a4740 1 -- 172.21.15.44:0/3501587713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 msgr2=0x556368da5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:14.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.424+0000 7fe1359a4740 1 --2- 172.21.15.44:0/3501587713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368da5bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00aa70 tx=0x7fe11c00af30 comp rx=0 tx=0).stop 2024-08-20T03:08:14.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.425+0000 7fe131b48640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe1180b3f40 0x7fe1180b6400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe12402aec0 tx=0x7fe124009380 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:14.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.425+0000 7fe1359a4740 1 -- 172.21.15.44:0/3501587713 shutdown_connections 2024-08-20T03:08:14.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.425+0000 7fe1359a4740 1 --2- 172.21.15.44:0/3501587713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368da6130 0x556368da8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.425+0000 7fe1359a4740 1 --2- 172.21.15.44:0/3501587713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368da5bf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.425+0000 7fe1359a4740 1 --2- 172.21.15.44:0/3501587713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 0x556368d99c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.425+0000 7fe1359a4740 1 -- 172.21.15.44:0/3501587713 >> 172.21.15.44:0/3501587713 conn(0x556368d976b0 msgr2=0x556368d98850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:14.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.425+0000 7fe1359a4740 1 -- 172.21.15.44:0/3501587713 shutdown_connections 2024-08-20T03:08:14.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.426+0000 7fe1359a4740 1 -- 172.21.15.44:0/3501587713 wait complete. 2024-08-20T03:08:14.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.426+0000 7fe1359a4740 1 -- start start 2024-08-20T03:08:14.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.427+0000 7fe1359a4740 1 Processor -- start 2024-08-20T03:08:14.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.428+0000 7fe1359a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d99880 0x556368da30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.428+0000 7fe131347640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d99880 0x556368da30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.428+0000 7fe131347640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d99880 0x556368da30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38824/0 (socket says 172.21.15.44:38824) 2024-08-20T03:08:14.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.428+0000 7fe131347640 1 -- 172.21.15.44:0/4264579090 learned_addr learned my addr 172.21.15.44:0/4264579090 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:14.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.428+0000 7fe1359a4740 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368da35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.428+0000 7fe1359a4740 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368da6130 0x556368da3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.429+0000 7fe1359a4740 1 -- 172.21.15.44:0/4264579090 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556368b62700 con 0x556368d99880 2024-08-20T03:08:14.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.429+0000 7fe1359a4740 1 -- 172.21.15.44:0/4264579090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556368c86810 con 0x556368d9a1a0 2024-08-20T03:08:14.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.429+0000 7fe1359a4740 1 -- 172.21.15.44:0/4264579090 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x556368da8e30 con 0x556368da6130 2024-08-20T03:08:14.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.429+0000 7fe1359a4740 10 client.0 _ll_register_callbacks cb 0x556368cf9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:14.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.429+0000 7fe130b46640 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368da35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.429+0000 7fe131b48640 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368da6130 0x556368da3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.430+0000 7fe130b46640 1 -- 172.21.15.44:0/4264579090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368da6130 msgr2=0x556368da3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:14.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.430+0000 7fe130b46640 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368da6130 0x556368da3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.430+0000 7fe130b46640 1 -- 172.21.15.44:0/4264579090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d99880 msgr2=0x556368da30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:14.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.430+0000 7fe130b46640 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d99880 0x556368da30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.430+0000 7fe130b46640 1 -- 172.21.15.44:0/4264579090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556368db22c0 con 0x556368d9a1a0 2024-08-20T03:08:14.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.431+0000 7fe130b46640 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368da35f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe1280e23f0 tx=0x7fe1280048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=180efbda9f35f6fd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:14.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.431+0000 7fe114ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:08:14.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.433+0000 7fe114ff9640 1 -- 172.21.15.44:0/4264579090 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12801d040 con 0x556368d9a1a0 2024-08-20T03:08:14.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.434+0000 7fe114ff9640 1 -- 172.21.15.44:0/4264579090 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe1280b46e0 con 0x556368d9a1a0 2024-08-20T03:08:14.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.436+0000 7fe1359a4740 1 -- 172.21.15.44:0/4264579090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556368caf150 con 0x556368d9a1a0 2024-08-20T03:08:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.440+0000 7fe114ff9640 1 -- 172.21.15.44:0/4264579090 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe1280b42a0 con 0x556368d9a1a0 2024-08-20T03:08:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.440+0000 7fe1359a4740 10 client.26108 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.440+0000 7fe1359a4740 1 -- 172.21.15.44:0/4264579090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556368db4ce0 con 0x556368d9a1a0 2024-08-20T03:08:14.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.442+0000 7fe114ff9640 1 -- 172.21.15.44:0/4264579090 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556368caf150 con 0x556368d9a1a0 2024-08-20T03:08:14.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.443+0000 7fe114ff9640 1 -- 172.21.15.44:0/4264579090 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe1280eb020 con 0x556368d9a1a0 2024-08-20T03:08:14.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.443+0000 7fe1359a4740 10 client.26108 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:14.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.443+0000 7fe1359a4740 10 client.26108 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:08:14.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.443+0000 7fe1359a4740 20 client.26108 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:14.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.443+0000 7fe1359a4740 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x556368db66d0 0x556368db8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.443+0000 7fe1359a4740 4 client.26108 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:08:14.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.443+0000 7fe1359a4740 1 -- 172.21.15.44:0/4264579090 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556368d12880 con 0x556368db66d0 2024-08-20T03:08:14.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.444+0000 7fe131347640 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x556368db66d0 0x556368db8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.448+0000 7fe131347640 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x556368db66d0 0x556368db8b10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab851b3eb16d98b1 server_cookie=d6ae6aaf4f0611ee in_seq=0 out_seq=0 2024-08-20T03:08:14.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.450+0000 7fe114ff9640 10 client.26108 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:08:14.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.452+0000 7fe114ff9640 1 -- 172.21.15.44:0/4264579090 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556368d12880 con 0x556368db66d0 2024-08-20T03:08:14.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.452+0000 7fe114ff9640 10 client.26108 handle_command_reply: tid=0 2024-08-20T03:08:14.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.452+0000 7fe1359a4740 1 client.26108 shutdown 2024-08-20T03:08:14.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.453+0000 7fe1359a4740 1 -- 172.21.15.44:0/4264579090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 msgr2=0x556368da35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:14.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.453+0000 7fe1359a4740 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368da35f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe1280e23f0 tx=0x7fe1280048f0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.462+0000 7fe1359a4740 1 -- 172.21.15.44:0/4264579090 shutdown_connections 2024-08-20T03:08:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.462+0000 7fe1359a4740 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x556368db66d0 0x556368db8b10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.462+0000 7fe1359a4740 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368da6130 0x556368da3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.462+0000 7fe1359a4740 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368da35f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.462+0000 7fe1359a4740 1 --2- 172.21.15.44:0/4264579090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d99880 0x556368da30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.462+0000 7fe114ff9640 0 client.26108 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:08:14.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.462+0000 7fe1359a4740 1 -- 172.21.15.44:0/4264579090 >> 172.21.15.44:0/4264579090 conn(0x556368d976b0 msgr2=0x556368d9d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:14.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.465+0000 7fe1359a4740 1 -- 172.21.15.44:0/4264579090 shutdown_connections 2024-08-20T03:08:14.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.465+0000 7fe1359a4740 1 -- 172.21.15.44:0/4264579090 wait complete. 2024-08-20T03:08:14.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.466+0000 7fe1359a4740 20 client.26108 trim_cache size 0 max 16384 2024-08-20T03:08:14.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.506+0000 7fe1359a4740 1 Processor -- start 2024-08-20T03:08:14.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.506+0000 7fe1359a4740 1 -- start start 2024-08-20T03:08:14.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.506+0000 7fe1359a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 0x556368d9d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.507+0000 7fe1359a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d9a1a0 0x556368da35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.507+0000 7fe1359a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368da6130 0x556368da3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.507+0000 7fe131347640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 0x556368d9d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.507+0000 7fe131347640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 0x556368d9d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:55994/0 (socket says 172.21.15.44:55994) 2024-08-20T03:08:14.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.507+0000 7fe131347640 1 -- 172.21.15.44:0/4276867822 learned_addr learned my addr 172.21.15.44:0/4276867822 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.507+0000 7fe1359a4740 1 -- 172.21.15.44:0/4276867822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe11000e510 con 0x556368d9a1a0 2024-08-20T03:08:14.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.507+0000 7fe1359a4740 1 -- 172.21.15.44:0/4276867822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe110010110 con 0x556368da6130 2024-08-20T03:08:14.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.507+0000 7fe1359a4740 1 -- 172.21.15.44:0/4276867822 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe11000ebe0 con 0x556368d99880 2024-08-20T03:08:14.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.508+0000 7fe130b46640 1 --2- 172.21.15.44:0/4276867822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d9a1a0 0x556368da35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.508+0000 7fe131347640 1 -- 172.21.15.44:0/4276867822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368da6130 msgr2=0x556368da3af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:08:14.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.508+0000 7fe131347640 1 --2- 172.21.15.44:0/4276867822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368da6130 0x556368da3af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.508+0000 7fe131347640 1 -- 172.21.15.44:0/4276867822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d9a1a0 msgr2=0x556368da35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:14.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.508+0000 7fe131347640 1 --2- 172.21.15.44:0/4276867822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d9a1a0 0x556368da35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.509+0000 7fe131347640 1 -- 172.21.15.44:0/4276867822 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556368da1930 con 0x556368d99880 2024-08-20T03:08:14.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.512+0000 7fe131347640 1 --2- 172.21.15.44:0/4276867822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 0x556368d9d8f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0047b0 tx=0x7fe11c006ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9f33a4cd7275f40 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:14.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.519+0000 7fe116ffd640 1 -- 172.21.15.44:0/4276867822 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c015040 con 0x556368d99880 2024-08-20T03:08:14.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.519+0000 7fe116ffd640 1 -- 172.21.15.44:0/4276867822 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe11c0044f0 con 0x556368d99880 2024-08-20T03:08:14.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.520+0000 7fe1359a4740 1 -- 172.21.15.44:0/4276867822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 msgr2=0x556368d9d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:14.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.520+0000 7fe1359a4740 1 --2- 172.21.15.44:0/4276867822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 0x556368d9d8f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0047b0 tx=0x7fe11c006ea0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.525+0000 7fe1359a4740 1 -- 172.21.15.44:0/4276867822 shutdown_connections 2024-08-20T03:08:14.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.525+0000 7fe1359a4740 1 --2- 172.21.15.44:0/4276867822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368da6130 0x556368da3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.525+0000 7fe1359a4740 1 --2- 172.21.15.44:0/4276867822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d9a1a0 0x556368da35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.525+0000 7fe1359a4740 1 --2- 172.21.15.44:0/4276867822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368d99880 0x556368d9d8f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.525+0000 7fe1359a4740 1 -- 172.21.15.44:0/4276867822 >> 172.21.15.44:0/4276867822 conn(0x556368d976b0 msgr2=0x556368da4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:14.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.530+0000 7fe1359a4740 1 -- 172.21.15.44:0/4276867822 shutdown_connections 2024-08-20T03:08:14.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.530+0000 7fe1359a4740 1 -- 172.21.15.44:0/4276867822 wait complete. 2024-08-20T03:08:14.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.530+0000 7fe1359a4740 1 -- start start 2024-08-20T03:08:14.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.531+0000 7fe1359a4740 1 Processor -- start 2024-08-20T03:08:14.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.532+0000 7fe1359a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d99880 0x556368daa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.532+0000 7fe1359a4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368daacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.532+0000 7fe1359a4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368da6130 0x556368dab230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.532+0000 7fe1359a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe11000f980 con 0x556368d99880 2024-08-20T03:08:14.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.532+0000 7fe1359a4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe11000f2b0 con 0x556368d9a1a0 2024-08-20T03:08:14.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.532+0000 7fe1359a4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe11000c5a0 con 0x556368da6130 2024-08-20T03:08:14.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.532+0000 7fe131347640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d99880 0x556368daa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.532+0000 7fe131347640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d99880 0x556368daa7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38850/0 (socket says 172.21.15.44:38850) 2024-08-20T03:08:14.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.532+0000 7fe131347640 1 -- 172.21.15.44:0/1246412221 learned_addr learned my addr 172.21.15.44:0/1246412221 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:14.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.532+0000 7fe1359a4740 10 client.0 _ll_register_callbacks cb 0x7fe1280a7860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:14.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.532+0000 7fe131b48640 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368da6130 0x556368dab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.532+0000 7fe130b46640 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368daacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.533+0000 7fe131b48640 1 -- 172.21.15.44:0/1246412221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 msgr2=0x556368daacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:14.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.533+0000 7fe131b48640 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368daacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.533+0000 7fe131b48640 1 -- 172.21.15.44:0/1246412221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d99880 msgr2=0x556368daa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:14.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.533+0000 7fe131b48640 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d99880 0x556368daa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.533+0000 7fe131b48640 1 -- 172.21.15.44:0/1246412221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556368e1f640 con 0x556368da6130 2024-08-20T03:08:14.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.533+0000 7fe131b48640 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368da6130 0x556368dab230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe124005e30 tx=0x7fe12402a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12c66f0cf83015ab server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:14.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.533+0000 7fe114ff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:08:14.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.535+0000 7fe114ff9640 1 -- 172.21.15.44:0/1246412221 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124004720 con 0x556368da6130 2024-08-20T03:08:14.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.535+0000 7fe1359a4740 1 -- 172.21.15.44:0/1246412221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe11c0044f0 con 0x556368da6130 2024-08-20T03:08:14.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.535+0000 7fe114ff9640 1 -- 172.21.15.44:0/1246412221 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe1240305d0 con 0x556368da6130 2024-08-20T03:08:14.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.543+0000 7fe114ff9640 1 -- 172.21.15.44:0/1246412221 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe1240307d0 con 0x556368da6130 2024-08-20T03:08:14.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.547+0000 7fe1359a4740 10 client.26092 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:14.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.547+0000 7fe1359a4740 1 -- 172.21.15.44:0/1246412221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556368e20900 con 0x556368da6130 2024-08-20T03:08:14.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.547+0000 7fe114ff9640 1 -- 172.21.15.44:0/1246412221 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe11c0044f0 con 0x556368da6130 2024-08-20T03:08:14.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.547+0000 7fe114ff9640 1 -- 172.21.15.44:0/1246412221 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe124051030 con 0x556368da6130 2024-08-20T03:08:14.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.547+0000 7fe1359a4740 10 client.26092 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:14.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.547+0000 7fe1359a4740 10 client.26092 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:08:14.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.547+0000 7fe1359a4740 20 client.26092 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:14.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.547+0000 7fe1359a4740 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x556368e21030 0x556368e23450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:14.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.547+0000 7fe1359a4740 4 client.26092 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:08:14.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.547+0000 7fe1359a4740 1 -- 172.21.15.44:0/1246412221 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556368d0e320 con 0x556368e21030 2024-08-20T03:08:14.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.551+0000 7fe131347640 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x556368e21030 0x556368e23450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:14.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.563+0000 7fe131347640 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x556368e21030 0x556368e23450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f335e73ee2ede0d2 server_cookie=83c5c261c9a46764 in_seq=0 out_seq=0 2024-08-20T03:08:14.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.564+0000 7fe114ff9640 10 client.26092 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:08:14.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.568+0000 7fe114ff9640 1 -- 172.21.15.44:0/1246412221 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556368d0e320 con 0x556368e21030 2024-08-20T03:08:14.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.568+0000 7fe114ff9640 10 client.26092 handle_command_reply: tid=0 2024-08-20T03:08:14.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.568+0000 7fe1359a4740 1 client.26092 shutdown 2024-08-20T03:08:14.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.580+0000 7fe1359a4740 1 -- 172.21.15.44:0/1246412221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368da6130 msgr2=0x556368dab230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:14.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.580+0000 7fe1359a4740 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368da6130 0x556368dab230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe124005e30 tx=0x7fe12402a8d0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.580+0000 7fe1359a4740 1 -- 172.21.15.44:0/1246412221 shutdown_connections 2024-08-20T03:08:14.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.580+0000 7fe1359a4740 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x556368e21030 0x556368e23450 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.580+0000 7fe1359a4740 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556368da6130 0x556368dab230 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.580+0000 7fe1359a4740 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556368d9a1a0 0x556368daacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.580+0000 7fe1359a4740 1 --2- 172.21.15.44:0/1246412221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556368d99880 0x556368daa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.580+0000 7fe114ff9640 0 client.26092 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:08:14.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.580+0000 7fe1359a4740 1 -- 172.21.15.44:0/1246412221 >> 172.21.15.44:0/1246412221 conn(0x556368d976b0 msgr2=0x556368d95f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:14.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.580+0000 7fe1359a4740 1 -- 172.21.15.44:0/1246412221 shutdown_connections 2024-08-20T03:08:14.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.583+0000 7fe1359a4740 1 -- 172.21.15.44:0/1246412221 wait complete. 2024-08-20T03:08:14.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.583+0000 7fe1359a4740 20 client.26092 trim_cache size 0 max 16384 2024-08-20T03:08:14.586 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:08:14.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.585+0000 7fe1335ab640 1 -- 172.21.15.44:0/2729973312 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe1180b3f40 msgr2=0x7fe1180b6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:14.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.585+0000 7fe1335ab640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe1180b3f40 0x7fe1180b6400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe12402aec0 tx=0x7fe124009380 comp rx=0 tx=0).stop 2024-08-20T03:08:14.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.585+0000 7fe1335ab640 1 -- 172.21.15.44:0/2729973312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe12c075b90 msgr2=0x7fe12c19fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:14.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.585+0000 7fe1335ab640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe12c075b90 0x7fe12c19fb50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe12800c970 tx=0x7fe12800ce30 comp rx=0 tx=0).stop 2024-08-20T03:08:14.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.586+0000 7fe1335ab640 1 -- 172.21.15.44:0/2729973312 shutdown_connections 2024-08-20T03:08:14.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.586+0000 7fe1335ab640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe1180b3f40 0x7fe1180b6400 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.586+0000 7fe1335ab640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe12c113820 0x7fe12c1a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.586+0000 7fe1335ab640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe12c075b90 0x7fe12c19fb50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.586+0000 7fe1335ab640 1 --2- 172.21.15.44:0/2729973312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe12c075270 0x7fe12c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:14.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.586+0000 7fe1335ab640 1 -- 172.21.15.44:0/2729973312 >> 172.21.15.44:0/2729973312 conn(0x7fe12c0fe590 msgr2=0x7fe12c0ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:14.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.587+0000 7fe1335ab640 1 -- 172.21.15.44:0/2729973312 shutdown_connections 2024-08-20T03:08:14.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:14.590+0000 7fe1335ab640 1 -- 172.21.15.44:0/2729973312 wait complete. 2024-08-20T03:08:15.967 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T03:08:16.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:16 smithi044 ceph-mon[18790]: pgmap v493: 97 pgs: 97 active+clean; 816 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 268 op/s 2024-08-20T03:08:16.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:16 smithi165 ceph-mon[22357]: pgmap v493: 97 pgs: 97 active+clean; 816 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 268 op/s 2024-08-20T03:08:16.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:16 smithi099 ceph-mon[21984]: pgmap v493: 97 pgs: 97 active+clean; 816 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 268 op/s 2024-08-20T03:08:17.005 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:08:17.480 INFO:tasks.workunit.client.0.smithi044.stdout: 205 1931005 449 1339765 376 1154710 1302 2024-08-20T03:08:17.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.680+0000 7f0d0a135640 1 -- 172.21.15.44:0/623878418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d041073a0 msgr2=0x7f0d041077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:17.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.680+0000 7f0d0a135640 1 --2- 172.21.15.44:0/623878418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d041073a0 0x7f0d041077e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009900 tx=0x7f0cf802a440 comp rx=0 tx=0).stop 2024-08-20T03:08:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.686+0000 7f0d0a135640 1 -- 172.21.15.44:0/623878418 shutdown_connections 2024-08-20T03:08:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.686+0000 7f0d0a135640 1 --2- 172.21.15.44:0/623878418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d04107d20 0x7f0d0410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.686+0000 7f0d0a135640 1 --2- 172.21.15.44:0/623878418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d041073a0 0x7f0d041077e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.686+0000 7f0d0a135640 1 --2- 172.21.15.44:0/623878418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d04101990 0x7f0d04101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.686+0000 7f0d0a135640 1 -- 172.21.15.44:0/623878418 >> 172.21.15.44:0/623878418 conn(0x7f0d040fd400 msgr2=0x7f0d040ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.686+0000 7f0d0a135640 1 -- 172.21.15.44:0/623878418 shutdown_connections 2024-08-20T03:08:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.688+0000 7f0d0a135640 1 -- 172.21.15.44:0/623878418 wait complete. 2024-08-20T03:08:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.688+0000 7f0d0a135640 1 Processor -- start 2024-08-20T03:08:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.689+0000 7f0d0a135640 1 -- start start 2024-08-20T03:08:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.689+0000 7f0d0a135640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d04101990 0x7f0d04115c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.689+0000 7f0d0a135640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d041073a0 0x7f0d04111f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.689+0000 7f0d0a135640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d04107d20 0x7f0d041125b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.689+0000 7f0d0a135640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d04110940 con 0x7f0d041073a0 2024-08-20T03:08:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.689+0000 7f0d0a135640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0d041107c0 con 0x7f0d04101990 2024-08-20T03:08:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.689+0000 7f0d0a135640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d04110ac0 con 0x7f0d04107d20 2024-08-20T03:08:17.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.690+0000 7f0d02ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d041073a0 0x7f0d04111f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.690+0000 7f0d02ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d041073a0 0x7f0d04111f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46962/0 (socket says 172.21.15.44:46962) 2024-08-20T03:08:17.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.690+0000 7f0d02ffd640 1 -- 172.21.15.44:0/569922856 learned_addr learned my addr 172.21.15.44:0/569922856 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:17.692 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.690+0000 7f0d03fff640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d04107d20 0x7f0d041125b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.691+0000 7f0d037fe640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d04101990 0x7f0d04115c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.692+0000 7f0d037fe640 1 -- 172.21.15.44:0/569922856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d04107d20 msgr2=0x7f0d041125b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:17.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.692+0000 7f0d037fe640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d04107d20 0x7f0d041125b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.692+0000 7f0d037fe640 1 -- 172.21.15.44:0/569922856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d041073a0 msgr2=0x7f0d04111f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:17.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.692+0000 7f0d037fe640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d041073a0 0x7f0d04111f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.692+0000 7f0d037fe640 1 -- 172.21.15.44:0/569922856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0406c8a0 con 0x7f0d04101990 2024-08-20T03:08:17.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.694+0000 7f0d037fe640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d04101990 0x7f0d04115c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400b3e0 tx=0x7f0cf400b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:17.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.696+0000 7f0d00ff9640 1 -- 172.21.15.44:0/569922856 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4015020 con 0x7f0d04101990 2024-08-20T03:08:17.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.697+0000 7f0d00ff9640 1 -- 172.21.15.44:0/569922856 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf4011ce0 con 0x7f0d04101990 2024-08-20T03:08:17.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.697+0000 7f0d0a135640 1 -- 172.21.15.44:0/569922856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0406cb90 con 0x7f0d04101990 2024-08-20T03:08:17.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.697+0000 7f0d0a135640 1 -- 172.21.15.44:0/569922856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d0406d0d0 con 0x7f0d04101990 2024-08-20T03:08:17.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.700+0000 7f0d0c52e740 1 Processor -- start 2024-08-20T03:08:17.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.700+0000 7f0d00ff9640 1 -- 172.21.15.44:0/569922856 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4010d10 con 0x7f0d04101990 2024-08-20T03:08:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.701+0000 7f0d0c52e740 1 -- start start 2024-08-20T03:08:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.701+0000 7f0d0c52e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90e9730 0x55f4c90e9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.701+0000 7f0d0c52e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c90f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.701+0000 7f0d0c52e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.701+0000 7f0d0c52e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f4c9060930 con 0x55f4c90f5fe0 2024-08-20T03:08:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.701+0000 7f0d0c52e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f4c904fe30 con 0x55f4c90e9730 2024-08-20T03:08:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.701+0000 7f0d0c52e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f4c9053e70 con 0x55f4c90ea050 2024-08-20T03:08:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.701+0000 7f0d02ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.701+0000 7f0d02ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46972/0 (socket says 172.21.15.44:46972) 2024-08-20T03:08:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.701+0000 7f0d02ffd640 1 -- 172.21.15.44:0/196075311 learned_addr learned my addr 172.21.15.44:0/196075311 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.701+0000 7f0d037fe640 1 --2- 172.21.15.44:0/196075311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c90f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.702+0000 7f0d03fff640 1 --2- 172.21.15.44:0/196075311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90e9730 0x55f4c90e9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.702+0000 7f0d02ffd640 1 -- 172.21.15.44:0/196075311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 msgr2=0x55f4c90f5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:17.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.702+0000 7f0d02ffd640 1 --2- 172.21.15.44:0/196075311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c90f5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.702+0000 7f0d02ffd640 1 -- 172.21.15.44:0/196075311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90e9730 msgr2=0x55f4c90e9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:17.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.702+0000 7f0d02ffd640 1 --2- 172.21.15.44:0/196075311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90e9730 0x55f4c90e9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.702+0000 7f0d02ffd640 1 -- 172.21.15.44:0/196075311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c8daf650 con 0x55f4c90f5fe0 2024-08-20T03:08:17.706 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.702+0000 7f0d02ffd640 1 --2- 172.21.15.44:0/196075311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f83d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8031870 tx=0x7f0cf80314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b523a7011ac69048 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:17.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.707+0000 7f0d00ff9640 1 -- 172.21.15.44:0/569922856 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0cf4018020 con 0x7f0d04101990 2024-08-20T03:08:17.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.708+0000 7f0d00ff9640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0cec0b3f50 0x7f0cec0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.709+0000 7f0d00ff9640 1 -- 172.21.15.44:0/569922856 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0cf40e37e0 con 0x7f0d04101990 2024-08-20T03:08:17.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.710+0000 7f0ce9ffb640 1 -- 172.21.15.44:0/196075311 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8035040 con 0x55f4c90f5fe0 2024-08-20T03:08:17.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.710+0000 7f0ce9ffb640 1 -- 172.21.15.44:0/196075311 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf8005500 con 0x55f4c90f5fe0 2024-08-20T03:08:17.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.710+0000 7f0d03fff640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0cec0b3f50 0x7f0cec0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.711+0000 7f0d0c52e740 1 -- 172.21.15.44:0/196075311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 msgr2=0x55f4c90f83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:17.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.711+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/196075311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f83d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8031870 tx=0x7f0cf80314a0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.712+0000 7f0d0c52e740 1 -- 172.21.15.44:0/196075311 shutdown_connections 2024-08-20T03:08:17.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.712+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/196075311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f83d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.712+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/196075311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c90f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.712+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/196075311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90e9730 0x55f4c90e9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.712+0000 7f0d0c52e740 1 -- 172.21.15.44:0/196075311 >> 172.21.15.44:0/196075311 conn(0x55f4c90e7560 msgr2=0x55f4c90e8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:17.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.712+0000 7f0d03fff640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0cec0b3f50 0x7f0cec0b6410 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc005fd0 tx=0x7f0cfc009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:17.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.714+0000 7f0d0c52e740 1 -- 172.21.15.44:0/196075311 shutdown_connections 2024-08-20T03:08:17.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.715+0000 7f0d0c52e740 1 -- 172.21.15.44:0/196075311 wait complete. 2024-08-20T03:08:17.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.715+0000 7f0d0c52e740 1 -- start start 2024-08-20T03:08:17.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.716+0000 7f0d0c52e740 1 Processor -- start 2024-08-20T03:08:17.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.717+0000 7f0d0c52e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90e9730 0x55f4c90f5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.717+0000 7f0d0c52e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c90f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.717+0000 7f0d0c52e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.717+0000 7f0d0c52e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f4c8eb2700 con 0x55f4c90f5fe0 2024-08-20T03:08:17.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.717+0000 7f0d0c52e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f4c8fd6810 con 0x55f4c90e9730 2024-08-20T03:08:17.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.717+0000 7f0d0c52e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f4c90f8cd0 con 0x55f4c90ea050 2024-08-20T03:08:17.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.717+0000 7f0d0c52e740 10 client.0 _ll_register_callbacks cb 0x55f4c9049700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:17.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.718+0000 7f0d02ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c90f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.718+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.718+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46978/0 (socket says 172.21.15.44:46978) 2024-08-20T03:08:17.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.718+0000 7f0d02ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c90f2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53178/0 (socket says 172.21.15.44:53178) 2024-08-20T03:08:17.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.718+0000 7f0d02ffd640 1 -- 172.21.15.44:0/1545709682 learned_addr learned my addr 172.21.15.44:0/1545709682 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:17.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.719+0000 7f0d037fe640 1 --2- 172.21.15.44:0/1545709682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90e9730 0x55f4c90f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.720+0000 7f0d03fff640 1 -- 172.21.15.44:0/1545709682 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 msgr2=0x55f4c90f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:17.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.720+0000 7f0d03fff640 1 --2- 172.21.15.44:0/1545709682 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c90f2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.720+0000 7f0d03fff640 1 -- 172.21.15.44:0/1545709682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90e9730 msgr2=0x55f4c90f5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:17.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.720+0000 7f0d03fff640 1 --2- 172.21.15.44:0/1545709682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90e9730 0x55f4c90f5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.720+0000 7f0d03fff640 1 -- 172.21.15.44:0/1545709682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c91020b0 con 0x55f4c90f5fe0 2024-08-20T03:08:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.721+0000 7f0d03fff640 1 --2- 172.21.15.44:0/1545709682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f3490 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc012cb0 tx=0x7f0cfc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f690bc87d653e65d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.721+0000 7f0ce2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:08:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.721+0000 7f0ce2ffd640 1 -- 172.21.15.44:0/1545709682 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc004b20 con 0x55f4c90f5fe0 2024-08-20T03:08:17.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.721+0000 7f0d0c52e740 1 -- 172.21.15.44:0/1545709682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f4c8fff150 con 0x55f4c90f5fe0 2024-08-20T03:08:17.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.722+0000 7f0ce2ffd640 1 -- 172.21.15.44:0/1545709682 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0cfc018030 con 0x55f4c90f5fe0 2024-08-20T03:08:17.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.722+0000 7f0ce2ffd640 1 -- 172.21.15.44:0/1545709682 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc004cc0 con 0x55f4c90f5fe0 2024-08-20T03:08:17.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.723+0000 7f0d0c52e740 10 client.16677 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:17.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.723+0000 7f0d0c52e740 1 -- 172.21.15.44:0/1545709682 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f4c91049b0 con 0x55f4c90f5fe0 2024-08-20T03:08:17.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.723+0000 7f0ce2ffd640 1 -- 172.21.15.44:0/1545709682 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f4c8fff150 con 0x55f4c90f5fe0 2024-08-20T03:08:17.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.723+0000 7f0ce2ffd640 1 -- 172.21.15.44:0/1545709682 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0cfc031030 con 0x55f4c90f5fe0 2024-08-20T03:08:17.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.724+0000 7f0d0c52e740 10 client.16677 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:17.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.724+0000 7f0d0c52e740 10 client.16677 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:08:17.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.724+0000 7f0d0c52e740 20 client.16677 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:17.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.724+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/1545709682 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55f4c9106340 0x55f4c91087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.724+0000 7f0d0c52e740 4 client.16677 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:08:17.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.724+0000 7f0d0c52e740 1 -- 172.21.15.44:0/1545709682 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4c9062880 con 0x55f4c9106340 2024-08-20T03:08:17.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.724+0000 7f0d037fe640 1 --2- 172.21.15.44:0/1545709682 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55f4c9106340 0x55f4c91087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.725+0000 7f0ce2ffd640 10 client.16677 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:08:17.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.726+0000 7f0d037fe640 1 --2- 172.21.15.44:0/1545709682 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55f4c9106340 0x55f4c91087e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4bf45aa1ff3ee53 server_cookie=adbf015f76ed0384 in_seq=0 out_seq=0 2024-08-20T03:08:17.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.728+0000 7f0ce2ffd640 1 -- 172.21.15.44:0/1545709682 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4c9062880 con 0x55f4c9106340 2024-08-20T03:08:17.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.728+0000 7f0ce2ffd640 10 client.16677 handle_command_reply: tid=0 2024-08-20T03:08:17.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.729+0000 7f0d0c52e740 1 client.16677 shutdown 2024-08-20T03:08:17.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.733+0000 7f0d0c52e740 1 -- 172.21.15.44:0/1545709682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 msgr2=0x55f4c90f3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:17.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.733+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/1545709682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f3490 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc012cb0 tx=0x7f0cfc004ab0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.734+0000 7f0d0c52e740 1 -- 172.21.15.44:0/1545709682 shutdown_connections 2024-08-20T03:08:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.734+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/1545709682 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55f4c9106340 0x55f4c91087e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.734+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/1545709682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f3490 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.734+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/1545709682 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c90f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.734+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/1545709682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90e9730 0x55f4c90f5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.734+0000 7f0d0c52e740 1 -- 172.21.15.44:0/1545709682 >> 172.21.15.44:0/1545709682 conn(0x55f4c90e7560 msgr2=0x55f4c90ed100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:17.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.734+0000 7f0ce2ffd640 0 client.16677 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:08:17.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.737+0000 7f0d0c52e740 1 -- 172.21.15.44:0/1545709682 shutdown_connections 2024-08-20T03:08:17.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.739+0000 7f0d0c52e740 1 -- 172.21.15.44:0/1545709682 wait complete. 2024-08-20T03:08:17.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.739+0000 7f0d0c52e740 20 client.16677 trim_cache size 0 max 16384 2024-08-20T03:08:17.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.772+0000 7f0d0c52e740 1 Processor -- start 2024-08-20T03:08:17.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.772+0000 7f0d0c52e740 1 -- start start 2024-08-20T03:08:17.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.772+0000 7f0d0c52e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90e9730 0x55f4c90f2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.772+0000 7f0d0c52e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c910e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.772+0000 7f0d0c52e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90f5fe0 0x55f4c910e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.772+0000 7f0d0c52e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf000e790 con 0x55f4c90e9730 2024-08-20T03:08:17.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.772+0000 7f0d0c52e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0010390 con 0x55f4c90f5fe0 2024-08-20T03:08:17.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.772+0000 7f0d0c52e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf000ee60 con 0x55f4c90ea050 2024-08-20T03:08:17.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.773+0000 7f0d02ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c910e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.773+0000 7f0d02ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c910e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53180/0 (socket says 172.21.15.44:53180) 2024-08-20T03:08:17.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.773+0000 7f0d02ffd640 1 -- 172.21.15.44:0/3670302617 learned_addr learned my addr 172.21.15.44:0/3670302617 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:17.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.773+0000 7f0d03fff640 1 --2- 172.21.15.44:0/3670302617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90f5fe0 0x55f4c910e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.773+0000 7f0d037fe640 1 --2- 172.21.15.44:0/3670302617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90e9730 0x55f4c90f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.774+0000 7f0d037fe640 1 -- 172.21.15.44:0/3670302617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 msgr2=0x55f4c910e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:17.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.774+0000 7f0d037fe640 1 --2- 172.21.15.44:0/3670302617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c910e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.774+0000 7f0d037fe640 1 -- 172.21.15.44:0/3670302617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90f5fe0 msgr2=0x55f4c910e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:17.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.774+0000 7f0d037fe640 1 --2- 172.21.15.44:0/3670302617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90f5fe0 0x55f4c910e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.774+0000 7f0d037fe640 1 -- 172.21.15.44:0/3670302617 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c90e9e80 con 0x55f4c90e9730 2024-08-20T03:08:17.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.774+0000 7f0d037fe640 1 --2- 172.21.15.44:0/3670302617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90e9730 0x55f4c90f2650 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40e1050 tx=0x7f0cf40cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805e43bdd7ebdc7e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:17.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.774+0000 7f0ce97fa640 1 -- 172.21.15.44:0/3670302617 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf40c3020 con 0x55f4c90e9730 2024-08-20T03:08:17.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.775+0000 7f0ce97fa640 1 -- 172.21.15.44:0/3670302617 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf40aa510 con 0x55f4c90e9730 2024-08-20T03:08:17.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.776+0000 7f0d0c52e740 1 -- 172.21.15.44:0/3670302617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90e9730 msgr2=0x55f4c90f2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:17.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.776+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/3670302617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90e9730 0x55f4c90f2650 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40e1050 tx=0x7f0cf40cd040 comp rx=0 tx=0).stop 2024-08-20T03:08:17.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.780+0000 7f0d0c52e740 1 -- 172.21.15.44:0/3670302617 shutdown_connections 2024-08-20T03:08:17.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.780+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/3670302617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90f5fe0 0x55f4c910e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.780+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/3670302617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90ea050 0x55f4c910e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.780+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/3670302617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90e9730 0x55f4c90f2650 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.780+0000 7f0d0c52e740 1 -- 172.21.15.44:0/3670302617 >> 172.21.15.44:0/3670302617 conn(0x55f4c90e7560 msgr2=0x55f4c90e64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:17.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.784+0000 7f0d0c52e740 1 -- 172.21.15.44:0/3670302617 shutdown_connections 2024-08-20T03:08:17.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.786+0000 7f0d0c52e740 1 -- 172.21.15.44:0/3670302617 wait complete. 2024-08-20T03:08:17.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.787+0000 7f0d0c52e740 1 -- start start 2024-08-20T03:08:17.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.787+0000 7f0d0c52e740 1 Processor -- start 2024-08-20T03:08:17.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.787+0000 7f0d0c52e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90e9730 0x55f4c90f6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.787+0000 7f0d0c52e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90ea050 0x55f4c90f6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.787+0000 7f0d0c52e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.787+0000 7f0d0c52e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf000fc00 con 0x55f4c90f5fe0 2024-08-20T03:08:17.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.787+0000 7f0d0c52e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf000f530 con 0x55f4c90ea050 2024-08-20T03:08:17.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.787+0000 7f0d0c52e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf000c770 con 0x55f4c90e9730 2024-08-20T03:08:17.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.788+0000 7f0d0c52e740 10 client.0 _ll_register_callbacks cb 0x7f0cfc009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:17.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.788+0000 7f0d037fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90e9730 0x55f4c90f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.789+0000 7f0d037fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90e9730 0x55f4c90f6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53192/0 (socket says 172.21.15.44:53192) 2024-08-20T03:08:17.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.789+0000 7f0d037fe640 1 -- 172.21.15.44:0/2587165192 learned_addr learned my addr 172.21.15.44:0/2587165192 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:17.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.789+0000 7f0d03fff640 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.789+0000 7f0d02ffd640 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90ea050 0x55f4c90f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.790+0000 7f0d03fff640 1 -- 172.21.15.44:0/2587165192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90e9730 msgr2=0x55f4c90f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.790+0000 7f0d03fff640 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90e9730 0x55f4c90f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.790+0000 7f0d03fff640 1 -- 172.21.15.44:0/2587165192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90ea050 msgr2=0x55f4c90f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.790+0000 7f0d03fff640 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90ea050 0x55f4c90f6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.790+0000 7f0d03fff640 1 -- 172.21.15.44:0/2587165192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c90fd1c0 con 0x55f4c90f5fe0 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.790+0000 7f0d03fff640 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f6e90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc019040 tx=0x7f0cfc005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1be6fe3b47d143 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.790+0000 7f0ce2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.791+0000 7f0ce2ffd640 1 -- 172.21.15.44:0/2587165192 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc041060 con 0x55f4c90f5fe0 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.791+0000 7f0d0c52e740 1 -- 172.21.15.44:0/2587165192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf40aa510 con 0x55f4c90f5fe0 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.791+0000 7f0ce2ffd640 1 -- 172.21.15.44:0/2587165192 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0cfc00f480 con 0x55f4c90f5fe0 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.791+0000 7f0ce2ffd640 1 -- 172.21.15.44:0/2587165192 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc004b20 con 0x55f4c90f5fe0 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.791+0000 7f0d0c52e740 10 client.16683 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:17.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.791+0000 7f0d0c52e740 1 -- 172.21.15.44:0/2587165192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f4c90fe480 con 0x55f4c90f5fe0 2024-08-20T03:08:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.793+0000 7f0ce2ffd640 1 -- 172.21.15.44:0/2587165192 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf40aa510 con 0x55f4c90f5fe0 2024-08-20T03:08:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.793+0000 7f0ce2ffd640 1 -- 172.21.15.44:0/2587165192 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0cfc044020 con 0x55f4c90f5fe0 2024-08-20T03:08:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.793+0000 7f0d0c52e740 10 client.16683 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.793+0000 7f0d0c52e740 10 client.16683 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:08:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.793+0000 7f0d0c52e740 20 client.16683 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.793+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55f4c9171080 0x55f4c9173470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.793+0000 7f0d0c52e740 4 client.16683 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:08:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.793+0000 7f0d0c52e740 1 -- 172.21.15.44:0/2587165192 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4c905e320 con 0x55f4c9171080 2024-08-20T03:08:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.794+0000 7f0d037fe640 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55f4c9171080 0x55f4c9173470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.796+0000 7f0d037fe640 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55f4c9171080 0x55f4c9173470 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52a24f1d71685263 server_cookie=6d401a12e1e0facd in_seq=0 out_seq=0 2024-08-20T03:08:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.796+0000 7f0ce2ffd640 10 client.16683 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:08:17.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.798+0000 7f0ce2ffd640 1 -- 172.21.15.44:0/2587165192 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4c905e320 con 0x55f4c9171080 2024-08-20T03:08:17.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.798+0000 7f0ce2ffd640 10 client.16683 handle_command_reply: tid=0 2024-08-20T03:08:17.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.798+0000 7f0d0c52e740 1 client.16683 shutdown 2024-08-20T03:08:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.799+0000 7f0d0c52e740 1 -- 172.21.15.44:0/2587165192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 msgr2=0x55f4c90f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.799+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f6e90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc019040 tx=0x7f0cfc005b80 comp rx=0 tx=0).stop 2024-08-20T03:08:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.799+0000 7f0d0c52e740 1 -- 172.21.15.44:0/2587165192 shutdown_connections 2024-08-20T03:08:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.799+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x55f4c9171080 0x55f4c9173470 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.799+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f4c90f5fe0 0x55f4c90f6e90 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.799+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4c90ea050 0x55f4c90f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.799+0000 7f0d0c52e740 1 --2- 172.21.15.44:0/2587165192 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f4c90e9730 0x55f4c90f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.799+0000 7f0d0c52e740 1 -- 172.21.15.44:0/2587165192 >> 172.21.15.44:0/2587165192 conn(0x55f4c90e7560 msgr2=0x55f4c90f43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.799+0000 7f0ce2ffd640 0 client.16683 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:08:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.803+0000 7f0d0c52e740 1 -- 172.21.15.44:0/2587165192 shutdown_connections 2024-08-20T03:08:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.803+0000 7f0d0c52e740 1 -- 172.21.15.44:0/2587165192 wait complete. 2024-08-20T03:08:17.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.805+0000 7f0d0c52e740 20 client.16683 trim_cache size 0 max 16384 2024-08-20T03:08:17.810 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:08:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.809+0000 7f0d0a135640 1 -- 172.21.15.44:0/569922856 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0cec0b3f50 msgr2=0x7f0cec0b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.809+0000 7f0d0a135640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0cec0b3f50 0x7f0cec0b6410 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc005fd0 tx=0x7f0cfc009500 comp rx=0 tx=0).stop 2024-08-20T03:08:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.809+0000 7f0d0a135640 1 -- 172.21.15.44:0/569922856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d04101990 msgr2=0x7f0d04115c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.809+0000 7f0d0a135640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d04101990 0x7f0d04115c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0cf400b3e0 tx=0x7f0cf400b7e0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.809+0000 7f0d0a135640 1 -- 172.21.15.44:0/569922856 shutdown_connections 2024-08-20T03:08:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.809+0000 7f0d0a135640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0cec0b3f50 0x7f0cec0b6410 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.809+0000 7f0d0a135640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d04107d20 0x7f0d041125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.809+0000 7f0d0a135640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d041073a0 0x7f0d04111f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.809+0000 7f0d0a135640 1 --2- 172.21.15.44:0/569922856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d04101990 0x7f0d04115c80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.809+0000 7f0d0a135640 1 -- 172.21.15.44:0/569922856 >> 172.21.15.44:0/569922856 conn(0x7f0d040fd400 msgr2=0x7f0d040ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.809+0000 7f0d0a135640 1 -- 172.21.15.44:0/569922856 shutdown_connections 2024-08-20T03:08:17.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:17.809+0000 7f0d0a135640 1 -- 172.21.15.44:0/569922856 wait complete. 2024-08-20T03:08:18.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:18 smithi165 ceph-mon[22357]: pgmap v494: 97 pgs: 97 active+clean; 818 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 389 op/s 2024-08-20T03:08:18.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:18 smithi099 ceph-mon[21984]: pgmap v494: 97 pgs: 97 active+clean; 818 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 389 op/s 2024-08-20T03:08:18.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:18 smithi044 ceph-mon[18790]: pgmap v494: 97 pgs: 97 active+clean; 818 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 389 op/s 2024-08-20T03:08:18.673 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T03:08:19.275 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:08:19.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.839+0000 7fda56d33640 1 -- 172.21.15.44:0/785765563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda50102aa0 msgr2=0x7fda50102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:19.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.839+0000 7fda56d33640 1 --2- 172.21.15.44:0/785765563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda50102aa0 0x7fda50102e80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fda44009900 tx=0x7fda4402a410 comp rx=0 tx=0).stop 2024-08-20T03:08:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.839+0000 7fda56d33640 1 -- 172.21.15.44:0/785765563 shutdown_connections 2024-08-20T03:08:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.839+0000 7fda56d33640 1 --2- 172.21.15.44:0/785765563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda50108e30 0x7fda5010f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.839+0000 7fda56d33640 1 --2- 172.21.15.44:0/785765563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda501084b0 0x7fda501088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.839+0000 7fda56d33640 1 --2- 172.21.15.44:0/785765563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda50102aa0 0x7fda50102e80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.839+0000 7fda56d33640 1 -- 172.21.15.44:0/785765563 >> 172.21.15.44:0/785765563 conn(0x7fda500fe550 msgr2=0x7fda50100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.839+0000 7fda56d33640 1 -- 172.21.15.44:0/785765563 shutdown_connections 2024-08-20T03:08:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.839+0000 7fda56d33640 1 -- 172.21.15.44:0/785765563 wait complete. 2024-08-20T03:08:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda56d33640 1 Processor -- start 2024-08-20T03:08:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda56d33640 1 -- start start 2024-08-20T03:08:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda56d33640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda50102aa0 0x7fda50198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda56d33640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda501084b0 0x7fda501994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda56d33640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda50108e30 0x7fda5019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda56d33640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda50112ab0 con 0x7fda50102aa0 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda56d33640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda50112930 con 0x7fda501084b0 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda56d33640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fda50112c30 con 0x7fda50108e30 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda54acf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda50102aa0 0x7fda50198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda54acf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda50102aa0 0x7fda50198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47008/0 (socket says 172.21.15.44:47008) 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda54acf640 1 -- 172.21.15.44:0/1765021473 learned_addr learned my addr 172.21.15.44:0/1765021473 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda4ffff640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda501084b0 0x7fda501994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.840+0000 7fda552d0640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda50108e30 0x7fda5019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.841+0000 7fda54acf640 1 -- 172.21.15.44:0/1765021473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda50108e30 msgr2=0x7fda5019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.841+0000 7fda54acf640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda50108e30 0x7fda5019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.841+0000 7fda54acf640 1 -- 172.21.15.44:0/1765021473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda501084b0 msgr2=0x7fda501994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.841+0000 7fda54acf640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda501084b0 0x7fda501994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.841+0000 7fda54acf640 1 -- 172.21.15.44:0/1765021473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5019f050 con 0x7fda50102aa0 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.841+0000 7fda4ffff640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda501084b0 0x7fda501994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.841+0000 7fda54acf640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda50102aa0 0x7fda50198fa0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fda44005940 tx=0x7fda440040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.841+0000 7fda4dffb640 1 -- 172.21.15.44:0/1765021473 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4403f030 con 0x7fda50102aa0 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.841+0000 7fda4dffb640 1 -- 172.21.15.44:0/1765021473 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda440044c0 con 0x7fda50102aa0 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.841+0000 7fda56d33640 1 -- 172.21.15.44:0/1765021473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5019f2e0 con 0x7fda50102aa0 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.841+0000 7fda4dffb640 1 -- 172.21.15.44:0/1765021473 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44035070 con 0x7fda50102aa0 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.841+0000 7fda56d33640 1 -- 172.21.15.44:0/1765021473 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda501a5a00 con 0x7fda50102aa0 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda4dffb640 1 -- 172.21.15.44:0/1765021473 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fda44042030 con 0x7fda50102aa0 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda5912c740 1 Processor -- start 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda5912c740 1 -- start start 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda5912c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 0x56487f823c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda5912c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f8241a0 0x56487f82fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda5912c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f830130 0x56487f832520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda5912c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56487f79a930 con 0x56487f8241a0 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda5912c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56487f789e30 con 0x56487f830130 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda5912c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56487f78de70 con 0x56487f823880 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda4dffb640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fda400b3d40 0x7fda400b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda54acf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f8241a0 0x56487f82fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda552d0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 0x56487f823c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda552d0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 0x56487f823c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53204/0 (socket says 172.21.15.44:53204) 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.844+0000 7fda54acf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f8241a0 0x56487f82fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47020/0 (socket says 172.21.15.44:47020) 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda4ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f830130 0x56487f832520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda54acf640 1 -- 172.21.15.44:0/71069680 learned_addr learned my addr 172.21.15.44:0/71069680 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda4dffb640 1 -- 172.21.15.44:0/1765021473 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fda44104310 con 0x7fda50102aa0 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda552d0640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fda400b3d40 0x7fda400b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda54acf640 1 -- 172.21.15.44:0/71069680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 msgr2=0x56487f823c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda54acf640 1 --2- 172.21.15.44:0/71069680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 0x56487f823c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda54acf640 1 -- 172.21.15.44:0/71069680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f830130 msgr2=0x56487f832520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda54acf640 1 --2- 172.21.15.44:0/71069680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f830130 0x56487f832520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda54acf640 1 -- 172.21.15.44:0/71069680 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56487f4e9650 con 0x56487f8241a0 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda4ffff640 1 --2- 172.21.15.44:0/71069680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f830130 0x56487f832520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda54acf640 1 --2- 172.21.15.44:0/71069680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f8241a0 0x56487f82fbf0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fda441019e0 tx=0x7fda44098730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7bedfd529e5041 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda552d0640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fda400b3d40 0x7fda400b6200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fda48005fd0 tx=0x7fda48009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda3affd640 1 -- 172.21.15.44:0/71069680 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44049020 con 0x56487f8241a0 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda3affd640 1 -- 172.21.15.44:0/71069680 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda440cafe0 con 0x56487f8241a0 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.845+0000 7fda3affd640 1 -- 172.21.15.44:0/71069680 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440dd8a0 con 0x56487f8241a0 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.846+0000 7fda5912c740 1 -- 172.21.15.44:0/71069680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f8241a0 msgr2=0x56487f82fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.846+0000 7fda5912c740 1 --2- 172.21.15.44:0/71069680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f8241a0 0x56487f82fbf0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fda441019e0 tx=0x7fda44098730 comp rx=0 tx=0).stop 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.846+0000 7fda5912c740 1 -- 172.21.15.44:0/71069680 shutdown_connections 2024-08-20T03:08:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.846+0000 7fda5912c740 1 --2- 172.21.15.44:0/71069680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f830130 0x56487f832520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.846+0000 7fda5912c740 1 --2- 172.21.15.44:0/71069680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f8241a0 0x56487f82fbf0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.846+0000 7fda5912c740 1 --2- 172.21.15.44:0/71069680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 0x56487f823c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.846+0000 7fda5912c740 1 -- 172.21.15.44:0/71069680 >> 172.21.15.44:0/71069680 conn(0x56487f8216b0 msgr2=0x56487f822850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.846+0000 7fda5912c740 1 -- 172.21.15.44:0/71069680 shutdown_connections 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.846+0000 7fda5912c740 1 -- 172.21.15.44:0/71069680 wait complete. 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.846+0000 7fda5912c740 1 -- start start 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda5912c740 1 Processor -- start 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda5912c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 0x56487f82d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda5912c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f8241a0 0x56487f82d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda5912c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f830130 0x56487f82dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda5912c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56487f5ec700 con 0x56487f830130 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda5912c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56487f710810 con 0x56487f8241a0 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda5912c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56487f832ba0 con 0x56487f823880 2024-08-20T03:08:19.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda5912c740 10 client.0 _ll_register_callbacks cb 0x56487f783700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda552d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f830130 0x56487f82dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda54acf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 0x56487f82d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda4ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f8241a0 0x56487f82d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda552d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f830130 0x56487f82dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47028/0 (socket says 172.21.15.44:47028) 2024-08-20T03:08:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda552d0640 1 -- 172.21.15.44:0/4173607127 learned_addr learned my addr 172.21.15.44:0/4173607127 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda54acf640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 0x56487f82d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53216/0 (socket says 172.21.15.44:53216) 2024-08-20T03:08:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda552d0640 1 -- 172.21.15.44:0/4173607127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 msgr2=0x56487f82d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda552d0640 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 0x56487f82d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda552d0640 1 -- 172.21.15.44:0/4173607127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f8241a0 msgr2=0x56487f82d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda552d0640 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f8241a0 0x56487f82d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda552d0640 1 -- 172.21.15.44:0/4173607127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56487f83be10 con 0x56487f830130 2024-08-20T03:08:19.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda54acf640 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 0x56487f82d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.847+0000 7fda4ffff640 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f8241a0 0x56487f82d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda552d0640 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f830130 0x56487f82dad0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fda4800ff20 tx=0x7fda48004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5555794467c4796e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda2ffff640 1 -- 172.21.15.44:0/4173607127 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4801e410 con 0x56487f830130 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda2ffff640 1 -- 172.21.15.44:0/4173607127 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fda4801e5b0 con 0x56487f830130 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda5912c740 1 -- 172.21.15.44:0/4173607127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56487f739150 con 0x56487f830130 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda2ffff640 1 -- 172.21.15.44:0/4173607127 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda4801e7b0 con 0x56487f830130 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda5912c740 10 client.16695 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda5912c740 1 -- 172.21.15.44:0/4173607127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56487f83e7c0 con 0x56487f830130 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda2ffff640 1 -- 172.21.15.44:0/4173607127 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56487f739150 con 0x56487f830130 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda2ffff640 1 -- 172.21.15.44:0/4173607127 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fda48032030 con 0x56487f830130 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda5912c740 10 client.16695 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda5912c740 10 client.16695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:08:19.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda5912c740 20 client.16695 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda5912c740 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56487f83fdc0 0x56487f8421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda5912c740 4 client.16695 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.848+0000 7fda5912c740 1 -- 172.21.15.44:0/4173607127 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56487f79c880 con 0x56487f83fdc0 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.849+0000 7fda54acf640 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56487f83fdc0 0x56487f8421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.849+0000 7fda54acf640 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56487f83fdc0 0x56487f8421e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a05d50c51b50d9a1 server_cookie=e13c96521c7322e0 in_seq=0 out_seq=0 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.849+0000 7fda2ffff640 10 client.16695 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.850+0000 7fda2ffff640 1 -- 172.21.15.44:0/4173607127 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56487f79c880 con 0x56487f83fdc0 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.850+0000 7fda2ffff640 10 client.16695 handle_command_reply: tid=0 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.850+0000 7fda5912c740 1 client.16695 shutdown 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.851+0000 7fda5912c740 1 -- 172.21.15.44:0/4173607127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f830130 msgr2=0x56487f82dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.851+0000 7fda5912c740 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f830130 0x56487f82dad0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fda4800ff20 tx=0x7fda48004ab0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.851+0000 7fda5912c740 1 -- 172.21.15.44:0/4173607127 shutdown_connections 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.851+0000 7fda5912c740 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56487f83fdc0 0x56487f8421e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.851+0000 7fda5912c740 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f830130 0x56487f82dad0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.851+0000 7fda5912c740 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f8241a0 0x56487f82d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.851+0000 7fda2ffff640 0 client.16695 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.851+0000 7fda5912c740 1 --2- 172.21.15.44:0/4173607127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f823880 0x56487f82d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.851+0000 7fda5912c740 1 -- 172.21.15.44:0/4173607127 >> 172.21.15.44:0/4173607127 conn(0x56487f8216b0 msgr2=0x56487f827250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.851+0000 7fda5912c740 1 -- 172.21.15.44:0/4173607127 shutdown_connections 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.851+0000 7fda5912c740 1 -- 172.21.15.44:0/4173607127 wait complete. 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.851+0000 7fda5912c740 20 client.16695 trim_cache size 0 max 16384 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.864+0000 7fda5912c740 1 Processor -- start 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.864+0000 7fda5912c740 1 -- start start 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.864+0000 7fda5912c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 0x56487f820660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.864+0000 7fda5912c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f8241a0 0x56487f82d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.864+0000 7fda5912c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f830130 0x56487f82da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda5912c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda501ae2b0 con 0x56487f823880 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda5912c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda501afeb0 con 0x56487f8241a0 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda5912c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fda501ae980 con 0x56487f830130 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda54acf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 0x56487f820660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda54acf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 0x56487f820660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47042/0 (socket says 172.21.15.44:47042) 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda54acf640 1 -- 172.21.15.44:0/3937180621 learned_addr learned my addr 172.21.15.44:0/3937180621 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda552d0640 1 --2- 172.21.15.44:0/3937180621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f830130 0x56487f82da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda4ffff640 1 --2- 172.21.15.44:0/3937180621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f8241a0 0x56487f82d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda54acf640 1 -- 172.21.15.44:0/3937180621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f830130 msgr2=0x56487f82da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda54acf640 1 --2- 172.21.15.44:0/3937180621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f830130 0x56487f82da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda54acf640 1 -- 172.21.15.44:0/3937180621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f8241a0 msgr2=0x56487f82d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda54acf640 1 --2- 172.21.15.44:0/3937180621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f8241a0 0x56487f82d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda54acf640 1 -- 172.21.15.44:0/3937180621 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56487f822130 con 0x56487f823880 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda4ffff640 1 --2- 172.21.15.44:0/3937180621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f8241a0 0x56487f82d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda54acf640 1 --2- 172.21.15.44:0/3937180621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 0x56487f820660 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fda440caf70 tx=0x7fda440ca850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4385472b37f2df49 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:19.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda3a7fc640 1 -- 172.21.15.44:0/3937180621 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44049040 con 0x56487f823880 2024-08-20T03:08:19.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda3a7fc640 1 -- 172.21.15.44:0/3937180621 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda440cf2f0 con 0x56487f823880 2024-08-20T03:08:19.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.865+0000 7fda3a7fc640 1 -- 172.21.15.44:0/3937180621 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440cb240 con 0x56487f823880 2024-08-20T03:08:19.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.866+0000 7fda5912c740 1 -- 172.21.15.44:0/3937180621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 msgr2=0x56487f820660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:19.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.866+0000 7fda5912c740 1 --2- 172.21.15.44:0/3937180621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 0x56487f820660 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fda440caf70 tx=0x7fda440ca850 comp rx=0 tx=0).stop 2024-08-20T03:08:19.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.866+0000 7fda5912c740 1 -- 172.21.15.44:0/3937180621 shutdown_connections 2024-08-20T03:08:19.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.866+0000 7fda5912c740 1 --2- 172.21.15.44:0/3937180621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f830130 0x56487f82da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.866+0000 7fda5912c740 1 --2- 172.21.15.44:0/3937180621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f8241a0 0x56487f82d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.866+0000 7fda5912c740 1 --2- 172.21.15.44:0/3937180621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 0x56487f820660 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.866+0000 7fda5912c740 1 -- 172.21.15.44:0/3937180621 >> 172.21.15.44:0/3937180621 conn(0x56487f8216b0 msgr2=0x56487f81ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:19.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.866+0000 7fda5912c740 1 -- 172.21.15.44:0/3937180621 shutdown_connections 2024-08-20T03:08:19.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.866+0000 7fda5912c740 1 -- 172.21.15.44:0/3937180621 wait complete. 2024-08-20T03:08:19.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.866+0000 7fda5912c740 1 -- start start 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda5912c740 1 Processor -- start 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda5912c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 0x56487f833720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda5912c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f8241a0 0x56487f833cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda5912c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f830130 0x56487f8a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda5912c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fda501af720 con 0x56487f823880 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda5912c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda501af050 con 0x56487f830130 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda5912c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fda501ac290 con 0x56487f8241a0 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda5912c740 10 client.0 _ll_register_callbacks cb 0x7fda4800fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda54acf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 0x56487f833720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda54acf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 0x56487f833720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47050/0 (socket says 172.21.15.44:47050) 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda54acf640 1 -- 172.21.15.44:0/342599591 learned_addr learned my addr 172.21.15.44:0/342599591 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda552d0640 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f830130 0x56487f8a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda4ffff640 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f8241a0 0x56487f833cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda54acf640 1 -- 172.21.15.44:0/342599591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f8241a0 msgr2=0x56487f833cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:19.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda54acf640 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f8241a0 0x56487f833cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda54acf640 1 -- 172.21.15.44:0/342599591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f830130 msgr2=0x56487f8a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda54acf640 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f830130 0x56487f8a2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.867+0000 7fda54acf640 1 -- 172.21.15.44:0/342599591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56487f827dd0 con 0x56487f823880 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda54acf640 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 0x56487f833720 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fda44006010 tx=0x7fda440c6f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2492b260a3af9a35 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda2ffff640 1 -- 172.21.15.44:0/342599591 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440af060 con 0x56487f823880 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda2ffff640 1 -- 172.21.15.44:0/342599591 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fda44101cf0 con 0x56487f823880 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda5912c740 1 -- 172.21.15.44:0/342599591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda440cf2f0 con 0x56487f823880 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda2ffff640 1 -- 172.21.15.44:0/342599591 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda44049070 con 0x56487f823880 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda5912c740 10 client.16701 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda5912c740 1 -- 172.21.15.44:0/342599591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56487f826830 con 0x56487f823880 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda2ffff640 1 -- 172.21.15.44:0/342599591 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda440cf2f0 con 0x56487f823880 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda2ffff640 1 -- 172.21.15.44:0/342599591 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fda48011ce0 con 0x56487f823880 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda5912c740 10 client.16701 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:19.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda5912c740 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda5912c740 20 client.16701 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda5912c740 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56487f8aa9b0 0x56487f8acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda5912c740 4 client.16701 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.868+0000 7fda5912c740 1 -- 172.21.15.44:0/342599591 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56487f798320 con 0x56487f8aa9b0 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.869+0000 7fda4ffff640 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56487f8aa9b0 0x56487f8acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.869+0000 7fda4ffff640 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56487f8aa9b0 0x56487f8acdd0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b21581d27bb50fb server_cookie=141b5fb3d1e31c4f in_seq=0 out_seq=0 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.869+0000 7fda2ffff640 10 client.16701 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.870+0000 7fda2ffff640 1 -- 172.21.15.44:0/342599591 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56487f798320 con 0x56487f8aa9b0 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.870+0000 7fda2ffff640 10 client.16701 handle_command_reply: tid=0 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.870+0000 7fda5912c740 1 client.16701 shutdown 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.870+0000 7fda5912c740 1 -- 172.21.15.44:0/342599591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 msgr2=0x56487f833720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.870+0000 7fda5912c740 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 0x56487f833720 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fda44006010 tx=0x7fda440c6f50 comp rx=0 tx=0).stop 2024-08-20T03:08:19.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.870+0000 7fda5912c740 1 -- 172.21.15.44:0/342599591 shutdown_connections 2024-08-20T03:08:19.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.870+0000 7fda5912c740 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56487f8aa9b0 0x56487f8acdd0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.870+0000 7fda5912c740 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56487f830130 0x56487f8a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.870+0000 7fda5912c740 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56487f8241a0 0x56487f833cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.870+0000 7fda2ffff640 0 client.16701 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:19.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.870+0000 7fda5912c740 1 --2- 172.21.15.44:0/342599591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56487f823880 0x56487f833720 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda5912c740 1 -- 172.21.15.44:0/342599591 >> 172.21.15.44:0/342599591 conn(0x56487f8216b0 msgr2=0x56487f83b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:19.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda5912c740 1 -- 172.21.15.44:0/342599591 shutdown_connections 2024-08-20T03:08:19.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda5912c740 1 -- 172.21.15.44:0/342599591 wait complete. 2024-08-20T03:08:19.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda5912c740 20 client.16701 trim_cache size 0 max 16384 2024-08-20T03:08:19.886 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:08:19.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda56d33640 1 -- 172.21.15.44:0/1765021473 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fda400b3d40 msgr2=0x7fda400b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:19.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda56d33640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fda400b3d40 0x7fda400b6200 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fda48005fd0 tx=0x7fda48009500 comp rx=0 tx=0).stop 2024-08-20T03:08:19.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda56d33640 1 -- 172.21.15.44:0/1765021473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda50102aa0 msgr2=0x7fda50198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:19.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda56d33640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda50102aa0 0x7fda50198fa0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fda44005940 tx=0x7fda440040c0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda56d33640 1 -- 172.21.15.44:0/1765021473 shutdown_connections 2024-08-20T03:08:19.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda56d33640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fda400b3d40 0x7fda400b6200 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda56d33640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda50108e30 0x7fda5019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda56d33640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda501084b0 0x7fda501994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda56d33640 1 --2- 172.21.15.44:0/1765021473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda50102aa0 0x7fda50198fa0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:19.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.871+0000 7fda56d33640 1 -- 172.21.15.44:0/1765021473 >> 172.21.15.44:0/1765021473 conn(0x7fda500fe550 msgr2=0x7fda50100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:19.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.872+0000 7fda56d33640 1 -- 172.21.15.44:0/1765021473 shutdown_connections 2024-08-20T03:08:19.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:19.872+0000 7fda56d33640 1 -- 172.21.15.44:0/1765021473 wait complete. 2024-08-20T03:08:20.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:20 smithi165 ceph-mon[22357]: pgmap v495: 97 pgs: 97 active+clean; 839 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.0 MiB/s wr, 890 op/s 2024-08-20T03:08:20.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:20 smithi099 ceph-mon[21984]: pgmap v495: 97 pgs: 97 active+clean; 839 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.0 MiB/s wr, 890 op/s 2024-08-20T03:08:20.477 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:20 smithi044 ceph-mon[18790]: pgmap v495: 97 pgs: 97 active+clean; 839 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.0 MiB/s wr, 890 op/s 2024-08-20T03:08:22.061 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T03:08:22.062 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T03:08:22.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:22 smithi165 ceph-mon[22357]: pgmap v496: 97 pgs: 97 active+clean; 858 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.3 MiB/s wr, 1.19k op/s 2024-08-20T03:08:22.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:22 smithi099 ceph-mon[21984]: pgmap v496: 97 pgs: 97 active+clean; 858 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.3 MiB/s wr, 1.19k op/s 2024-08-20T03:08:22.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:22 smithi044 ceph-mon[18790]: pgmap v496: 97 pgs: 97 active+clean; 858 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.3 MiB/s wr, 1.19k op/s 2024-08-20T03:08:23.347 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:08:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.907+0000 7f04e49fb640 1 -- 172.21.15.44:0/3316560135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e01073a0 msgr2=0x7f04e01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.907+0000 7f04e49fb640 1 --2- 172.21.15.44:0/3316560135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e01073a0 0x7f04e01077e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009860 tx=0x7f04d402a340 comp rx=0 tx=0).stop 2024-08-20T03:08:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.907+0000 7f04e49fb640 1 -- 172.21.15.44:0/3316560135 shutdown_connections 2024-08-20T03:08:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.907+0000 7f04e49fb640 1 --2- 172.21.15.44:0/3316560135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04e0107d20 0x7f04e010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.907+0000 7f04e49fb640 1 --2- 172.21.15.44:0/3316560135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e01073a0 0x7f04e01077e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.907+0000 7f04e49fb640 1 --2- 172.21.15.44:0/3316560135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e0101990 0x7f04e0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.907+0000 7f04e49fb640 1 -- 172.21.15.44:0/3316560135 >> 172.21.15.44:0/3316560135 conn(0x7f04e00fd400 msgr2=0x7f04e00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:23.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.907+0000 7f04e49fb640 1 -- 172.21.15.44:0/3316560135 shutdown_connections 2024-08-20T03:08:23.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.907+0000 7f04e49fb640 1 -- 172.21.15.44:0/3316560135 wait complete. 2024-08-20T03:08:23.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.908+0000 7f04e49fb640 1 Processor -- start 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.908+0000 7f04e49fb640 1 -- start start 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04e49fb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e0101990 0x7f04e0198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04e49fb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e01073a0 0x7f04e01994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04e49fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04e0107d20 0x7f04e019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04e49fb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0112ad0 con 0x7f04e01073a0 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04e49fb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0112950 con 0x7f04e0107d20 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04e49fb640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0112c50 con 0x7f04e0101990 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04de59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e0101990 0x7f04e0198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04de59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e0101990 0x7f04e0198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53258/0 (socket says 172.21.15.44:53258) 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04de59c640 1 -- 172.21.15.44:0/684218425 learned_addr learned my addr 172.21.15.44:0/684218425 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e01073a0 0x7f04e01994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04ded9d640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04e0107d20 0x7f04e019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04de59c640 1 -- 172.21.15.44:0/684218425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04e0107d20 msgr2=0x7f04e019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04de59c640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04e0107d20 0x7f04e019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04de59c640 1 -- 172.21.15.44:0/684218425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e01073a0 msgr2=0x7f04e01994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.909+0000 7f04de59c640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e01073a0 0x7f04e01994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.910+0000 7f04de59c640 1 -- 172.21.15.44:0/684218425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e019f030 con 0x7f04e0101990 2024-08-20T03:08:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.910+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e01073a0 0x7f04e01994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.910+0000 7f04de59c640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e0101990 0x7f04e0198f80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f04d000e9c0 tx=0x7f04d000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.910+0000 7f04cf7fe640 1 -- 172.21.15.44:0/684218425 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d000cde0 con 0x7f04e0101990 2024-08-20T03:08:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.910+0000 7f04cf7fe640 1 -- 172.21.15.44:0/684218425 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d00035f0 con 0x7f04e0101990 2024-08-20T03:08:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.910+0000 7f04e49fb640 1 -- 172.21.15.44:0/684218425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e019f320 con 0x7f04e0101990 2024-08-20T03:08:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.910+0000 7f04cf7fe640 1 -- 172.21.15.44:0/684218425 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0010690 con 0x7f04e0101990 2024-08-20T03:08:23.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.910+0000 7f04e49fb640 1 -- 172.21.15.44:0/684218425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e01a59e0 con 0x7f04e0101990 2024-08-20T03:08:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.914+0000 7f04cf7fe640 1 -- 172.21.15.44:0/684218425 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f04d0004000 con 0x7f04e0101990 2024-08-20T03:08:23.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.914+0000 7f04e6df4740 1 Processor -- start 2024-08-20T03:08:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.916+0000 7f04e6df4740 1 -- start start 2024-08-20T03:08:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.916+0000 7f04e6df4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 0x559be7039790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.916+0000 7f04e6df4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be7039cd0 0x559be7049b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.916+0000 7f04e6df4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be704a070 0x559be704c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.916+0000 7f04e6df4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559be6faf940 con 0x559be7039cd0 2024-08-20T03:08:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.916+0000 7f04e6df4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559be6f9ee40 con 0x559be704a070 2024-08-20T03:08:23.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.916+0000 7f04e6df4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559be6fa2e80 con 0x559be70393b0 2024-08-20T03:08:23.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.916+0000 7f04ded9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 0x559be7039790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.916+0000 7f04ded9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 0x559be7039790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53266/0 (socket says 172.21.15.44:53266) 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.916+0000 7f04ded9d640 1 -- 172.21.15.44:0/3193624895 learned_addr learned my addr 172.21.15.44:0/3193624895 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.917+0000 7f04de59c640 1 --2- 172.21.15.44:0/3193624895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be7039cd0 0x559be7049b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.917+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/3193624895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be704a070 0x559be704c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.917+0000 7f04ded9d640 1 -- 172.21.15.44:0/3193624895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be704a070 msgr2=0x559be704c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.917+0000 7f04ded9d640 1 --2- 172.21.15.44:0/3193624895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be704a070 0x559be704c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.917+0000 7f04ded9d640 1 -- 172.21.15.44:0/3193624895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be7039cd0 msgr2=0x559be7049b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.917+0000 7f04ded9d640 1 --2- 172.21.15.44:0/3193624895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be7039cd0 0x559be7049b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.917+0000 7f04ded9d640 1 -- 172.21.15.44:0/3193624895 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be6f3b520 con 0x559be70393b0 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.917+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/3193624895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be704a070 0x559be704c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.917+0000 7f04ded9d640 1 --2- 172.21.15.44:0/3193624895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 0x559be7039790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f04d8011440 tx=0x7f04d8011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b22164949421d455 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.917+0000 7f04ccff9640 1 -- 172.21.15.44:0/3193624895 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8004d20 con 0x559be70393b0 2024-08-20T03:08:23.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.917+0000 7f04ccff9640 1 -- 172.21.15.44:0/3193624895 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d8004050 con 0x559be70393b0 2024-08-20T03:08:23.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.917+0000 7f04ccff9640 1 -- 172.21.15.44:0/3193624895 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d800f7a0 con 0x559be70393b0 2024-08-20T03:08:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.918+0000 7f04e6df4740 1 -- 172.21.15.44:0/3193624895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 msgr2=0x559be7039790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.918+0000 7f04e6df4740 1 --2- 172.21.15.44:0/3193624895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 0x559be7039790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f04d8011440 tx=0x7f04d8011900 comp rx=0 tx=0).stop 2024-08-20T03:08:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.918+0000 7f04e6df4740 1 -- 172.21.15.44:0/3193624895 shutdown_connections 2024-08-20T03:08:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.918+0000 7f04cf7fe640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f04c80b3d40 0x7f04c80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.918+0000 7f04e6df4740 1 --2- 172.21.15.44:0/3193624895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be704a070 0x559be704c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.918+0000 7f04e6df4740 1 --2- 172.21.15.44:0/3193624895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be7039cd0 0x559be7049b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.918+0000 7f04e6df4740 1 --2- 172.21.15.44:0/3193624895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 0x559be7039790 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.918+0000 7f04e6df4740 1 -- 172.21.15.44:0/3193624895 >> 172.21.15.44:0/3193624895 conn(0x559be70371e0 msgr2=0x559be7038380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.919+0000 7f04ded9d640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f04c80b3d40 0x7f04c80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.919+0000 7f04cf7fe640 1 -- 172.21.15.44:0/684218425 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f04d00e4290 con 0x7f04e0101990 2024-08-20T03:08:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.919+0000 7f04ded9d640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f04c80b3d40 0x7f04c80b6200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f04d80048f0 tx=0x7f04d8000f40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.920+0000 7f04e6df4740 1 -- 172.21.15.44:0/3193624895 shutdown_connections 2024-08-20T03:08:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.920+0000 7f04e6df4740 1 -- 172.21.15.44:0/3193624895 wait complete. 2024-08-20T03:08:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.920+0000 7f04e6df4740 1 -- start start 2024-08-20T03:08:23.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.920+0000 7f04e6df4740 1 Processor -- start 2024-08-20T03:08:23.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.920+0000 7f04e6df4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 0x559be7047090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.920+0000 7f04e6df4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be7039cd0 0x559be70475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04e6df4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be704a070 0x559be7047b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04e6df4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559be6e01890 con 0x559be7039cd0 2024-08-20T03:08:23.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04e6df4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559be6f25700 con 0x559be704a070 2024-08-20T03:08:23.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04e6df4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x559be704cdc0 con 0x559be70393b0 2024-08-20T03:08:23.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04de59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 0x559be7047090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04de59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 0x559be7047090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53276/0 (socket says 172.21.15.44:53276) 2024-08-20T03:08:23.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04de59c640 1 -- 172.21.15.44:0/3579573944 learned_addr learned my addr 172.21.15.44:0/3579573944 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:23.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be7039cd0 0x559be70475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04ded9d640 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be704a070 0x559be7047b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04de59c640 1 -- 172.21.15.44:0/3579573944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be704a070 msgr2=0x559be7047b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04de59c640 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be704a070 0x559be7047b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04de59c640 1 -- 172.21.15.44:0/3579573944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be7039cd0 msgr2=0x559be70475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04de59c640 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be7039cd0 0x559be70475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04e6df4740 10 client.0 _ll_register_callbacks cb 0x559be6f9a6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04de59c640 1 -- 172.21.15.44:0/3579573944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be6cfe930 con 0x559be70393b0 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.921+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be7039cd0 0x559be70475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.922+0000 7f04de59c640 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 0x559be7047090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f04d00aac10 tx=0x7f04d00aada0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30782374fcdfd3d6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.922+0000 7f04e6df4740 1 -- 172.21.15.44:0/3579573944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d8004050 con 0x559be70393b0 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.922+0000 7f04c5ffb640 10 client.26134 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.922+0000 7f04c5ffb640 1 -- 172.21.15.44:0/3579573944 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00aaa20 con 0x559be70393b0 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.922+0000 7f04c5ffb640 1 -- 172.21.15.44:0/3579573944 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f04d00b3b30 con 0x559be70393b0 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.922+0000 7f04c5ffb640 1 -- 172.21.15.44:0/3579573944 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04d00b3d50 con 0x559be70393b0 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.922+0000 7f04e6df4740 10 client.26134 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:23.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.922+0000 7f04e6df4740 1 -- 172.21.15.44:0/3579573944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({fsmap=30}) -- 0x559be7040d90 con 0x559be70393b0 2024-08-20T03:08:23.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.922+0000 7f04c5ffb640 1 -- 172.21.15.44:0/3579573944 <== mon.1 v2:172.21.15.165:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f04d00aa170 con 0x559be70393b0 2024-08-20T03:08:23.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.923+0000 7f04e6df4740 10 client.26134 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:23.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.923+0000 7f04e6df4740 10 client.26134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:08:23.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.923+0000 7f04e6df4740 20 client.26134 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:23.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.923+0000 7f04e6df4740 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559be70426e0 0x559be7042b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.923+0000 7f04e6df4740 4 client.26134 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:08:23.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.923+0000 7f04e6df4740 1 -- 172.21.15.44:0/3579573944 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559be6fb1890 con 0x559be70426e0 2024-08-20T03:08:23.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.923+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559be70426e0 0x559be7042b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.924+0000 7f04c5ffb640 10 client.26134 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:23.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.924+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559be70426e0 0x559be7042b90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=865ebcecd211f73c server_cookie=d8ba480af8b9e2fa in_seq=0 out_seq=0 2024-08-20T03:08:23.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.930+0000 7f04c5ffb640 1 -- 172.21.15.44:0/3579573944 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559be6fb1890 con 0x559be70426e0 2024-08-20T03:08:23.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.930+0000 7f04c5ffb640 10 client.26134 handle_command_reply: tid=0 2024-08-20T03:08:23.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.930+0000 7f04e6df4740 1 client.26134 shutdown 2024-08-20T03:08:23.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.930+0000 7f04e6df4740 1 -- 172.21.15.44:0/3579573944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 msgr2=0x559be7047090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:23.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.930+0000 7f04e6df4740 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 0x559be7047090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f04d00aac10 tx=0x7f04d00aada0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.931+0000 7f04e6df4740 1 -- 172.21.15.44:0/3579573944 shutdown_connections 2024-08-20T03:08:23.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.931+0000 7f04e6df4740 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559be70426e0 0x559be7042b90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.931+0000 7f04e6df4740 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be704a070 0x559be7047b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.931+0000 7f04e6df4740 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be7039cd0 0x559be70475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.931+0000 7f04c5ffb640 0 client.26134 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:23.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.931+0000 7f04e6df4740 1 --2- 172.21.15.44:0/3579573944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be70393b0 0x559be7047090 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.931+0000 7f04e6df4740 1 -- 172.21.15.44:0/3579573944 >> 172.21.15.44:0/3579573944 conn(0x559be70371e0 msgr2=0x559be7036300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:23.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.931+0000 7f04e6df4740 1 -- 172.21.15.44:0/3579573944 shutdown_connections 2024-08-20T03:08:23.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.931+0000 7f04e6df4740 1 -- 172.21.15.44:0/3579573944 wait complete. 2024-08-20T03:08:23.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.931+0000 7f04e6df4740 20 client.26134 trim_cache size 0 max 16384 2024-08-20T03:08:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.969+0000 7f04e6df4740 1 Processor -- start 2024-08-20T03:08:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.969+0000 7f04e6df4740 1 -- start start 2024-08-20T03:08:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.969+0000 7f04e6df4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 0x559be7044440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.969+0000 7f04e6df4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 0x559be7044980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.969+0000 7f04e6df4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be70426e0 0x559be7055050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.969+0000 7f04e6df4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01ac100 con 0x559be70393b0 2024-08-20T03:08:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.969+0000 7f04e6df4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01add00 con 0x559be70426e0 2024-08-20T03:08:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.969+0000 7f04e6df4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01ac7d0 con 0x559be7039cd0 2024-08-20T03:08:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.970+0000 7f04de59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 0x559be7044440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.970+0000 7f04de59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 0x559be7044440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:47082/0 (socket says 172.21.15.44:47082) 2024-08-20T03:08:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.970+0000 7f04de59c640 1 -- 172.21.15.44:0/4172530655 learned_addr learned my addr 172.21.15.44:0/4172530655 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:23.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.970+0000 7f04ddd9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 0x559be7044980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.970+0000 7f04ded9d640 1 --2- 172.21.15.44:0/4172530655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be70426e0 0x559be7055050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.970+0000 7f04ddd9b640 1 -- 172.21.15.44:0/4172530655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be70426e0 msgr2=0x559be7055050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.970+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/4172530655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be70426e0 0x559be7055050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.970+0000 7f04ddd9b640 1 -- 172.21.15.44:0/4172530655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 msgr2=0x559be7044440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.970+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/4172530655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 0x559be7044440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.970+0000 7f04ddd9b640 1 -- 172.21.15.44:0/4172530655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559be7032b10 con 0x559be7039cd0 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.970+0000 7f04de59c640 1 --2- 172.21.15.44:0/4172530655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 0x559be7044440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.970+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/4172530655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 0x559be7044980 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f04d4035040 tx=0x7f04d4030340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=248bfd0c32f4f241 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.971+0000 7f04c7fff640 1 -- 172.21.15.44:0/4172530655 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d404d020 con 0x559be7039cd0 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.971+0000 7f04c7fff640 1 -- 172.21.15.44:0/4172530655 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d4041dc0 con 0x559be7039cd0 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.971+0000 7f04c7fff640 1 -- 172.21.15.44:0/4172530655 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4037040 con 0x559be7039cd0 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.972+0000 7f04e6df4740 1 -- 172.21.15.44:0/4172530655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 msgr2=0x559be7044980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.972+0000 7f04e6df4740 1 --2- 172.21.15.44:0/4172530655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 0x559be7044980 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f04d4035040 tx=0x7f04d4030340 comp rx=0 tx=0).stop 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.972+0000 7f04e6df4740 1 -- 172.21.15.44:0/4172530655 shutdown_connections 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.972+0000 7f04e6df4740 1 --2- 172.21.15.44:0/4172530655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be70426e0 0x559be7055050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.972+0000 7f04e6df4740 1 --2- 172.21.15.44:0/4172530655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 0x559be7044980 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.972+0000 7f04e6df4740 1 --2- 172.21.15.44:0/4172530655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 0x559be7044440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.972+0000 7f04e6df4740 1 -- 172.21.15.44:0/4172530655 >> 172.21.15.44:0/4172530655 conn(0x559be70371e0 msgr2=0x559be7048ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:23.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.972+0000 7f04e6df4740 1 -- 172.21.15.44:0/4172530655 shutdown_connections 2024-08-20T03:08:23.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.972+0000 7f04e6df4740 1 -- 172.21.15.44:0/4172530655 wait complete. 2024-08-20T03:08:23.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.972+0000 7f04e6df4740 1 -- start start 2024-08-20T03:08:23.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.972+0000 7f04e6df4740 1 Processor -- start 2024-08-20T03:08:23.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04e6df4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 0x559be7059960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04e6df4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 0x559be7059ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04e6df4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be70426e0 0x559be70bf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04e6df4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01ad570 con 0x559be70393b0 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04e6df4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01acea0 con 0x559be70426e0 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04e6df4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f04e01aa0e0 con 0x559be7039cd0 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04e6df4740 10 client.0 _ll_register_callbacks cb 0x7f04d00a6dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04ddd9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 0x559be7059ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04ddd9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 0x559be7059ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:53304/0 (socket says 172.21.15.44:53304) 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04ddd9b640 1 -- 172.21.15.44:0/1922181377 learned_addr learned my addr 172.21.15.44:0/1922181377 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04de59c640 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 0x559be7059960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04ded9d640 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be70426e0 0x559be70bf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04ddd9b640 1 -- 172.21.15.44:0/1922181377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be70426e0 msgr2=0x559be70bf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be70426e0 0x559be70bf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04ddd9b640 1 -- 172.21.15.44:0/1922181377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 msgr2=0x559be7059960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:23.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 0x559be7059960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04ddd9b640 1 -- 172.21.15.44:0/1922181377 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559be70bf7d0 con 0x559be7039cd0 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.973+0000 7f04de59c640 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 0x559be7059960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.974+0000 7f04ddd9b640 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 0x559be7059ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005b60 tx=0x7f04d4041cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f78c4aba58ced978 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.974+0000 7f04c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.974+0000 7f04c5ffb640 1 -- 172.21.15.44:0/1922181377 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d404d030 con 0x559be7039cd0 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.974+0000 7f04c5ffb640 1 -- 172.21.15.44:0/1922181377 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f04d4004400 con 0x559be7039cd0 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.974+0000 7f04e6df4740 1 -- 172.21.15.44:0/1922181377 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d4041dc0 con 0x559be7039cd0 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.974+0000 7f04c5ffb640 1 -- 172.21.15.44:0/1922181377 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04d4061050 con 0x559be7039cd0 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.975+0000 7f04e6df4740 10 client.26140 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.975+0000 7f04e6df4740 1 -- 172.21.15.44:0/1922181377 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559be70c0af0 con 0x559be7039cd0 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.975+0000 7f04c5ffb640 1 -- 172.21.15.44:0/1922181377 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d4041dc0 con 0x559be7039cd0 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.975+0000 7f04c5ffb640 1 -- 172.21.15.44:0/1922181377 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f04d40035f0 con 0x559be7039cd0 2024-08-20T03:08:23.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.975+0000 7f04e6df4740 10 client.26140 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:23.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.975+0000 7f04e6df4740 10 client.26140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:08:23.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.975+0000 7f04e6df4740 20 client.26140 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:23.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.975+0000 7f04e6df4740 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559be704a070 0x559be70c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:23.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.975+0000 7f04e6df4740 4 client.26140 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T03:08:23.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.975+0000 7f04e6df4740 1 -- 172.21.15.44:0/1922181377 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559be6fad330 con 0x559be704a070 2024-08-20T03:08:23.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.975+0000 7f04de59c640 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559be704a070 0x559be70c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:23.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.976+0000 7f04de59c640 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559be704a070 0x559be70c3270 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de7c79f202660e0 server_cookie=45848a976b3ce433 in_seq=0 out_seq=0 2024-08-20T03:08:23.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.976+0000 7f04c5ffb640 10 client.26140 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:23.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.991+0000 7f04c5ffb640 1 -- 172.21.15.44:0/1922181377 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559be6fad330 con 0x559be704a070 2024-08-20T03:08:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.991+0000 7f04c5ffb640 10 client.26140 handle_command_reply: tid=0 2024-08-20T03:08:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.991+0000 7f04e6df4740 1 client.26140 shutdown 2024-08-20T03:08:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.992+0000 7f04e6df4740 1 -- 172.21.15.44:0/1922181377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 msgr2=0x559be7059ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:23.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.992+0000 7f04e6df4740 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 0x559be7059ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005b60 tx=0x7f04d4041cd0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.991+0000 7fe39d7fa640 1 -- 172.21.15.44:0/2005122847 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe3b00a64b0 con 0x55fc08293150 2024-08-20T03:08:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.992+0000 7fe39d7fa640 10 client.26102 handle_command_reply: tid=0 2024-08-20T03:08:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.992+0000 7fe3bee5a740 1 client.26102 shutdown 2024-08-20T03:08:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.993+0000 7fe3bee5a740 1 -- 172.21.15.44:0/2005122847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 msgr2=0x55fc082209a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.993+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc082209a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55fc0822ac70 tx=0x7fe3a40046c0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.995+0000 7fe3bee5a740 1 -- 172.21.15.44:0/2005122847 shutdown_connections 2024-08-20T03:08:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.995+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55fc08293150 0x55fc08295570 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.995+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fc082185f0 0x55fc0821c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.995+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc0820c660 0x55fc08220f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.995+0000 7fe3bee5a740 1 --2- 172.21.15.44:0/2005122847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fc0820bd40 0x55fc082209a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.995+0000 7fe3bee5a740 1 -- 172.21.15.44:0/2005122847 >> 172.21.15.44:0/2005122847 conn(0x55fc08209b70 msgr2=0x55fc082168a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.996+0000 7fe39d7fa640 0 client.26102 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.996+0000 7fe3bee5a740 1 -- 172.21.15.44:0/2005122847 shutdown_connections 2024-08-20T03:08:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.996+0000 7fe3bee5a740 1 -- 172.21.15.44:0/2005122847 wait complete. 2024-08-20T03:08:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.996+0000 7fe3bee5a740 20 client.26102 trim_cache size 0 max 16384 2024-08-20T03:08:24.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.996+0000 7f04e6df4740 1 -- 172.21.15.44:0/1922181377 shutdown_connections 2024-08-20T03:08:24.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.996+0000 7f04e6df4740 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x559be704a070 0x559be70c3270 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.996+0000 7f04e6df4740 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559be70426e0 0x559be70bf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.996+0000 7f04e6df4740 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559be7039cd0 0x559be7059ea0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.996+0000 7f04e6df4740 1 --2- 172.21.15.44:0/1922181377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559be70393b0 0x559be7059960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.996+0000 7f04e6df4740 1 -- 172.21.15.44:0/1922181377 >> 172.21.15.44:0/1922181377 conn(0x559be70371e0 msgr2=0x559be703c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:08:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:08:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:08:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:08:24.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.997+0000 7f04c5ffb640 0 client.26140 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:24.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.997+0000 7f04e6df4740 1 -- 172.21.15.44:0/1922181377 shutdown_connections 2024-08-20T03:08:24.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.997+0000 7f04e6df4740 1 -- 172.21.15.44:0/1922181377 wait complete. 2024-08-20T03:08:24.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.997+0000 7f04e6df4740 20 client.26140 trim_cache size 0 max 16384 2024-08-20T03:08:24.002 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:08:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-08-20T03:08:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "e9d17694-8eff-44c0-a730-895f2597ab32", 2024-08-20T03:08:24.002 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-08-20T03:08:24.002 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:08:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.997+0000 7fe3bad2d640 1 -- 172.21.15.44:0/2500890283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe3a00b3f50 msgr2=0x7fe3a00b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.997+0000 7fe3bad2d640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe3a00b3f50 0x7fe3a00b6410 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55fc0820eec0 tx=0x7fe3a400b390 comp rx=0 tx=0).stop 2024-08-20T03:08:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.997+0000 7fe3bad2d640 1 -- 172.21.15.44:0/2500890283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b41ad1c0 msgr2=0x7fe3b41ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.997+0000 7fe3bad2d640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b41ad1c0 0x7fe3b41ad670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000d9e0 tx=0x7fe3b000dea0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.997+0000 7fe3bad2d640 1 -- 172.21.15.44:0/2500890283 shutdown_connections 2024-08-20T03:08:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.997+0000 7fe3bad2d640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe3a00b3f50 0x7fe3a00b6410 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.998+0000 7fe3bad2d640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b41ad1c0 0x7fe3b41ad670 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.998+0000 7fe3bad2d640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3b4115380 0x7fe3b41a7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.998+0000 7fe3bad2d640 1 --2- 172.21.15.44:0/2500890283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3b4074ca0 0x7fe3b41a78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.998+0000 7fe3bad2d640 1 -- 172.21.15.44:0/2500890283 >> 172.21.15.44:0/2500890283 conn(0x7fe3b406d8b0 msgr2=0x7fe3b40739c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.998+0000 7fe3bad2d640 1 -- 172.21.15.44:0/2500890283 shutdown_connections 2024-08-20T03:08:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:23.998+0000 7fe3bad2d640 1 -- 172.21.15.44:0/2500890283 wait complete. 2024-08-20T03:08:24.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.999+0000 7f04e49fb640 1 -- 172.21.15.44:0/684218425 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f04c80b3d40 msgr2=0x7f04c80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:24.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.999+0000 7f04e49fb640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f04c80b3d40 0x7f04c80b6200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f04d80048f0 tx=0x7f04d8000f40 comp rx=0 tx=0).stop 2024-08-20T03:08:24.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.999+0000 7f04e49fb640 1 -- 172.21.15.44:0/684218425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e0101990 msgr2=0x7f04e0198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:24.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.999+0000 7f04e49fb640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e0101990 0x7f04e0198f80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f04d000e9c0 tx=0x7f04d000ee80 comp rx=0 tx=0).stop 2024-08-20T03:08:24.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.999+0000 7f04e49fb640 1 -- 172.21.15.44:0/684218425 shutdown_connections 2024-08-20T03:08:24.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.999+0000 7f04e49fb640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f04c80b3d40 0x7f04c80b6200 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.999+0000 7f04e49fb640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04e0107d20 0x7f04e019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.999+0000 7f04e49fb640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04e01073a0 0x7f04e01994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.999+0000 7f04e49fb640 1 --2- 172.21.15.44:0/684218425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04e0101990 0x7f04e0198f80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:24.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:23.999+0000 7f04e49fb640 1 -- 172.21.15.44:0/684218425 >> 172.21.15.44:0/684218425 conn(0x7f04e00fd400 msgr2=0x7f04e00ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:24.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:24.000+0000 7f04e49fb640 1 -- 172.21.15.44:0/684218425 shutdown_connections 2024-08-20T03:08:24.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:24.000+0000 7f04e49fb640 1 -- 172.21.15.44:0/684218425 wait complete. 2024-08-20T03:08:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:24 smithi165 ceph-mon[22357]: pgmap v497: 97 pgs: 97 active+clean; 868 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 1.30k op/s 2024-08-20T03:08:24.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:24 smithi099 ceph-mon[21984]: pgmap v497: 97 pgs: 97 active+clean; 868 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 1.30k op/s 2024-08-20T03:08:24.485 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:24 smithi044 ceph-mon[18790]: pgmap v497: 97 pgs: 97 active+clean; 868 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 1.30k op/s 2024-08-20T03:08:25.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub queued for path: / 2024-08-20T03:08:25.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:25.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:25.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:25.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:25.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:25.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.256 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:25.257 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.257 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:25.257 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.257 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:25.257 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:25 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.258 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:08:25.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub queued for path: / 2024-08-20T03:08:25.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:25.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:25.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:25.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:25 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub queued for path: / 2024-08-20T03:08:25.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:25.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:25.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:25.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:25 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: pgmap v498: 97 pgs: 97 active+clean; 898 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.0 MiB/s wr, 1.58k op/s 2024-08-20T03:08:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:26.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:26 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:26.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: pgmap v498: 97 pgs: 97 active+clean; 898 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.0 MiB/s wr, 1.58k op/s 2024-08-20T03:08:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:26.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:26.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:26.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:26.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.416 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:26.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:26.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:26.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.417 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:26.418 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:26 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:26.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:26.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: pgmap v498: 97 pgs: 97 active+clean; 898 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 9.0 MiB/s wr, 1.58k op/s 2024-08-20T03:08:26.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:26.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:26.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:26.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:26.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:26.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:26.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:26.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:26.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:26.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:26 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:27.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:27 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:08:27.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:27 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:27.480 INFO:tasks.workunit.client.0.smithi044.stdout: 210 1476378 287 1016337 292 886686 1099 2024-08-20T03:08:27.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:27 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:08:27.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:27 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:27.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:27 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:08:27.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:27 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:27.654 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:08:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.157+0000 7f5185aa7640 1 -- 172.21.15.44:0/672865180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5180107d60 msgr2=0x7f518010e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.157+0000 7f5185aa7640 1 --2- 172.21.15.44:0/672865180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5180107d60 0x7f518010e560 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5178009ca0 tx=0x7f517802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.157+0000 7f5185aa7640 1 -- 172.21.15.44:0/672865180 shutdown_connections 2024-08-20T03:08:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.157+0000 7f5185aa7640 1 --2- 172.21.15.44:0/672865180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5180107d60 0x7f518010e560 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.157+0000 7f5185aa7640 1 --2- 172.21.15.44:0/672865180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51801073e0 0x7f5180107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.157+0000 7f5185aa7640 1 --2- 172.21.15.44:0/672865180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51801019d0 0x7f5180101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.157+0000 7f5185aa7640 1 -- 172.21.15.44:0/672865180 >> 172.21.15.44:0/672865180 conn(0x7f51800fd440 msgr2=0x7f51800ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.157+0000 7f5185aa7640 1 -- 172.21.15.44:0/672865180 shutdown_connections 2024-08-20T03:08:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.157+0000 7f5185aa7640 1 -- 172.21.15.44:0/672865180 wait complete. 2024-08-20T03:08:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.158+0000 7f5185aa7640 1 Processor -- start 2024-08-20T03:08:28.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.158+0000 7f5185aa7640 1 -- start start 2024-08-20T03:08:28.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.158+0000 7f5185aa7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51801019d0 0x7f518019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.158+0000 7f5185aa7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51801073e0 0x7f518019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.158+0000 7f5185aa7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5180107d60 0x7f51801a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.158+0000 7f5185aa7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f51801119e0 con 0x7f51801073e0 2024-08-20T03:08:28.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.158+0000 7f5185aa7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5180111860 con 0x7f5180107d60 2024-08-20T03:08:28.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.158+0000 7f5185aa7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5180111b60 con 0x7f51801019d0 2024-08-20T03:08:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.158+0000 7f5176ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51801073e0 0x7f518019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.158+0000 7f517f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51801019d0 0x7f518019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.158+0000 7f517f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51801019d0 0x7f518019b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:38848/0 (socket says 172.21.15.44:38848) 2024-08-20T03:08:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.158+0000 7f517f7fe640 1 -- 172.21.15.44:0/1227642978 learned_addr learned my addr 172.21.15.44:0/1227642978 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f517ffff640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5180107d60 0x7f51801a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f517f7fe640 1 -- 172.21.15.44:0/1227642978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5180107d60 msgr2=0x7f51801a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f517f7fe640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5180107d60 0x7f51801a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f517f7fe640 1 -- 172.21.15.44:0/1227642978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51801073e0 msgr2=0x7f518019b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f517f7fe640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51801073e0 0x7f518019b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f517f7fe640 1 -- 172.21.15.44:0/1227642978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51801a1220 con 0x7f51801019d0 2024-08-20T03:08:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f5176ffd640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51801073e0 0x7f518019b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:28.163 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f517f7fe640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51801019d0 0x7f518019b170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f517000e9f0 tx=0x7f517000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:28.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f517d7fa640 1 -- 172.21.15.44:0/1227642978 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517000cd80 con 0x7f51801019d0 2024-08-20T03:08:28.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f5185aa7640 1 -- 172.21.15.44:0/1227642978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51801a1510 con 0x7f51801019d0 2024-08-20T03:08:28.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f517d7fa640 1 -- 172.21.15.44:0/1227642978 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51700108e0 con 0x7f51801019d0 2024-08-20T03:08:28.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f517d7fa640 1 -- 172.21.15.44:0/1227642978 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170018980 con 0x7f51801019d0 2024-08-20T03:08:28.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.159+0000 7f5185aa7640 1 -- 172.21.15.44:0/1227642978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51801a7da0 con 0x7f51801019d0 2024-08-20T03:08:28.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.162+0000 7f5187ea0740 1 Processor -- start 2024-08-20T03:08:28.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.162+0000 7f5187ea0740 1 -- start start 2024-08-20T03:08:28.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.162+0000 7f5187ea0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad118d830 0x55bad118dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.169 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.162+0000 7f5187ea0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad1199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.162+0000 7f5187ea0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad119a0e0 0x55bad119c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.162+0000 7f5187ea0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bad11048e0 con 0x55bad118d830 2024-08-20T03:08:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.162+0000 7f5187ea0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bad10f3de0 con 0x55bad119a0e0 2024-08-20T03:08:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.162+0000 7f5187ea0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bad10f7e20 con 0x55bad118e150 2024-08-20T03:08:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f517ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad118d830 0x55bad118dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f517d7fa640 1 -- 172.21.15.44:0/1227642978 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5170010430 con 0x7f51801019d0 2024-08-20T03:08:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f517ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad118d830 0x55bad118dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46464/0 (socket says 172.21.15.44:46464) 2024-08-20T03:08:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f517ffff640 1 -- 172.21.15.44:0/1231792859 learned_addr learned my addr 172.21.15.44:0/1231792859 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f5176ffd640 1 --2- 172.21.15.44:0/1231792859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad119a0e0 0x55bad119c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f517f7fe640 1 --2- 172.21.15.44:0/1231792859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad1199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f517ffff640 1 -- 172.21.15.44:0/1231792859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 msgr2=0x55bad1199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:28.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f517ffff640 1 --2- 172.21.15.44:0/1231792859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad1199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f517ffff640 1 -- 172.21.15.44:0/1231792859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad119a0e0 msgr2=0x55bad119c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f517ffff640 1 --2- 172.21.15.44:0/1231792859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad119a0e0 0x55bad119c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f517ffff640 1 -- 172.21.15.44:0/1231792859 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad0e53650 con 0x55bad118d830 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f5176ffd640 1 --2- 172.21.15.44:0/1231792859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad119a0e0 0x55bad119c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f517ffff640 1 --2- 172.21.15.44:0/1231792859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad118d830 0x55bad118dc10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5178005ef0 tx=0x7f5178005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6bc5d35164c745 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.163+0000 7f5175ffb640 1 -- 172.21.15.44:0/1231792859 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178031cf0 con 0x55bad118d830 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f5175ffb640 1 -- 172.21.15.44:0/1231792859 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5178036070 con 0x55bad118d830 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f5175ffb640 1 -- 172.21.15.44:0/1231792859 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51780042a0 con 0x55bad118d830 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f517d7fa640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f516c0b3d40 0x7f516c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f517f7fe640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f516c0b3d40 0x7f516c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f517d7fa640 1 -- 172.21.15.44:0/1227642978 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f51700e42a0 con 0x7f51801019d0 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f5187ea0740 1 -- 172.21.15.44:0/1231792859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad118d830 msgr2=0x55bad118dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f5187ea0740 1 --2- 172.21.15.44:0/1231792859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad118d830 0x55bad118dc10 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5178005ef0 tx=0x7f5178005f20 comp rx=0 tx=0).stop 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f5187ea0740 1 -- 172.21.15.44:0/1231792859 shutdown_connections 2024-08-20T03:08:28.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f5187ea0740 1 --2- 172.21.15.44:0/1231792859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad119a0e0 0x55bad119c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f5187ea0740 1 --2- 172.21.15.44:0/1231792859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad1199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f5187ea0740 1 --2- 172.21.15.44:0/1231792859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad118d830 0x55bad118dc10 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f5187ea0740 1 -- 172.21.15.44:0/1231792859 >> 172.21.15.44:0/1231792859 conn(0x55bad118b660 msgr2=0x55bad118c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f517f7fe640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f516c0b3d40 0x7f516c0b6200 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f51700e1b10 tx=0x7f5170000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f5187ea0740 1 -- 172.21.15.44:0/1231792859 shutdown_connections 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.164+0000 7f5187ea0740 1 -- 172.21.15.44:0/1231792859 wait complete. 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f5187ea0740 1 -- start start 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f5187ea0740 1 Processor -- start 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f5187ea0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad118d830 0x55bad11970b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f5187ea0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad11975f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f5187ea0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 0x55bad1197b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f5187ea0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55bad0f56700 con 0x55bad119a0e0 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f5187ea0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bad107a7c0 con 0x55bad118d830 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f5187ea0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55bad119cbb0 con 0x55bad118e150 2024-08-20T03:08:28.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f5187ea0740 10 client.0 _ll_register_callbacks cb 0x55bad10ed6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f517ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 0x55bad1197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f5176ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad11975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f517ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 0x55bad1197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46466/0 (socket says 172.21.15.44:46466) 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.165+0000 7f517ffff640 1 -- 172.21.15.44:0/1221068239 learned_addr learned my addr 172.21.15.44:0/1221068239 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f517ffff640 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad118d830 0x55bad11970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f5176ffd640 1 -- 172.21.15.44:0/1221068239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad118d830 msgr2=0x55bad11970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f5176ffd640 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad118d830 0x55bad11970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f5176ffd640 1 -- 172.21.15.44:0/1221068239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 msgr2=0x55bad1197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f5176ffd640 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 0x55bad1197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f5176ffd640 1 -- 172.21.15.44:0/1221068239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bad11a5f90 con 0x55bad118e150 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f517ffff640 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad118d830 0x55bad11970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f5176ffd640 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad11975f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5180103dd0 tx=0x7f5168012e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4aa585a6477a6c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f515effd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f515effd640 1 -- 172.21.15.44:0/1221068239 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516800f980 con 0x55bad118e150 2024-08-20T03:08:28.173 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f5187ea0740 1 -- 172.21.15.44:0/1221068239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bad10a3100 con 0x55bad118e150 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f515effd640 1 -- 172.21.15.44:0/1221068239 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f516800fb20 con 0x55bad118e150 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f515effd640 1 -- 172.21.15.44:0/1221068239 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f516800fd20 con 0x55bad118e150 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f5187ea0740 10 client.26152 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.166+0000 7f5187ea0740 1 -- 172.21.15.44:0/1221068239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bad11a89f0 con 0x55bad118e150 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.167+0000 7f515effd640 1 -- 172.21.15.44:0/1221068239 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bad10a3100 con 0x55bad118e150 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.167+0000 7f515effd640 1 -- 172.21.15.44:0/1221068239 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5168024020 con 0x55bad118e150 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.167+0000 7f5187ea0740 10 client.26152 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.167+0000 7f5187ea0740 10 client.26152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.167+0000 7f5187ea0740 20 client.26152 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.167+0000 7f5187ea0740 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55bad11a9fd0 0x55bad11ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.167+0000 7f5187ea0740 4 client.26152 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.167+0000 7f5187ea0740 1 -- 172.21.15.44:0/1221068239 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bad1106830 con 0x55bad11a9fd0 2024-08-20T03:08:28.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.167+0000 7f517ffff640 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55bad11a9fd0 0x55bad11ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.167+0000 7f515effd640 10 client.26152 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:28.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.167+0000 7f517ffff640 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55bad11a9fd0 0x55bad11ac3c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90e4d9b5fd9170c7 server_cookie=6bbb6df48d739ced in_seq=0 out_seq=0 2024-08-20T03:08:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f515effd640 1 -- 172.21.15.44:0/1221068239 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bad1106830 con 0x55bad11a9fd0 2024-08-20T03:08:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f515effd640 10 client.26152 handle_command_reply: tid=0 2024-08-20T03:08:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f5187ea0740 1 client.26152 shutdown 2024-08-20T03:08:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f5187ea0740 1 -- 172.21.15.44:0/1221068239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 msgr2=0x55bad11975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f5187ea0740 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad11975f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5180103dd0 tx=0x7f5168012e70 comp rx=0 tx=0).stop 2024-08-20T03:08:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f5187ea0740 1 -- 172.21.15.44:0/1221068239 shutdown_connections 2024-08-20T03:08:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f5187ea0740 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55bad11a9fd0 0x55bad11ac3c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f5187ea0740 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 0x55bad1197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f5187ea0740 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad11975f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f515effd640 0 client.26152 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:28.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f5187ea0740 1 --2- 172.21.15.44:0/1221068239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad118d830 0x55bad11970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f5187ea0740 1 -- 172.21.15.44:0/1221068239 >> 172.21.15.44:0/1221068239 conn(0x55bad118b660 msgr2=0x55bad11911d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:28.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f5187ea0740 1 -- 172.21.15.44:0/1221068239 shutdown_connections 2024-08-20T03:08:28.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f5187ea0740 1 -- 172.21.15.44:0/1221068239 wait complete. 2024-08-20T03:08:28.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.169+0000 7f5187ea0740 20 client.26152 trim_cache size 0 max 16384 2024-08-20T03:08:28.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.188+0000 7f5187ea0740 1 Processor -- start 2024-08-20T03:08:28.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.188+0000 7f5187ea0740 1 -- start start 2024-08-20T03:08:28.190 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:28 smithi044 ceph-mon[18790]: pgmap v499: 97 pgs: 97 active+clean; 882 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.3 MiB/s wr, 1.57k op/s 2024-08-20T03:08:28.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.188+0000 7f5187ea0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118d830 0x55bad1196810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.188+0000 7f5187ea0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad118e150 0x55bad11b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.188+0000 7f5187ea0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 0x55bad11b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.188+0000 7f5187ea0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f51801b0390 con 0x55bad119a0e0 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.188+0000 7f5187ea0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f51801b1f90 con 0x55bad118e150 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.188+0000 7f5187ea0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f51801b0a60 con 0x55bad118d830 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.188+0000 7f517ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 0x55bad11b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f517ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 0x55bad11b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46468/0 (socket says 172.21.15.44:46468) 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f517ffff640 1 -- 172.21.15.44:0/3903345759 learned_addr learned my addr 172.21.15.44:0/3903345759 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f517ffff640 1 --2- 172.21.15.44:0/3903345759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118d830 0x55bad1196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f5176ffd640 1 --2- 172.21.15.44:0/3903345759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad118e150 0x55bad11b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f517ffff640 1 -- 172.21.15.44:0/3903345759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118d830 msgr2=0x55bad1196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f517ffff640 1 --2- 172.21.15.44:0/3903345759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118d830 0x55bad1196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f517ffff640 1 -- 172.21.15.44:0/3903345759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad118e150 msgr2=0x55bad11b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f517ffff640 1 --2- 172.21.15.44:0/3903345759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad118e150 0x55bad11b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f517ffff640 1 -- 172.21.15.44:0/3903345759 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad118c0e0 con 0x55bad119a0e0 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f5176ffd640 1 --2- 172.21.15.44:0/3903345759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad118e150 0x55bad11b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:08:28.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f517ffff640 1 --2- 172.21.15.44:0/3903345759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 0x55bad11b21b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5178003de0 tx=0x7f517802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f944a90834d6e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f51757fa640 1 -- 172.21.15.44:0/3903345759 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178057070 con 0x55bad119a0e0 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f51757fa640 1 -- 172.21.15.44:0/3903345759 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5178058040 con 0x55bad119a0e0 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.189+0000 7f51757fa640 1 -- 172.21.15.44:0/3903345759 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178004840 con 0x55bad119a0e0 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.190+0000 7f5187ea0740 1 -- 172.21.15.44:0/3903345759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 msgr2=0x55bad11b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.190+0000 7f5187ea0740 1 --2- 172.21.15.44:0/3903345759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 0x55bad11b21b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5178003de0 tx=0x7f517802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.190+0000 7f5187ea0740 1 -- 172.21.15.44:0/3903345759 shutdown_connections 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.190+0000 7f5187ea0740 1 --2- 172.21.15.44:0/3903345759 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad119a0e0 0x55bad11b21b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.190+0000 7f5187ea0740 1 --2- 172.21.15.44:0/3903345759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad118e150 0x55bad11b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.190+0000 7f5187ea0740 1 --2- 172.21.15.44:0/3903345759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118d830 0x55bad1196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.190+0000 7f5187ea0740 1 -- 172.21.15.44:0/3903345759 >> 172.21.15.44:0/3903345759 conn(0x55bad118b660 msgr2=0x55bad1198f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.190+0000 7f5187ea0740 1 -- 172.21.15.44:0/3903345759 shutdown_connections 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.190+0000 7f5187ea0740 1 -- 172.21.15.44:0/3903345759 wait complete. 2024-08-20T03:08:28.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.190+0000 7f5187ea0740 1 -- start start 2024-08-20T03:08:28.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f5187ea0740 1 Processor -- start 2024-08-20T03:08:28.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f5187ea0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad118d830 0x55bad11a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f5187ea0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad119ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f5187ea0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad119a0e0 0x55bad119d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f5187ea0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f51801b1800 con 0x55bad118d830 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f5187ea0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f51801b1130 con 0x55bad119a0e0 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f5187ea0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f51801ae370 con 0x55bad118e150 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f5187ea0740 10 client.0 _ll_register_callbacks cb 0x7f5168009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f5176ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad119ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f517ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad118d830 0x55bad11a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f5176ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad119ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:38866/0 (socket says 172.21.15.44:38866) 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f5176ffd640 1 -- 172.21.15.44:0/3818497298 learned_addr learned my addr 172.21.15.44:0/3818497298 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.191+0000 7f517ffff640 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad119a0e0 0x55bad119d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f5176ffd640 1 -- 172.21.15.44:0/3818497298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad119a0e0 msgr2=0x55bad119d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f5176ffd640 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad119a0e0 0x55bad119d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f5176ffd640 1 -- 172.21.15.44:0/3818497298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad118d830 msgr2=0x55bad11a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f5176ffd640 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad118d830 0x55bad11a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f5176ffd640 1 -- 172.21.15.44:0/3818497298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bad1212c30 con 0x55bad118e150 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f517ffff640 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad119a0e0 0x55bad119d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f5176ffd640 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad119ce00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5168009460 tx=0x7f5168005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc29287a73a037b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f515effd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f515effd640 1 -- 172.21.15.44:0/3818497298 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168026ae0 con 0x55bad118e150 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f5187ea0740 1 -- 172.21.15.44:0/3818497298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5178058040 con 0x55bad118e150 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f515effd640 1 -- 172.21.15.44:0/3818497298 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f5168026c80 con 0x55bad118e150 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f515effd640 1 -- 172.21.15.44:0/3818497298 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5168005040 con 0x55bad118e150 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f5187ea0740 10 client.26158 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f5187ea0740 1 -- 172.21.15.44:0/3818497298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bad1213db0 con 0x55bad118e150 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.192+0000 7f515effd640 1 -- 172.21.15.44:0/3818497298 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5178058040 con 0x55bad118e150 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.193+0000 7f515effd640 1 -- 172.21.15.44:0/3818497298 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5168024070 con 0x55bad118e150 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.193+0000 7f5187ea0740 10 client.26158 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.193+0000 7f5187ea0740 10 client.26158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:08:28.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.193+0000 7f5187ea0740 20 client.26158 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.193+0000 7f5187ea0740 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55bad1214560 0x55bad1216980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.193+0000 7f5187ea0740 4 client.26158 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.193+0000 7f5187ea0740 1 -- 172.21.15.44:0/3818497298 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bad11022d0 con 0x55bad1214560 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.193+0000 7f517ffff640 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55bad1214560 0x55bad1216980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.193+0000 7f517ffff640 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55bad1214560 0x55bad1216980 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41e4b52894d16165 server_cookie=4cba544806fe8a1d in_seq=0 out_seq=0 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.193+0000 7f515effd640 10 client.26158 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.194+0000 7f515effd640 1 -- 172.21.15.44:0/3818497298 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55bad11022d0 con 0x55bad1214560 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.194+0000 7f515effd640 10 client.26158 handle_command_reply: tid=0 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.194+0000 7f5187ea0740 1 client.26158 shutdown 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5187ea0740 1 -- 172.21.15.44:0/3818497298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 msgr2=0x55bad119ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5187ea0740 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad119ce00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5168009460 tx=0x7f5168005d30 comp rx=0 tx=0).stop 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5187ea0740 1 -- 172.21.15.44:0/3818497298 shutdown_connections 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5187ea0740 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55bad1214560 0x55bad1216980 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5187ea0740 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bad119a0e0 0x55bad119d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5187ea0740 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bad118e150 0x55bad119ce00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f515effd640 0 client.26158 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5187ea0740 1 --2- 172.21.15.44:0/3818497298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bad118d830 0x55bad11a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5187ea0740 1 -- 172.21.15.44:0/3818497298 >> 172.21.15.44:0/3818497298 conn(0x55bad118b660 msgr2=0x55bad11908a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5187ea0740 1 -- 172.21.15.44:0/3818497298 shutdown_connections 2024-08-20T03:08:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5187ea0740 1 -- 172.21.15.44:0/3818497298 wait complete. 2024-08-20T03:08:28.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5187ea0740 20 client.26158 trim_cache size 0 max 16384 2024-08-20T03:08:28.201 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:08:28.201 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "scrub active (31289 inodes in the stack)", 2024-08-20T03:08:28.201 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-08-20T03:08:28.201 INFO:teuthology.orchestra.run.smithi044.stdout: "e9d17694-8eff-44c0-a730-895f2597ab32": { 2024-08-20T03:08:28.201 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-08-20T03:08:28.202 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "e9d17694-8eff-44c0-a730-895f2597ab32", 2024-08-20T03:08:28.202 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-08-20T03:08:28.202 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:08:28.202 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:08:28.202 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:08:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5185aa7640 1 -- 172.21.15.44:0/1227642978 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f516c0b3d40 msgr2=0x7f516c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5185aa7640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f516c0b3d40 0x7f516c0b6200 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f51700e1b10 tx=0x7f5170000bc0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.195+0000 7f5185aa7640 1 -- 172.21.15.44:0/1227642978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51801019d0 msgr2=0x7f518019b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.196+0000 7f5185aa7640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51801019d0 0x7f518019b170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f517000e9f0 tx=0x7f517000eeb0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.196+0000 7f5185aa7640 1 -- 172.21.15.44:0/1227642978 shutdown_connections 2024-08-20T03:08:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.196+0000 7f5185aa7640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f516c0b3d40 0x7f516c0b6200 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.196+0000 7f5185aa7640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5180107d60 0x7f51801a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.196+0000 7f5185aa7640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51801073e0 0x7f518019b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.196+0000 7f5185aa7640 1 --2- 172.21.15.44:0/1227642978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51801019d0 0x7f518019b170 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.196+0000 7f5185aa7640 1 -- 172.21.15.44:0/1227642978 >> 172.21.15.44:0/1227642978 conn(0x7f51800fd440 msgr2=0x7f518006b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.196+0000 7f5185aa7640 1 -- 172.21.15.44:0/1227642978 shutdown_connections 2024-08-20T03:08:28.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:08:28.196+0000 7f5185aa7640 1 -- 172.21.15.44:0/1227642978 wait complete. 2024-08-20T03:08:28.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:28 smithi099 ceph-mon[21984]: pgmap v499: 97 pgs: 97 active+clean; 882 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.3 MiB/s wr, 1.57k op/s 2024-08-20T03:08:28.449 INFO:tasks.workunit.client.0.smithi044.stdout: 2024-08-20T03:08:28.450 INFO:tasks.workunit.client.0.smithi044.stdout:Final score for writes: 210 2024-08-20T03:08:28.450 INFO:tasks.workunit.client.0.smithi044.stdout:Final score for reads : 851643 2024-08-20T03:08:28.450 INFO:tasks.workunit.client.0.smithi044.stdout: 2024-08-20T03:08:28.450 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-20T03:08:28.450 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-20T03:08:28.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:28 smithi165 ceph-mon[22357]: pgmap v499: 97 pgs: 97 active+clean; 882 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.3 MiB/s wr, 1.57k op/s 2024-08-20T03:08:29.599 INFO:tasks.cephfs.filesystem:scrub status for tag:e9d17694-8eff-44c0-a730-895f2597ab32 - {'path': '/', 'tag': 'e9d17694-8eff-44c0-a730-895f2597ab32', 'options': 'recursive,force'} 2024-08-20T03:08:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:30 smithi099 ceph-mon[21984]: pgmap v500: 97 pgs: 97 active+clean; 856 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.5 MiB/s wr, 1.44k op/s 2024-08-20T03:08:30.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:30 smithi044 ceph-mon[18790]: pgmap v500: 97 pgs: 97 active+clean; 856 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.5 MiB/s wr, 1.44k op/s 2024-08-20T03:08:30.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:30 smithi165 ceph-mon[22357]: pgmap v500: 97 pgs: 97 active+clean; 856 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.5 MiB/s wr, 1.44k op/s 2024-08-20T03:08:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:32 smithi099 ceph-mon[21984]: pgmap v501: 97 pgs: 97 active+clean; 848 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 1.02k op/s 2024-08-20T03:08:32.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:32 smithi044 ceph-mon[18790]: pgmap v501: 97 pgs: 97 active+clean; 848 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 1.02k op/s 2024-08-20T03:08:32.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:32 smithi165 ceph-mon[22357]: pgmap v501: 97 pgs: 97 active+clean; 848 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 1.02k op/s 2024-08-20T03:08:34.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:34 smithi099 ceph-mon[21984]: pgmap v502: 97 pgs: 97 active+clean; 853 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 882 op/s 2024-08-20T03:08:34.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:34 smithi044 ceph-mon[18790]: pgmap v502: 97 pgs: 97 active+clean; 853 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 882 op/s 2024-08-20T03:08:34.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:34 smithi165 ceph-mon[22357]: pgmap v502: 97 pgs: 97 active+clean; 853 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 882 op/s 2024-08-20T03:08:35.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:35 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:08:35.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:35 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:08:35.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:35 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:08:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:36 smithi099 ceph-mon[21984]: pgmap v503: 97 pgs: 97 active+clean; 854 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 1.26k op/s 2024-08-20T03:08:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T03:08:36.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T03:08:36.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T03:08:36.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T03:08:36.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:08:36.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:08:36.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:08:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:36 smithi044 ceph-mon[18790]: pgmap v503: 97 pgs: 97 active+clean; 854 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 1.26k op/s 2024-08-20T03:08:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T03:08:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T03:08:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T03:08:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T03:08:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:08:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:08:36.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:08:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:36 smithi165 ceph-mon[22357]: pgmap v503: 97 pgs: 97 active+clean; 854 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 1.26k op/s 2024-08-20T03:08:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T03:08:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T03:08:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T03:08:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T03:08:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:08:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:08:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:08:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.097+0000 7f1cc64e5640 1 -- 172.21.15.44:0/1000080948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc01072d0 msgr2=0x7f1cc0107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.097+0000 7f1cc64e5640 1 --2- 172.21.15.44:0/1000080948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc01072d0 0x7f1cc0107710 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009900 tx=0x7f1ca802a410 comp rx=0 tx=0).stop 2024-08-20T03:08:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.097+0000 7f1cc64e5640 1 -- 172.21.15.44:0/1000080948 shutdown_connections 2024-08-20T03:08:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.097+0000 7f1cc64e5640 1 --2- 172.21.15.44:0/1000080948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc0107c50 0x7f1cc010e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.097+0000 7f1cc64e5640 1 --2- 172.21.15.44:0/1000080948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc01072d0 0x7f1cc0107710 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.097+0000 7f1cc64e5640 1 --2- 172.21.15.44:0/1000080948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cc0101920 0x7f1cc0101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.097+0000 7f1cc64e5640 1 -- 172.21.15.44:0/1000080948 >> 172.21.15.44:0/1000080948 conn(0x7f1cc0078620 msgr2=0x7f1cc00ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.097+0000 7f1cc64e5640 1 -- 172.21.15.44:0/1000080948 shutdown_connections 2024-08-20T03:08:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.097+0000 7f1cc64e5640 1 -- 172.21.15.44:0/1000080948 wait complete. 2024-08-20T03:08:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.098+0000 7f1cc64e5640 1 Processor -- start 2024-08-20T03:08:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.098+0000 7f1cc64e5640 1 -- start start 2024-08-20T03:08:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.098+0000 7f1cc64e5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cc0101920 0x7f1cc0198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.098+0000 7f1cc64e5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc01072d0 0x7f1cc0199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.098+0000 7f1cc64e5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc0107c50 0x7f1cc019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.098+0000 7f1cc64e5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0111910 con 0x7f1cc01072d0 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.098+0000 7f1cc64e5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0111790 con 0x7f1cc0107c50 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.098+0000 7f1cc64e5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc0111a90 con 0x7f1cc0101920 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.098+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc01072d0 0x7f1cc0199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.098+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc01072d0 0x7f1cc0199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46494/0 (socket says 172.21.15.44:46494) 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cc4a82640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc0107c50 0x7f1cc019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.098+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cc0101920 0x7f1cc0198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cbf7fe640 1 -- 172.21.15.44:0/2531988347 learned_addr learned my addr 172.21.15.44:0/2531988347 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cbf7fe640 1 -- 172.21.15.44:0/2531988347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cc0101920 msgr2=0x7f1cc0198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cbf7fe640 1 --2- 172.21.15.44:0/2531988347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cc0101920 0x7f1cc0198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cbf7fe640 1 -- 172.21.15.44:0/2531988347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc0107c50 msgr2=0x7f1cc019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cbf7fe640 1 --2- 172.21.15.44:0/2531988347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc0107c50 0x7f1cc019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cbf7fe640 1 -- 172.21.15.44:0/2531988347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc019eff0 con 0x7f1cc01072d0 2024-08-20T03:08:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cbf7fe640 1 --2- 172.21.15.44:0/2531988347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc01072d0 0x7f1cc0199490 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8005940 tx=0x7f1ca80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cbd7fa640 1 -- 172.21.15.44:0/2531988347 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca803f030 con 0x7f1cc01072d0 2024-08-20T03:08:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cbd7fa640 1 -- 172.21.15.44:0/2531988347 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca80044c0 con 0x7f1cc01072d0 2024-08-20T03:08:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cbd7fa640 1 -- 172.21.15.44:0/2531988347 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8035070 con 0x7f1cc01072d0 2024-08-20T03:08:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cc64e5640 1 -- 172.21.15.44:0/2531988347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc019f280 con 0x7f1cc01072d0 2024-08-20T03:08:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.099+0000 7f1cc64e5640 1 -- 172.21.15.44:0/2531988347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01a59a0 con 0x7f1cc01072d0 2024-08-20T03:08:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cbd7fa640 1 -- 172.21.15.44:0/2531988347 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1ca8042030 con 0x7f1cc01072d0 2024-08-20T03:08:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cc93be740 1 Processor -- start 2024-08-20T03:08:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cc93be740 1 -- start start 2024-08-20T03:08:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cc93be740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f020ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cc93be740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f020f270 0x5648f021acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cc93be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f021b200 0x5648f021d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cc93be740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5648f0184360 con 0x5648f020e950 2024-08-20T03:08:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cc93be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5648f0173860 con 0x5648f021b200 2024-08-20T03:08:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cc93be740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5648f01778a0 con 0x5648f020f270 2024-08-20T03:08:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cc4a82640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f020ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cc4a82640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f020ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46506/0 (socket says 172.21.15.44:46506) 2024-08-20T03:08:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cc4a82640 1 -- 172.21.15.44:0/3614742280 learned_addr learned my addr 172.21.15.44:0/3614742280 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cbffff640 1 --2- 172.21.15.44:0/3614742280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f020f270 0x5648f021acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.102+0000 7f1cbf7fe640 1 --2- 172.21.15.44:0/3614742280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f021b200 0x5648f021d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.103+0000 7f1cc4a82640 1 -- 172.21.15.44:0/3614742280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f020f270 msgr2=0x5648f021acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.103+0000 7f1cc4a82640 1 --2- 172.21.15.44:0/3614742280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f020f270 0x5648f021acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.103+0000 7f1cc4a82640 1 -- 172.21.15.44:0/3614742280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f021b200 msgr2=0x5648f021d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.103+0000 7f1cc4a82640 1 --2- 172.21.15.44:0/3614742280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f021b200 0x5648f021d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.103+0000 7f1cc4a82640 1 -- 172.21.15.44:0/3614742280 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f011ae40 con 0x5648f020e950 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.103+0000 7f1cbd7fa640 1 --2- 172.21.15.44:0/2531988347 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1cac0b3d40 0x7f1cac0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.103+0000 7f1cc4a82640 1 --2- 172.21.15.44:0/3614742280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f020ed30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8011440 tx=0x7f1cb8011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8b69176115d470 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.103+0000 7f1cbffff640 1 --2- 172.21.15.44:0/2531988347 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1cac0b3d40 0x7f1cac0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.103+0000 7f1cb67fc640 1 -- 172.21.15.44:0/3614742280 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8003c30 con 0x5648f020e950 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.103+0000 7f1cbd7fa640 1 -- 172.21.15.44:0/2531988347 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1ca8104280 con 0x7f1cc01072d0 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.103+0000 7f1cb67fc640 1 -- 172.21.15.44:0/3614742280 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb8003dd0 con 0x5648f020e950 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.103+0000 7f1cb67fc640 1 -- 172.21.15.44:0/3614742280 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb800fa20 con 0x5648f020e950 2024-08-20T03:08:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.104+0000 7f1cc93be740 1 -- 172.21.15.44:0/3614742280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 msgr2=0x5648f020ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.104+0000 7f1cbffff640 1 --2- 172.21.15.44:0/2531988347 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1cac0b3d40 0x7f1cac0b6200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5648f0211ad0 tx=0x7f1cb0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.104+0000 7f1cc93be740 1 --2- 172.21.15.44:0/3614742280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f020ed30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8011440 tx=0x7f1cb8011900 comp rx=0 tx=0).stop 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.104+0000 7f1cc93be740 1 -- 172.21.15.44:0/3614742280 shutdown_connections 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.104+0000 7f1cc93be740 1 --2- 172.21.15.44:0/3614742280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f021b200 0x5648f021d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.104+0000 7f1cc93be740 1 --2- 172.21.15.44:0/3614742280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f020f270 0x5648f021acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.104+0000 7f1cc93be740 1 --2- 172.21.15.44:0/3614742280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f020ed30 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.104+0000 7f1cc93be740 1 -- 172.21.15.44:0/3614742280 >> 172.21.15.44:0/3614742280 conn(0x5648f020c730 msgr2=0x5648f020d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.104+0000 7f1cc93be740 1 -- 172.21.15.44:0/3614742280 shutdown_connections 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.104+0000 7f1cc93be740 1 -- 172.21.15.44:0/3614742280 wait complete. 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.104+0000 7f1cc93be740 1 -- start start 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.104+0000 7f1cc93be740 1 Processor -- start 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cc93be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f020e950 0x5648f021ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cc93be740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020f270 0x5648f0217fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cc93be740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f021b200 0x5648f02186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cc93be740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5648f00acab0 con 0x5648f020f270 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cc93be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5648f00fa780 con 0x5648f020e950 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cc93be740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5648f021dee0 con 0x5648f021b200 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cc93be740 10 client.0 _ll_register_callbacks cb 0x5648efe82a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020f270 0x5648f0217fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020f270 0x5648f0217fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46516/0 (socket says 172.21.15.44:46516) 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f020e950 0x5648f021ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cbf7fe640 1 -- 172.21.15.44:0/1712251381 learned_addr learned my addr 172.21.15.44:0/1712251381 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cc4a82640 1 --2- 172.21.15.44:0/1712251381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f021b200 0x5648f02186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cbf7fe640 1 -- 172.21.15.44:0/1712251381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f021b200 msgr2=0x5648f02186a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cbf7fe640 1 --2- 172.21.15.44:0/1712251381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f021b200 0x5648f02186a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cbf7fe640 1 -- 172.21.15.44:0/1712251381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f020e950 msgr2=0x5648f021ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cbf7fe640 1 --2- 172.21.15.44:0/1712251381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f020e950 0x5648f021ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cbf7fe640 1 -- 172.21.15.44:0/1712251381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648efed6790 con 0x5648f020f270 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cbf7fe640 1 --2- 172.21.15.44:0/1712251381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020f270 0x5648f0217fd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8006010 tx=0x7f1ca8101ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3288c7569c7074 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1c9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1c9f7fe640 1 -- 172.21.15.44:0/1712251381 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8049070 con 0x5648f020f270 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1c9f7fe640 1 -- 172.21.15.44:0/1712251381 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1ca80cb370 con 0x5648f020f270 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cc93be740 1 -- 172.21.15.44:0/1712251381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5648f0122e00 con 0x5648f020f270 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1c9f7fe640 1 -- 172.21.15.44:0/1712251381 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1ca80ca850 con 0x5648f020f270 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cc93be740 10 client.16749 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.105+0000 7f1cc93be740 1 -- 172.21.15.44:0/1712251381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648eff4f620 con 0x5648f020f270 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.106+0000 7f1c9f7fe640 1 -- 172.21.15.44:0/1712251381 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5648f0122e00 con 0x5648f020f270 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.106+0000 7f1c9f7fe640 1 -- 172.21.15.44:0/1712251381 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1ca80d3b20 con 0x5648f020f270 2024-08-20T03:08:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.106+0000 7f1cc93be740 10 client.16749 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.106+0000 7f1cc93be740 10 client.16749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:08:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.106+0000 7f1cc93be740 20 client.16749 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.106+0000 7f1cc93be740 1 --2- 172.21.15.44:0/1712251381 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5648f022ab90 0x5648f022d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.106+0000 7f1cc93be740 4 client.16749 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:08:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.106+0000 7f1cc93be740 1 -- 172.21.15.44:0/1712251381 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648f01862b0 con 0x5648f022ab90 2024-08-20T03:08:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.106+0000 7f1cbffff640 1 --2- 172.21.15.44:0/1712251381 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5648f022ab90 0x5648f022d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.106+0000 7f1cbffff640 1 --2- 172.21.15.44:0/1712251381 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5648f022ab90 0x5648f022d020 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9bf531f18dab646 server_cookie=757303ef4e447273 in_seq=0 out_seq=0 2024-08-20T03:08:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.107+0000 7f1c9f7fe640 10 client.16749 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1c9f7fe640 1 -- 172.21.15.44:0/1712251381 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648f01862b0 con 0x5648f022ab90 2024-08-20T03:08:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1c9f7fe640 10 client.16749 handle_command_reply: tid=0 2024-08-20T03:08:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1cc93be740 1 client.16749 shutdown 2024-08-20T03:08:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1cc93be740 1 -- 172.21.15.44:0/1712251381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020f270 msgr2=0x5648f0217fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1cc93be740 1 --2- 172.21.15.44:0/1712251381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020f270 0x5648f0217fd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8006010 tx=0x7f1ca8101ff0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1cc93be740 1 -- 172.21.15.44:0/1712251381 shutdown_connections 2024-08-20T03:08:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1cc93be740 1 --2- 172.21.15.44:0/1712251381 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5648f022ab90 0x5648f022d020 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1cc93be740 1 --2- 172.21.15.44:0/1712251381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f021b200 0x5648f02186a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1cc93be740 1 --2- 172.21.15.44:0/1712251381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020f270 0x5648f0217fd0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1c9f7fe640 0 client.16749 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1cc93be740 1 --2- 172.21.15.44:0/1712251381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f020e950 0x5648f021ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1cc93be740 1 -- 172.21.15.44:0/1712251381 >> 172.21.15.44:0/1712251381 conn(0x5648f020c730 msgr2=0x5648f0212320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1cc93be740 1 -- 172.21.15.44:0/1712251381 shutdown_connections 2024-08-20T03:08:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1cc93be740 1 -- 172.21.15.44:0/1712251381 wait complete. 2024-08-20T03:08:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.109+0000 7f1cc93be740 20 client.16749 trim_cache size 0 max 16384 2024-08-20T03:08:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.124+0000 7f1cc93be740 1 Processor -- start 2024-08-20T03:08:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.124+0000 7f1cc93be740 1 -- start start 2024-08-20T03:08:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.124+0000 7f1cc93be740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f0227000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.124+0000 7f1cc93be740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f020f270 0x5648f0219d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.124+0000 7f1cc93be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f021b200 0x5648f0232930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.124+0000 7f1cc93be740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01ae160 con 0x5648f020e950 2024-08-20T03:08:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.124+0000 7f1cc93be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01afd60 con 0x5648f021b200 2024-08-20T03:08:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.124+0000 7f1cc93be740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01ae830 con 0x5648f020f270 2024-08-20T03:08:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.124+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f0227000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.124+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f0227000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46522/0 (socket says 172.21.15.44:46522) 2024-08-20T03:08:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.124+0000 7f1cbffff640 1 -- 172.21.15.44:0/279512911 learned_addr learned my addr 172.21.15.44:0/279512911 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.124+0000 7f1cbf7fe640 1 --2- 172.21.15.44:0/279512911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f020f270 0x5648f0219d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.125+0000 7f1cc4a82640 1 --2- 172.21.15.44:0/279512911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f021b200 0x5648f0232930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.125+0000 7f1cbffff640 1 -- 172.21.15.44:0/279512911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f020f270 msgr2=0x5648f0219d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.125+0000 7f1cbffff640 1 --2- 172.21.15.44:0/279512911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f020f270 0x5648f0219d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.125+0000 7f1cbffff640 1 -- 172.21.15.44:0/279512911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f021b200 msgr2=0x5648f0232930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.125+0000 7f1cbffff640 1 --2- 172.21.15.44:0/279512911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f021b200 0x5648f0232930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.125+0000 7f1cbffff640 1 -- 172.21.15.44:0/279512911 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f020f0a0 con 0x5648f020e950 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.125+0000 7f1cbffff640 1 --2- 172.21.15.44:0/279512911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f0227000 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x5648f022d8d0 tx=0x7f1cb001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f2b659e37b2544 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.125+0000 7f1cb5ffb640 1 -- 172.21.15.44:0/279512911 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb002b420 con 0x5648f020e950 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.125+0000 7f1cb5ffb640 1 -- 172.21.15.44:0/279512911 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb002ba70 con 0x5648f020e950 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.125+0000 7f1cb5ffb640 1 -- 172.21.15.44:0/279512911 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb001fab0 con 0x5648f020e950 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.126+0000 7f1cc93be740 1 -- 172.21.15.44:0/279512911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 msgr2=0x5648f0227000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.126+0000 7f1cc93be740 1 --2- 172.21.15.44:0/279512911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f0227000 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x5648f022d8d0 tx=0x7f1cb001b000 comp rx=0 tx=0).stop 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.126+0000 7f1cc93be740 1 -- 172.21.15.44:0/279512911 shutdown_connections 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.126+0000 7f1cc93be740 1 --2- 172.21.15.44:0/279512911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f021b200 0x5648f0232930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.126+0000 7f1cc93be740 1 --2- 172.21.15.44:0/279512911 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f020f270 0x5648f0219d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.126+0000 7f1cc93be740 1 --2- 172.21.15.44:0/279512911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f0227000 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.126+0000 7f1cc93be740 1 -- 172.21.15.44:0/279512911 >> 172.21.15.44:0/279512911 conn(0x5648f020c730 msgr2=0x5648f0219600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.126+0000 7f1cc93be740 1 -- 172.21.15.44:0/279512911 shutdown_connections 2024-08-20T03:08:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.126+0000 7f1cc93be740 1 -- 172.21.15.44:0/279512911 wait complete. 2024-08-20T03:08:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.126+0000 7f1cc93be740 1 -- start start 2024-08-20T03:08:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.126+0000 7f1cc93be740 1 Processor -- start 2024-08-20T03:08:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cc93be740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f0223a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cc93be740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f020f270 0x5648f021c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cc93be740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f021b200 0x5648f021caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cc93be740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01af5d0 con 0x5648f020e950 2024-08-20T03:08:37.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cc93be740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01aef00 con 0x5648f020f270 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cc93be740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc01ac140 con 0x5648f021b200 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cc93be740 10 client.0 _ll_register_callbacks cb 0x7f1ca8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f0223a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f0223a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46528/0 (socket says 172.21.15.44:46528) 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cbffff640 1 -- 172.21.15.44:0/3522291337 learned_addr learned my addr 172.21.15.44:0/3522291337 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cbf7fe640 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f020f270 0x5648f021c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cc4a82640 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f021b200 0x5648f021caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cbffff640 1 -- 172.21.15.44:0/3522291337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f021b200 msgr2=0x5648f021caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cbffff640 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f021b200 0x5648f021caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cbffff640 1 -- 172.21.15.44:0/3522291337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f020f270 msgr2=0x5648f021c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cbffff640 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f020f270 0x5648f021c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cbffff640 1 -- 172.21.15.44:0/3522291337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f0222620 con 0x5648f020e950 2024-08-20T03:08:37.136 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cbf7fe640 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f020f270 0x5648f021c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cbffff640 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f0223a40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009fa0 tx=0x7f1cb0009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51000bfc705e28a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1c9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1c9f7fe640 1 -- 172.21.15.44:0/3522291337 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb001f400 con 0x5648f020e950 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1c9f7fe640 1 -- 172.21.15.44:0/3522291337 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f1cb002b420 con 0x5648f020e950 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.127+0000 7f1cc93be740 1 -- 172.21.15.44:0/3522291337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb002ba70 con 0x5648f020e950 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1c9f7fe640 1 -- 172.21.15.44:0/3522291337 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1cb0046050 con 0x5648f020e950 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1cc93be740 10 client.16755 fetch_fsmap learned FSMap version 30 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1cc93be740 1 -- 172.21.15.44:0/3522291337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648f0295bb0 con 0x5648f020e950 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1c9f7fe640 1 -- 172.21.15.44:0/3522291337 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb002ba70 con 0x5648f020e950 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1c9f7fe640 1 -- 172.21.15.44:0/3522291337 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1cb0033070 con 0x5648f020e950 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1cc93be740 10 client.16755 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1cc93be740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:08:37.137 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1cc93be740 20 client.16755 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1cc93be740 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5648f0296330 0x5648f0298750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1cc93be740 4 client.16755 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:08:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1cc93be740 1 -- 172.21.15.44:0/3522291337 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5648f0181d50 con 0x5648f0296330 2024-08-20T03:08:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1cbf7fe640 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5648f0296330 0x5648f0298750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1c9f7fe640 10 client.16755 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:37.138 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:37.128+0000 7f1cbf7fe640 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5648f0296330 0x5648f0298750 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed27f89881c7aafb server_cookie=bfe1bc1bfa85861b in_seq=0 out_seq=0 2024-08-20T03:08:37.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:37 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi044 to 177.0M 2024-08-20T03:08:37.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:37 smithi099 ceph-mon[21984]: Unable to set osd_memory_target on smithi044 to 185701376: error parsing value: Value '185701376' is below minimum 939524096 2024-08-20T03:08:37.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:37 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi044 to 177.0M 2024-08-20T03:08:37.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:37 smithi044 ceph-mon[18790]: Unable to set osd_memory_target on smithi044 to 185701376: error parsing value: Value '185701376' is below minimum 939524096 2024-08-20T03:08:37.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:37 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi044 to 177.0M 2024-08-20T03:08:37.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:37 smithi165 ceph-mon[22357]: Unable to set osd_memory_target on smithi044 to 185701376: error parsing value: Value '185701376' is below minimum 939524096 2024-08-20T03:08:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:38 smithi099 ceph-mon[21984]: pgmap v504: 97 pgs: 97 active+clean; 859 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 1.08k op/s 2024-08-20T03:08:38.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:38 smithi044 ceph-mon[18790]: pgmap v504: 97 pgs: 97 active+clean; 859 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 1.08k op/s 2024-08-20T03:08:38.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:38 smithi165 ceph-mon[22357]: pgmap v504: 97 pgs: 97 active+clean; 859 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 1.08k op/s 2024-08-20T03:08:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1c9f7fe640 1 -- 172.21.15.44:0/3522291337 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5648f0181d50 con 0x5648f0296330 2024-08-20T03:08:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1c9f7fe640 10 client.16755 handle_command_reply: tid=0 2024-08-20T03:08:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1cc93be740 1 client.16755 shutdown 2024-08-20T03:08:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1cc93be740 1 -- 172.21.15.44:0/3522291337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 msgr2=0x5648f0223a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1cc93be740 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f0223a40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009fa0 tx=0x7f1cb0009fd0 comp rx=0 tx=0).stop 2024-08-20T03:08:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1cc93be740 1 -- 172.21.15.44:0/3522291337 shutdown_connections 2024-08-20T03:08:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1cc93be740 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5648f0296330 0x5648f0298750 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1cc93be740 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5648f021b200 0x5648f021caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1c9f7fe640 0 client.16755 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1cc93be740 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648f020f270 0x5648f021c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1cc93be740 1 --2- 172.21.15.44:0/3522291337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5648f020e950 0x5648f0223a40 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1cc93be740 1 -- 172.21.15.44:0/3522291337 >> 172.21.15.44:0/3522291337 conn(0x5648f020c730 msgr2=0x5648f0219e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.326+0000 7f1cc93be740 1 -- 172.21.15.44:0/3522291337 shutdown_connections 2024-08-20T03:08:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc93be740 1 -- 172.21.15.44:0/3522291337 wait complete. 2024-08-20T03:08:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc93be740 20 client.16755 trim_cache size 0 max 16384 2024-08-20T03:08:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:08:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:08:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:08:39.332 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:08:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc64e5640 1 -- 172.21.15.44:0/2531988347 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1cac0b3d40 msgr2=0x7f1cac0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc64e5640 1 --2- 172.21.15.44:0/2531988347 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1cac0b3d40 0x7f1cac0b6200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5648f0211ad0 tx=0x7f1cb0005f90 comp rx=0 tx=0).stop 2024-08-20T03:08:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc64e5640 1 -- 172.21.15.44:0/2531988347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc01072d0 msgr2=0x7f1cc0199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc64e5640 1 --2- 172.21.15.44:0/2531988347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc01072d0 0x7f1cc0199490 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8005940 tx=0x7f1ca80040c0 comp rx=0 tx=0).stop 2024-08-20T03:08:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc64e5640 1 -- 172.21.15.44:0/2531988347 shutdown_connections 2024-08-20T03:08:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc64e5640 1 --2- 172.21.15.44:0/2531988347 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1cac0b3d40 0x7f1cac0b6200 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc64e5640 1 --2- 172.21.15.44:0/2531988347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cc0107c50 0x7f1cc019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc64e5640 1 --2- 172.21.15.44:0/2531988347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1cc01072d0 0x7f1cc0199490 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc64e5640 1 --2- 172.21.15.44:0/2531988347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cc0101920 0x7f1cc0198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc64e5640 1 -- 172.21.15.44:0/2531988347 >> 172.21.15.44:0/2531988347 conn(0x7f1cc0078620 msgr2=0x7f1cc0100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc64e5640 1 -- 172.21.15.44:0/2531988347 shutdown_connections 2024-08-20T03:08:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:39.327+0000 7f1cc64e5640 1 -- 172.21.15.44:0/2531988347 wait complete. 2024-08-20T03:08:40.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:40 smithi099 ceph-mon[21984]: pgmap v505: 97 pgs: 97 active+clean; 871 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 1.41k op/s 2024-08-20T03:08:40.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:40 smithi044 ceph-mon[18790]: pgmap v505: 97 pgs: 97 active+clean; 871 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 1.41k op/s 2024-08-20T03:08:40.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:40 smithi165 ceph-mon[22357]: pgmap v505: 97 pgs: 97 active+clean; 871 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 1.41k op/s 2024-08-20T03:08:42.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:42 smithi099 ceph-mon[21984]: pgmap v506: 97 pgs: 97 active+clean; 879 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 1.55k op/s 2024-08-20T03:08:42.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:42 smithi044 ceph-mon[18790]: pgmap v506: 97 pgs: 97 active+clean; 879 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 1.55k op/s 2024-08-20T03:08:42.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:42 smithi165 ceph-mon[22357]: pgmap v506: 97 pgs: 97 active+clean; 879 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 1.55k op/s 2024-08-20T03:08:44.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:44 smithi044 ceph-mon[18790]: pgmap v507: 97 pgs: 97 active+clean; 868 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 1.81k op/s 2024-08-20T03:08:44.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:44 smithi165 ceph-mon[22357]: pgmap v507: 97 pgs: 97 active+clean; 868 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 1.81k op/s 2024-08-20T03:08:44.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:44 smithi099 ceph-mon[21984]: pgmap v507: 97 pgs: 97 active+clean; 868 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.5 MiB/s wr, 1.81k op/s 2024-08-20T03:08:46.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:46 smithi044 ceph-mon[18790]: pgmap v508: 97 pgs: 97 active+clean; 845 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 2.25k op/s 2024-08-20T03:08:46.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:46 smithi165 ceph-mon[22357]: pgmap v508: 97 pgs: 97 active+clean; 845 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 2.25k op/s 2024-08-20T03:08:46.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:46 smithi099 ceph-mon[21984]: pgmap v508: 97 pgs: 97 active+clean; 845 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.0 MiB/s wr, 2.25k op/s 2024-08-20T03:08:48.466 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:48 smithi099 ceph-mon[21984]: pgmap v509: 97 pgs: 97 active+clean; 846 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 1.96k op/s 2024-08-20T03:08:48.466 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:48 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:08:48.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:48 smithi044 ceph-mon[18790]: pgmap v509: 97 pgs: 97 active+clean; 846 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 1.96k op/s 2024-08-20T03:08:48.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:48 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:08:48.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:48 smithi165 ceph-mon[22357]: pgmap v509: 97 pgs: 97 active+clean; 846 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.8 MiB/s wr, 1.96k op/s 2024-08-20T03:08:48.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:48 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:08:50.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:50 smithi044 ceph-mon[18790]: pgmap v510: 97 pgs: 97 active+clean; 844 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.4 MiB/s wr, 2.36k op/s 2024-08-20T03:08:50.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:50 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:08:50.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:50 smithi165 ceph-mon[22357]: pgmap v510: 97 pgs: 97 active+clean; 844 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.4 MiB/s wr, 2.36k op/s 2024-08-20T03:08:50.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:50 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:08:50.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:50 smithi099 ceph-mon[21984]: pgmap v510: 97 pgs: 97 active+clean; 844 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.4 MiB/s wr, 2.36k op/s 2024-08-20T03:08:50.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:50 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:08:51.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:51 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:51.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:51 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:08:51.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:51 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:08:51.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:51 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:08:51.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:51 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:08:51.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:51 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:51.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:51 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:08:51.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:51 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:08:51.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:51 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:08:51.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:51 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:08:51.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:51 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:08:51.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:51 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:08:51.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:51 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:08:51.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:51 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:08:51.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:51 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:08:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.422+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/3260918453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc104210 msgr2=0x7f6fdc10c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.422+0000 7f6fe3b7b640 1 --2- 172.21.15.44:0/3260918453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc104210 0x7f6fdc10c6f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009900 tx=0x7f6fcc02a410 comp rx=0 tx=0).stop 2024-08-20T03:08:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.422+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/3260918453 shutdown_connections 2024-08-20T03:08:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.422+0000 7f6fe3b7b640 1 --2- 172.21.15.44:0/3260918453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fdc10cd40 0x7f6fdc10f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.422+0000 7f6fe3b7b640 1 --2- 172.21.15.44:0/3260918453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc104210 0x7f6fdc10c6f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.422+0000 7f6fe3b7b640 1 --2- 172.21.15.44:0/3260918453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc1038f0 0x7f6fdc103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.422+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/3260918453 >> 172.21.15.44:0/3260918453 conn(0x7f6fdc0fd340 msgr2=0x7f6fdc0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.422+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/3260918453 shutdown_connections 2024-08-20T03:08:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.422+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/3260918453 wait complete. 2024-08-20T03:08:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.423+0000 7f6fe3b7b640 1 Processor -- start 2024-08-20T03:08:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.423+0000 7f6fe3b7b640 1 -- start start 2024-08-20T03:08:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.423+0000 7f6fe3b7b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc1038f0 0x7f6fdc198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.423+0000 7f6fe3b7b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fdc104210 0x7f6fdc199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.423+0000 7f6fe3b7b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc10cd40 0x7f6fdc19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.423+0000 7f6fe3b7b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc1129d0 con 0x7f6fdc10cd40 2024-08-20T03:08:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.423+0000 7f6fe3b7b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc112850 con 0x7f6fdc1038f0 2024-08-20T03:08:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.423+0000 7f6fe3b7b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc112b50 con 0x7f6fdc104210 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.423+0000 7f6fe2118640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc10cd40 0x7f6fdc19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.423+0000 7f6fe2118640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc10cd40 0x7f6fdc19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55766/0 (socket says 172.21.15.44:55766) 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.423+0000 7f6fe2118640 1 -- 172.21.15.44:0/1760042541 learned_addr learned my addr 172.21.15.44:0/1760042541 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.423+0000 7f6fe1116640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fdc104210 0x7f6fdc199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fe1917640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc1038f0 0x7f6fdc198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fe2118640 1 -- 172.21.15.44:0/1760042541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fdc104210 msgr2=0x7f6fdc199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fe2118640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fdc104210 0x7f6fdc199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fe2118640 1 -- 172.21.15.44:0/1760042541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc1038f0 msgr2=0x7f6fdc198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fe2118640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc1038f0 0x7f6fdc198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fe2118640 1 -- 172.21.15.44:0/1760042541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc19efd0 con 0x7f6fdc10cd40 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fe1917640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc1038f0 0x7f6fdc198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fe2118640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc10cd40 0x7f6fdc19e800 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400eb10 tx=0x7f6fd400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fd2ffd640 1 -- 172.21.15.44:0/1760042541 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4004b20 con 0x7f6fdc10cd40 2024-08-20T03:08:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fd2ffd640 1 -- 172.21.15.44:0/1760042541 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd4004cc0 con 0x7f6fdc10cd40 2024-08-20T03:08:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/1760042541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc19f2c0 con 0x7f6fdc10cd40 2024-08-20T03:08:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fd2ffd640 1 -- 172.21.15.44:0/1760042541 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4010670 con 0x7f6fdc10cd40 2024-08-20T03:08:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.424+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/1760042541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc1a5980 con 0x7f6fdc10cd40 2024-08-20T03:08:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.427+0000 7f6fe6a46740 1 Processor -- start 2024-08-20T03:08:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.427+0000 7f6fd2ffd640 1 -- 172.21.15.44:0/1760042541 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f6fd4005020 con 0x7f6fdc10cd40 2024-08-20T03:08:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe6a46740 1 -- start start 2024-08-20T03:08:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe6a46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29e4bf0 0x55dfb29e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe6a46740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e5510 0x55dfb29f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe6a46740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 0x55dfb29f3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe6a46740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55dfb295a360 con 0x55dfb29f14a0 2024-08-20T03:08:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe6a46740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dfb2949860 con 0x55dfb29e5510 2024-08-20T03:08:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe6a46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dfb294d8a0 con 0x55dfb29e4bf0 2024-08-20T03:08:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe1116640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 0x55dfb29f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe1116640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 0x55dfb29f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55780/0 (socket says 172.21.15.44:55780) 2024-08-20T03:08:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe1116640 1 -- 172.21.15.44:0/3307935228 learned_addr learned my addr 172.21.15.44:0/3307935228 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe1116640 1 --2- 172.21.15.44:0/3307935228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29e4bf0 0x55dfb29e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe1917640 1 --2- 172.21.15.44:0/3307935228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e5510 0x55dfb29f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fd2ffd640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6fc80b3d40 0x7f6fc80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe1116640 1 -- 172.21.15.44:0/3307935228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29e4bf0 msgr2=0x55dfb29e4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:52.441 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe1116640 1 --2- 172.21.15.44:0/3307935228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29e4bf0 0x55dfb29e4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe1116640 1 -- 172.21.15.44:0/3307935228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e5510 msgr2=0x55dfb29f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe1116640 1 --2- 172.21.15.44:0/3307935228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e5510 0x55dfb29f0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe1116640 1 -- 172.21.15.44:0/3307935228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb28f0e40 con 0x55dfb29f14a0 2024-08-20T03:08:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.428+0000 7f6fe1917640 1 --2- 172.21.15.44:0/3307935228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e5510 0x55dfb29f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:08:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.429+0000 7f6fe2118640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6fc80b3d40 0x7f6fc80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.429+0000 7f6fd2ffd640 1 -- 172.21.15.44:0/1760042541 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6fd40e4500 con 0x7f6fdc10cd40 2024-08-20T03:08:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.429+0000 7f6fe1116640 1 --2- 172.21.15.44:0/3307935228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 0x55dfb29f3890 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc06bf30 tx=0x7f6fcc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795c675cdd9c3c9d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.429+0000 7f6fc7fff640 1 -- 172.21.15.44:0/3307935228 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc035070 con 0x55dfb29f14a0 2024-08-20T03:08:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.429+0000 7f6fe2118640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6fc80b3d40 0x7f6fc80b6200 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40aae80 tx=0x7f6fd409fa00 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.429+0000 7f6fc7fff640 1 -- 172.21.15.44:0/3307935228 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fcc004030 con 0x55dfb29f14a0 2024-08-20T03:08:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.429+0000 7f6fc7fff640 1 -- 172.21.15.44:0/3307935228 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc003850 con 0x55dfb29f14a0 2024-08-20T03:08:52.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.430+0000 7f6fe6a46740 1 -- 172.21.15.44:0/3307935228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 msgr2=0x55dfb29f3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.430+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/3307935228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 0x55dfb29f3890 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc06bf30 tx=0x7f6fcc005780 comp rx=0 tx=0).stop 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.430+0000 7f6fe6a46740 1 -- 172.21.15.44:0/3307935228 shutdown_connections 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.430+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/3307935228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 0x55dfb29f3890 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.430+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/3307935228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e5510 0x55dfb29f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.430+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/3307935228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29e4bf0 0x55dfb29e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.430+0000 7f6fe6a46740 1 -- 172.21.15.44:0/3307935228 >> 172.21.15.44:0/3307935228 conn(0x55dfb29e2a20 msgr2=0x55dfb29e3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.430+0000 7f6fe6a46740 1 -- 172.21.15.44:0/3307935228 shutdown_connections 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.430+0000 7f6fe6a46740 1 -- 172.21.15.44:0/3307935228 wait complete. 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.430+0000 7f6fe6a46740 1 -- start start 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.430+0000 7f6fe6a46740 1 Processor -- start 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe6a46740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb29f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe6a46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29e5510 0x55dfb29ee280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe6a46740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 0x55dfb29ee930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe6a46740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55dfb2882ab0 con 0x55dfb29f14a0 2024-08-20T03:08:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe6a46740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dfb28d0780 con 0x55dfb29e4bf0 2024-08-20T03:08:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe6a46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55dfb29f8530 con 0x55dfb29e5510 2024-08-20T03:08:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe6a46740 10 client.0 _ll_register_callbacks cb 0x55dfb2658a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe1116640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 0x55dfb29ee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe1116640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 0x55dfb29ee930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55786/0 (socket says 172.21.15.44:55786) 2024-08-20T03:08:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe1116640 1 -- 172.21.15.44:0/996439821 learned_addr learned my addr 172.21.15.44:0/996439821 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe1917640 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29e5510 0x55dfb29ee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe1116640 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb29f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe1116640 1 -- 172.21.15.44:0/996439821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29e5510 msgr2=0x55dfb29ee280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe1116640 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29e5510 0x55dfb29ee280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe1116640 1 -- 172.21.15.44:0/996439821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 msgr2=0x55dfb29f0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe1116640 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb29f0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe1116640 1 -- 172.21.15.44:0/996439821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb26ac790 con 0x55dfb29f14a0 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe1116640 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 0x55dfb29ee930 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009900 tx=0x7f6fcc032e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7907cbf979a36703 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fc57fa640 1 -- 172.21.15.44:0/996439821 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc050060 con 0x55dfb29f14a0 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fc57fa640 1 -- 172.21.15.44:0/996439821 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6fcc031d10 con 0x55dfb29f14a0 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe6a46740 1 -- 172.21.15.44:0/996439821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dfb28f8e00 con 0x55dfb29f14a0 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fc57fa640 1 -- 172.21.15.44:0/996439821 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6fcc035040 con 0x55dfb29f14a0 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe6a46740 10 client.16767 fetch_fsmap learned FSMap version 31 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.431+0000 7f6fe6a46740 1 -- 172.21.15.44:0/996439821 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dfb2725620 con 0x55dfb29f14a0 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.432+0000 7f6fc57fa640 1 -- 172.21.15.44:0/996439821 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dfb28f8e00 con 0x55dfb29f14a0 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.432+0000 7f6fc57fa640 1 -- 172.21.15.44:0/996439821 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f6fcc053020 con 0x55dfb29f14a0 2024-08-20T03:08:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.432+0000 7f6fe6a46740 10 client.16767 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.432+0000 7f6fe6a46740 10 client.16767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.432+0000 7f6fe6a46740 20 client.16767 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.432+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55dfb2a050c0 0x55dfb2a074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.432+0000 7f6fe6a46740 4 client.16767 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.432+0000 7f6fe6a46740 1 -- 172.21.15.44:0/996439821 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfb295c2b0 con 0x55dfb2a050c0 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.432+0000 7f6fe1917640 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55dfb2a050c0 0x55dfb2a074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.432+0000 7f6fc57fa640 10 client.16767 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.432+0000 7f6fe1917640 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55dfb2a050c0 0x55dfb2a074b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9850ccb5604f2e0 server_cookie=9c9a9c0be28daba6 in_seq=0 out_seq=0 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.434+0000 7f6fc57fa640 1 -- 172.21.15.44:0/996439821 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfb295c2b0 con 0x55dfb2a050c0 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fc57fa640 10 client.16767 handle_command_reply: tid=0 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fe6a46740 1 client.16767 shutdown 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fe6a46740 1 -- 172.21.15.44:0/996439821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 msgr2=0x55dfb29ee930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 0x55dfb29ee930 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009900 tx=0x7f6fcc032e00 comp rx=0 tx=0).stop 2024-08-20T03:08:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fe6a46740 1 -- 172.21.15.44:0/996439821 shutdown_connections 2024-08-20T03:08:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55dfb2a050c0 0x55dfb2a074b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29f14a0 0x55dfb29ee930 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29e5510 0x55dfb29ee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fc57fa640 0 client.16767 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/996439821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb29f0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fe6a46740 1 -- 172.21.15.44:0/996439821 >> 172.21.15.44:0/996439821 conn(0x55dfb29e2a20 msgr2=0x55dfb29e85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fe6a46740 1 -- 172.21.15.44:0/996439821 shutdown_connections 2024-08-20T03:08:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fe6a46740 1 -- 172.21.15.44:0/996439821 wait complete. 2024-08-20T03:08:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.435+0000 7f6fe6a46740 20 client.16767 trim_cache size 0 max 16384 2024-08-20T03:08:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe6a46740 1 Processor -- start 2024-08-20T03:08:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe6a46740 1 -- start start 2024-08-20T03:08:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe6a46740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb29edaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe6a46740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 0x55dfb29e22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe6a46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29f14a0 0x55dfb29f2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe6a46740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc1ac080 con 0x55dfb29e5510 2024-08-20T03:08:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe6a46740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc1adc80 con 0x55dfb29e4bf0 2024-08-20T03:08:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe6a46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc1ac750 con 0x55dfb29f14a0 2024-08-20T03:08:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe1917640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 0x55dfb29e22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe1917640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 0x55dfb29e22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55794/0 (socket says 172.21.15.44:55794) 2024-08-20T03:08:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe1917640 1 -- 172.21.15.44:0/3041248578 learned_addr learned my addr 172.21.15.44:0/3041248578 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe1116640 1 --2- 172.21.15.44:0/3041248578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb29edaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe1116640 1 --2- 172.21.15.44:0/3041248578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29f14a0 0x55dfb29f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe1917640 1 -- 172.21.15.44:0/3041248578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29f14a0 msgr2=0x55dfb29f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe1917640 1 --2- 172.21.15.44:0/3041248578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29f14a0 0x55dfb29f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe1917640 1 -- 172.21.15.44:0/3041248578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 msgr2=0x55dfb29edaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe1917640 1 --2- 172.21.15.44:0/3041248578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb29edaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe1917640 1 -- 172.21.15.44:0/3041248578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb29f8470 con 0x55dfb29e5510 2024-08-20T03:08:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe1116640 1 --2- 172.21.15.44:0/3041248578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb29edaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fe1917640 1 --2- 172.21.15.44:0/3041248578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 0x55dfb29e22d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800bfd0 tx=0x7f6fd8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75db07f8a6663877 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fc77fe640 1 -- 172.21.15.44:0/3041248578 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8015070 con 0x55dfb29e5510 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fc77fe640 1 -- 172.21.15.44:0/3041248578 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd80054d0 con 0x55dfb29e5510 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.450+0000 7f6fc77fe640 1 -- 172.21.15.44:0/3041248578 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd801d610 con 0x55dfb29e5510 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.451+0000 7f6fe6a46740 1 -- 172.21.15.44:0/3041248578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 msgr2=0x55dfb29e22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.451+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/3041248578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 0x55dfb29e22d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800bfd0 tx=0x7f6fd8004530 comp rx=0 tx=0).stop 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.451+0000 7f6fe6a46740 1 -- 172.21.15.44:0/3041248578 shutdown_connections 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.451+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/3041248578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29f14a0 0x55dfb29f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.451+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/3041248578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 0x55dfb29e22d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.451+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/3041248578 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb29edaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.451+0000 7f6fe6a46740 1 -- 172.21.15.44:0/3041248578 >> 172.21.15.44:0/3041248578 conn(0x55dfb29e2a20 msgr2=0x55dfb29f00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.451+0000 7f6fe6a46740 1 -- 172.21.15.44:0/3041248578 shutdown_connections 2024-08-20T03:08:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.451+0000 7f6fe6a46740 1 -- 172.21.15.44:0/3041248578 wait complete. 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.451+0000 7f6fe6a46740 1 -- start start 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.451+0000 7f6fe6a46740 1 Processor -- start 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe6a46740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb2a0a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe6a46740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 0x55dfb2a0ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe6a46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29f14a0 0x55dfb2a05f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe6a46740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc1ad4f0 con 0x55dfb29e5510 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe6a46740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc1ace20 con 0x55dfb29e4bf0 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe6a46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc1aa060 con 0x55dfb29f14a0 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe6a46740 10 client.0 _ll_register_callbacks cb 0x7f6fd800b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe1917640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 0x55dfb2a0ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe1917640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 0x55dfb2a0ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55806/0 (socket says 172.21.15.44:55806) 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe1917640 1 -- 172.21.15.44:0/977915479 learned_addr learned my addr 172.21.15.44:0/977915479 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:08:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe1116640 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29f14a0 0x55dfb2a05f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe1116640 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb2a0a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe1917640 1 -- 172.21.15.44:0/977915479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29f14a0 msgr2=0x55dfb2a05f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe1917640 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29f14a0 0x55dfb2a05f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe1917640 1 -- 172.21.15.44:0/977915479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 msgr2=0x55dfb2a0a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe1917640 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb2a0a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe1917640 1 -- 172.21.15.44:0/977915479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb2a06760 con 0x55dfb29e5510 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fe1917640 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 0x55dfb2a0ada0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8000c00 tx=0x7f6fd8004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f5f8828144f66d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.452+0000 7f6fc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fc57fa640 1 -- 172.21.15.44:0/977915479 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd802c070 con 0x55dfb29e5510 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fc57fa640 1 -- 172.21.15.44:0/977915479 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f6fd8005220 con 0x55dfb29e5510 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fe6a46740 1 -- 172.21.15.44:0/977915479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fd80054d0 con 0x55dfb29e5510 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fc57fa640 1 -- 172.21.15.44:0/977915479 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6fd8038050 con 0x55dfb29e5510 2024-08-20T03:08:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fe6a46740 10 client.16773 fetch_fsmap learned FSMap version 31 2024-08-20T03:08:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fe6a46740 1 -- 172.21.15.44:0/977915479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dfb2a6f710 con 0x55dfb29e5510 2024-08-20T03:08:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fc57fa640 1 -- 172.21.15.44:0/977915479 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd80054d0 con 0x55dfb29e5510 2024-08-20T03:08:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fc57fa640 1 -- 172.21.15.44:0/977915479 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f6fd801dad0 con 0x55dfb29e5510 2024-08-20T03:08:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fe6a46740 10 client.16773 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T03:08:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fe6a46740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:08:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fe6a46740 20 client.16773 populate_metadata read hostname 'smithi044' 2024-08-20T03:08:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55dfb2a6fec0 0x55dfb2a722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:08:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fe6a46740 4 client.16773 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:08:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fe6a46740 1 -- 172.21.15.44:0/977915479 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfb2957d50 con 0x55dfb2a6fec0 2024-08-20T03:08:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.453+0000 7f6fe1116640 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55dfb2a6fec0 0x55dfb2a722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:08:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.454+0000 7f6fe1116640 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55dfb2a6fec0 0x55dfb2a722e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=243424ba25de7433 server_cookie=5c78caf90ac97c6d in_seq=0 out_seq=0 2024-08-20T03:08:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:52.454+0000 7f6fc57fa640 10 client.16773 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:52.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:52 smithi044 ceph-mon[18790]: pgmap v511: 97 pgs: 97 active+clean; 843 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 2.23k op/s 2024-08-20T03:08:52.537 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:52 smithi165 ceph-mon[22357]: pgmap v511: 97 pgs: 97 active+clean; 843 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 2.23k op/s 2024-08-20T03:08:52.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:52 smithi099 ceph-mon[21984]: pgmap v511: 97 pgs: 97 active+clean; 843 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 2.23k op/s 2024-08-20T03:08:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fc57fa640 1 -- 172.21.15.44:0/977915479 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfb2957d50 con 0x55dfb2a6fec0 2024-08-20T03:08:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fc57fa640 10 client.16773 handle_command_reply: tid=0 2024-08-20T03:08:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe6a46740 1 client.16773 shutdown 2024-08-20T03:08:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe6a46740 1 -- 172.21.15.44:0/977915479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 msgr2=0x55dfb2a0ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 0x55dfb2a0ada0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8000c00 tx=0x7f6fd8004e90 comp rx=0 tx=0).stop 2024-08-20T03:08:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe6a46740 1 -- 172.21.15.44:0/977915479 shutdown_connections 2024-08-20T03:08:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55dfb2a6fec0 0x55dfb2a722e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dfb29f14a0 0x55dfb2a05f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55dfb29e5510 0x55dfb2a0ada0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fc57fa640 0 client.16773 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:08:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe6a46740 1 --2- 172.21.15.44:0/977915479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfb29e4bf0 0x55dfb2a0a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe6a46740 1 -- 172.21.15.44:0/977915479 >> 172.21.15.44:0/977915479 conn(0x55dfb29e2a20 msgr2=0x55dfb29e7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe6a46740 1 -- 172.21.15.44:0/977915479 shutdown_connections 2024-08-20T03:08:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe6a46740 1 -- 172.21.15.44:0/977915479 wait complete. 2024-08-20T03:08:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe6a46740 20 client.16773 trim_cache size 0 max 16384 2024-08-20T03:08:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:08:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:08:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:08:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:08:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/1760042541 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6fc80b3d40 msgr2=0x7f6fc80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe3b7b640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6fc80b3d40 0x7f6fc80b6200 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40aae80 tx=0x7f6fd409fa00 comp rx=0 tx=0).stop 2024-08-20T03:08:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/1760042541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc10cd40 msgr2=0x7f6fdc19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:08:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe3b7b640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc10cd40 0x7f6fdc19e800 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400eb10 tx=0x7f6fd400c2d0 comp rx=0 tx=0).stop 2024-08-20T03:08:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/1760042541 shutdown_connections 2024-08-20T03:08:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe3b7b640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f6fc80b3d40 0x7f6fc80b6200 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe3b7b640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fdc10cd40 0x7f6fdc19e800 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe3b7b640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fdc104210 0x7f6fdc199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe3b7b640 1 --2- 172.21.15.44:0/1760042541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fdc1038f0 0x7f6fdc198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:08:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.396+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/1760042541 >> 172.21.15.44:0/1760042541 conn(0x7f6fdc0fd340 msgr2=0x7f6fdc0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:08:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.397+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/1760042541 shutdown_connections 2024-08-20T03:08:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:08:53.397+0000 7f6fe3b7b640 1 -- 172.21.15.44:0/1760042541 wait complete. 2024-08-20T03:08:54.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:54 smithi044 ceph-mon[18790]: pgmap v512: 97 pgs: 97 active+clean; 849 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 2.18k op/s 2024-08-20T03:08:54.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:54 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:08:54.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:54 smithi165 ceph-mon[22357]: pgmap v512: 97 pgs: 97 active+clean; 849 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 2.18k op/s 2024-08-20T03:08:54.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:54 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:08:54.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:54 smithi099 ceph-mon[21984]: pgmap v512: 97 pgs: 97 active+clean; 849 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 2.18k op/s 2024-08-20T03:08:54.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:54 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:08:56.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:56 smithi044 ceph-mon[18790]: pgmap v513: 97 pgs: 97 active+clean; 841 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 2.14k op/s 2024-08-20T03:08:56.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:56 smithi165 ceph-mon[22357]: pgmap v513: 97 pgs: 97 active+clean; 841 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 2.14k op/s 2024-08-20T03:08:56.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:56 smithi099 ceph-mon[21984]: pgmap v513: 97 pgs: 97 active+clean; 841 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 2.14k op/s 2024-08-20T03:08:58.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:08:58 smithi044 ceph-mon[18790]: pgmap v514: 97 pgs: 97 active+clean; 828 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 1.55k op/s 2024-08-20T03:08:58.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:08:58 smithi165 ceph-mon[22357]: pgmap v514: 97 pgs: 97 active+clean; 828 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 1.55k op/s 2024-08-20T03:08:58.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:08:58 smithi099 ceph-mon[21984]: pgmap v514: 97 pgs: 97 active+clean; 828 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 1.55k op/s 2024-08-20T03:08:59.601 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:08:59.911 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:00.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:00 smithi044 ceph-mon[18790]: pgmap v515: 97 pgs: 97 active+clean; 827 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.49k op/s 2024-08-20T03:09:00.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:00 smithi165 ceph-mon[22357]: pgmap v515: 97 pgs: 97 active+clean; 827 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.49k op/s 2024-08-20T03:09:00.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:00 smithi099 ceph-mon[21984]: pgmap v515: 97 pgs: 97 active+clean; 827 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 1.49k op/s 2024-08-20T03:09:00.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.872+0000 7f0275fbe640 1 -- 172.21.15.44:0/4186442991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0270107e10 msgr2=0x7f027010e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:00.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.872+0000 7f0275fbe640 1 --2- 172.21.15.44:0/4186442991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0270107e10 0x7f027010e560 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0268009ca0 tx=0x7f026802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.872+0000 7f0275fbe640 1 -- 172.21.15.44:0/4186442991 shutdown_connections 2024-08-20T03:09:00.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.872+0000 7f0275fbe640 1 --2- 172.21.15.44:0/4186442991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0270107e10 0x7f027010e560 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.872+0000 7f0275fbe640 1 --2- 172.21.15.44:0/4186442991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0270107470 0x7f02701078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.872+0000 7f0275fbe640 1 --2- 172.21.15.44:0/4186442991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0270102610 0x7f02701029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.872+0000 7f0275fbe640 1 -- 172.21.15.44:0/4186442991 >> 172.21.15.44:0/4186442991 conn(0x7f0270076cc0 msgr2=0x7f02700770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.872+0000 7f0275fbe640 1 -- 172.21.15.44:0/4186442991 shutdown_connections 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.872+0000 7f0275fbe640 1 -- 172.21.15.44:0/4186442991 wait complete. 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0275fbe640 1 Processor -- start 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0275fbe640 1 -- start start 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0275fbe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0270102610 0x7f0270198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0275fbe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0270107470 0x7f02701994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0275fbe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0270107e10 0x7f027019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0275fbe640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0270111990 con 0x7f0270107470 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0275fbe640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0270111810 con 0x7f0270107e10 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0275fbe640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0270111b10 con 0x7f0270102610 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0267fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0270107470 0x7f02701994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f026f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0270102610 0x7f0270198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0267fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0270107470 0x7f02701994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54354/0 (socket says 172.21.15.44:54354) 2024-08-20T03:09:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f026ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0270107e10 0x7f027019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0267fff640 1 -- 172.21.15.44:0/92926570 learned_addr learned my addr 172.21.15.44:0/92926570 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0267fff640 1 -- 172.21.15.44:0/92926570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0270102610 msgr2=0x7f0270198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0267fff640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0270102610 0x7f0270198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0267fff640 1 -- 172.21.15.44:0/92926570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0270107e10 msgr2=0x7f027019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0267fff640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0270107e10 0x7f027019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f0267fff640 1 -- 172.21.15.44:0/92926570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027019f010 con 0x7f0270107470 2024-08-20T03:09:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.873+0000 7f026f7fe640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0270102610 0x7f0270198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.874+0000 7f0267fff640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0270107470 0x7f02701994a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f025800e9f0 tx=0x7f025800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.874+0000 7f026d7fa640 1 -- 172.21.15.44:0/92926570 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025800cd80 con 0x7f0270107470 2024-08-20T03:09:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.874+0000 7f0275fbe640 1 -- 172.21.15.44:0/92926570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027019f300 con 0x7f0270107470 2024-08-20T03:09:00.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.874+0000 7f026d7fa640 1 -- 172.21.15.44:0/92926570 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02580108e0 con 0x7f0270107470 2024-08-20T03:09:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.874+0000 7f026d7fa640 1 -- 172.21.15.44:0/92926570 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258018980 con 0x7f0270107470 2024-08-20T03:09:00.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.874+0000 7f0275fbe640 1 -- 172.21.15.44:0/92926570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02701a5b90 con 0x7f0270107470 2024-08-20T03:09:00.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.876+0000 7f026d7fa640 1 -- 172.21.15.44:0/92926570 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0258010430 con 0x7f0270107470 2024-08-20T03:09:00.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.876+0000 7f02783b7740 1 Processor -- start 2024-08-20T03:09:00.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f02783b7740 1 -- start start 2024-08-20T03:09:00.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f02783b7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb58830 0x55b95cb58c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f02783b7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb59150 0x55b95cb68fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f02783b7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb694f0 0x55b95cb6b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f02783b7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b95cacf8e0 con 0x55b95cb694f0 2024-08-20T03:09:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f02783b7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b95cabede0 con 0x55b95cb58830 2024-08-20T03:09:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f02783b7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b95cac2e20 con 0x55b95cb59150 2024-08-20T03:09:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f0267fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb694f0 0x55b95cb6b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f026f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb59150 0x55b95cb68fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f026ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb58830 0x55b95cb58c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f026ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb58830 0x55b95cb58c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53474/0 (socket says 172.21.15.44:53474) 2024-08-20T03:09:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f026f7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb59150 0x55b95cb68fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47776/0 (socket says 172.21.15.44:47776) 2024-08-20T03:09:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f026ffff640 1 -- 172.21.15.44:0/3429035963 learned_addr learned my addr 172.21.15.44:0/3429035963 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f026ffff640 1 -- 172.21.15.44:0/3429035963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb59150 msgr2=0x55b95cb68fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f026ffff640 1 --2- 172.21.15.44:0/3429035963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb59150 0x55b95cb68fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f026ffff640 1 -- 172.21.15.44:0/3429035963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb694f0 msgr2=0x55b95cb6b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f026ffff640 1 --2- 172.21.15.44:0/3429035963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb694f0 0x55b95cb6b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f026d7fa640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f025c0b3dc0 0x7f025c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.877+0000 7f026ffff640 1 -- 172.21.15.44:0/3429035963 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b95c81e650 con 0x55b95cb58830 2024-08-20T03:09:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.878+0000 7f026f7fe640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f025c0b3dc0 0x7f025c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.878+0000 7f026d7fa640 1 -- 172.21.15.44:0/92926570 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f02580e4290 con 0x7f0270107470 2024-08-20T03:09:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.878+0000 7f026ffff640 1 --2- 172.21.15.44:0/3429035963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb58830 0x55b95cb58c10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0268009410 tx=0x7f0268005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9150d08e5c1aa92 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.878+0000 7f0265ffb640 1 -- 172.21.15.44:0/3429035963 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268031cf0 con 0x55b95cb58830 2024-08-20T03:09:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.878+0000 7f0265ffb640 1 -- 172.21.15.44:0/3429035963 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0268036030 con 0x55b95cb58830 2024-08-20T03:09:00.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.878+0000 7f0265ffb640 1 -- 172.21.15.44:0/3429035963 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02680042a0 con 0x55b95cb58830 2024-08-20T03:09:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.879+0000 7f02783b7740 1 -- 172.21.15.44:0/3429035963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb58830 msgr2=0x55b95cb58c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.879+0000 7f02783b7740 1 --2- 172.21.15.44:0/3429035963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb58830 0x55b95cb58c10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0268009410 tx=0x7f0268005ef0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.879+0000 7f026f7fe640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f025c0b3dc0 0x7f025c0b6280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55b95cb5ca70 tx=0x7f0260005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.879+0000 7f02783b7740 1 -- 172.21.15.44:0/3429035963 shutdown_connections 2024-08-20T03:09:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.879+0000 7f02783b7740 1 --2- 172.21.15.44:0/3429035963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb694f0 0x55b95cb6b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.879+0000 7f02783b7740 1 --2- 172.21.15.44:0/3429035963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb59150 0x55b95cb68fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.879+0000 7f02783b7740 1 --2- 172.21.15.44:0/3429035963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb58830 0x55b95cb58c10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.879+0000 7f02783b7740 1 -- 172.21.15.44:0/3429035963 >> 172.21.15.44:0/3429035963 conn(0x55b95cb56660 msgr2=0x55b95cb57800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:00.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.879+0000 7f02783b7740 1 -- 172.21.15.44:0/3429035963 shutdown_connections 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.879+0000 7f02783b7740 1 -- 172.21.15.44:0/3429035963 wait complete. 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.879+0000 7f02783b7740 1 -- start start 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.879+0000 7f02783b7740 1 Processor -- start 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f02783b7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb58830 0x55b95cb66450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f02783b7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb59150 0x55b95cb66990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f02783b7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb694f0 0x55b95cb66ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f02783b7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b95c921700 con 0x55b95cb58830 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f02783b7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b95ca457c0 con 0x55b95cb59150 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f02783b7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b95cb6bfc0 con 0x55b95cb694f0 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f026f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb58830 0x55b95cb66450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f026f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb58830 0x55b95cb66450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54370/0 (socket says 172.21.15.44:54370) 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f026f7fe640 1 -- 172.21.15.44:0/1183877428 learned_addr learned my addr 172.21.15.44:0/1183877428 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:00.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f02783b7740 10 client.0 _ll_register_callbacks cb 0x55b95cab86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f0267fff640 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb59150 0x55b95cb66990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f026ffff640 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb694f0 0x55b95cb66ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f026f7fe640 1 -- 172.21.15.44:0/1183877428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb694f0 msgr2=0x55b95cb66ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f026f7fe640 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb694f0 0x55b95cb66ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f026f7fe640 1 -- 172.21.15.44:0/1183877428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb59150 msgr2=0x55b95cb66990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f026f7fe640 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb59150 0x55b95cb66990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f026f7fe640 1 -- 172.21.15.44:0/1183877428 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b95cb75370 con 0x55b95cb58830 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f0267fff640 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb59150 0x55b95cb66990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f026f7fe640 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb58830 0x55b95cb66450 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f02600045a0 tx=0x7f0260004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5406412d7d56d9a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f024effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f024effd640 1 -- 172.21.15.44:0/1183877428 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026000f830 con 0x55b95cb58830 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.880+0000 7f024effd640 1 -- 172.21.15.44:0/1183877428 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f026000f9d0 con 0x55b95cb58830 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f02783b7740 1 -- 172.21.15.44:0/1183877428 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b95ca6e100 con 0x55b95cb58830 2024-08-20T03:09:00.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f024effd640 1 -- 172.21.15.44:0/1183877428 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f026000fc10 con 0x55b95cb58830 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f02783b7740 10 client.16785 fetch_fsmap learned FSMap version 32 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f02783b7740 1 -- 172.21.15.44:0/1183877428 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b95cb77d80 con 0x55b95cb58830 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f024effd640 1 -- 172.21.15.44:0/1183877428 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b95ca6e100 con 0x55b95cb58830 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f024effd640 1 -- 172.21.15.44:0/1183877428 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f0260031020 con 0x55b95cb58830 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f02783b7740 10 client.16785 fetch_fsmap finished waiting for FSMap version 32 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f02783b7740 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f02783b7740 20 client.16785 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f02783b7740 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b95cb79230 0x55b95cb7b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f02783b7740 4 client.16785 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f02783b7740 1 -- 172.21.15.44:0/1183877428 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b95cad1830 con 0x55b95cb79230 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.881+0000 7f0267fff640 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b95cb79230 0x55b95cb7b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.882+0000 7f024effd640 10 client.16785 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:00.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.882+0000 7f0267fff640 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b95cb79230 0x55b95cb7b6b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c7ed7e4854d288 server_cookie=4a10fc8e9a273f43 in_seq=0 out_seq=0 2024-08-20T03:09:00.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f024effd640 1 -- 172.21.15.44:0/1183877428 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b95cad1830 con 0x55b95cb79230 2024-08-20T03:09:00.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f024effd640 10 client.16785 handle_command_reply: tid=0 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f02783b7740 1 client.16785 shutdown 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f02783b7740 1 -- 172.21.15.44:0/1183877428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb58830 msgr2=0x55b95cb66450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f02783b7740 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb58830 0x55b95cb66450 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f02600045a0 tx=0x7f0260004730 comp rx=0 tx=0).stop 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f02783b7740 1 -- 172.21.15.44:0/1183877428 shutdown_connections 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f02783b7740 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b95cb79230 0x55b95cb7b6b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f02783b7740 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb694f0 0x55b95cb66ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f02783b7740 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb59150 0x55b95cb66990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f02783b7740 1 --2- 172.21.15.44:0/1183877428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb58830 0x55b95cb66450 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f024effd640 0 client.16785 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f02783b7740 1 -- 172.21.15.44:0/1183877428 >> 172.21.15.44:0/1183877428 conn(0x55b95cb56660 msgr2=0x55b95cb5c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f02783b7740 1 -- 172.21.15.44:0/1183877428 shutdown_connections 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f02783b7740 1 -- 172.21.15.44:0/1183877428 wait complete. 2024-08-20T03:09:00.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.904+0000 7f02783b7740 20 client.16785 trim_cache size 0 max 16384 2024-08-20T03:09:00.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f02783b7740 1 Processor -- start 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f02783b7740 1 -- start start 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f02783b7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb58830 0x55b95cb63f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f02783b7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb59150 0x55b95cb57e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f02783b7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb694f0 0x55b95cb55f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f02783b7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f02701abfb0 con 0x55b95cb59150 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f02783b7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f02701adbe0 con 0x55b95cb694f0 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f02783b7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02701ac680 con 0x55b95cb58830 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f0267fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb59150 0x55b95cb57e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f0267fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb59150 0x55b95cb57e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54378/0 (socket says 172.21.15.44:54378) 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f0267fff640 1 -- 172.21.15.44:0/1814393355 learned_addr learned my addr 172.21.15.44:0/1814393355 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f026f7fe640 1 --2- 172.21.15.44:0/1814393355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb58830 0x55b95cb63f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.924+0000 7f026ffff640 1 --2- 172.21.15.44:0/1814393355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb694f0 0x55b95cb55f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.925+0000 7f0267fff640 1 -- 172.21.15.44:0/1814393355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb58830 msgr2=0x55b95cb63f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:00.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.925+0000 7f0267fff640 1 --2- 172.21.15.44:0/1814393355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb58830 0x55b95cb63f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.925+0000 7f0267fff640 1 -- 172.21.15.44:0/1814393355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb694f0 msgr2=0x55b95cb55f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:00.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.925+0000 7f0267fff640 1 --2- 172.21.15.44:0/1814393355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb694f0 0x55b95cb55f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.925+0000 7f0267fff640 1 -- 172.21.15.44:0/1814393355 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b95cb54aa0 con 0x55b95cb59150 2024-08-20T03:09:00.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.925+0000 7f0267fff640 1 --2- 172.21.15.44:0/1814393355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb59150 0x55b95cb57e30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f02580aaef0 tx=0x7f0258000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b299aee8475159 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:00.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.925+0000 7f02657fa640 1 -- 172.21.15.44:0/1814393355 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258010720 con 0x55b95cb59150 2024-08-20T03:09:00.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.925+0000 7f02657fa640 1 -- 172.21.15.44:0/1814393355 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02580aaa70 con 0x55b95cb59150 2024-08-20T03:09:00.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.925+0000 7f02657fa640 1 -- 172.21.15.44:0/1814393355 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258018b20 con 0x55b95cb59150 2024-08-20T03:09:00.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.926+0000 7f02783b7740 1 -- 172.21.15.44:0/1814393355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb59150 msgr2=0x55b95cb57e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:00.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.926+0000 7f02783b7740 1 --2- 172.21.15.44:0/1814393355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb59150 0x55b95cb57e30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f02580aaef0 tx=0x7f0258000bc0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.926+0000 7f02783b7740 1 -- 172.21.15.44:0/1814393355 shutdown_connections 2024-08-20T03:09:00.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.926+0000 7f02783b7740 1 --2- 172.21.15.44:0/1814393355 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb694f0 0x55b95cb55f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.926+0000 7f02783b7740 1 --2- 172.21.15.44:0/1814393355 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb59150 0x55b95cb57e30 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.926+0000 7f02783b7740 1 --2- 172.21.15.44:0/1814393355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb58830 0x55b95cb63f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.926+0000 7f02783b7740 1 -- 172.21.15.44:0/1814393355 >> 172.21.15.44:0/1814393355 conn(0x55b95cb56660 msgr2=0x55b95cb57720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:00.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.926+0000 7f02783b7740 1 -- 172.21.15.44:0/1814393355 shutdown_connections 2024-08-20T03:09:00.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.926+0000 7f02783b7740 1 -- 172.21.15.44:0/1814393355 wait complete. 2024-08-20T03:09:00.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.926+0000 7f02783b7740 1 -- start start 2024-08-20T03:09:00.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.926+0000 7f02783b7740 1 Processor -- start 2024-08-20T03:09:00.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f02783b7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb58830 0x55b95cb80f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f02783b7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb59150 0x55b95cb814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f02783b7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb694f0 0x55b95cb71280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f02783b7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f02701ad450 con 0x55b95cb694f0 2024-08-20T03:09:00.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f02783b7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f02701acd50 con 0x55b95cb59150 2024-08-20T03:09:00.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f02783b7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f02701a9f90 con 0x55b95cb58830 2024-08-20T03:09:00.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f02783b7740 10 client.0 _ll_register_callbacks cb 0x7f0260009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:00.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f026ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb694f0 0x55b95cb71280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f0267fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb59150 0x55b95cb814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f026ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb694f0 0x55b95cb71280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54384/0 (socket says 172.21.15.44:54384) 2024-08-20T03:09:00.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f026ffff640 1 -- 172.21.15.44:0/2752309020 learned_addr learned my addr 172.21.15.44:0/2752309020 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:00.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f026f7fe640 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb58830 0x55b95cb80f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f026ffff640 1 -- 172.21.15.44:0/2752309020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb58830 msgr2=0x55b95cb80f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:00.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f026ffff640 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb58830 0x55b95cb80f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f026ffff640 1 -- 172.21.15.44:0/2752309020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb59150 msgr2=0x55b95cb814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:00.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f026ffff640 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb59150 0x55b95cb814c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f026ffff640 1 -- 172.21.15.44:0/2752309020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b95cb5b430 con 0x55b95cb694f0 2024-08-20T03:09:00.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f0267fff640 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb59150 0x55b95cb814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:00.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f026ffff640 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb694f0 0x55b95cb71280 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0268005cc0 tx=0x7f0268005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5d39895d65c508 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:00.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f024effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:00.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.927+0000 7f024effd640 1 -- 172.21.15.44:0/2752309020 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268053070 con 0x55b95cb694f0 2024-08-20T03:09:00.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f024effd640 1 -- 172.21.15.44:0/2752309020 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7f0268031660 con 0x55b95cb694f0 2024-08-20T03:09:00.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f02783b7740 1 -- 172.21.15.44:0/2752309020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02580aaa70 con 0x55b95cb694f0 2024-08-20T03:09:00.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f024effd640 1 -- 172.21.15.44:0/2752309020 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f026805f050 con 0x55b95cb694f0 2024-08-20T03:09:00.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f02783b7740 10 client.16791 fetch_fsmap learned FSMap version 32 2024-08-20T03:09:00.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f02783b7740 1 -- 172.21.15.44:0/2752309020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b95cb59fb0 con 0x55b95cb694f0 2024-08-20T03:09:00.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f024effd640 1 -- 172.21.15.44:0/2752309020 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02580aaa70 con 0x55b95cb694f0 2024-08-20T03:09:00.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f024effd640 1 -- 172.21.15.44:0/2752309020 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f0268043070 con 0x55b95cb694f0 2024-08-20T03:09:00.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f02783b7740 10 client.16791 fetch_fsmap finished waiting for FSMap version 32 2024-08-20T03:09:00.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f02783b7740 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:00.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f02783b7740 20 client.16791 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:00.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f02783b7740 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b95cb5a4a0 0x55b95cbe5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:00.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f02783b7740 4 client.16791 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:09:00.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f02783b7740 1 -- 172.21.15.44:0/2752309020 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b95cacd2d0 con 0x55b95cb5a4a0 2024-08-20T03:09:00.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.928+0000 7f026f7fe640 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b95cb5a4a0 0x55b95cbe5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:00.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.929+0000 7f024effd640 10 client.16791 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:00.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.929+0000 7f026f7fe640 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b95cb5a4a0 0x55b95cbe5f90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f3878816bfe8234 server_cookie=2559b235b21f8d93 in_seq=0 out_seq=0 2024-08-20T03:09:00.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.938+0000 7f024effd640 1 -- 172.21.15.44:0/2752309020 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b95cacd2d0 con 0x55b95cb5a4a0 2024-08-20T03:09:00.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f024effd640 10 client.16791 handle_command_reply: tid=0 2024-08-20T03:09:00.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f02783b7740 1 client.16791 shutdown 2024-08-20T03:09:00.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f02783b7740 1 -- 172.21.15.44:0/2752309020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb694f0 msgr2=0x55b95cb71280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:00.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f02783b7740 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb694f0 0x55b95cb71280 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f0268005cc0 tx=0x7f0268005c50 comp rx=0 tx=0).stop 2024-08-20T03:09:00.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f02783b7740 1 -- 172.21.15.44:0/2752309020 shutdown_connections 2024-08-20T03:09:00.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f02783b7740 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b95cb5a4a0 0x55b95cbe5f90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f02783b7740 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b95cb694f0 0x55b95cb71280 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f024effd640 0 client.16791 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:00.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f02783b7740 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b95cb59150 0x55b95cb814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f02783b7740 1 --2- 172.21.15.44:0/2752309020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b95cb58830 0x55b95cb80f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f02783b7740 1 -- 172.21.15.44:0/2752309020 >> 172.21.15.44:0/2752309020 conn(0x55b95cb56660 msgr2=0x55b95cb663a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:00.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f02783b7740 1 -- 172.21.15.44:0/2752309020 shutdown_connections 2024-08-20T03:09:00.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f02783b7740 1 -- 172.21.15.44:0/2752309020 wait complete. 2024-08-20T03:09:00.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.939+0000 7f02783b7740 20 client.16791 trim_cache size 0 max 16384 2024-08-20T03:09:00.943 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:09:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-08-20T03:09:00.943 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-08-20T03:09:00.943 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:09:00.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.940+0000 7f0275fbe640 1 -- 172.21.15.44:0/92926570 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f025c0b3dc0 msgr2=0x7f025c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:00.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.940+0000 7f0275fbe640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f025c0b3dc0 0x7f025c0b6280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55b95cb5ca70 tx=0x7f0260005f90 comp rx=0 tx=0).stop 2024-08-20T03:09:00.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.940+0000 7f0275fbe640 1 -- 172.21.15.44:0/92926570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0270107470 msgr2=0x7f02701994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:00.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.940+0000 7f0275fbe640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0270107470 0x7f02701994a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f025800e9f0 tx=0x7f025800eeb0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.940+0000 7f0275fbe640 1 -- 172.21.15.44:0/92926570 shutdown_connections 2024-08-20T03:09:00.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.940+0000 7f0275fbe640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f025c0b3dc0 0x7f025c0b6280 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.940+0000 7f0275fbe640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0270107e10 0x7f027019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.940+0000 7f0275fbe640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0270107470 0x7f02701994a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.940+0000 7f0275fbe640 1 --2- 172.21.15.44:0/92926570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0270102610 0x7f0270198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:00.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.940+0000 7f0275fbe640 1 -- 172.21.15.44:0/92926570 >> 172.21.15.44:0/92926570 conn(0x7f0270076cc0 msgr2=0x7f0270105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:00.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.940+0000 7f0275fbe640 1 -- 172.21.15.44:0/92926570 shutdown_connections 2024-08-20T03:09:00.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:00.940+0000 7f0275fbe640 1 -- 172.21.15.44:0/92926570 wait complete. 2024-08-20T03:09:02.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:02 smithi044 ceph-mon[18790]: pgmap v516: 97 pgs: 97 active+clean; 830 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 989 op/s 2024-08-20T03:09:02.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:02 smithi165 ceph-mon[22357]: pgmap v516: 97 pgs: 97 active+clean; 830 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 989 op/s 2024-08-20T03:09:02.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:02 smithi099 ceph-mon[21984]: pgmap v516: 97 pgs: 97 active+clean; 830 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 989 op/s 2024-08-20T03:09:02.860 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T03:09:02.860 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:09:03.121 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:03.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.581+0000 7fe56d465640 1 -- 172.21.15.44:0/187502763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568107d20 msgr2=0x7fe56810e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:03.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.581+0000 7fe56d465640 1 --2- 172.21.15.44:0/187502763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568107d20 0x7fe56810e520 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe560009ca0 tx=0x7fe56002a8c0 comp rx=0 tx=0).stop 2024-08-20T03:09:03.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.581+0000 7fe56d465640 1 -- 172.21.15.44:0/187502763 shutdown_connections 2024-08-20T03:09:03.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.581+0000 7fe56d465640 1 --2- 172.21.15.44:0/187502763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568107d20 0x7fe56810e520 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:03.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.581+0000 7fe56d465640 1 --2- 172.21.15.44:0/187502763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5681073a0 0x7fe5681077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:03.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.581+0000 7fe56d465640 1 --2- 172.21.15.44:0/187502763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe568101990 0x7fe568101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:03.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.581+0000 7fe56d465640 1 -- 172.21.15.44:0/187502763 >> 172.21.15.44:0/187502763 conn(0x7fe5680fd400 msgr2=0x7fe5680ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.581+0000 7fe56d465640 1 -- 172.21.15.44:0/187502763 shutdown_connections 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.581+0000 7fe56d465640 1 -- 172.21.15.44:0/187502763 wait complete. 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.581+0000 7fe56d465640 1 Processor -- start 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.581+0000 7fe56d465640 1 -- start start 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.581+0000 7fe56d465640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe568101990 0x7fe568198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe56d465640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5681073a0 0x7fe568199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe56d465640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe568107d20 0x7fe56819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe56d465640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe568112ae0 con 0x7fe5681073a0 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe56d465640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe568112960 con 0x7fe568107d20 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe56d465640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe568112c60 con 0x7fe568101990 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe55ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5681073a0 0x7fe568199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe566ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe568101990 0x7fe568198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe566ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe568101990 0x7fe568198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47808/0 (socket says 172.21.15.44:47808) 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe566ffd640 1 -- 172.21.15.44:0/4066192932 learned_addr learned my addr 172.21.15.44:0/4066192932 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:03.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe55ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5681073a0 0x7fe568199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54408/0 (socket says 172.21.15.44:54408) 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe5677fe640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe568107d20 0x7fe56819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe55ffff640 1 -- 172.21.15.44:0/4066192932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe568101990 msgr2=0x7fe568198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe55ffff640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe568101990 0x7fe568198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe55ffff640 1 -- 172.21.15.44:0/4066192932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe568107d20 msgr2=0x7fe56819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe55ffff640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe568107d20 0x7fe56819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe55ffff640 1 -- 172.21.15.44:0/4066192932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56819efc0 con 0x7fe5681073a0 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe55ffff640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5681073a0 0x7fe568199450 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe55000c970 tx=0x7fe55000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe564ff9640 1 -- 172.21.15.44:0/4066192932 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550007bb0 con 0x7fe5681073a0 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe564ff9640 1 -- 172.21.15.44:0/4066192932 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe550007d50 con 0x7fe5681073a0 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe56d465640 1 -- 172.21.15.44:0/4066192932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56819f2b0 con 0x7fe5681073a0 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.582+0000 7fe564ff9640 1 -- 172.21.15.44:0/4066192932 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550004df0 con 0x7fe5681073a0 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.583+0000 7fe56d465640 1 -- 172.21.15.44:0/4066192932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5681a5970 con 0x7fe5681073a0 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.583+0000 7fe56d465640 1 -- 172.21.15.44:0/4066192932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe568101d70 con 0x7fe5681073a0 2024-08-20T03:09:03.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.585+0000 7fe564ff9640 1 -- 172.21.15.44:0/4066192932 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe550005000 con 0x7fe5681073a0 2024-08-20T03:09:03.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.586+0000 7fe564ff9640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe5540b3d40 0x7fe5540b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:03.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.586+0000 7fe566ffd640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe5540b3d40 0x7fe5540b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:03.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.586+0000 7fe564ff9640 1 -- 172.21.15.44:0/4066192932 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8405+0+0 (secure 0 0 0) 0x7fe5500e3ff0 con 0x7fe5681073a0 2024-08-20T03:09:03.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.586+0000 7fe566ffd640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe5540b3d40 0x7fe5540b6200 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe558009c90 tx=0x7fe558005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:03.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.589+0000 7fe564ff9640 1 -- 172.21.15.44:0/4066192932 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe550014030 con 0x7fe5681073a0 2024-08-20T03:09:03.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.931+0000 7fe56d465640 1 -- 172.21.15.44:0/4066192932 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe56819a040 con 0x7fe5681073a0 2024-08-20T03:09:03.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.931+0000 7fe564ff9640 1 -- 172.21.15.44:0/4066192932 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11976 (secure 0 0 0) 0x7fe5500aad20 con 0x7fe5681073a0 2024-08-20T03:09:03.934 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:09:03.934 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":33,"btime":"2024-08-20T03:09:03:207277+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:09:03.139393+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24604,24589,24596,24583,24569]},"id":1}]} 2024-08-20T03:09:03.934 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 33 2024-08-20T03:09:03.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.934+0000 7fe56d465640 1 -- 172.21.15.44:0/4066192932 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe5540b3d40 msgr2=0x7fe5540b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:03.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.934+0000 7fe56d465640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe5540b3d40 0x7fe5540b6200 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe558009c90 tx=0x7fe558005f90 comp rx=0 tx=0).stop 2024-08-20T03:09:03.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.934+0000 7fe56d465640 1 -- 172.21.15.44:0/4066192932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5681073a0 msgr2=0x7fe568199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:03.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.934+0000 7fe56d465640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5681073a0 0x7fe568199450 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe55000c970 tx=0x7fe55000ce30 comp rx=0 tx=0).stop 2024-08-20T03:09:03.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.934+0000 7fe56d465640 1 -- 172.21.15.44:0/4066192932 shutdown_connections 2024-08-20T03:09:03.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.934+0000 7fe56d465640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe5540b3d40 0x7fe5540b6200 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:03.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.934+0000 7fe56d465640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe568107d20 0x7fe56819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:03.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.934+0000 7fe56d465640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe5681073a0 0x7fe568199450 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:03.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.934+0000 7fe56d465640 1 --2- 172.21.15.44:0/4066192932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe568101990 0x7fe568198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:03.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.934+0000 7fe56d465640 1 -- 172.21.15.44:0/4066192932 >> 172.21.15.44:0/4066192932 conn(0x7fe5680fd400 msgr2=0x7fe5680ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:03.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.934+0000 7fe56d465640 1 -- 172.21.15.44:0/4066192932 shutdown_connections 2024-08-20T03:09:03.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:03.934+0000 7fe56d465640 1 -- 172.21.15.44:0/4066192932 wait complete. 2024-08-20T03:09:04.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:04 smithi044 ceph-mon[18790]: pgmap v517: 97 pgs: 97 active+clean; 835 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 798 op/s 2024-08-20T03:09:04.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:04 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:04.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:04 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/4066192932' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:09:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:04 smithi165 ceph-mon[22357]: pgmap v517: 97 pgs: 97 active+clean; 835 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 798 op/s 2024-08-20T03:09:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:04 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:04 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/4066192932' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:09:04.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:04 smithi099 ceph-mon[21984]: pgmap v517: 97 pgs: 97 active+clean; 835 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 798 op/s 2024-08-20T03:09:04.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:04 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:04.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:04 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/4066192932' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:09:04.672 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-20T03:09:04.966 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:05.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.523+0000 7f39af59e640 1 -- 172.21.15.44:0/417397901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b010a260 msgr2=0x7f39b010a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:05.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.523+0000 7f39af59e640 1 --2- 172.21.15.44:0/417397901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b010a260 0x7f39b010a6a0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009900 tx=0x7f39a002a400 comp rx=0 tx=0).stop 2024-08-20T03:09:05.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.526+0000 7f39af59e640 1 -- 172.21.15.44:0/417397901 shutdown_connections 2024-08-20T03:09:05.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.526+0000 7f39af59e640 1 --2- 172.21.15.44:0/417397901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b011bdd0 0x7f39b011e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.526+0000 7f39af59e640 1 --2- 172.21.15.44:0/417397901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b010a260 0x7f39b010a6a0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.526+0000 7f39af59e640 1 --2- 172.21.15.44:0/417397901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39b0109e80 0x7f39b0110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.526+0000 7f39af59e640 1 -- 172.21.15.44:0/417397901 >> 172.21.15.44:0/417397901 conn(0x7f39b006c830 msgr2=0x7f39b006cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:05.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.527+0000 7f39af59e640 1 -- 172.21.15.44:0/417397901 shutdown_connections 2024-08-20T03:09:05.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.529+0000 7f39af59e640 1 -- 172.21.15.44:0/417397901 wait complete. 2024-08-20T03:09:05.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.530+0000 7f39af59e640 1 Processor -- start 2024-08-20T03:09:05.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.530+0000 7f39af59e640 1 -- start start 2024-08-20T03:09:05.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.531+0000 7f39af59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39b0109e80 0x7f39b01a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.531+0000 7f39af59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b010a260 0x7f39b01a80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.531+0000 7f39af59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b011bdd0 0x7f39b01ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.531+0000 7f39af59e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f39b0121720 con 0x7f39b011bdd0 2024-08-20T03:09:05.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.532+0000 7f39add9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b010a260 0x7f39b01a80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:05.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.532+0000 7f39add9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b010a260 0x7f39b01a80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47830/0 (socket says 172.21.15.44:47830) 2024-08-20T03:09:05.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.532+0000 7f39add9b640 1 -- 172.21.15.44:0/2056054399 learned_addr learned my addr 172.21.15.44:0/2056054399 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:05.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.532+0000 7f39af59e640 1 -- 172.21.15.44:0/2056054399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01215a0 con 0x7f39b0109e80 2024-08-20T03:09:05.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.532+0000 7f39af59e640 1 -- 172.21.15.44:0/2056054399 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01218a0 con 0x7f39b010a260 2024-08-20T03:09:05.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.532+0000 7f39aed9d640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b011bdd0 0x7f39b01ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:05.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.533+0000 7f39aed9d640 1 -- 172.21.15.44:0/2056054399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b010a260 msgr2=0x7f39b01a80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:05.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.533+0000 7f39aed9d640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b010a260 0x7f39b01a80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.533+0000 7f39aed9d640 1 -- 172.21.15.44:0/2056054399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39b0109e80 msgr2=0x7f39b01a7ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T03:09:05.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.533+0000 7f39aed9d640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39b0109e80 0x7f39b01a7ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.533+0000 7f39aed9d640 1 -- 172.21.15.44:0/2056054399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b01adc50 con 0x7f39b011bdd0 2024-08-20T03:09:05.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.533+0000 7f39aed9d640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b011bdd0 0x7f39b01ad480 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f39a400d680 tx=0x7f39a400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:05.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.533+0000 7f399f7fe640 1 -- 172.21.15.44:0/2056054399 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4017070 con 0x7f39b011bdd0 2024-08-20T03:09:05.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.534+0000 7f399f7fe640 1 -- 172.21.15.44:0/2056054399 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a4009820 con 0x7f39b011bdd0 2024-08-20T03:09:05.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.534+0000 7f399f7fe640 1 -- 172.21.15.44:0/2056054399 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4004ce0 con 0x7f39b011bdd0 2024-08-20T03:09:05.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.534+0000 7f39af59e640 1 -- 172.21.15.44:0/2056054399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b01adf40 con 0x7f39b011bdd0 2024-08-20T03:09:05.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.534+0000 7f39af59e640 1 -- 172.21.15.44:0/2056054399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b01b4600 con 0x7f39b011bdd0 2024-08-20T03:09:05.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.538+0000 7f399f7fe640 1 -- 172.21.15.44:0/2056054399 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f39a40099d0 con 0x7f39b011bdd0 2024-08-20T03:09:05.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.539+0000 7f399f7fe640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f39980b3e60 0x7f39980b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.539+0000 7f399f7fe640 1 -- 172.21.15.44:0/2056054399 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f39a40e39c0 con 0x7f39b011bdd0 2024-08-20T03:09:05.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.539+0000 7f39ae59c640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f39980b3e60 0x7f39980b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:05.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.539+0000 7f39ae59c640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f39980b3e60 0x7f39980b6320 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f39a8005c40 tx=0x7f39a8005bd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:05.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.544+0000 7f39b6f58740 1 Processor -- start 2024-08-20T03:09:05.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.545+0000 7f39b6f58740 1 -- start start 2024-08-20T03:09:05.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.545+0000 7f39b6f58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877a910 0x55e65877acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.545+0000 7f39b6f58740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877b230 0x55e658786c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.545+0000 7f39b6f58740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e6587895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.545+0000 7f39b6f58740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e6586f1930 con 0x55e6587871c0 2024-08-20T03:09:05.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.545+0000 7f39b6f58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e6586e0e30 con 0x55e65877a910 2024-08-20T03:09:05.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.545+0000 7f39b6f58740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e6586e4e70 con 0x55e65877b230 2024-08-20T03:09:05.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.545+0000 7f39aed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e6587895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:05.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.545+0000 7f39aed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e6587895b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54446/0 (socket says 172.21.15.44:54446) 2024-08-20T03:09:05.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.546+0000 7f39aed9d640 1 -- 172.21.15.44:0/427236875 learned_addr learned my addr 172.21.15.44:0/427236875 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:05.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.546+0000 7f39add9b640 1 --2- 172.21.15.44:0/427236875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877b230 0x55e658786c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:05.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.547+0000 7f39add9b640 1 -- 172.21.15.44:0/427236875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877a910 msgr2=0x55e65877acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:05.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.547+0000 7f39add9b640 1 --2- 172.21.15.44:0/427236875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877a910 0x55e65877acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.547+0000 7f39add9b640 1 -- 172.21.15.44:0/427236875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 msgr2=0x55e6587895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:05.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.547+0000 7f39add9b640 1 --2- 172.21.15.44:0/427236875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e6587895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.547+0000 7f39add9b640 1 -- 172.21.15.44:0/427236875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e658440650 con 0x55e65877b230 2024-08-20T03:09:05.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.549+0000 7f39add9b640 1 --2- 172.21.15.44:0/427236875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877b230 0x55e658786c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009230 tx=0x7f39a0032bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=360e1e2859842bf4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:05.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.551+0000 7f399cff9640 1 -- 172.21.15.44:0/427236875 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0035030 con 0x55e65877b230 2024-08-20T03:09:05.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.551+0000 7f399cff9640 1 -- 172.21.15.44:0/427236875 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a0004030 con 0x55e65877b230 2024-08-20T03:09:05.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.551+0000 7f399cff9640 1 -- 172.21.15.44:0/427236875 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0005590 con 0x55e65877b230 2024-08-20T03:09:05.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.553+0000 7f39b6f58740 1 -- 172.21.15.44:0/427236875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877b230 msgr2=0x55e658786c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.553+0000 7f39b6f58740 1 --2- 172.21.15.44:0/427236875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877b230 0x55e658786c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009230 tx=0x7f39a0032bb0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.553+0000 7f39b6f58740 1 -- 172.21.15.44:0/427236875 shutdown_connections 2024-08-20T03:09:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.553+0000 7f39b6f58740 1 --2- 172.21.15.44:0/427236875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e6587895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.553+0000 7f39b6f58740 1 --2- 172.21.15.44:0/427236875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877b230 0x55e658786c80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.553+0000 7f39b6f58740 1 --2- 172.21.15.44:0/427236875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877a910 0x55e65877acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.553+0000 7f39b6f58740 1 -- 172.21.15.44:0/427236875 >> 172.21.15.44:0/427236875 conn(0x55e658778740 msgr2=0x55e6587798e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:05.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.555+0000 7f39b6f58740 1 -- 172.21.15.44:0/427236875 shutdown_connections 2024-08-20T03:09:05.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.555+0000 7f39b6f58740 1 -- 172.21.15.44:0/427236875 wait complete. 2024-08-20T03:09:05.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.555+0000 7f39b6f58740 1 -- start start 2024-08-20T03:09:05.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.556+0000 7f39b6f58740 1 Processor -- start 2024-08-20T03:09:05.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.556+0000 7f39b6f58740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877a910 0x55e6587841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.556+0000 7f39b6f58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877b230 0x55e658784700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.556+0000 7f39b6f58740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e658784c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.556+0000 7f39b6f58740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e658543700 con 0x55e6587871c0 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.556+0000 7f39b6f58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e658667810 con 0x55e65877b230 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.556+0000 7f39b6f58740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e658789e10 con 0x55e65877a910 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.556+0000 7f39b6f58740 10 client.0 _ll_register_callbacks cb 0x55e6586da700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.556+0000 7f39add9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877b230 0x55e658784700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.556+0000 7f39add9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877b230 0x55e658784700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53546/0 (socket says 172.21.15.44:53546) 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.556+0000 7f39add9b640 1 -- 172.21.15.44:0/3326073772 learned_addr learned my addr 172.21.15.44:0/3326073772 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.556+0000 7f39aed9d640 1 --2- 172.21.15.44:0/3326073772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e658784c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.557+0000 7f39aed9d640 1 -- 172.21.15.44:0/3326073772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877a910 msgr2=0x55e6587841c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.557+0000 7f39aed9d640 1 --2- 172.21.15.44:0/3326073772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877a910 0x55e6587841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.557+0000 7f39aed9d640 1 -- 172.21.15.44:0/3326073772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877b230 msgr2=0x55e658784700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:05.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.557+0000 7f39aed9d640 1 --2- 172.21.15.44:0/3326073772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877b230 0x55e658784700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.557+0000 7f39aed9d640 1 -- 172.21.15.44:0/3326073772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6587a4250 con 0x55e6587871c0 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.557+0000 7f39aed9d640 1 --2- 172.21.15.44:0/3326073772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e658784c40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f39a4005880 tx=0x7f39a40aa160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5891a8d4a3812e05 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.557+0000 7f3995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.558+0000 7f3995ffb640 1 -- 172.21.15.44:0/3326073772 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a40c4060 con 0x55e6587871c0 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.558+0000 7f3995ffb640 1 -- 172.21.15.44:0/3326073772 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f39a40e1350 con 0x55e6587871c0 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.558+0000 7f39b6f58740 1 -- 172.21.15.44:0/3326073772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e658690150 con 0x55e6587871c0 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.558+0000 7f3995ffb640 1 -- 172.21.15.44:0/3326073772 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f39a4020040 con 0x55e6587871c0 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.558+0000 7f39b6f58740 10 client.16809 fetch_fsmap learned FSMap version 33 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.558+0000 7f39b6f58740 1 -- 172.21.15.44:0/3326073772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e658793140 con 0x55e6587871c0 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.559+0000 7f3995ffb640 1 -- 172.21.15.44:0/3326073772 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a40e1550 con 0x55e6587871c0 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.559+0000 7f3995ffb640 1 -- 172.21.15.44:0/3326073772 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f39a40a9260 con 0x55e6587871c0 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.559+0000 7f39b6f58740 10 client.16809 fetch_fsmap finished waiting for FSMap version 33 2024-08-20T03:09:05.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.559+0000 7f39b6f58740 10 client.16809 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:09:05.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.559+0000 7f39b6f58740 20 client.16809 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:05.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.559+0000 7f39b6f58740 1 --2- 172.21.15.44:0/3326073772 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55e6587945a0 0x55e6587969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.559+0000 7f39b6f58740 4 client.16809 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:05.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.559+0000 7f39b6f58740 1 -- 172.21.15.44:0/3326073772 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e658690150 con 0x55e6587945a0 2024-08-20T03:09:05.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.563+0000 7f39ae59c640 1 --2- 172.21.15.44:0/3326073772 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55e6587945a0 0x55e6587969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:05.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.565+0000 7f39ae59c640 1 --2- 172.21.15.44:0/3326073772 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55e6587945a0 0x55e6587969f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3520978388f4d04e server_cookie=d65a227095b0f261 in_seq=0 out_seq=0 2024-08-20T03:09:05.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.565+0000 7f3995ffb640 10 client.16809 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:09:05.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.571+0000 7f3995ffb640 1 -- 172.21.15.44:0/3326073772 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e658690150 con 0x55e6587945a0 2024-08-20T03:09:05.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.571+0000 7f3995ffb640 10 client.16809 handle_command_reply: tid=0 2024-08-20T03:09:05.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.571+0000 7f39b6f58740 1 client.16809 shutdown 2024-08-20T03:09:05.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.572+0000 7f39b6f58740 1 -- 172.21.15.44:0/3326073772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 msgr2=0x55e658784c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:05.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.572+0000 7f39b6f58740 1 --2- 172.21.15.44:0/3326073772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e658784c40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f39a4005880 tx=0x7f39a40aa160 comp rx=0 tx=0).stop 2024-08-20T03:09:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.573+0000 7f39b6f58740 1 -- 172.21.15.44:0/3326073772 shutdown_connections 2024-08-20T03:09:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.573+0000 7f39b6f58740 1 --2- 172.21.15.44:0/3326073772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e658784c40 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.573+0000 7f39b6f58740 1 --2- 172.21.15.44:0/3326073772 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55e6587945a0 0x55e6587969f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.573+0000 7f39b6f58740 1 --2- 172.21.15.44:0/3326073772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877b230 0x55e658784700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.573+0000 7f3995ffb640 0 client.16809 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:09:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.573+0000 7f39b6f58740 1 --2- 172.21.15.44:0/3326073772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877a910 0x55e6587841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.573+0000 7f39b6f58740 1 -- 172.21.15.44:0/3326073772 >> 172.21.15.44:0/3326073772 conn(0x55e658778740 msgr2=0x55e65877e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:05.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.574+0000 7f39b6f58740 1 -- 172.21.15.44:0/3326073772 shutdown_connections 2024-08-20T03:09:05.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.574+0000 7f39b6f58740 1 -- 172.21.15.44:0/3326073772 wait complete. 2024-08-20T03:09:05.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.575+0000 7f39b6f58740 20 client.16809 trim_cache size 0 max 16384 2024-08-20T03:09:05.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.606+0000 7f39b6f58740 1 Processor -- start 2024-08-20T03:09:05.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.606+0000 7f39b6f58740 1 -- start start 2024-08-20T03:09:05.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.606+0000 7f39b6f58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877a910 0x55e65879c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.606+0000 7f39b6f58740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e65877b230 0x55e65879c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.606+0000 7f39b6f58740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e6587871c0 0x55e6587813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.606+0000 7f39b6f58740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01b48e0 con 0x55e65877b230 2024-08-20T03:09:05.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.606+0000 7f39b6f58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01b6540 con 0x55e65877a910 2024-08-20T03:09:05.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.606+0000 7f39b6f58740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01b4fb0 con 0x55e6587871c0 2024-08-20T03:09:05.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.606+0000 7f39add9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e65877b230 0x55e65879c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:05.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.606+0000 7f39add9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e65877b230 0x55e65879c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54468/0 (socket says 172.21.15.44:54468) 2024-08-20T03:09:05.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.606+0000 7f39add9b640 1 -- 172.21.15.44:0/3918837566 learned_addr learned my addr 172.21.15.44:0/3918837566 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:05.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.607+0000 7f39add9b640 1 -- 172.21.15.44:0/3918837566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e6587871c0 msgr2=0x55e6587813d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:05.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.607+0000 7f39add9b640 1 --2- 172.21.15.44:0/3918837566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e6587871c0 0x55e6587813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.607+0000 7f39add9b640 1 -- 172.21.15.44:0/3918837566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877a910 msgr2=0x55e65879c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:05.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.607+0000 7f39add9b640 1 --2- 172.21.15.44:0/3918837566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877a910 0x55e65879c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.607+0000 7f39add9b640 1 -- 172.21.15.44:0/3918837566 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6587791c0 con 0x55e65877b230 2024-08-20T03:09:05.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.609+0000 7f39add9b640 1 --2- 172.21.15.44:0/3918837566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e65877b230 0x55e65879c800 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f39a0032fd0 tx=0x7f39a0032b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e3186a2124031c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:05.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.610+0000 7f3997fff640 1 -- 172.21.15.44:0/3918837566 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0035040 con 0x55e65877b230 2024-08-20T03:09:05.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.610+0000 7f3997fff640 1 -- 172.21.15.44:0/3918837566 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a004b070 con 0x55e65877b230 2024-08-20T03:09:05.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.610+0000 7f3997fff640 1 -- 172.21.15.44:0/3918837566 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0004920 con 0x55e65877b230 2024-08-20T03:09:05.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.611+0000 7f39b6f58740 1 -- 172.21.15.44:0/3918837566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e65877b230 msgr2=0x55e65879c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:05.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.611+0000 7f39b6f58740 1 --2- 172.21.15.44:0/3918837566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e65877b230 0x55e65879c800 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f39a0032fd0 tx=0x7f39a0032b30 comp rx=0 tx=0).stop 2024-08-20T03:09:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.615+0000 7f39b6f58740 1 -- 172.21.15.44:0/3918837566 shutdown_connections 2024-08-20T03:09:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.615+0000 7f39b6f58740 1 --2- 172.21.15.44:0/3918837566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e6587871c0 0x55e6587813d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.615+0000 7f39b6f58740 1 --2- 172.21.15.44:0/3918837566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e65877b230 0x55e65879c800 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.615+0000 7f39b6f58740 1 --2- 172.21.15.44:0/3918837566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877a910 0x55e65879c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.615+0000 7f39b6f58740 1 -- 172.21.15.44:0/3918837566 >> 172.21.15.44:0/3918837566 conn(0x55e658778740 msgr2=0x55e6587776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:05.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.616+0000 7f39b6f58740 1 -- 172.21.15.44:0/3918837566 shutdown_connections 2024-08-20T03:09:05.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.616+0000 7f39b6f58740 1 -- 172.21.15.44:0/3918837566 wait complete. 2024-08-20T03:09:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.616+0000 7f39b6f58740 1 -- start start 2024-08-20T03:09:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.617+0000 7f39b6f58740 1 Processor -- start 2024-08-20T03:09:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.617+0000 7f39b6f58740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877a910 0x55e65878f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.617+0000 7f39b6f58740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877b230 0x55e65878ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.617+0000 7f39b6f58740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e65878bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.617+0000 7f39b6f58740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01b5db0 con 0x55e6587871c0 2024-08-20T03:09:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.617+0000 7f39b6f58740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01b5680 con 0x55e65877a910 2024-08-20T03:09:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.617+0000 7f39b6f58740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f39b0117f90 con 0x55e65877b230 2024-08-20T03:09:05.620 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.617+0000 7f39b6f58740 10 client.0 _ll_register_callbacks cb 0x7f39a4004e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.618+0000 7f39aed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e65878bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.618+0000 7f39aed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e65878bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54484/0 (socket says 172.21.15.44:54484) 2024-08-20T03:09:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.618+0000 7f39aed9d640 1 -- 172.21.15.44:0/806962605 learned_addr learned my addr 172.21.15.44:0/806962605 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:05.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.619+0000 7f39aed9d640 1 -- 172.21.15.44:0/806962605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877b230 msgr2=0x55e65878ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:09:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.619+0000 7f39aed9d640 1 --2- 172.21.15.44:0/806962605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877b230 0x55e65878ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.619+0000 7f39aed9d640 1 -- 172.21.15.44:0/806962605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877a910 msgr2=0x55e65878f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.619+0000 7f39aed9d640 1 --2- 172.21.15.44:0/806962605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877a910 0x55e65878f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.619+0000 7f39aed9d640 1 -- 172.21.15.44:0/806962605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65878c550 con 0x55e6587871c0 2024-08-20T03:09:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.619+0000 7f39aed9d640 1 --2- 172.21.15.44:0/806962605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e65878bf40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f39a40aa290 tx=0x7f39a40e1580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde86d3f5519e2d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:05.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.619+0000 7f3995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:05.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.622+0000 7f3995ffb640 1 -- 172.21.15.44:0/806962605 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a40c4070 con 0x55e6587871c0 2024-08-20T03:09:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.623+0000 7f3995ffb640 1 -- 172.21.15.44:0/806962605 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f39a40a6b80 con 0x55e6587871c0 2024-08-20T03:09:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.623+0000 7f39b6f58740 1 -- 172.21.15.44:0/806962605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39a004b070 con 0x55e6587871c0 2024-08-20T03:09:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.623+0000 7f3995ffb640 1 -- 172.21.15.44:0/806962605 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f39a40a9ee0 con 0x55e6587871c0 2024-08-20T03:09:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.623+0000 7f39b6f58740 10 client.16815 fetch_fsmap learned FSMap version 33 2024-08-20T03:09:05.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.623+0000 7f39b6f58740 1 -- 172.21.15.44:0/806962605 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e658801c20 con 0x55e6587871c0 2024-08-20T03:09:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.625+0000 7f3995ffb640 1 -- 172.21.15.44:0/806962605 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a004b070 con 0x55e6587871c0 2024-08-20T03:09:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.625+0000 7f3995ffb640 1 -- 172.21.15.44:0/806962605 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f39a40afd20 con 0x55e6587871c0 2024-08-20T03:09:05.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.626+0000 7f39b6f58740 10 client.16815 fetch_fsmap finished waiting for FSMap version 33 2024-08-20T03:09:05.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.626+0000 7f39b6f58740 10 client.16815 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:09:05.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.626+0000 7f39b6f58740 20 client.16815 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:05.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.626+0000 7f39b6f58740 1 --2- 172.21.15.44:0/806962605 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55e6587945a0 0x55e658804170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:05.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.626+0000 7f39b6f58740 4 client.16815 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:09:05.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.626+0000 7f39b6f58740 1 -- 172.21.15.44:0/806962605 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6586f3880 con 0x55e6587945a0 2024-08-20T03:09:05.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.627+0000 7f39ae59c640 1 --2- 172.21.15.44:0/806962605 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55e6587945a0 0x55e658804170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:05.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.629+0000 7f39ae59c640 1 --2- 172.21.15.44:0/806962605 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55e6587945a0 0x55e658804170 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9649a3461fd869ef server_cookie=91822497a85d0ba in_seq=0 out_seq=0 2024-08-20T03:09:05.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.629+0000 7f3995ffb640 10 client.16815 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:09:05.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.629+0000 7f3995ffb640 1 -- 172.21.15.44:0/806962605 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6586f3880 con 0x55e6587945a0 2024-08-20T03:09:05.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.629+0000 7f3995ffb640 10 client.16815 handle_command_reply: tid=0 2024-08-20T03:09:05.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.629+0000 7f39b6f58740 1 client.16815 shutdown 2024-08-20T03:09:05.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.633+0000 7f39b6f58740 1 -- 172.21.15.44:0/806962605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 msgr2=0x55e65878bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.633+0000 7f39b6f58740 1 --2- 172.21.15.44:0/806962605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e65878bf40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f39a40aa290 tx=0x7f39a40e1580 comp rx=0 tx=0).stop 2024-08-20T03:09:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.633+0000 7f39b6f58740 1 -- 172.21.15.44:0/806962605 shutdown_connections 2024-08-20T03:09:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.633+0000 7f39b6f58740 1 --2- 172.21.15.44:0/806962605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e6587871c0 0x55e65878bf40 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.633+0000 7f39b6f58740 1 --2- 172.21.15.44:0/806962605 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e65877b230 0x55e65878ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.633+0000 7f39b6f58740 1 --2- 172.21.15.44:0/806962605 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55e6587945a0 0x55e658804170 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.633+0000 7f39b6f58740 1 --2- 172.21.15.44:0/806962605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e65877a910 0x55e65878f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.633+0000 7f3995ffb640 0 client.16815 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:09:05.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.633+0000 7f39b6f58740 1 -- 172.21.15.44:0/806962605 >> 172.21.15.44:0/806962605 conn(0x55e658778740 msgr2=0x55e6587857f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.633+0000 7f39b6f58740 1 -- 172.21.15.44:0/806962605 shutdown_connections 2024-08-20T03:09:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.635+0000 7f39b6f58740 1 -- 172.21.15.44:0/806962605 wait complete. 2024-08-20T03:09:05.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.635+0000 7f39b6f58740 20 client.16815 trim_cache size 0 max 16384 2024-08-20T03:09:05.638 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:09:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.636+0000 7f39af59e640 1 -- 172.21.15.44:0/2056054399 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f39980b3e60 msgr2=0x7f39980b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.636+0000 7f39af59e640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f39980b3e60 0x7f39980b6320 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f39a8005c40 tx=0x7f39a8005bd0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.636+0000 7f39af59e640 1 -- 172.21.15.44:0/2056054399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b011bdd0 msgr2=0x7f39b01ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.636+0000 7f39af59e640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b011bdd0 0x7f39b01ad480 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f39a400d680 tx=0x7f39a400db40 comp rx=0 tx=0).stop 2024-08-20T03:09:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.636+0000 7f39af59e640 1 -- 172.21.15.44:0/2056054399 shutdown_connections 2024-08-20T03:09:05.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.636+0000 7f39af59e640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f39980b3e60 0x7f39980b6320 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.636+0000 7f39af59e640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39b011bdd0 0x7f39b01ad480 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.636+0000 7f39af59e640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b010a260 0x7f39b01a80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.636+0000 7f39af59e640 1 --2- 172.21.15.44:0/2056054399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39b0109e80 0x7f39b01a7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.636+0000 7f39af59e640 1 -- 172.21.15.44:0/2056054399 >> 172.21.15.44:0/2056054399 conn(0x7f39b006c830 msgr2=0x7f39b011c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.637+0000 7f39af59e640 1 -- 172.21.15.44:0/2056054399 shutdown_connections 2024-08-20T03:09:05.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:05.637+0000 7f39af59e640 1 -- 172.21.15.44:0/2056054399 wait complete. 2024-08-20T03:09:06.264 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:06 smithi044 ceph-mon[18790]: pgmap v518: 97 pgs: 97 active+clean; 851 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 712 op/s 2024-08-20T03:09:06.264 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:06 smithi044 ceph-mon[18790]: osdmap e78: 12 total, 12 up, 12 in 2024-08-20T03:09:06.452 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-20T03:09:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.545+0000 7f64f9bc2640 1 -- 172.21.15.44:0/954576203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64f40755f0 msgr2=0x7f64f407bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.545+0000 7f64f9bc2640 1 --2- 172.21.15.44:0/954576203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64f40755f0 0x7f64f407bd40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009be0 tx=0x7f64ec02a6c0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.545+0000 7f64f9bc2640 1 -- 172.21.15.44:0/954576203 shutdown_connections 2024-08-20T03:09:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.545+0000 7f64f9bc2640 1 --2- 172.21.15.44:0/954576203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64f40755f0 0x7f64f407bd40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.545+0000 7f64f9bc2640 1 --2- 172.21.15.44:0/954576203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64f4074c70 0x7f64f40750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.545+0000 7f64f9bc2640 1 --2- 172.21.15.44:0/954576203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f410a0f0 0x7f64f410a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.545+0000 7f64f9bc2640 1 -- 172.21.15.44:0/954576203 >> 172.21.15.44:0/954576203 conn(0x7f64f406daa0 msgr2=0x7f64f406deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.548+0000 7f64f9bc2640 1 -- 172.21.15.44:0/954576203 shutdown_connections 2024-08-20T03:09:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.548+0000 7f64f9bc2640 1 -- 172.21.15.44:0/954576203 wait complete. 2024-08-20T03:09:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.549+0000 7f64f9bc2640 1 Processor -- start 2024-08-20T03:09:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.553+0000 7f64f9bc2640 1 -- start start 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.553+0000 7f64f9bc2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64f4074c70 0x7f64f4085580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.553+0000 7f64f9bc2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f40755f0 0x7f64f4085ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.553+0000 7f64f9bc2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64f410a0f0 0x7f64f4080960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.553+0000 7f64f9bc2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f64f407ef20 con 0x7f64f4074c70 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.553+0000 7f64f9bc2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64f407eda0 con 0x7f64f40755f0 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.553+0000 7f64f9bc2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64f407f0a0 con 0x7f64f410a0f0 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.553+0000 7f64f37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64f4074c70 0x7f64f4085580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.553+0000 7f64f37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64f4074c70 0x7f64f4085580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54502/0 (socket says 172.21.15.44:54502) 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.554+0000 7f64f37fe640 1 -- 172.21.15.44:0/248824420 learned_addr learned my addr 172.21.15.44:0/248824420 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.554+0000 7f64f2ffd640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f40755f0 0x7f64f4085ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.554+0000 7f64f37fe640 1 -- 172.21.15.44:0/248824420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64f410a0f0 msgr2=0x7f64f4080960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.554+0000 7f64f37fe640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64f410a0f0 0x7f64f4080960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.554+0000 7f64f37fe640 1 -- 172.21.15.44:0/248824420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f40755f0 msgr2=0x7f64f4085ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.554+0000 7f64f37fe640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f40755f0 0x7f64f4085ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.554+0000 7f64f37fe640 1 -- 172.21.15.44:0/248824420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f4081220 con 0x7f64f4074c70 2024-08-20T03:09:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.556+0000 7f64f37fe640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64f4074c70 0x7f64f4085580 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f64e400d950 tx=0x7f64e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.557+0000 7f64f0ff9640 1 -- 172.21.15.44:0/248824420 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4017070 con 0x7f64f4074c70 2024-08-20T03:09:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.557+0000 7f64f0ff9640 1 -- 172.21.15.44:0/248824420 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e4009c30 con 0x7f64f4074c70 2024-08-20T03:09:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.557+0000 7f64f0ff9640 1 -- 172.21.15.44:0/248824420 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e400f620 con 0x7f64f4074c70 2024-08-20T03:09:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.558+0000 7f64f9bc2640 1 -- 172.21.15.44:0/248824420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f4081510 con 0x7f64f4074c70 2024-08-20T03:09:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.558+0000 7f64f9bc2640 1 -- 172.21.15.44:0/248824420 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f4131180 con 0x7f64f4074c70 2024-08-20T03:09:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.564+0000 7f64fca8d740 1 Processor -- start 2024-08-20T03:09:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.564+0000 7f64fca8d740 1 -- start start 2024-08-20T03:09:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.564+0000 7f64fca8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c601ebf0 0x5641c601efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.564+0000 7f64fca8d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c602af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.564+0000 7f64fca8d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c602b4a0 0x5641c602d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.564+0000 7f64fca8d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5641c5f94360 con 0x5641c602b4a0 2024-08-20T03:09:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.564+0000 7f64fca8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5641c5f83860 con 0x5641c601ebf0 2024-08-20T03:09:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.564+0000 7f64fca8d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641c5f878a0 con 0x5641c601f510 2024-08-20T03:09:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.566+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c601ebf0 0x5641c601efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.566+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c601ebf0 0x5641c601efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53604/0 (socket says 172.21.15.44:53604) 2024-08-20T03:09:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.566+0000 7f64f3fff640 1 -- 172.21.15.44:0/3370048765 learned_addr learned my addr 172.21.15.44:0/3370048765 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.567+0000 7f64f37fe640 1 --2- 172.21.15.44:0/3370048765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c602af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.567+0000 7f64f0ff9640 1 -- 172.21.15.44:0/248824420 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f64e400f7c0 con 0x7f64f4074c70 2024-08-20T03:09:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.567+0000 7f64f3fff640 1 -- 172.21.15.44:0/3370048765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 msgr2=0x5641c602af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.568+0000 7f64f3fff640 1 --2- 172.21.15.44:0/3370048765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c602af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.568+0000 7f64f3fff640 1 -- 172.21.15.44:0/3370048765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c602b4a0 msgr2=0x5641c602d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.568+0000 7f64f3fff640 1 --2- 172.21.15.44:0/3370048765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c602b4a0 0x5641c602d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.568+0000 7f64f3fff640 1 -- 172.21.15.44:0/3370048765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641c5f2ae40 con 0x5641c601ebf0 2024-08-20T03:09:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.568+0000 7f64f0ff9640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f64dc0b3f10 0x7f64dc0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.568+0000 7f64f0ff9640 1 -- 172.21.15.44:0/248824420 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f64e40e3a00 con 0x7f64f4074c70 2024-08-20T03:09:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.568+0000 7f64f37fe640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f64dc0b3f10 0x7f64dc0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.568+0000 7f64f3fff640 1 --2- 172.21.15.44:0/3370048765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c601ebf0 0x5641c601efd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f64ec004af0 tx=0x7f64ec030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b15928816382bd0a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.569+0000 7f64d9ffb640 1 -- 172.21.15.44:0/3370048765 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec040020 con 0x5641c601ebf0 2024-08-20T03:09:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.569+0000 7f64d9ffb640 1 -- 172.21.15.44:0/3370048765 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64ec003e20 con 0x5641c601ebf0 2024-08-20T03:09:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.570+0000 7f64fca8d740 1 -- 172.21.15.44:0/3370048765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c601ebf0 msgr2=0x5641c601efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.570+0000 7f64fca8d740 1 --2- 172.21.15.44:0/3370048765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c601ebf0 0x5641c601efd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f64ec004af0 tx=0x7f64ec030430 comp rx=0 tx=0).stop 2024-08-20T03:09:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.571+0000 7f64fca8d740 1 -- 172.21.15.44:0/3370048765 shutdown_connections 2024-08-20T03:09:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.571+0000 7f64fca8d740 1 --2- 172.21.15.44:0/3370048765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c602b4a0 0x5641c602d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.571+0000 7f64fca8d740 1 --2- 172.21.15.44:0/3370048765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c602af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.571+0000 7f64fca8d740 1 --2- 172.21.15.44:0/3370048765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c601ebf0 0x5641c601efd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.571+0000 7f64fca8d740 1 -- 172.21.15.44:0/3370048765 >> 172.21.15.44:0/3370048765 conn(0x5641c601ca20 msgr2=0x5641c601dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.572+0000 7f64f37fe640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f64dc0b3f10 0x7f64dc0b63d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5641c6021d70 tx=0x7f64e4005f70 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.576+0000 7f64fca8d740 1 -- 172.21.15.44:0/3370048765 shutdown_connections 2024-08-20T03:09:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.579+0000 7f64fca8d740 1 -- 172.21.15.44:0/3370048765 wait complete. 2024-08-20T03:09:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.579+0000 7f64fca8d740 1 -- start start 2024-08-20T03:09:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.579+0000 7f64fca8d740 1 Processor -- start 2024-08-20T03:09:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.580+0000 7f64fca8d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c601f510 0x5641c602aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.580+0000 7f64fca8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c602b4a0 0x5641c6027f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.580+0000 7f64fca8d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c6028470 0x5641c60288f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.580+0000 7f64fca8d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5641c5ebcab0 con 0x5641c601f510 2024-08-20T03:09:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.580+0000 7f64fca8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5641c5f0a780 con 0x5641c602b4a0 2024-08-20T03:09:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.580+0000 7f64fca8d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641c602e1c0 con 0x5641c6028470 2024-08-20T03:09:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.580+0000 7f64fca8d740 10 client.0 _ll_register_callbacks cb 0x5641c5c92a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.581+0000 7f64f2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c602b4a0 0x5641c6027f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.581+0000 7f64f2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c602b4a0 0x5641c6027f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:53608/0 (socket says 172.21.15.44:53608) 2024-08-20T03:09:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.581+0000 7f64f2ffd640 1 -- 172.21.15.44:0/1816677191 learned_addr learned my addr 172.21.15.44:0/1816677191 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.581+0000 7f64f3fff640 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c6028470 0x5641c60288f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.581+0000 7f64f3fff640 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c601f510 0x5641c602aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.582+0000 7f64f3fff640 1 -- 172.21.15.44:0/1816677191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c6028470 msgr2=0x5641c60288f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.582+0000 7f64f3fff640 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c6028470 0x5641c60288f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.582+0000 7f64f3fff640 1 -- 172.21.15.44:0/1816677191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c602b4a0 msgr2=0x5641c6027f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.582+0000 7f64f3fff640 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c602b4a0 0x5641c6027f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.582+0000 7f64f3fff640 1 -- 172.21.15.44:0/1816677191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641c5ce6790 con 0x5641c601f510 2024-08-20T03:09:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.583+0000 7f64f3fff640 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c601f510 0x5641c602aaf0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009be0 tx=0x7f64ec005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663d6a02fb01defc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.583+0000 7f64d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.584+0000 7f64d2ffd640 1 -- 172.21.15.44:0/1816677191 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec040030 con 0x5641c601f510 2024-08-20T03:09:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.584+0000 7f64fca8d740 1 -- 172.21.15.44:0/1816677191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5641c5f32e00 con 0x5641c601f510 2024-08-20T03:09:06.586 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.584+0000 7f64d2ffd640 1 -- 172.21.15.44:0/1816677191 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f64ec004360 con 0x5641c601f510 2024-08-20T03:09:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.584+0000 7f64d2ffd640 1 -- 172.21.15.44:0/1816677191 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f64ec057050 con 0x5641c601f510 2024-08-20T03:09:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.585+0000 7f64fca8d740 10 client.16824 fetch_fsmap learned FSMap version 33 2024-08-20T03:09:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.585+0000 7f64fca8d740 1 -- 172.21.15.44:0/1816677191 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641c5d5f620 con 0x5641c601f510 2024-08-20T03:09:06.587 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.585+0000 7f64d2ffd640 1 -- 172.21.15.44:0/1816677191 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5641c5f32e00 con 0x5641c601f510 2024-08-20T03:09:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.586+0000 7f64d2ffd640 1 -- 172.21.15.44:0/1816677191 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f64ec03c5b0 con 0x5641c601f510 2024-08-20T03:09:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.586+0000 7f64fca8d740 10 client.16824 fetch_fsmap finished waiting for FSMap version 33 2024-08-20T03:09:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.586+0000 7f64fca8d740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.586+0000 7f64fca8d740 20 client.16824 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:06.588 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.586+0000 7f64fca8d740 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641c603b260 0x5641c603d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.586+0000 7f64fca8d740 4 client.16824 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:06.589 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.586+0000 7f64fca8d740 1 -- 172.21.15.44:0/1816677191 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641c5f962b0 con 0x5641c603b260 2024-08-20T03:09:06.590 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.588+0000 7f64f2ffd640 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641c603b260 0x5641c603d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.590+0000 7f64f2ffd640 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641c603b260 0x5641c603d700 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e363fa5de57fe4 server_cookie=9df30c8387ab5612 in_seq=0 out_seq=0 2024-08-20T03:09:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.590+0000 7f64d2ffd640 10 client.16824 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.594+0000 7f64d2ffd640 1 -- 172.21.15.44:0/1816677191 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641c5f962b0 con 0x5641c603b260 2024-08-20T03:09:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.594+0000 7f64d2ffd640 10 client.16824 handle_command_reply: tid=0 2024-08-20T03:09:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.594+0000 7f64fca8d740 1 client.16824 shutdown 2024-08-20T03:09:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.596+0000 7f64fca8d740 1 -- 172.21.15.44:0/1816677191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c601f510 msgr2=0x5641c602aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.596+0000 7f64fca8d740 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c601f510 0x5641c602aaf0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009be0 tx=0x7f64ec005fd0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.596+0000 7f64fca8d740 1 -- 172.21.15.44:0/1816677191 shutdown_connections 2024-08-20T03:09:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.596+0000 7f64fca8d740 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641c603b260 0x5641c603d700 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.596+0000 7f64fca8d740 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c6028470 0x5641c60288f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.596+0000 7f64fca8d740 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c602b4a0 0x5641c6027f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.596+0000 7f64fca8d740 1 --2- 172.21.15.44:0/1816677191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c601f510 0x5641c602aaf0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.596+0000 7f64fca8d740 1 -- 172.21.15.44:0/1816677191 >> 172.21.15.44:0/1816677191 conn(0x5641c601ca20 msgr2=0x5641c6021da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.596+0000 7f64d2ffd640 0 client.16824 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.596+0000 7f64fca8d740 1 -- 172.21.15.44:0/1816677191 shutdown_connections 2024-08-20T03:09:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.596+0000 7f64fca8d740 1 -- 172.21.15.44:0/1816677191 wait complete. 2024-08-20T03:09:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.597+0000 7f64fca8d740 20 client.16824 trim_cache size 0 max 16384 2024-08-20T03:09:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.633+0000 7f64fca8d740 1 Processor -- start 2024-08-20T03:09:06.635 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:06 smithi165 ceph-mon[22357]: pgmap v518: 97 pgs: 97 active+clean; 851 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 712 op/s 2024-08-20T03:09:06.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:06 smithi165 ceph-mon[22357]: osdmap e78: 12 total, 12 up, 12 in 2024-08-20T03:09:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.633+0000 7f64fca8d740 1 -- start start 2024-08-20T03:09:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64fca8d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c6027670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64fca8d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c6028470 0x5641c6042fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64fca8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c602b4a0 0x5641c6043510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64fca8d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f64e000e730 con 0x5641c6028470 2024-08-20T03:09:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64fca8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64e0010390 con 0x5641c602b4a0 2024-08-20T03:09:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64fca8d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64e000ee00 con 0x5641c601f510 2024-08-20T03:09:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c6027670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c6027670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:47914/0 (socket says 172.21.15.44:47914) 2024-08-20T03:09:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64f3fff640 1 -- 172.21.15.44:0/3690906515 learned_addr learned my addr 172.21.15.44:0/3690906515 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64f3fff640 1 --2- 172.21.15.44:0/3690906515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c602b4a0 0x5641c6043510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64f3fff640 1 -- 172.21.15.44:0/3690906515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c602b4a0 msgr2=0x5641c6043510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64f3fff640 1 --2- 172.21.15.44:0/3690906515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c602b4a0 0x5641c6043510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64f3fff640 1 -- 172.21.15.44:0/3690906515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c6028470 msgr2=0x5641c6042fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:09:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64f3fff640 1 --2- 172.21.15.44:0/3690906515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c6028470 0x5641c6042fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.634+0000 7f64f3fff640 1 -- 172.21.15.44:0/3690906515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641c601d2a0 con 0x5641c601f510 2024-08-20T03:09:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.635+0000 7f64f3fff640 1 --2- 172.21.15.44:0/3690906515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c6027670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f64ec03cea0 tx=0x7f64ec03e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a35c38176971aa7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.636+0000 7f64d97fa640 1 -- 172.21.15.44:0/3690906515 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec040070 con 0x5641c601f510 2024-08-20T03:09:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.636+0000 7f64d97fa640 1 -- 172.21.15.44:0/3690906515 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64ec03cb70 con 0x5641c601f510 2024-08-20T03:09:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.637+0000 7f64fca8d740 1 -- 172.21.15.44:0/3690906515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 msgr2=0x5641c6027670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.637+0000 7f64fca8d740 1 --2- 172.21.15.44:0/3690906515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c6027670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f64ec03cea0 tx=0x7f64ec03e040 comp rx=0 tx=0).stop 2024-08-20T03:09:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.637+0000 7f64fca8d740 1 -- 172.21.15.44:0/3690906515 shutdown_connections 2024-08-20T03:09:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.637+0000 7f64fca8d740 1 --2- 172.21.15.44:0/3690906515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c602b4a0 0x5641c6043510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.637+0000 7f64fca8d740 1 --2- 172.21.15.44:0/3690906515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c6028470 0x5641c6042fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.637+0000 7f64fca8d740 1 --2- 172.21.15.44:0/3690906515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c6027670 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.637+0000 7f64fca8d740 1 -- 172.21.15.44:0/3690906515 >> 172.21.15.44:0/3690906515 conn(0x5641c601ca20 msgr2=0x5641c6017780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.637+0000 7f64fca8d740 1 -- 172.21.15.44:0/3690906515 shutdown_connections 2024-08-20T03:09:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.638+0000 7f64fca8d740 1 -- 172.21.15.44:0/3690906515 wait complete. 2024-08-20T03:09:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.638+0000 7f64fca8d740 1 -- start start 2024-08-20T03:09:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.638+0000 7f64fca8d740 1 Processor -- start 2024-08-20T03:09:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.639+0000 7f64fca8d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c6033500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.639+0000 7f64fca8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c6028470 0x5641c6033a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.639+0000 7f64fca8d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c602b4a0 0x5641c6024180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.639+0000 7f64fca8d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f64e000fc00 con 0x5641c602b4a0 2024-08-20T03:09:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.639+0000 7f64fca8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64e000f4d0 con 0x5641c6028470 2024-08-20T03:09:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.639+0000 7f64fca8d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f64e000c7c0 con 0x5641c601f510 2024-08-20T03:09:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.639+0000 7f64fca8d740 10 client.0 _ll_register_callbacks cb 0x7f64ec009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.639+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c602b4a0 0x5641c6024180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.639+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c602b4a0 0x5641c6024180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:54542/0 (socket says 172.21.15.44:54542) 2024-08-20T03:09:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.639+0000 7f64f3fff640 1 -- 172.21.15.44:0/266808549 learned_addr learned my addr 172.21.15.44:0/266808549 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.640+0000 7f64f3fff640 1 -- 172.21.15.44:0/266808549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 msgr2=0x5641c6033500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.640+0000 7f64f3fff640 1 --2- 172.21.15.44:0/266808549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c6033500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.640+0000 7f64f3fff640 1 -- 172.21.15.44:0/266808549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c6028470 msgr2=0x5641c6033a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:09:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.640+0000 7f64f3fff640 1 --2- 172.21.15.44:0/266808549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c6028470 0x5641c6033a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.640+0000 7f64f3fff640 1 -- 172.21.15.44:0/266808549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641c6023a20 con 0x5641c602b4a0 2024-08-20T03:09:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.640+0000 7f64f3fff640 1 --2- 172.21.15.44:0/266808549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c602b4a0 0x5641c6024180 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009be0 tx=0x7f64ec000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893a054bdb7dc446 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.641+0000 7f64d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.641+0000 7f64d2ffd640 1 -- 172.21.15.44:0/266808549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec040040 con 0x5641c602b4a0 2024-08-20T03:09:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.641+0000 7f64fca8d740 1 -- 172.21.15.44:0/266808549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64ec03cb70 con 0x5641c602b4a0 2024-08-20T03:09:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.641+0000 7f64d2ffd640 1 -- 172.21.15.44:0/266808549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f64ec03c470 con 0x5641c602b4a0 2024-08-20T03:09:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.642+0000 7f64d2ffd640 1 -- 172.21.15.44:0/266808549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f64ec06a050 con 0x5641c602b4a0 2024-08-20T03:09:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.643+0000 7f64fca8d740 10 client.16827 fetch_fsmap learned FSMap version 33 2024-08-20T03:09:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.643+0000 7f64fca8d740 1 -- 172.21.15.44:0/266808549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5641c602e290 con 0x5641c602b4a0 2024-08-20T03:09:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.647+0000 7f64d2ffd640 1 -- 172.21.15.44:0/266808549 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64ec03cb70 con 0x5641c602b4a0 2024-08-20T03:09:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.647+0000 7f64d2ffd640 1 -- 172.21.15.44:0/266808549 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f64ec05a070 con 0x5641c602b4a0 2024-08-20T03:09:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.648+0000 7f64fca8d740 10 client.16827 fetch_fsmap finished waiting for FSMap version 33 2024-08-20T03:09:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.648+0000 7f64fca8d740 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.648+0000 7f64fca8d740 20 client.16827 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.648+0000 7f64fca8d740 1 --2- 172.21.15.44:0/266808549 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641c60a57d0 0x5641c60a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.648+0000 7f64fca8d740 4 client.16827 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:09:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.648+0000 7f64fca8d740 1 -- 172.21.15.44:0/266808549 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641c5f91d50 con 0x5641c60a57d0 2024-08-20T03:09:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.648+0000 7f64f2ffd640 1 --2- 172.21.15.44:0/266808549 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641c60a57d0 0x5641c60a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.649+0000 7f64d2ffd640 10 client.16827 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:06.650+0000 7f64f2ffd640 1 --2- 172.21.15.44:0/266808549 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641c60a57d0 0x5641c60a7bf0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aa279f0575fbb2e server_cookie=3b4a4299a38ccea2 in_seq=0 out_seq=0 2024-08-20T03:09:06.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:06 smithi099 ceph-mon[21984]: pgmap v518: 97 pgs: 97 active+clean; 851 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 712 op/s 2024-08-20T03:09:06.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:06 smithi099 ceph-mon[21984]: osdmap e78: 12 total, 12 up, 12 in 2024-08-20T03:09:06.934 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.535+0000 7f0fbd48d640 1 -- 172.21.15.44:0/670875840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb8109e80 msgr2=0x7f0fb8110ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.535+0000 7f0fbd48d640 1 --2- 172.21.15.44:0/670875840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb8109e80 0x7f0fb8110ee0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009900 tx=0x7f0fa802a4b0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.535+0000 7f0fbd48d640 1 -- 172.21.15.44:0/670875840 shutdown_connections 2024-08-20T03:09:07.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.535+0000 7f0fbd48d640 1 --2- 172.21.15.44:0/670875840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fb811bdd0 0x7f0fb811e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.535+0000 7f0fbd48d640 1 --2- 172.21.15.44:0/670875840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fb810a260 0x7f0fb810a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.535+0000 7f0fbd48d640 1 --2- 172.21.15.44:0/670875840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb8109e80 0x7f0fb8110ee0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.535+0000 7f0fbd48d640 1 -- 172.21.15.44:0/670875840 >> 172.21.15.44:0/670875840 conn(0x7f0fb806dcc0 msgr2=0x7f0fb806e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:07.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.537+0000 7f0fbd48d640 1 -- 172.21.15.44:0/670875840 shutdown_connections 2024-08-20T03:09:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.540+0000 7f0fbd48d640 1 -- 172.21.15.44:0/670875840 wait complete. 2024-08-20T03:09:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.541+0000 7f0fbd48d640 1 Processor -- start 2024-08-20T03:09:07.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.542+0000 7f0fbd48d640 1 -- start start 2024-08-20T03:09:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.542+0000 7f0fbd48d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fb8109e80 0x7f0fb81a7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.542+0000 7f0fbd48d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb810a260 0x7f0fb81a8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.542+0000 7f0fbd48d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fb811bdd0 0x7f0fb81ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.542+0000 7f0fbd48d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb81218b0 con 0x7f0fb810a260 2024-08-20T03:09:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.542+0000 7f0fbd48d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8121730 con 0x7f0fb811bdd0 2024-08-20T03:09:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.542+0000 7f0fbd48d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8121a30 con 0x7f0fb8109e80 2024-08-20T03:09:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.542+0000 7f0fb77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fb811bdd0 0x7f0fb81ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.542+0000 7f0fb77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fb811bdd0 0x7f0fb81ad6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:46662/0 (socket says 172.21.15.44:46662) 2024-08-20T03:09:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.542+0000 7f0fb77fe640 1 -- 172.21.15.44:0/2033294008 learned_addr learned my addr 172.21.15.44:0/2033294008 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:07.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.543+0000 7f0fb67fc640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb810a260 0x7f0fb81a8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.544+0000 7f0fb67fc640 1 -- 172.21.15.44:0/2033294008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fb8109e80 msgr2=0x7f0fb81a7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.544+0000 7f0fb67fc640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fb8109e80 0x7f0fb81a7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.544+0000 7f0fb67fc640 1 -- 172.21.15.44:0/2033294008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fb811bdd0 msgr2=0x7f0fb81ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.544+0000 7f0fb67fc640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fb811bdd0 0x7f0fb81ad6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.544+0000 7f0fb67fc640 1 -- 172.21.15.44:0/2033294008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fb81ade70 con 0x7f0fb810a260 2024-08-20T03:09:07.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.545+0000 7f0fb67fc640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb810a260 0x7f0fb81a8300 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000cce0 tx=0x7f0fb00075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:07.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.546+0000 7f0f9ffff640 1 -- 172.21.15.44:0/2033294008 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0004b20 con 0x7f0fb810a260 2024-08-20T03:09:07.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.546+0000 7f0f9ffff640 1 -- 172.21.15.44:0/2033294008 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fb0004cc0 con 0x7f0fb810a260 2024-08-20T03:09:07.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.547+0000 7f0f9ffff640 1 -- 172.21.15.44:0/2033294008 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000f6b0 con 0x7f0fb810a260 2024-08-20T03:09:07.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.547+0000 7f0fbd48d640 1 -- 172.21.15.44:0/2033294008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fb81ae160 con 0x7f0fb810a260 2024-08-20T03:09:07.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.548+0000 7f0fbd48d640 1 -- 172.21.15.44:0/2033294008 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fb81115b0 con 0x7f0fb810a260 2024-08-20T03:09:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.552+0000 7f0fbf886740 1 Processor -- start 2024-08-20T03:09:07.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.552+0000 7f0fbf886740 1 -- start start 2024-08-20T03:09:07.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.554+0000 7f0fbf886740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ed880 0x5604613edc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.554+0000 7f0fbf886740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ee1a0 0x5604613f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.554+0000 7f0fbf886740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x5604613fc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.554+0000 7f0fbf886740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x560461364930 con 0x5604613fa130 2024-08-20T03:09:07.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.554+0000 7f0fbf886740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560461353e30 con 0x5604613ed880 2024-08-20T03:09:07.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.554+0000 7f0fbf886740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x560461357e70 con 0x5604613ee1a0 2024-08-20T03:09:07.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.554+0000 7f0fb77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ed880 0x5604613edc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.554+0000 7f0fb77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ed880 0x5604613edc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:46674/0 (socket says 172.21.15.44:46674) 2024-08-20T03:09:07.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.554+0000 7f0fb77fe640 1 -- 172.21.15.44:0/73482795 learned_addr learned my addr 172.21.15.44:0/73482795 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:07.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.554+0000 7f0f9ffff640 1 -- 172.21.15.44:0/2033294008 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0fb000f850 con 0x7f0fb810a260 2024-08-20T03:09:07.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.555+0000 7f0fb67fc640 1 --2- 172.21.15.44:0/73482795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x5604613fc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.555+0000 7f0fb6ffd640 1 --2- 172.21.15.44:0/73482795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ee1a0 0x5604613f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.556+0000 7f0f9ffff640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0fa00b3e20 0x7f0fa00b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.557+0000 7f0fb77fe640 1 -- 172.21.15.44:0/73482795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ee1a0 msgr2=0x5604613f9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.557+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/73482795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ee1a0 0x5604613f9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.558+0000 7f0fb77fe640 1 -- 172.21.15.44:0/73482795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 msgr2=0x5604613fc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.558+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/73482795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x5604613fc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.558+0000 7f0fb77fe640 1 -- 172.21.15.44:0/73482795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604610b3650 con 0x5604613ed880 2024-08-20T03:09:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.558+0000 7f0f9ffff640 1 -- 172.21.15.44:0/2033294008 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f0fb00e4250 con 0x7f0fb810a260 2024-08-20T03:09:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.558+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0fa00b3e20 0x7f0fa00b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.558+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/73482795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ed880 0x5604613edc60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00ac80 tx=0x7f0fac00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a5924ec5be2480 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.559+0000 7f0f9d7fa640 1 -- 172.21.15.44:0/73482795 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac01a910 con 0x5604613ed880 2024-08-20T03:09:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.559+0000 7f0f9d7fa640 1 -- 172.21.15.44:0/73482795 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fac005050 con 0x5604613ed880 2024-08-20T03:09:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.559+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0fa00b3e20 0x7f0fa00b62e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0fac01f000 tx=0x7f0fac020000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:07.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.559+0000 7f0f9d7fa640 1 -- 172.21.15.44:0/73482795 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac004a00 con 0x5604613ed880 2024-08-20T03:09:07.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.560+0000 7f0fbf886740 1 -- 172.21.15.44:0/73482795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ed880 msgr2=0x5604613edc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:07.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.560+0000 7f0fbf886740 1 --2- 172.21.15.44:0/73482795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ed880 0x5604613edc60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00ac80 tx=0x7f0fac00f790 comp rx=0 tx=0).stop 2024-08-20T03:09:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.562+0000 7f0fbf886740 1 -- 172.21.15.44:0/73482795 shutdown_connections 2024-08-20T03:09:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.562+0000 7f0fbf886740 1 --2- 172.21.15.44:0/73482795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x5604613fc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.562+0000 7f0fbf886740 1 --2- 172.21.15.44:0/73482795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ee1a0 0x5604613f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.562+0000 7f0fbf886740 1 --2- 172.21.15.44:0/73482795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ed880 0x5604613edc60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.562+0000 7f0fbf886740 1 -- 172.21.15.44:0/73482795 >> 172.21.15.44:0/73482795 conn(0x5604613eb6b0 msgr2=0x5604613ec850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:07.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.563+0000 7f0fbf886740 1 -- 172.21.15.44:0/73482795 shutdown_connections 2024-08-20T03:09:07.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.566+0000 7f0fbf886740 1 -- 172.21.15.44:0/73482795 wait complete. 2024-08-20T03:09:07.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.566+0000 7f0fbf886740 1 -- start start 2024-08-20T03:09:07.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.567+0000 7f0fbf886740 1 Processor -- start 2024-08-20T03:09:07.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.567+0000 7f0fbf886740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613ed880 0x5604613f7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.567+0000 7f0fbf886740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ee1a0 0x5604613f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.567+0000 7f0fbf886740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613fa130 0x5604613f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.567+0000 7f0fbf886740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5604611b6700 con 0x5604613ed880 2024-08-20T03:09:07.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.567+0000 7f0fbf886740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5604612da810 con 0x5604613ee1a0 2024-08-20T03:09:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.567+0000 7f0fbf886740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5604613fcde0 con 0x5604613fa130 2024-08-20T03:09:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.568+0000 7f0fbf886740 10 client.0 _ll_register_callbacks cb 0x56046134d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.568+0000 7f0fb67fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ee1a0 0x5604613f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.568+0000 7f0fb67fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ee1a0 0x5604613f76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:46676/0 (socket says 172.21.15.44:46676) 2024-08-20T03:09:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.568+0000 7f0fb67fc640 1 -- 172.21.15.44:0/362704381 learned_addr learned my addr 172.21.15.44:0/362704381 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.568+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/362704381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613fa130 0x5604613f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.569+0000 7f0fb77fe640 1 -- 172.21.15.44:0/362704381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ee1a0 msgr2=0x5604613f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.569+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/362704381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ee1a0 0x5604613f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.569+0000 7f0fb77fe640 1 -- 172.21.15.44:0/362704381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613ed880 msgr2=0x5604613f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.569+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/362704381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613ed880 0x5604613f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.569+0000 7f0fb77fe640 1 -- 172.21.15.44:0/362704381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560461406240 con 0x5604613fa130 2024-08-20T03:09:07.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.571+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/362704381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613fa130 0x5604613f7bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00a880 tx=0x7f0fac0056c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bced90412d80066b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:07.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.571+0000 7f0f9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:09:07.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.572+0000 7f0f9aff5640 1 -- 172.21.15.44:0/362704381 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac028030 con 0x5604613fa130 2024-08-20T03:09:07.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.572+0000 7f0fbf886740 1 -- 172.21.15.44:0/362704381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560461303150 con 0x5604613fa130 2024-08-20T03:09:07.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.573+0000 7f0f9aff5640 1 -- 172.21.15.44:0/362704381 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f0fac01a910 con 0x5604613fa130 2024-08-20T03:09:07.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.573+0000 7f0f9aff5640 1 -- 172.21.15.44:0/362704381 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0fac05f050 con 0x5604613fa130 2024-08-20T03:09:07.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.573+0000 7f0fbf886740 10 client.26239 fetch_fsmap learned FSMap version 33 2024-08-20T03:09:07.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.573+0000 7f0fbf886740 1 -- 172.21.15.44:0/362704381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560461408bf0 con 0x5604613fa130 2024-08-20T03:09:07.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.577+0000 7f0f9aff5640 1 -- 172.21.15.44:0/362704381 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560461303150 con 0x5604613fa130 2024-08-20T03:09:07.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.577+0000 7f0f9aff5640 1 -- 172.21.15.44:0/362704381 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f0fac051440 con 0x5604613fa130 2024-08-20T03:09:07.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.577+0000 7f0fbf886740 10 client.26239 fetch_fsmap finished waiting for FSMap version 33 2024-08-20T03:09:07.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.577+0000 7f0fbf886740 10 client.26239 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:09:07.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.577+0000 7f0fbf886740 20 client.26239 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:07.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.577+0000 7f0fbf886740 1 --2- 172.21.15.44:0/362704381 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x56046140a4e0 0x56046140c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.577+0000 7f0fbf886740 4 client.26239 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:07.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.577+0000 7f0fbf886740 1 -- 172.21.15.44:0/362704381 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560461366880 con 0x56046140a4e0 2024-08-20T03:09:07.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.578+0000 7f0fb6ffd640 1 --2- 172.21.15.44:0/362704381 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x56046140a4e0 0x56046140c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.598+0000 7f0fb6ffd640 1 --2- 172.21.15.44:0/362704381 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x56046140a4e0 0x56046140c980 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c2e9c371f6b041c server_cookie=5f6eb2b2bb72a93b in_seq=0 out_seq=0 2024-08-20T03:09:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.598+0000 7f0f9aff5640 10 client.26239 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:09:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.600+0000 7f0f9aff5640 1 -- 172.21.15.44:0/362704381 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560461366880 con 0x56046140a4e0 2024-08-20T03:09:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.600+0000 7f0f9aff5640 10 client.26239 handle_command_reply: tid=0 2024-08-20T03:09:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.601+0000 7f0fbf886740 1 client.26239 shutdown 2024-08-20T03:09:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.601+0000 7f0fbf886740 1 -- 172.21.15.44:0/362704381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613fa130 msgr2=0x5604613f7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:07.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.601+0000 7f0fbf886740 1 --2- 172.21.15.44:0/362704381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613fa130 0x5604613f7bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00a880 tx=0x7f0fac0056c0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.601+0000 7f0fbf886740 1 -- 172.21.15.44:0/362704381 shutdown_connections 2024-08-20T03:09:07.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.601+0000 7f0fbf886740 1 --2- 172.21.15.44:0/362704381 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x56046140a4e0 0x56046140c980 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.601+0000 7f0fbf886740 1 --2- 172.21.15.44:0/362704381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613fa130 0x5604613f7bf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.601+0000 7f0fbf886740 1 --2- 172.21.15.44:0/362704381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ee1a0 0x5604613f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.601+0000 7f0fbf886740 1 --2- 172.21.15.44:0/362704381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613ed880 0x5604613f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.601+0000 7f0fbf886740 1 -- 172.21.15.44:0/362704381 >> 172.21.15.44:0/362704381 conn(0x5604613eb6b0 msgr2=0x5604613f1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:07.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.602+0000 7f0f9aff5640 0 client.26239 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:09:07.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.603+0000 7f0fbf886740 1 -- 172.21.15.44:0/362704381 shutdown_connections 2024-08-20T03:09:07.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.603+0000 7f0fbf886740 1 -- 172.21.15.44:0/362704381 wait complete. 2024-08-20T03:09:07.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.603+0000 7f0fbf886740 20 client.26239 trim_cache size 0 max 16384 2024-08-20T03:09:07.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.629+0000 7f0fbf886740 1 Processor -- start 2024-08-20T03:09:07.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.629+0000 7f0fbf886740 1 -- start start 2024-08-20T03:09:07.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.629+0000 7f0fbf886740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ed880 0x5604613f7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.632 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.629+0000 7f0fbf886740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ee1a0 0x5604613f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.629+0000 7f0fbf886740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x560461412290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.629+0000 7f0fbf886740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8075100 con 0x5604613fa130 2024-08-20T03:09:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.629+0000 7f0fbf886740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8076d60 con 0x5604613ee1a0 2024-08-20T03:09:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.629+0000 7f0fbf886740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb80757d0 con 0x5604613ed880 2024-08-20T03:09:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.631+0000 7f0fb77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x560461412290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.633 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.631+0000 7f0fb77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x560461412290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51308/0 (socket says 172.21.15.44:51308) 2024-08-20T03:09:07.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.631+0000 7f0fb77fe640 1 -- 172.21.15.44:0/334673399 learned_addr learned my addr 172.21.15.44:0/334673399 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.632+0000 7f0fb6ffd640 1 --2- 172.21.15.44:0/334673399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ed880 0x5604613f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.632+0000 7f0fb67fc640 1 --2- 172.21.15.44:0/334673399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ee1a0 0x5604613f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.633+0000 7f0fb77fe640 1 -- 172.21.15.44:0/334673399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ed880 msgr2=0x5604613f7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.633+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/334673399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ed880 0x5604613f7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.633+0000 7f0fb77fe640 1 -- 172.21.15.44:0/334673399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ee1a0 msgr2=0x5604613f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.633+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/334673399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ee1a0 0x5604613f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.633+0000 7f0fb77fe640 1 -- 172.21.15.44:0/334673399 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604613edfd0 con 0x5604613fa130 2024-08-20T03:09:07.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.634+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/334673399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x560461412290 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0fac001eb0 tx=0x7f0fac004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5526dbe17feba1a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.635+0000 7f0f9cff9640 1 -- 172.21.15.44:0/334673399 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac028040 con 0x5604613fa130 2024-08-20T03:09:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.635+0000 7f0f9cff9640 1 -- 172.21.15.44:0/334673399 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fac068070 con 0x5604613fa130 2024-08-20T03:09:07.637 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.635+0000 7f0f9cff9640 1 -- 172.21.15.44:0/334673399 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac01ae60 con 0x5604613fa130 2024-08-20T03:09:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.636+0000 7f0fbf886740 1 -- 172.21.15.44:0/334673399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 msgr2=0x560461412290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:07.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.636+0000 7f0fbf886740 1 --2- 172.21.15.44:0/334673399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x560461412290 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0fac001eb0 tx=0x7f0fac004bf0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.641+0000 7f0fbf886740 1 -- 172.21.15.44:0/334673399 shutdown_connections 2024-08-20T03:09:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.641+0000 7f0fbf886740 1 --2- 172.21.15.44:0/334673399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x560461412290 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.641+0000 7f0fbf886740 1 --2- 172.21.15.44:0/334673399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ee1a0 0x5604613f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.641+0000 7f0fbf886740 1 --2- 172.21.15.44:0/334673399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ed880 0x5604613f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.641+0000 7f0fbf886740 1 -- 172.21.15.44:0/334673399 >> 172.21.15.44:0/334673399 conn(0x5604613eb6b0 msgr2=0x5604613f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:07.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.642+0000 7f0fbf886740 1 -- 172.21.15.44:0/334673399 shutdown_connections 2024-08-20T03:09:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.645+0000 7f0fbf886740 1 -- 172.21.15.44:0/334673399 wait complete. 2024-08-20T03:09:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.645+0000 7f0fbf886740 1 -- start start 2024-08-20T03:09:07.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.645+0000 7f0fbf886740 1 Processor -- start 2024-08-20T03:09:07.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.648+0000 7f0fbf886740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ed880 0x560461402070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.648+0000 7f0fbf886740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ee1a0 0x5604614025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.648+0000 7f0fbf886740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x5604613f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.648+0000 7f0fbf886740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb80765d0 con 0x5604613fa130 2024-08-20T03:09:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.648+0000 7f0fbf886740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8075ea0 con 0x5604613ed880 2024-08-20T03:09:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.648+0000 7f0fbf886740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0fb8117e00 con 0x5604613ee1a0 2024-08-20T03:09:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.648+0000 7f0fbf886740 10 client.0 _ll_register_callbacks cb 0x7f0fac00ab00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.649+0000 7f0fb77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x5604613f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.649+0000 7f0fb77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x5604613f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51318/0 (socket says 172.21.15.44:51318) 2024-08-20T03:09:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.649+0000 7f0fb77fe640 1 -- 172.21.15.44:0/1510701705 learned_addr learned my addr 172.21.15.44:0/1510701705 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.649+0000 7f0fb67fc640 1 --2- 172.21.15.44:0/1510701705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ee1a0 0x5604614025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.649+0000 7f0fb77fe640 1 -- 172.21.15.44:0/1510701705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ee1a0 msgr2=0x5604614025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:07.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.649+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/1510701705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ee1a0 0x5604614025b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.649+0000 7f0fb77fe640 1 -- 172.21.15.44:0/1510701705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ed880 msgr2=0x560461402070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:09:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.649+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/1510701705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ed880 0x560461402070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.649+0000 7f0fb77fe640 1 -- 172.21.15.44:0/1510701705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604613f3b00 con 0x5604613fa130 2024-08-20T03:09:07.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.650+0000 7f0fb77fe640 1 --2- 172.21.15.44:0/1510701705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x5604613f35c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0fac010f70 tx=0x7f0fac010670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1cb1552f0d236d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.650+0000 7f0f9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.650+0000 7f0f9aff5640 1 -- 172.21.15.44:0/1510701705 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac028040 con 0x5604613fa130 2024-08-20T03:09:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.651+0000 7f0f9aff5640 1 -- 172.21.15.44:0/1510701705 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f0fac051640 con 0x5604613fa130 2024-08-20T03:09:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.651+0000 7f0fbf886740 1 -- 172.21.15.44:0/1510701705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fac068070 con 0x5604613fa130 2024-08-20T03:09:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.651+0000 7f0f9aff5640 1 -- 172.21.15.44:0/1510701705 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0fac0519b0 con 0x5604613fa130 2024-08-20T03:09:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.651+0000 7f0fbf886740 10 client.16842 fetch_fsmap learned FSMap version 33 2024-08-20T03:09:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.651+0000 7f0fbf886740 1 -- 172.21.15.44:0/1510701705 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560461474910 con 0x5604613fa130 2024-08-20T03:09:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.652+0000 7f0f9aff5640 1 -- 172.21.15.44:0/1510701705 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fac068070 con 0x5604613fa130 2024-08-20T03:09:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.652+0000 7f0f9aff5640 1 -- 172.21.15.44:0/1510701705 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f0fac0053c0 con 0x5604613fa130 2024-08-20T03:09:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.652+0000 7f0fbf886740 10 client.16842 fetch_fsmap finished waiting for FSMap version 33 2024-08-20T03:09:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.652+0000 7f0fbf886740 10 client.16842 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:09:07.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.652+0000 7f0fbf886740 20 client.16842 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:07.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.652+0000 7f0fbf886740 1 --2- 172.21.15.44:0/1510701705 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x560461474fb0 0x5604614773d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:07.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.652+0000 7f0fbf886740 4 client.16842 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:09:07.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.652+0000 7f0fbf886740 1 -- 172.21.15.44:0/1510701705 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560461362320 con 0x560461474fb0 2024-08-20T03:09:07.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.655+0000 7f0fb6ffd640 1 --2- 172.21.15.44:0/1510701705 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x560461474fb0 0x5604614773d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.658+0000 7f0fb6ffd640 1 --2- 172.21.15.44:0/1510701705 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x560461474fb0 0x5604614773d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9351c09deb34ea54 server_cookie=ec660e900ef7aa3f in_seq=0 out_seq=0 2024-08-20T03:09:07.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.658+0000 7f0f9aff5640 10 client.16842 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:09:07.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.659+0000 7f0f9aff5640 1 -- 172.21.15.44:0/1510701705 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560461362320 con 0x560461474fb0 2024-08-20T03:09:07.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.659+0000 7f0f9aff5640 10 client.16842 handle_command_reply: tid=0 2024-08-20T03:09:07.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.659+0000 7f0fbf886740 1 client.16842 shutdown 2024-08-20T03:09:07.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.660+0000 7f0fbf886740 1 -- 172.21.15.44:0/1510701705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 msgr2=0x5604613f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:07.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.660+0000 7f0fbf886740 1 --2- 172.21.15.44:0/1510701705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x5604613f35c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0fac010f70 tx=0x7f0fac010670 comp rx=0 tx=0).stop 2024-08-20T03:09:07.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.662+0000 7f0fbf886740 1 -- 172.21.15.44:0/1510701705 shutdown_connections 2024-08-20T03:09:07.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.662+0000 7f0fbf886740 1 --2- 172.21.15.44:0/1510701705 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x560461474fb0 0x5604614773d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.662+0000 7f0fbf886740 1 --2- 172.21.15.44:0/1510701705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604613fa130 0x5604613f35c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.662+0000 7f0fbf886740 1 --2- 172.21.15.44:0/1510701705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604613ee1a0 0x5604614025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.662+0000 7f0fbf886740 1 --2- 172.21.15.44:0/1510701705 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604613ed880 0x560461402070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.662+0000 7f0fbf886740 1 -- 172.21.15.44:0/1510701705 >> 172.21.15.44:0/1510701705 conn(0x5604613eb6b0 msgr2=0x5604613f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:07.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.662+0000 7f0f9aff5640 0 client.16842 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:09:07.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.663+0000 7f0fbf886740 1 -- 172.21.15.44:0/1510701705 shutdown_connections 2024-08-20T03:09:07.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.663+0000 7f0fbf886740 1 -- 172.21.15.44:0/1510701705 wait complete. 2024-08-20T03:09:07.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.665+0000 7f0fbf886740 20 client.16842 trim_cache size 0 max 16384 2024-08-20T03:09:07.668 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:09:07.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.667+0000 7f0fbd48d640 1 -- 172.21.15.44:0/2033294008 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0fa00b3e20 msgr2=0x7f0fa00b62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:07.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.667+0000 7f0fbd48d640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0fa00b3e20 0x7f0fa00b62e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0fac01f000 tx=0x7f0fac020000 comp rx=0 tx=0).stop 2024-08-20T03:09:07.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.668+0000 7f0fbd48d640 1 -- 172.21.15.44:0/2033294008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb810a260 msgr2=0x7f0fb81a8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:07.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.668+0000 7f0fbd48d640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb810a260 0x7f0fb81a8300 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000cce0 tx=0x7f0fb00075f0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.671+0000 7f0fbd48d640 1 -- 172.21.15.44:0/2033294008 shutdown_connections 2024-08-20T03:09:07.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.671+0000 7f0fbd48d640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0fa00b3e20 0x7f0fa00b62e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.671+0000 7f0fbd48d640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fb811bdd0 0x7f0fb81ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.671+0000 7f0fbd48d640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0fb810a260 0x7f0fb81a8300 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.671+0000 7f0fbd48d640 1 --2- 172.21.15.44:0/2033294008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0fb8109e80 0x7f0fb81a7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:07.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.671+0000 7f0fbd48d640 1 -- 172.21.15.44:0/2033294008 >> 172.21.15.44:0/2033294008 conn(0x7f0fb806dcc0 msgr2=0x7f0fb811c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:07.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.673+0000 7f0fbd48d640 1 -- 172.21.15.44:0/2033294008 shutdown_connections 2024-08-20T03:09:07.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:07.675+0000 7f0fbd48d640 1 -- 172.21.15.44:0/2033294008 wait complete. 2024-08-20T03:09:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64d2ffd640 1 -- 172.21.15.44:0/266808549 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5641c5f91d50 con 0x5641c60a57d0 2024-08-20T03:09:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64d2ffd640 10 client.16827 handle_command_reply: tid=0 2024-08-20T03:09:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64fca8d740 1 client.16827 shutdown 2024-08-20T03:09:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64fca8d740 1 -- 172.21.15.44:0/266808549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c602b4a0 msgr2=0x5641c6024180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64fca8d740 1 --2- 172.21.15.44:0/266808549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c602b4a0 0x5641c6024180 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009be0 tx=0x7f64ec000f40 comp rx=0 tx=0).stop 2024-08-20T03:09:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64fca8d740 1 -- 172.21.15.44:0/266808549 shutdown_connections 2024-08-20T03:09:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64fca8d740 1 --2- 172.21.15.44:0/266808549 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641c60a57d0 0x5641c60a7bf0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64fca8d740 1 --2- 172.21.15.44:0/266808549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641c602b4a0 0x5641c6024180 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64fca8d740 1 --2- 172.21.15.44:0/266808549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641c6028470 0x5641c6033a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64fca8d740 1 --2- 172.21.15.44:0/266808549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641c601f510 0x5641c6033500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64fca8d740 1 -- 172.21.15.44:0/266808549 >> 172.21.15.44:0/266808549 conn(0x5641c601ca20 msgr2=0x5641c6021da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64d2ffd640 0 client.16827 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64fca8d740 1 -- 172.21.15.44:0/266808549 shutdown_connections 2024-08-20T03:09:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64fca8d740 1 -- 172.21.15.44:0/266808549 wait complete. 2024-08-20T03:09:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.281+0000 7f64fca8d740 20 client.16827 trim_cache size 0 max 16384 2024-08-20T03:09:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:09:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:09:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:09:08.285 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:09:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.282+0000 7f64f9bc2640 1 -- 172.21.15.44:0/248824420 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f64dc0b3f10 msgr2=0x7f64dc0b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.282+0000 7f64f9bc2640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f64dc0b3f10 0x7f64dc0b63d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5641c6021d70 tx=0x7f64e4005f70 comp rx=0 tx=0).stop 2024-08-20T03:09:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.282+0000 7f64f9bc2640 1 -- 172.21.15.44:0/248824420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64f4074c70 msgr2=0x7f64f4085580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.282+0000 7f64f9bc2640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64f4074c70 0x7f64f4085580 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f64e400d950 tx=0x7f64e400de10 comp rx=0 tx=0).stop 2024-08-20T03:09:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.285+0000 7f64f9bc2640 1 -- 172.21.15.44:0/248824420 shutdown_connections 2024-08-20T03:09:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.285+0000 7f64f9bc2640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f64dc0b3f10 0x7f64dc0b63d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.285+0000 7f64f9bc2640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f64f410a0f0 0x7f64f4080960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.285+0000 7f64f9bc2640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64f40755f0 0x7f64f4085ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.285+0000 7f64f9bc2640 1 --2- 172.21.15.44:0/248824420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64f4074c70 0x7f64f4085580 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.285+0000 7f64f9bc2640 1 -- 172.21.15.44:0/248824420 >> 172.21.15.44:0/248824420 conn(0x7f64f406daa0 msgr2=0x7f64f410d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.285+0000 7f64f9bc2640 1 -- 172.21.15.44:0/248824420 shutdown_connections 2024-08-20T03:09:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:08.285+0000 7f64f9bc2640 1 -- 172.21.15.44:0/248824420 wait complete. 2024-08-20T03:09:08.399 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:08 smithi044 ceph-mon[18790]: pgmap v520: 97 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 84 active+clean; 852 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 443 op/s 2024-08-20T03:09:08.400 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T03:09:08.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:08 smithi165 ceph-mon[22357]: pgmap v520: 97 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 84 active+clean; 852 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 443 op/s 2024-08-20T03:09:08.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:08 smithi099 ceph-mon[21984]: pgmap v520: 97 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 84 active+clean; 852 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 443 op/s 2024-08-20T03:09:08.887 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:09.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.489+0000 7f7853286640 1 -- 172.21.15.44:0/2918300666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f784c11bdd0 msgr2=0x7f784c11e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:09.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.489+0000 7f7853286640 1 --2- 172.21.15.44:0/2918300666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f784c11bdd0 0x7f784c11e1c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f783c009860 tx=0x7f783c02a3f0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.490+0000 7f7853286640 1 -- 172.21.15.44:0/2918300666 shutdown_connections 2024-08-20T03:09:09.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.490+0000 7f7853286640 1 --2- 172.21.15.44:0/2918300666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f784c11bdd0 0x7f784c11e1c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.490+0000 7f7853286640 1 --2- 172.21.15.44:0/2918300666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f784c10a260 0x7f784c10a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.490+0000 7f7853286640 1 --2- 172.21.15.44:0/2918300666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c109e80 0x7f784c110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.490+0000 7f7853286640 1 -- 172.21.15.44:0/2918300666 >> 172.21.15.44:0/2918300666 conn(0x7f784c06dcc0 msgr2=0x7f784c06e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:09.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.496+0000 7f7853286640 1 -- 172.21.15.44:0/2918300666 shutdown_connections 2024-08-20T03:09:09.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.497+0000 7f7853286640 1 -- 172.21.15.44:0/2918300666 wait complete. 2024-08-20T03:09:09.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.499+0000 7f7853286640 1 Processor -- start 2024-08-20T03:09:09.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.499+0000 7f7853286640 1 -- start start 2024-08-20T03:09:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.499+0000 7f7853286640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c109e80 0x7f784c1175e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.499+0000 7f7853286640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f784c10a260 0x7f784c117b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.499+0000 7f7853286640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f784c11bdd0 0x7f784c1a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.499+0000 7f7853286640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f784c113e40 con 0x7f784c109e80 2024-08-20T03:09:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.499+0000 7f7853286640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f784c113cc0 con 0x7f784c11bdd0 2024-08-20T03:09:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.499+0000 7f7853286640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f784c113fc0 con 0x7f784c10a260 2024-08-20T03:09:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.499+0000 7f7851022640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c109e80 0x7f784c1175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.499+0000 7f7851022640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c109e80 0x7f784c1175e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51340/0 (socket says 172.21.15.44:51340) 2024-08-20T03:09:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.500+0000 7f7851022640 1 -- 172.21.15.44:0/3662904435 learned_addr learned my addr 172.21.15.44:0/3662904435 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.500+0000 7f7851823640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f784c11bdd0 0x7f784c1a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.500+0000 7f7850821640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f784c10a260 0x7f784c117b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.500+0000 7f7851823640 1 -- 172.21.15.44:0/3662904435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f784c10a260 msgr2=0x7f784c117b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:09.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.500+0000 7f7851823640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f784c10a260 0x7f784c117b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.500+0000 7f7851823640 1 -- 172.21.15.44:0/3662904435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c109e80 msgr2=0x7f784c1175e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:09.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.500+0000 7f7851823640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c109e80 0x7f784c1175e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.500+0000 7f7851823640 1 -- 172.21.15.44:0/3662904435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c1a55f0 con 0x7f784c11bdd0 2024-08-20T03:09:09.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.501+0000 7f7851823640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f784c11bdd0 0x7f784c1a4e20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f783c02a910 tx=0x7f783c030ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:09.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.501+0000 7f78427fc640 1 -- 172.21.15.44:0/3662904435 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c040070 con 0x7f784c11bdd0 2024-08-20T03:09:09.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.502+0000 7f7853286640 1 -- 172.21.15.44:0/3662904435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784c1a5880 con 0x7f784c11bdd0 2024-08-20T03:09:09.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.502+0000 7f7853286640 1 -- 172.21.15.44:0/3662904435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784c1cdfb0 con 0x7f784c11bdd0 2024-08-20T03:09:09.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.502+0000 7f78427fc640 1 -- 172.21.15.44:0/3662904435 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783c005570 con 0x7f784c11bdd0 2024-08-20T03:09:09.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.502+0000 7f78427fc640 1 -- 172.21.15.44:0/3662904435 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c0046c0 con 0x7f784c11bdd0 2024-08-20T03:09:09.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.506+0000 7f785567f740 1 Processor -- start 2024-08-20T03:09:09.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.508+0000 7f785567f740 1 -- start start 2024-08-20T03:09:09.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.508+0000 7f785567f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 0x5582dcd98c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.508+0000 7f785567f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 0x5582dcda4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.508+0000 7f785567f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcda5130 0x5582dcda7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.508+0000 7f785567f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5582dcd0f930 con 0x5582dcd991a0 2024-08-20T03:09:09.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.508+0000 7f785567f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5582dccfee30 con 0x5582dcd98880 2024-08-20T03:09:09.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.508+0000 7f785567f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5582dcd02e70 con 0x5582dcda5130 2024-08-20T03:09:09.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.508+0000 7f78427fc640 1 -- 172.21.15.44:0/3662904435 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f783c043070 con 0x7f784c11bdd0 2024-08-20T03:09:09.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.510+0000 7f78427fc640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f78380b3f10 0x7f78380b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.510+0000 7f7851022640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 0x5582dcda4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.510+0000 7f7850821640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 0x5582dcd98c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.510+0000 7f7851022640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 0x5582dcda4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51344/0 (socket says 172.21.15.44:51344) 2024-08-20T03:09:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.510+0000 7f7851022640 1 -- 172.21.15.44:0/295068033 learned_addr learned my addr 172.21.15.44:0/295068033 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.510+0000 7f7850821640 1 --2- 172.21.15.44:0/295068033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcda5130 0x5582dcda7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.510+0000 7f78427fc640 1 -- 172.21.15.44:0/3662904435 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f783c1046d0 con 0x7f784c11bdd0 2024-08-20T03:09:09.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.511+0000 7f7851823640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f78380b3f10 0x7f78380b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.513+0000 7f7851022640 1 -- 172.21.15.44:0/295068033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcda5130 msgr2=0x5582dcda7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.513+0000 7f7851022640 1 --2- 172.21.15.44:0/295068033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcda5130 0x5582dcda7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.513+0000 7f7851022640 1 -- 172.21.15.44:0/295068033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 msgr2=0x5582dcd98c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.513+0000 7f7851022640 1 --2- 172.21.15.44:0/295068033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 0x5582dcd98c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.513+0000 7f7851022640 1 -- 172.21.15.44:0/295068033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582dca5e650 con 0x5582dcd991a0 2024-08-20T03:09:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.513+0000 7f7850821640 1 --2- 172.21.15.44:0/295068033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 0x5582dcd98c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:09.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.513+0000 7f7851823640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f78380b3f10 0x7f78380b63d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f783c031000 tx=0x7f783c02a3f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:09.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.515+0000 7f7851022640 1 --2- 172.21.15.44:0/295068033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 0x5582dcda4bf0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f78440114c0 tx=0x7f7844011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04222016a87acfc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.516+0000 7f78377fe640 1 -- 172.21.15.44:0/295068033 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844012770 con 0x5582dcd991a0 2024-08-20T03:09:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.516+0000 7f78377fe640 1 -- 172.21.15.44:0/295068033 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7844012dc0 con 0x5582dcd991a0 2024-08-20T03:09:09.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.516+0000 7f78377fe640 1 -- 172.21.15.44:0/295068033 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784401bdf0 con 0x5582dcd991a0 2024-08-20T03:09:09.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.517+0000 7f785567f740 1 -- 172.21.15.44:0/295068033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 msgr2=0x5582dcda4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:09.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.517+0000 7f785567f740 1 --2- 172.21.15.44:0/295068033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 0x5582dcda4bf0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f78440114c0 tx=0x7f7844011980 comp rx=0 tx=0).stop 2024-08-20T03:09:09.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.517+0000 7f785567f740 1 -- 172.21.15.44:0/295068033 shutdown_connections 2024-08-20T03:09:09.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.517+0000 7f785567f740 1 --2- 172.21.15.44:0/295068033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcda5130 0x5582dcda7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.517+0000 7f785567f740 1 --2- 172.21.15.44:0/295068033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 0x5582dcda4bf0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.517+0000 7f785567f740 1 --2- 172.21.15.44:0/295068033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 0x5582dcd98c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.517+0000 7f785567f740 1 -- 172.21.15.44:0/295068033 >> 172.21.15.44:0/295068033 conn(0x5582dcd966b0 msgr2=0x5582dcd97850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:09.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.518+0000 7f785567f740 1 -- 172.21.15.44:0/295068033 shutdown_connections 2024-08-20T03:09:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.518+0000 7f785567f740 1 -- 172.21.15.44:0/295068033 wait complete. 2024-08-20T03:09:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.518+0000 7f785567f740 1 -- start start 2024-08-20T03:09:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.518+0000 7f785567f740 1 Processor -- start 2024-08-20T03:09:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.519+0000 7f785567f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd98880 0x5582dcda2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.519+0000 7f785567f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcd991a0 0x5582dcda25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.519+0000 7f785567f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcda5130 0x5582dcda2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.519+0000 7f7850821640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd98880 0x5582dcda2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.520+0000 7f7850821640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd98880 0x5582dcda2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51358/0 (socket says 172.21.15.44:51358) 2024-08-20T03:09:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.520+0000 7f7850821640 1 -- 172.21.15.44:0/834972513 learned_addr learned my addr 172.21.15.44:0/834972513 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.520+0000 7f785567f740 1 -- 172.21.15.44:0/834972513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5582dcb61700 con 0x5582dcd98880 2024-08-20T03:09:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.520+0000 7f785567f740 1 -- 172.21.15.44:0/834972513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5582dcc85810 con 0x5582dcda5130 2024-08-20T03:09:09.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.520+0000 7f785567f740 1 -- 172.21.15.44:0/834972513 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5582dcda7e30 con 0x5582dcd991a0 2024-08-20T03:09:09.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.520+0000 7f785567f740 10 client.0 _ll_register_callbacks cb 0x5582dccf8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:09.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.521+0000 7f7850821640 1 -- 172.21.15.44:0/834972513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcd991a0 msgr2=0x5582dcda25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:09.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.521+0000 7f7850821640 1 --2- 172.21.15.44:0/834972513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcd991a0 0x5582dcda25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.521+0000 7f7850821640 1 -- 172.21.15.44:0/834972513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcda5130 msgr2=0x5582dcda2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:09.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.521+0000 7f7850821640 1 --2- 172.21.15.44:0/834972513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcda5130 0x5582dcda2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.521+0000 7f7850821640 1 -- 172.21.15.44:0/834972513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582dcdb1220 con 0x5582dcd98880 2024-08-20T03:09:09.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.521+0000 7f7850821640 1 --2- 172.21.15.44:0/834972513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd98880 0x5582dcda2080 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f784801afd0 tx=0x7f7848013cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79cbe1ee89c2f34 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:09.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.521+0000 7f7834ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:09.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.521+0000 7f7834ff9640 1 -- 172.21.15.44:0/834972513 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784801a050 con 0x5582dcd98880 2024-08-20T03:09:09.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.522+0000 7f7834ff9640 1 -- 172.21.15.44:0/834972513 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f7848010a60 con 0x5582dcd98880 2024-08-20T03:09:09.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.523+0000 7f785567f740 1 -- 172.21.15.44:0/834972513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5582dccae150 con 0x5582dcd98880 2024-08-20T03:09:09.525 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.523+0000 7f7834ff9640 1 -- 172.21.15.44:0/834972513 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7848010c60 con 0x5582dcd98880 2024-08-20T03:09:09.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.524+0000 7f785567f740 10 client.16854 fetch_fsmap learned FSMap version 33 2024-08-20T03:09:09.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.524+0000 7f785567f740 1 -- 172.21.15.44:0/834972513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5582dcdb3c40 con 0x5582dcd98880 2024-08-20T03:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.525+0000 7f7834ff9640 1 -- 172.21.15.44:0/834972513 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5582dccae150 con 0x5582dcd98880 2024-08-20T03:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.525+0000 7f7834ff9640 1 -- 172.21.15.44:0/834972513 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f784800f040 con 0x5582dcd98880 2024-08-20T03:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.525+0000 7f785567f740 10 client.16854 fetch_fsmap finished waiting for FSMap version 33 2024-08-20T03:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.525+0000 7f785567f740 10 client.16854 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.526+0000 7f785567f740 20 client.16854 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.526+0000 7f785567f740 1 --2- 172.21.15.44:0/834972513 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x5582dcdb51a0 0x5582dcdb7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.526+0000 7f785567f740 4 client.16854 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:09.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.526+0000 7f785567f740 1 -- 172.21.15.44:0/834972513 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582dcd11880 con 0x5582dcdb51a0 2024-08-20T03:09:09.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.527+0000 7f7851022640 1 --2- 172.21.15.44:0/834972513 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x5582dcdb51a0 0x5582dcdb7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.534+0000 7f7851022640 1 --2- 172.21.15.44:0/834972513 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x5582dcdb51a0 0x5582dcdb7630 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a7a5ac2a3d21d4f server_cookie=50dc091978440769 in_seq=0 out_seq=0 2024-08-20T03:09:09.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.534+0000 7f7834ff9640 10 client.16854 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:09:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.537+0000 7f7834ff9640 1 -- 172.21.15.44:0/834972513 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582dcd11880 con 0x5582dcdb51a0 2024-08-20T03:09:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.537+0000 7f7834ff9640 10 client.16854 handle_command_reply: tid=0 2024-08-20T03:09:09.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.537+0000 7f785567f740 1 client.16854 shutdown 2024-08-20T03:09:09.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.537+0000 7f785567f740 1 -- 172.21.15.44:0/834972513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd98880 msgr2=0x5582dcda2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:09.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.537+0000 7f785567f740 1 --2- 172.21.15.44:0/834972513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd98880 0x5582dcda2080 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f784801afd0 tx=0x7f7848013cb0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.538+0000 7f785567f740 1 -- 172.21.15.44:0/834972513 shutdown_connections 2024-08-20T03:09:09.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.538+0000 7f785567f740 1 --2- 172.21.15.44:0/834972513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcda5130 0x5582dcda2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.538+0000 7f785567f740 1 --2- 172.21.15.44:0/834972513 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x5582dcdb51a0 0x5582dcdb7630 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.538+0000 7f7834ff9640 0 client.16854 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:09:09.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.538+0000 7f785567f740 1 --2- 172.21.15.44:0/834972513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcd991a0 0x5582dcda25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.538+0000 7f785567f740 1 --2- 172.21.15.44:0/834972513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd98880 0x5582dcda2080 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.538+0000 7f785567f740 1 -- 172.21.15.44:0/834972513 >> 172.21.15.44:0/834972513 conn(0x5582dcd966b0 msgr2=0x5582dcd9c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:09.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.539+0000 7f785567f740 1 -- 172.21.15.44:0/834972513 shutdown_connections 2024-08-20T03:09:09.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.540+0000 7f785567f740 1 -- 172.21.15.44:0/834972513 wait complete. 2024-08-20T03:09:09.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.540+0000 7f785567f740 20 client.16854 trim_cache size 0 max 16384 2024-08-20T03:09:09.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.565+0000 7f785567f740 1 Processor -- start 2024-08-20T03:09:09.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.566+0000 7f785567f740 1 -- start start 2024-08-20T03:09:09.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.566+0000 7f785567f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 0x5582dcda3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.566+0000 7f785567f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 0x5582dcdbcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.566+0000 7f785567f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcda5130 0x5582dcdbd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.566+0000 7f785567f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f783000e860 con 0x5582dcd991a0 2024-08-20T03:09:09.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.566+0000 7f785567f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78300104c0 con 0x5582dcd98880 2024-08-20T03:09:09.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.566+0000 7f785567f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f783000ef30 con 0x5582dcda5130 2024-08-20T03:09:09.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.568+0000 7f7850821640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 0x5582dcda3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.568+0000 7f7850821640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 0x5582dcda3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:46752/0 (socket says 172.21.15.44:46752) 2024-08-20T03:09:09.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.568+0000 7f7850821640 1 -- 172.21.15.44:0/425525878 learned_addr learned my addr 172.21.15.44:0/425525878 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:09.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.568+0000 7f7850821640 1 --2- 172.21.15.44:0/425525878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcda5130 0x5582dcdbd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.568+0000 7f7850821640 1 -- 172.21.15.44:0/425525878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcda5130 msgr2=0x5582dcdbd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:09.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.568+0000 7f7850821640 1 --2- 172.21.15.44:0/425525878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcda5130 0x5582dcdbd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.568+0000 7f7850821640 1 -- 172.21.15.44:0/425525878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 msgr2=0x5582dcdbcf00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:09:09.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.568+0000 7f7850821640 1 --2- 172.21.15.44:0/425525878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 0x5582dcdbcf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.568+0000 7f7850821640 1 -- 172.21.15.44:0/425525878 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582dcd98fd0 con 0x5582dcd98880 2024-08-20T03:09:09.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.571+0000 7f7850821640 1 --2- 172.21.15.44:0/425525878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 0x5582dcda3dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7848010820 tx=0x7f78480035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c1411f303201a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:09.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.572+0000 7f7836ffd640 1 -- 172.21.15.44:0/425525878 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784802f070 con 0x5582dcd98880 2024-08-20T03:09:09.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.572+0000 7f7836ffd640 1 -- 172.21.15.44:0/425525878 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7848003b50 con 0x5582dcd98880 2024-08-20T03:09:09.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.572+0000 7f7836ffd640 1 -- 172.21.15.44:0/425525878 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78480044b0 con 0x5582dcd98880 2024-08-20T03:09:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.573+0000 7f785567f740 1 -- 172.21.15.44:0/425525878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 msgr2=0x5582dcda3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:09.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.573+0000 7f785567f740 1 --2- 172.21.15.44:0/425525878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 0x5582dcda3dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7848010820 tx=0x7f78480035f0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.574+0000 7f785567f740 1 -- 172.21.15.44:0/425525878 shutdown_connections 2024-08-20T03:09:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.574+0000 7f785567f740 1 --2- 172.21.15.44:0/425525878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcda5130 0x5582dcdbd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.574+0000 7f785567f740 1 --2- 172.21.15.44:0/425525878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 0x5582dcdbcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.574+0000 7f785567f740 1 --2- 172.21.15.44:0/425525878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcd98880 0x5582dcda3dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.575+0000 7f785567f740 1 -- 172.21.15.44:0/425525878 >> 172.21.15.44:0/425525878 conn(0x5582dcd966b0 msgr2=0x5582dcda36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:09.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.576+0000 7f785567f740 1 -- 172.21.15.44:0/425525878 shutdown_connections 2024-08-20T03:09:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.577+0000 7f785567f740 1 -- 172.21.15.44:0/425525878 wait complete. 2024-08-20T03:09:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.578+0000 7f785567f740 1 -- start start 2024-08-20T03:09:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.578+0000 7f785567f740 1 Processor -- start 2024-08-20T03:09:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f785567f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcd98880 0x5582dcda9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f785567f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 0x5582dcda97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f785567f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcda5130 0x5582dcda9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f785567f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f783000fd30 con 0x5582dcd991a0 2024-08-20T03:09:09.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f785567f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f783000f600 con 0x5582dcda5130 2024-08-20T03:09:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f785567f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f783000c860 con 0x5582dcd98880 2024-08-20T03:09:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f785567f740 10 client.0 _ll_register_callbacks cb 0x7f784801add0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f7850821640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcda5130 0x5582dcda9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f7850821640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcda5130 0x5582dcda9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:46762/0 (socket says 172.21.15.44:46762) 2024-08-20T03:09:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f7850821640 1 -- 172.21.15.44:0/3467933707 learned_addr learned my addr 172.21.15.44:0/3467933707 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f7850821640 1 --2- 172.21.15.44:0/3467933707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcd98880 0x5582dcda9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f7850821640 1 -- 172.21.15.44:0/3467933707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcd98880 msgr2=0x5582dcda9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f7850821640 1 --2- 172.21.15.44:0/3467933707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcd98880 0x5582dcda9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f7850821640 1 -- 172.21.15.44:0/3467933707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 msgr2=0x5582dcda97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:09.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f7850821640 1 --2- 172.21.15.44:0/3467933707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 0x5582dcda97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.579+0000 7f7850821640 1 -- 172.21.15.44:0/3467933707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582dce1f1c0 con 0x5582dcda5130 2024-08-20T03:09:09.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.582+0000 7f7850821640 1 --2- 172.21.15.44:0/3467933707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcda5130 0x5582dcda9d50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f784801a730 tx=0x7f7848005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470738813a01a7be server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:09.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.583+0000 7f7834ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:09:09.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.583+0000 7f7834ff9640 1 -- 172.21.15.44:0/3467933707 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784802f070 con 0x5582dcda5130 2024-08-20T03:09:09.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.583+0000 7f7834ff9640 1 -- 172.21.15.44:0/3467933707 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f7848004810 con 0x5582dcda5130 2024-08-20T03:09:09.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.583+0000 7f785567f740 1 -- 172.21.15.44:0/3467933707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7848003b50 con 0x5582dcda5130 2024-08-20T03:09:09.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.583+0000 7f7834ff9640 1 -- 172.21.15.44:0/3467933707 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f78480036a0 con 0x5582dcda5130 2024-08-20T03:09:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.583+0000 7f785567f740 10 client.26279 fetch_fsmap learned FSMap version 33 2024-08-20T03:09:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.583+0000 7f785567f740 1 -- 172.21.15.44:0/3467933707 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5582dce204c0 con 0x5582dcda5130 2024-08-20T03:09:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.584+0000 7f7834ff9640 1 -- 172.21.15.44:0/3467933707 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7848003b50 con 0x5582dcda5130 2024-08-20T03:09:09.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.584+0000 7f7834ff9640 1 -- 172.21.15.44:0/3467933707 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f7848020cb0 con 0x5582dcda5130 2024-08-20T03:09:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.585+0000 7f785567f740 10 client.26279 fetch_fsmap finished waiting for FSMap version 33 2024-08-20T03:09:09.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.585+0000 7f785567f740 10 client.26279 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:09:09.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.585+0000 7f785567f740 20 client.26279 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:09.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.585+0000 7f785567f740 1 --2- 172.21.15.44:0/3467933707 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x5582dcdb51a0 0x5582dce22b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:09.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.585+0000 7f785567f740 4 client.26279 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:09:09.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.585+0000 7f785567f740 1 -- 172.21.15.44:0/3467933707 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582dcd0d320 con 0x5582dcdb51a0 2024-08-20T03:09:09.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.586+0000 7f7851022640 1 --2- 172.21.15.44:0/3467933707 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x5582dcdb51a0 0x5582dce22b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:09.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.589+0000 7f7851022640 1 --2- 172.21.15.44:0/3467933707 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x5582dcdb51a0 0x5582dce22b10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2e073d170292844 server_cookie=7063c0e7088db34d in_seq=0 out_seq=0 2024-08-20T03:09:09.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.589+0000 7f7834ff9640 10 client.26279 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:09:09.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.590+0000 7f7834ff9640 1 -- 172.21.15.44:0/3467933707 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582dcd0d320 con 0x5582dcdb51a0 2024-08-20T03:09:09.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.590+0000 7f7834ff9640 10 client.26279 handle_command_reply: tid=0 2024-08-20T03:09:09.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.590+0000 7f785567f740 1 client.26279 shutdown 2024-08-20T03:09:09.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.595+0000 7f785567f740 1 -- 172.21.15.44:0/3467933707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcda5130 msgr2=0x5582dcda9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:09.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.595+0000 7f785567f740 1 --2- 172.21.15.44:0/3467933707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcda5130 0x5582dcda9d50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f784801a730 tx=0x7f7848005970 comp rx=0 tx=0).stop 2024-08-20T03:09:09.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.595+0000 7f785567f740 1 -- 172.21.15.44:0/3467933707 shutdown_connections 2024-08-20T03:09:09.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.595+0000 7f785567f740 1 --2- 172.21.15.44:0/3467933707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5582dcda5130 0x5582dcda9d50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.595+0000 7f785567f740 1 --2- 172.21.15.44:0/3467933707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582dcd991a0 0x5582dcda97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.595+0000 7f785567f740 1 --2- 172.21.15.44:0/3467933707 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x5582dcdb51a0 0x5582dce22b10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.595+0000 7f785567f740 1 --2- 172.21.15.44:0/3467933707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5582dcd98880 0x5582dcda9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.595+0000 7f7834ff9640 0 client.26279 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:09:09.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.595+0000 7f785567f740 1 -- 172.21.15.44:0/3467933707 >> 172.21.15.44:0/3467933707 conn(0x5582dcd966b0 msgr2=0x5582dcda3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:09.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.598+0000 7f785567f740 1 -- 172.21.15.44:0/3467933707 shutdown_connections 2024-08-20T03:09:09.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.598+0000 7f785567f740 1 -- 172.21.15.44:0/3467933707 wait complete. 2024-08-20T03:09:09.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.598+0000 7f785567f740 20 client.26279 trim_cache size 0 max 16384 2024-08-20T03:09:09.600 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:09:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.599+0000 7f7853286640 1 -- 172.21.15.44:0/3662904435 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f78380b3f10 msgr2=0x7f78380b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.599+0000 7f7853286640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f78380b3f10 0x7f78380b63d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f783c031000 tx=0x7f783c02a3f0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.599+0000 7f7853286640 1 -- 172.21.15.44:0/3662904435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f784c11bdd0 msgr2=0x7f784c1a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.599+0000 7f7853286640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f784c11bdd0 0x7f784c1a4e20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f783c02a910 tx=0x7f783c030ef0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.599+0000 7f7853286640 1 -- 172.21.15.44:0/3662904435 shutdown_connections 2024-08-20T03:09:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.599+0000 7f7853286640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f78380b3f10 0x7f78380b63d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.599+0000 7f7853286640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f784c11bdd0 0x7f784c1a4e20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.599+0000 7f7853286640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f784c10a260 0x7f784c117b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.599+0000 7f7853286640 1 --2- 172.21.15.44:0/3662904435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f784c109e80 0x7f784c1175e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.599+0000 7f7853286640 1 -- 172.21.15.44:0/3662904435 >> 172.21.15.44:0/3662904435 conn(0x7f784c06dcc0 msgr2=0x7f784c071510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.599+0000 7f7853286640 1 -- 172.21.15.44:0/3662904435 shutdown_connections 2024-08-20T03:09:09.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:09.599+0000 7f7853286640 1 -- 172.21.15.44:0/3662904435 wait complete. 2024-08-20T03:09:10.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T03:09:10.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:10 smithi044 ceph-mon[18790]: pgmap v521: 97 pgs: 18 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 49 active+clean; 858 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 406 op/s 2024-08-20T03:09:10.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:10 smithi165 ceph-mon[22357]: pgmap v521: 97 pgs: 18 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 49 active+clean; 858 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 406 op/s 2024-08-20T03:09:10.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:10 smithi099 ceph-mon[21984]: pgmap v521: 97 pgs: 18 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 49 active+clean; 858 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 406 op/s 2024-08-20T03:09:10.780 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:11.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.433+0000 7f50e44eb640 1 -- 172.21.15.44:0/3834585846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50dc101990 msgr2=0x7f50dc101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:11.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.433+0000 7f50e44eb640 1 --2- 172.21.15.44:0/3834585846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50dc101990 0x7f50dc101d70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009900 tx=0x7f50d802a410 comp rx=0 tx=0).stop 2024-08-20T03:09:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.434+0000 7f50e44eb640 1 -- 172.21.15.44:0/3834585846 shutdown_connections 2024-08-20T03:09:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.434+0000 7f50e44eb640 1 --2- 172.21.15.44:0/3834585846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50dc107d20 0x7f50dc10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.434+0000 7f50e44eb640 1 --2- 172.21.15.44:0/3834585846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50dc1073a0 0x7f50dc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.434+0000 7f50e44eb640 1 --2- 172.21.15.44:0/3834585846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50dc101990 0x7f50dc101d70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.434+0000 7f50e44eb640 1 -- 172.21.15.44:0/3834585846 >> 172.21.15.44:0/3834585846 conn(0x7f50dc0fd400 msgr2=0x7f50dc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.435+0000 7f50e44eb640 1 -- 172.21.15.44:0/3834585846 shutdown_connections 2024-08-20T03:09:11.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.435+0000 7f50e44eb640 1 -- 172.21.15.44:0/3834585846 wait complete. 2024-08-20T03:09:11.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.436+0000 7f50e44eb640 1 Processor -- start 2024-08-20T03:09:11.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.436+0000 7f50e44eb640 1 -- start start 2024-08-20T03:09:11.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.436+0000 7f50e44eb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50dc101990 0x7f50dc10e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.436+0000 7f50e44eb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50dc1073a0 0x7f50dc10a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.436+0000 7f50e44eb640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50dc107d20 0x7f50dc10ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.436+0000 7f50e44eb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc110940 con 0x7f50dc1073a0 2024-08-20T03:09:11.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.436+0000 7f50e44eb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1107c0 con 0x7f50dc101990 2024-08-20T03:09:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.436+0000 7f50e44eb640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc110ac0 con 0x7f50dc107d20 2024-08-20T03:09:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.436+0000 7f50e2287640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50dc101990 0x7f50dc10e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.436+0000 7f50e2287640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50dc101990 0x7f50dc10e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:46770/0 (socket says 172.21.15.44:46770) 2024-08-20T03:09:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.436+0000 7f50e2287640 1 -- 172.21.15.44:0/2865783510 learned_addr learned my addr 172.21.15.44:0/2865783510 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:11.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.436+0000 7f50e2a88640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50dc107d20 0x7f50dc10ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.437+0000 7f50e2a88640 1 -- 172.21.15.44:0/2865783510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50dc101990 msgr2=0x7f50dc10e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.437+0000 7f50e2a88640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50dc101990 0x7f50dc10e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.437+0000 7f50e2a88640 1 -- 172.21.15.44:0/2865783510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50dc1073a0 msgr2=0x7f50dc10a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.437+0000 7f50e2a88640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50dc1073a0 0x7f50dc10a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.437+0000 7f50e2a88640 1 -- 172.21.15.44:0/2865783510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50dc06c8a0 con 0x7f50dc107d20 2024-08-20T03:09:11.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.438+0000 7f50e2a88640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50dc107d20 0x7f50dc10ac20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f50d400e940 tx=0x7f50d400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:11.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.438+0000 7f50d37fe640 1 -- 172.21.15.44:0/2865783510 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4004d20 con 0x7f50dc107d20 2024-08-20T03:09:11.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.438+0000 7f50e44eb640 1 -- 172.21.15.44:0/2865783510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50dc06cb90 con 0x7f50dc107d20 2024-08-20T03:09:11.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.438+0000 7f50e44eb640 1 -- 172.21.15.44:0/2865783510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50dc06d080 con 0x7f50dc107d20 2024-08-20T03:09:11.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.438+0000 7f50d37fe640 1 -- 172.21.15.44:0/2865783510 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50d4005020 con 0x7f50dc107d20 2024-08-20T03:09:11.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.439+0000 7f50d37fe640 1 -- 172.21.15.44:0/2865783510 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d40116d0 con 0x7f50dc107d20 2024-08-20T03:09:11.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.442+0000 7f50e68e4740 1 Processor -- start 2024-08-20T03:09:11.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.445+0000 7f50d37fe640 1 -- 172.21.15.44:0/2865783510 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f50d400cb20 con 0x7f50dc107d20 2024-08-20T03:09:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.446+0000 7f50e68e4740 1 -- start start 2024-08-20T03:09:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.446+0000 7f50e68e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f1880 0x564e9d6f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.446+0000 7f50e68e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d6fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.446+0000 7f50e68e4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6fe130 0x564e9d700520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.446+0000 7f50e68e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564e9d668930 con 0x564e9d6f1880 2024-08-20T03:09:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.446+0000 7f50e68e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564e9d657e30 con 0x564e9d6f21a0 2024-08-20T03:09:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.446+0000 7f50e68e4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564e9d65be70 con 0x564e9d6fe130 2024-08-20T03:09:11.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.447+0000 7f50e1a86640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f1880 0x564e9d6f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.447+0000 7f50e1a86640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6fe130 0x564e9d700520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.447+0000 7f50e1a86640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6fe130 0x564e9d700520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40946/0 (socket says 172.21.15.44:40946) 2024-08-20T03:09:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.447+0000 7f50e1a86640 1 -- 172.21.15.44:0/265602692 learned_addr learned my addr 172.21.15.44:0/265602692 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.447+0000 7f50e1a86640 1 -- 172.21.15.44:0/265602692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6fe130 msgr2=0x564e9d700520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.447+0000 7f50e1a86640 1 --2- 172.21.15.44:0/265602692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6fe130 0x564e9d700520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.447+0000 7f50e1a86640 1 -- 172.21.15.44:0/265602692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 msgr2=0x564e9d6fdbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:09:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.447+0000 7f50e1a86640 1 --2- 172.21.15.44:0/265602692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d6fdbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.447+0000 7f50e1a86640 1 -- 172.21.15.44:0/265602692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e9d3b7650 con 0x564e9d6f1880 2024-08-20T03:09:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.448+0000 7f50e1a86640 1 --2- 172.21.15.44:0/265602692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f1880 0x564e9d6f1c60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f50cc012830 tx=0x7f50cc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb9aec1121b509e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.448+0000 7f50d37fe640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f50c80b3e00 0x7f50c80b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.448+0000 7f50d37fe640 1 -- 172.21.15.44:0/2865783510 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f50d40e4030 con 0x7f50dc107d20 2024-08-20T03:09:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.448+0000 7f50d0ff9640 1 -- 172.21.15.44:0/265602692 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00a850 con 0x564e9d6f1880 2024-08-20T03:09:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.449+0000 7f50d0ff9640 1 -- 172.21.15.44:0/265602692 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50cc00f460 con 0x564e9d6f1880 2024-08-20T03:09:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.450+0000 7f50e68e4740 1 -- 172.21.15.44:0/265602692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f1880 msgr2=0x564e9d6f1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.450+0000 7f50e68e4740 1 --2- 172.21.15.44:0/265602692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f1880 0x564e9d6f1c60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f50cc012830 tx=0x7f50cc005e50 comp rx=0 tx=0).stop 2024-08-20T03:09:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.451+0000 7f50e68e4740 1 -- 172.21.15.44:0/265602692 shutdown_connections 2024-08-20T03:09:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.451+0000 7f50e68e4740 1 --2- 172.21.15.44:0/265602692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6fe130 0x564e9d700520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.451+0000 7f50e68e4740 1 --2- 172.21.15.44:0/265602692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d6fdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.451+0000 7f50e68e4740 1 --2- 172.21.15.44:0/265602692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f1880 0x564e9d6f1c60 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.451+0000 7f50e68e4740 1 -- 172.21.15.44:0/265602692 >> 172.21.15.44:0/265602692 conn(0x564e9d6ef6b0 msgr2=0x564e9d6f0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:11.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.452+0000 7f50e2287640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f50c80b3e00 0x7f50c80b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.453+0000 7f50e2287640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f50c80b3e00 0x7f50c80b62c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f50d8005ab0 tx=0x7f50d8005a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:11.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.454+0000 7f50e68e4740 1 -- 172.21.15.44:0/265602692 shutdown_connections 2024-08-20T03:09:11.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.454+0000 7f50e68e4740 1 -- 172.21.15.44:0/265602692 wait complete. 2024-08-20T03:09:11.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.454+0000 7f50e68e4740 1 -- start start 2024-08-20T03:09:11.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.455+0000 7f50e68e4740 1 Processor -- start 2024-08-20T03:09:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.461+0000 7f50e68e4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 0x564e9d6fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.461+0000 7f50e68e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f21a0 0x564e9d6fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.461+0000 7f50e68e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6fe130 0x564e9d6fb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.461+0000 7f50e68e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564e9d4ba700 con 0x564e9d6f21a0 2024-08-20T03:09:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.461+0000 7f50e68e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564e9d5de810 con 0x564e9d6fe130 2024-08-20T03:09:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.461+0000 7f50e68e4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564e9d700e30 con 0x564e9d6f1880 2024-08-20T03:09:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.461+0000 7f50e68e4740 10 client.0 _ll_register_callbacks cb 0x564e9d651700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.461+0000 7f50e1a86640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f21a0 0x564e9d6fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.461+0000 7f50e1a86640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f21a0 0x564e9d6fafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51422/0 (socket says 172.21.15.44:51422) 2024-08-20T03:09:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.461+0000 7f50e1a86640 1 -- 172.21.15.44:0/2380431840 learned_addr learned my addr 172.21.15.44:0/2380431840 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:11.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.461+0000 7f50e2a88640 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6fe130 0x564e9d6fb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.461+0000 7f50e2287640 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 0x564e9d6fdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.462+0000 7f50e1a86640 1 -- 172.21.15.44:0/2380431840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 msgr2=0x564e9d6fdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.462+0000 7f50e1a86640 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 0x564e9d6fdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.462+0000 7f50e1a86640 1 -- 172.21.15.44:0/2380431840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6fe130 msgr2=0x564e9d6fb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.462+0000 7f50e1a86640 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6fe130 0x564e9d6fb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.462+0000 7f50e1a86640 1 -- 172.21.15.44:0/2380431840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e9d70a240 con 0x564e9d6f21a0 2024-08-20T03:09:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.462+0000 7f50e1a86640 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f21a0 0x564e9d6fafe0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0122a0 tx=0x7f50cc0122d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cabd3ab771a313 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.462+0000 7f50c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:11.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.463+0000 7f50c5ffb640 1 -- 172.21.15.44:0/2380431840 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00ecb0 con 0x564e9d6f21a0 2024-08-20T03:09:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.463+0000 7f50e68e4740 1 -- 172.21.15.44:0/2380431840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564e9d607150 con 0x564e9d6f21a0 2024-08-20T03:09:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.463+0000 7f50c5ffb640 1 -- 172.21.15.44:0/2380431840 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f50cc00f460 con 0x564e9d6f21a0 2024-08-20T03:09:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.464+0000 7f50c5ffb640 1 -- 172.21.15.44:0/2380431840 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f50cc00ee50 con 0x564e9d6f21a0 2024-08-20T03:09:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.465+0000 7f50e68e4740 10 client.16863 fetch_fsmap learned FSMap version 33 2024-08-20T03:09:11.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.465+0000 7f50e68e4740 1 -- 172.21.15.44:0/2380431840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564e9d70cba0 con 0x564e9d6f21a0 2024-08-20T03:09:11.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.466+0000 7f50c5ffb640 1 -- 172.21.15.44:0/2380431840 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564e9d607150 con 0x564e9d6f21a0 2024-08-20T03:09:11.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.466+0000 7f50c5ffb640 1 -- 172.21.15.44:0/2380431840 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f50cc00a7f0 con 0x564e9d6f21a0 2024-08-20T03:09:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.466+0000 7f50e68e4740 10 client.16863 fetch_fsmap finished waiting for FSMap version 33 2024-08-20T03:09:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.466+0000 7f50e68e4740 10 client.16863 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:09:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.466+0000 7f50e68e4740 20 client.16863 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.466+0000 7f50e68e4740 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564e9d70df50 0x564e9d710340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.466+0000 7f50e68e4740 4 client.16863 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.466+0000 7f50e68e4740 1 -- 172.21.15.44:0/2380431840 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e9d66a880 con 0x564e9d70df50 2024-08-20T03:09:11.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.467+0000 7f50e2287640 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564e9d70df50 0x564e9d710340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.468+0000 7f50e2287640 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564e9d70df50 0x564e9d710340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94ef83599d7528af server_cookie=74c7e6f38cf91ceb in_seq=0 out_seq=0 2024-08-20T03:09:11.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.468+0000 7f50c5ffb640 10 client.16863 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:09:11.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.470+0000 7f50c5ffb640 1 -- 172.21.15.44:0/2380431840 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e9d66a880 con 0x564e9d70df50 2024-08-20T03:09:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.471+0000 7f50c5ffb640 10 client.16863 handle_command_reply: tid=0 2024-08-20T03:09:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.471+0000 7f50e68e4740 1 client.16863 shutdown 2024-08-20T03:09:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.471+0000 7f50e68e4740 1 -- 172.21.15.44:0/2380431840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f21a0 msgr2=0x564e9d6fafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:11.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.471+0000 7f50e68e4740 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f21a0 0x564e9d6fafe0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0122a0 tx=0x7f50cc0122d0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.474+0000 7f50e68e4740 1 -- 172.21.15.44:0/2380431840 shutdown_connections 2024-08-20T03:09:11.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.474+0000 7f50e68e4740 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564e9d70df50 0x564e9d710340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.474+0000 7f50e68e4740 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6fe130 0x564e9d6fb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.474+0000 7f50e68e4740 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6f21a0 0x564e9d6fafe0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.474+0000 7f50e68e4740 1 --2- 172.21.15.44:0/2380431840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 0x564e9d6fdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.474+0000 7f50e68e4740 1 -- 172.21.15.44:0/2380431840 >> 172.21.15.44:0/2380431840 conn(0x564e9d6ef6b0 msgr2=0x564e9d6f5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:11.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.474+0000 7f50c5ffb640 0 client.16863 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:09:11.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.474+0000 7f50e68e4740 1 -- 172.21.15.44:0/2380431840 shutdown_connections 2024-08-20T03:09:11.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.475+0000 7f50e68e4740 1 -- 172.21.15.44:0/2380431840 wait complete. 2024-08-20T03:09:11.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.475+0000 7f50e68e4740 20 client.16863 trim_cache size 0 max 16384 2024-08-20T03:09:11.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.502+0000 7f50e68e4740 1 Processor -- start 2024-08-20T03:09:11.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.504+0000 7f50e68e4740 1 -- start start 2024-08-20T03:09:11.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.504+0000 7f50e68e4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 0x564e9d6ea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.504+0000 7f50e68e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d6f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.504+0000 7f50e68e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6fe130 0x564e9d6f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.504+0000 7f50e68e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50c000e790 con 0x564e9d6fe130 2024-08-20T03:09:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.504+0000 7f50e68e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50c00103f0 con 0x564e9d6f21a0 2024-08-20T03:09:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.504+0000 7f50e68e4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50c000ee60 con 0x564e9d6f1880 2024-08-20T03:09:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.504+0000 7f50e1a86640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d6f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.504+0000 7f50e1a86640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d6f9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:46798/0 (socket says 172.21.15.44:46798) 2024-08-20T03:09:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.504+0000 7f50e1a86640 1 -- 172.21.15.44:0/294803396 learned_addr learned my addr 172.21.15.44:0/294803396 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.505+0000 7f50e2287640 1 --2- 172.21.15.44:0/294803396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 0x564e9d6ea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.505+0000 7f50e1a86640 1 -- 172.21.15.44:0/294803396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 msgr2=0x564e9d6ea1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.505+0000 7f50e1a86640 1 --2- 172.21.15.44:0/294803396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 0x564e9d6ea1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.505+0000 7f50e1a86640 1 -- 172.21.15.44:0/294803396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6fe130 msgr2=0x564e9d6f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.505+0000 7f50e1a86640 1 --2- 172.21.15.44:0/294803396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6fe130 0x564e9d6f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.505+0000 7f50e1a86640 1 -- 172.21.15.44:0/294803396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e9d6f0130 con 0x564e9d6f21a0 2024-08-20T03:09:11.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.506+0000 7f50e1a86640 1 --2- 172.21.15.44:0/294803396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d6f9860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e870 tx=0x7f50cc00e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e55c3dba804925d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:11.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.510+0000 7f50c7fff640 1 -- 172.21.15.44:0/294803396 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00fe60 con 0x564e9d6f21a0 2024-08-20T03:09:11.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.510+0000 7f50c7fff640 1 -- 172.21.15.44:0/294803396 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50cc00cb90 con 0x564e9d6f21a0 2024-08-20T03:09:11.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.511+0000 7f50c7fff640 1 -- 172.21.15.44:0/294803396 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00f460 con 0x564e9d6f21a0 2024-08-20T03:09:11.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.511+0000 7f50e68e4740 1 -- 172.21.15.44:0/294803396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 msgr2=0x564e9d6f9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:11.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.511+0000 7f50e68e4740 1 --2- 172.21.15.44:0/294803396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d6f9860 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e870 tx=0x7f50cc00e7e0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.512+0000 7f50e68e4740 1 -- 172.21.15.44:0/294803396 shutdown_connections 2024-08-20T03:09:11.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.512+0000 7f50e68e4740 1 --2- 172.21.15.44:0/294803396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6fe130 0x564e9d6f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.512+0000 7f50e68e4740 1 --2- 172.21.15.44:0/294803396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d6f9860 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.512+0000 7f50e68e4740 1 --2- 172.21.15.44:0/294803396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 0x564e9d6ea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.512+0000 7f50e68e4740 1 -- 172.21.15.44:0/294803396 >> 172.21.15.44:0/294803396 conn(0x564e9d6ef6b0 msgr2=0x564e9d70ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:11.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.513+0000 7f50e68e4740 1 -- 172.21.15.44:0/294803396 shutdown_connections 2024-08-20T03:09:11.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.513+0000 7f50e68e4740 1 -- 172.21.15.44:0/294803396 wait complete. 2024-08-20T03:09:11.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.514+0000 7f50e68e4740 1 -- start start 2024-08-20T03:09:11.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.514+0000 7f50e68e4740 1 Processor -- start 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.514+0000 7f50e68e4740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 0x564e9d701360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.514+0000 7f50e68e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d7018a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.515+0000 7f50e68e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6fe130 0x564e9d776b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.515+0000 7f50e68e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50c000fc60 con 0x564e9d6fe130 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.515+0000 7f50e68e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50c000f530 con 0x564e9d6f21a0 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.515+0000 7f50e68e4740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f50c000c790 con 0x564e9d6f1880 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.515+0000 7f50e68e4740 10 client.0 _ll_register_callbacks cb 0x7f50cc00cdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.515+0000 7f50e1a86640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d7018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.515+0000 7f50e1a86640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d7018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:46812/0 (socket says 172.21.15.44:46812) 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.515+0000 7f50e1a86640 1 -- 172.21.15.44:0/716998629 learned_addr learned my addr 172.21.15.44:0/716998629 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.515+0000 7f50e1a86640 1 -- 172.21.15.44:0/716998629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 msgr2=0x564e9d701360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.515+0000 7f50e1a86640 1 --2- 172.21.15.44:0/716998629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 0x564e9d701360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.515+0000 7f50e1a86640 1 -- 172.21.15.44:0/716998629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6fe130 msgr2=0x564e9d776b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:11.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.515+0000 7f50e1a86640 1 --2- 172.21.15.44:0/716998629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6fe130 0x564e9d776b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.516+0000 7f50e1a86640 1 -- 172.21.15.44:0/716998629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e9d777060 con 0x564e9d6f21a0 2024-08-20T03:09:11.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.516+0000 7f50e1a86640 1 --2- 172.21.15.44:0/716998629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d7018a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00c6a0 tx=0x7f50cc00fed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=779a2bf777121ef8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:11.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.516+0000 7f50c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:09:11.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.517+0000 7f50c5ffb640 1 -- 172.21.15.44:0/716998629 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00eab0 con 0x564e9d6f21a0 2024-08-20T03:09:11.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.517+0000 7f50c5ffb640 1 -- 172.21.15.44:0/716998629 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f50cc00a620 con 0x564e9d6f21a0 2024-08-20T03:09:11.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.517+0000 7f50e68e4740 1 -- 172.21.15.44:0/716998629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50cc00cb90 con 0x564e9d6f21a0 2024-08-20T03:09:11.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.519+0000 7f50c5ffb640 1 -- 172.21.15.44:0/716998629 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f50cc00a820 con 0x564e9d6f21a0 2024-08-20T03:09:11.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.519+0000 7f50e68e4740 10 client.26294 fetch_fsmap learned FSMap version 33 2024-08-20T03:09:11.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.519+0000 7f50e68e4740 1 -- 172.21.15.44:0/716998629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564e9d7781e0 con 0x564e9d6f21a0 2024-08-20T03:09:11.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.520+0000 7f50c5ffb640 1 -- 172.21.15.44:0/716998629 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50cc00cb90 con 0x564e9d6f21a0 2024-08-20T03:09:11.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.520+0000 7f50c5ffb640 1 -- 172.21.15.44:0/716998629 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f50cc004b20 con 0x564e9d6f21a0 2024-08-20T03:09:11.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.521+0000 7f50e68e4740 10 client.26294 fetch_fsmap finished waiting for FSMap version 33 2024-08-20T03:09:11.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.521+0000 7f50e68e4740 10 client.26294 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:09:11.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.521+0000 7f50e68e4740 20 client.26294 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:11.523 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.521+0000 7f50e68e4740 1 --2- 172.21.15.44:0/716998629 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564e9d7788e0 0x564e9d77ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:11.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.521+0000 7f50e68e4740 4 client.26294 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:09:11.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.521+0000 7f50e68e4740 1 -- 172.21.15.44:0/716998629 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e9d6ea880 con 0x564e9d7788e0 2024-08-20T03:09:11.524 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.522+0000 7f50e2287640 1 --2- 172.21.15.44:0/716998629 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564e9d7788e0 0x564e9d77ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:11.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.527+0000 7f50c5ffb640 10 client.26294 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:09:11.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.527+0000 7f50e2287640 1 --2- 172.21.15.44:0/716998629 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564e9d7788e0 0x564e9d77ad00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e34d2fe0b3d373af server_cookie=3eeab19048b5091b in_seq=0 out_seq=0 2024-08-20T03:09:11.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.528+0000 7f50c5ffb640 1 -- 172.21.15.44:0/716998629 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e9d6ea880 con 0x564e9d7788e0 2024-08-20T03:09:11.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.528+0000 7f50c5ffb640 10 client.26294 handle_command_reply: tid=0 2024-08-20T03:09:11.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50e68e4740 1 client.26294 shutdown 2024-08-20T03:09:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50e68e4740 1 -- 172.21.15.44:0/716998629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 msgr2=0x564e9d7018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50e68e4740 1 --2- 172.21.15.44:0/716998629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d7018a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00c6a0 tx=0x7f50cc00fed0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50e68e4740 1 -- 172.21.15.44:0/716998629 shutdown_connections 2024-08-20T03:09:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50e68e4740 1 --2- 172.21.15.44:0/716998629 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x564e9d7788e0 0x564e9d77ad00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50e68e4740 1 --2- 172.21.15.44:0/716998629 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e9d6fe130 0x564e9d776b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50e68e4740 1 --2- 172.21.15.44:0/716998629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564e9d6f21a0 0x564e9d7018a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50e68e4740 1 --2- 172.21.15.44:0/716998629 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564e9d6f1880 0x564e9d701360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50e68e4740 1 -- 172.21.15.44:0/716998629 >> 172.21.15.44:0/716998629 conn(0x564e9d6ef6b0 msgr2=0x564e9d6f97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:11.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50c5ffb640 0 client.26294 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:09:11.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50e68e4740 1 -- 172.21.15.44:0/716998629 shutdown_connections 2024-08-20T03:09:11.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50e68e4740 1 -- 172.21.15.44:0/716998629 wait complete. 2024-08-20T03:09:11.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.529+0000 7f50e68e4740 20 client.26294 trim_cache size 0 max 16384 2024-08-20T03:09:11.534 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:09:11.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.531+0000 7f50e44eb640 1 -- 172.21.15.44:0/2865783510 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f50c80b3e00 msgr2=0x7f50c80b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:11.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.531+0000 7f50e44eb640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f50c80b3e00 0x7f50c80b62c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f50d8005ab0 tx=0x7f50d8005a40 comp rx=0 tx=0).stop 2024-08-20T03:09:11.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.531+0000 7f50e44eb640 1 -- 172.21.15.44:0/2865783510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50dc107d20 msgr2=0x7f50dc10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.531+0000 7f50e44eb640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50dc107d20 0x7f50dc10ac20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f50d400e940 tx=0x7f50d400ee00 comp rx=0 tx=0).stop 2024-08-20T03:09:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.531+0000 7f50e44eb640 1 -- 172.21.15.44:0/2865783510 shutdown_connections 2024-08-20T03:09:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.531+0000 7f50e44eb640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f50c80b3e00 0x7f50c80b62c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.531+0000 7f50e44eb640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50dc107d20 0x7f50dc10ac20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.531+0000 7f50e44eb640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50dc1073a0 0x7f50dc10a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.531+0000 7f50e44eb640 1 --2- 172.21.15.44:0/2865783510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50dc101990 0x7f50dc10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.531+0000 7f50e44eb640 1 -- 172.21.15.44:0/2865783510 >> 172.21.15.44:0/2865783510 conn(0x7f50dc0fd400 msgr2=0x7f50dc0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.531+0000 7f50e44eb640 1 -- 172.21.15.44:0/2865783510 shutdown_connections 2024-08-20T03:09:11.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:11.531+0000 7f50e44eb640 1 -- 172.21.15.44:0/2865783510 wait complete. 2024-08-20T03:09:12.238 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T03:09:12.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:12 smithi044 ceph-mon[18790]: pgmap v522: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 847 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 401 op/s 2024-08-20T03:09:12.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:12 smithi165 ceph-mon[22357]: pgmap v522: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 847 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 401 op/s 2024-08-20T03:09:12.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:12 smithi099 ceph-mon[21984]: pgmap v522: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 847 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 401 op/s 2024-08-20T03:09:12.693 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.283+0000 7fd382475640 1 -- 172.21.15.44:0/805759092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd37c11bdd0 msgr2=0x7fd37c11e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.283+0000 7fd382475640 1 --2- 172.21.15.44:0/805759092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd37c11bdd0 0x7fd37c11e1c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd374009be0 tx=0x7fd37402a790 comp rx=0 tx=0).stop 2024-08-20T03:09:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.284+0000 7fd382475640 1 -- 172.21.15.44:0/805759092 shutdown_connections 2024-08-20T03:09:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.284+0000 7fd382475640 1 --2- 172.21.15.44:0/805759092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd37c11bdd0 0x7fd37c11e1c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.284+0000 7fd382475640 1 --2- 172.21.15.44:0/805759092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd37c10a260 0x7fd37c10a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.284+0000 7fd382475640 1 --2- 172.21.15.44:0/805759092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd37c109e80 0x7fd37c110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.284+0000 7fd382475640 1 -- 172.21.15.44:0/805759092 >> 172.21.15.44:0/805759092 conn(0x7fd37c06dcc0 msgr2=0x7fd37c06e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:13.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.284+0000 7fd382475640 1 -- 172.21.15.44:0/805759092 shutdown_connections 2024-08-20T03:09:13.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.284+0000 7fd382475640 1 -- 172.21.15.44:0/805759092 wait complete. 2024-08-20T03:09:13.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.285+0000 7fd382475640 1 Processor -- start 2024-08-20T03:09:13.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.285+0000 7fd382475640 1 -- start start 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.286+0000 7fd382475640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd37c109e80 0x7fd37c1a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.286+0000 7fd382475640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd37c10a260 0x7fd37c1a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.286+0000 7fd382475640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd37c11bdd0 0x7fd37c1ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.286+0000 7fd382475640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c121860 con 0x7fd37c109e80 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.286+0000 7fd382475640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c1216e0 con 0x7fd37c11bdd0 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.286+0000 7fd382475640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c1219e0 con 0x7fd37c10a260 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.286+0000 7fd37bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd37c109e80 0x7fd37c1a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.286+0000 7fd37bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd37c109e80 0x7fd37c1a7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51466/0 (socket says 172.21.15.44:51466) 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.286+0000 7fd37bfff640 1 -- 172.21.15.44:0/3219868760 learned_addr learned my addr 172.21.15.44:0/3219868760 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.286+0000 7fd380a12640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd37c11bdd0 0x7fd37c1ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.286+0000 7fd37b7fe640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd37c10a260 0x7fd37c1a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.287+0000 7fd380a12640 1 -- 172.21.15.44:0/3219868760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd37c10a260 msgr2=0x7fd37c1a82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:13.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.287+0000 7fd380a12640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd37c10a260 0x7fd37c1a82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.287+0000 7fd380a12640 1 -- 172.21.15.44:0/3219868760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd37c109e80 msgr2=0x7fd37c1a7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.287+0000 7fd380a12640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd37c109e80 0x7fd37c1a7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.287+0000 7fd380a12640 1 -- 172.21.15.44:0/3219868760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37c1ade20 con 0x7fd37c11bdd0 2024-08-20T03:09:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.288+0000 7fd380a12640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd37c11bdd0 0x7fd37c1ad650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd374009b30 tx=0x7fd374003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.288+0000 7fd3797fa640 1 -- 172.21.15.44:0/3219868760 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374041020 con 0x7fd37c11bdd0 2024-08-20T03:09:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.288+0000 7fd382475640 1 -- 172.21.15.44:0/3219868760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37c1ae0b0 con 0x7fd37c11bdd0 2024-08-20T03:09:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.288+0000 7fd382475640 1 -- 172.21.15.44:0/3219868760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd37c1b47d0 con 0x7fd37c11bdd0 2024-08-20T03:09:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.289+0000 7fd3797fa640 1 -- 172.21.15.44:0/3219868760 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3740318f0 con 0x7fd37c11bdd0 2024-08-20T03:09:13.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.289+0000 7fd3797fa640 1 -- 172.21.15.44:0/3219868760 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374036040 con 0x7fd37c11bdd0 2024-08-20T03:09:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.293+0000 7fd3797fa640 1 -- 172.21.15.44:0/3219868760 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd374044020 con 0x7fd37c11bdd0 2024-08-20T03:09:13.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.293+0000 7fd38486e740 1 Processor -- start 2024-08-20T03:09:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.294+0000 7fd3797fa640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd3640b3ed0 0x7fd3640b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.294+0000 7fd38486e740 1 -- start start 2024-08-20T03:09:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.295+0000 7fd38486e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca4090880 0x561ca4090c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.295+0000 7fd38486e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40911a0 0x561ca409cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.295+0000 7fd38486e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca409d130 0x561ca409f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.295+0000 7fd38486e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561ca4007930 con 0x561ca409d130 2024-08-20T03:09:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.295+0000 7fd38486e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561ca3ff6e30 con 0x561ca4090880 2024-08-20T03:09:13.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.295+0000 7fd38486e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561ca3ffae70 con 0x561ca40911a0 2024-08-20T03:09:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.295+0000 7fd3797fa640 1 -- 172.21.15.44:0/3219868760 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7fd374104290 con 0x7fd37c11bdd0 2024-08-20T03:09:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.295+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40911a0 0x561ca409cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.295+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40911a0 0x561ca409cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:40992/0 (socket says 172.21.15.44:40992) 2024-08-20T03:09:13.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.295+0000 7fd37b7fe640 1 -- 172.21.15.44:0/238482698 learned_addr learned my addr 172.21.15.44:0/238482698 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:13.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.296+0000 7fd37b7fe640 1 -- 172.21.15.44:0/238482698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca4090880 msgr2=0x561ca4090c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:09:13.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.296+0000 7fd37b7fe640 1 --2- 172.21.15.44:0/238482698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca4090880 0x561ca4090c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.296+0000 7fd37b7fe640 1 -- 172.21.15.44:0/238482698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca409d130 msgr2=0x561ca409f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:13.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.296+0000 7fd380a12640 1 --2- 172.21.15.44:0/238482698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca409d130 0x561ca409f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.296+0000 7fd37b7fe640 1 --2- 172.21.15.44:0/238482698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca409d130 0x561ca409f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.296+0000 7fd37b7fe640 1 -- 172.21.15.44:0/238482698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ca3d56650 con 0x561ca40911a0 2024-08-20T03:09:13.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.296+0000 7fd37b7fe640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd3640b3ed0 0x7fd3640b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.296+0000 7fd37b7fe640 1 --2- 172.21.15.44:0/238482698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40911a0 0x561ca409cbf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009dc0 tx=0x7fd36c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46802f11f925201e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:13.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.297+0000 7fd3627fc640 1 -- 172.21.15.44:0/238482698 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c011e40 con 0x561ca40911a0 2024-08-20T03:09:13.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.297+0000 7fd37b7fe640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd3640b3ed0 0x7fd3640b6390 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd36c023000 tx=0x7fd36c020000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:13.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.297+0000 7fd3627fc640 1 -- 172.21.15.44:0/238482698 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd36c00ed50 con 0x561ca40911a0 2024-08-20T03:09:13.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.298+0000 7fd38486e740 1 -- 172.21.15.44:0/238482698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40911a0 msgr2=0x561ca409cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:13.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.298+0000 7fd38486e740 1 --2- 172.21.15.44:0/238482698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40911a0 0x561ca409cbf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009dc0 tx=0x7fd36c011810 comp rx=0 tx=0).stop 2024-08-20T03:09:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.300+0000 7fd38486e740 1 -- 172.21.15.44:0/238482698 shutdown_connections 2024-08-20T03:09:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.300+0000 7fd38486e740 1 --2- 172.21.15.44:0/238482698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca409d130 0x561ca409f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.300+0000 7fd38486e740 1 --2- 172.21.15.44:0/238482698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40911a0 0x561ca409cbf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.300+0000 7fd38486e740 1 --2- 172.21.15.44:0/238482698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca4090880 0x561ca4090c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.300+0000 7fd38486e740 1 -- 172.21.15.44:0/238482698 >> 172.21.15.44:0/238482698 conn(0x561ca408e6b0 msgr2=0x561ca408f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.300+0000 7fd38486e740 1 -- 172.21.15.44:0/238482698 shutdown_connections 2024-08-20T03:09:13.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.301+0000 7fd38486e740 1 -- 172.21.15.44:0/238482698 wait complete. 2024-08-20T03:09:13.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.301+0000 7fd38486e740 1 -- start start 2024-08-20T03:09:13.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.301+0000 7fd38486e740 1 Processor -- start 2024-08-20T03:09:13.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.302+0000 7fd38486e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca4090880 0x561ca409a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.302+0000 7fd38486e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca40911a0 0x561ca409a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.302+0000 7fd38486e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca409d130 0x561ca409abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.302+0000 7fd38486e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561ca3e59700 con 0x561ca4090880 2024-08-20T03:09:13.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.302+0000 7fd38486e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561ca3f7d810 con 0x561ca40911a0 2024-08-20T03:09:13.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.302+0000 7fd38486e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561ca409fe30 con 0x561ca409d130 2024-08-20T03:09:13.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.302+0000 7fd38486e740 10 client.0 _ll_register_callbacks cb 0x561ca3ff0700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:13.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.303+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca40911a0 0x561ca409a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.303+0000 7fd37b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca40911a0 0x561ca409a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:46834/0 (socket says 172.21.15.44:46834) 2024-08-20T03:09:13.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.303+0000 7fd37b7fe640 1 -- 172.21.15.44:0/176532166 learned_addr learned my addr 172.21.15.44:0/176532166 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:13.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.303+0000 7fd37bfff640 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca4090880 0x561ca409a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.303+0000 7fd380a12640 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca409d130 0x561ca409abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.303+0000 7fd37bfff640 1 -- 172.21.15.44:0/176532166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca409d130 msgr2=0x561ca409abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:13.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.303+0000 7fd37bfff640 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca409d130 0x561ca409abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.303+0000 7fd37bfff640 1 -- 172.21.15.44:0/176532166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca40911a0 msgr2=0x561ca409a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:13.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.303+0000 7fd37bfff640 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca40911a0 0x561ca409a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.303+0000 7fd37bfff640 1 -- 172.21.15.44:0/176532166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ca40a9310 con 0x561ca4090880 2024-08-20T03:09:13.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.304+0000 7fd37bfff640 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca4090880 0x561ca409a140 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd370005fd0 tx=0x7fd37000fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c7cf3640ce3401 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.307+0000 7fd35b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:13.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.307+0000 7fd35b7fe640 1 -- 172.21.15.44:0/176532166 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37000daa0 con 0x561ca4090880 2024-08-20T03:09:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.307+0000 7fd38486e740 1 -- 172.21.15.44:0/176532166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561ca3fa6150 con 0x561ca4090880 2024-08-20T03:09:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.307+0000 7fd35b7fe640 1 -- 172.21.15.44:0/176532166 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7fd37000dc40 con 0x561ca4090880 2024-08-20T03:09:13.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.307+0000 7fd35b7fe640 1 -- 172.21.15.44:0/176532166 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd37000de40 con 0x561ca4090880 2024-08-20T03:09:13.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.308+0000 7fd38486e740 10 client.16872 fetch_fsmap learned FSMap version 33 2024-08-20T03:09:13.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.308+0000 7fd38486e740 1 -- 172.21.15.44:0/176532166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561ca40abde0 con 0x561ca4090880 2024-08-20T03:09:13.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.308+0000 7fd35b7fe640 1 -- 172.21.15.44:0/176532166 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561ca3fa6150 con 0x561ca4090880 2024-08-20T03:09:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.309+0000 7fd35b7fe640 1 -- 172.21.15.44:0/176532166 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7fd37001f020 con 0x561ca4090880 2024-08-20T03:09:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.309+0000 7fd38486e740 10 client.16872 fetch_fsmap finished waiting for FSMap version 33 2024-08-20T03:09:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.309+0000 7fd38486e740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.309+0000 7fd38486e740 20 client.16872 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.309+0000 7fd38486e740 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561ca40ad770 0x561ca40afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.309+0000 7fd38486e740 4 client.16872 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.309+0000 7fd38486e740 1 -- 172.21.15.44:0/176532166 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ca4009880 con 0x561ca40ad770 2024-08-20T03:09:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.309+0000 7fd37b7fe640 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561ca40ad770 0x561ca40afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.310+0000 7fd37b7fe640 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561ca40ad770 0x561ca40afc30 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d623a068a16543d9 server_cookie=104c5f924afbea38 in_seq=0 out_seq=0 2024-08-20T03:09:13.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.310+0000 7fd35b7fe640 10 client.16872 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:13.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.312+0000 7fd35b7fe640 1 -- 172.21.15.44:0/176532166 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ca4009880 con 0x561ca40ad770 2024-08-20T03:09:13.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.312+0000 7fd35b7fe640 10 client.16872 handle_command_reply: tid=0 2024-08-20T03:09:13.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.312+0000 7fd38486e740 1 client.16872 shutdown 2024-08-20T03:09:13.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.313+0000 7fd38486e740 1 -- 172.21.15.44:0/176532166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca4090880 msgr2=0x561ca409a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:13.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.313+0000 7fd38486e740 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca4090880 0x561ca409a140 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd370005fd0 tx=0x7fd37000fc10 comp rx=0 tx=0).stop 2024-08-20T03:09:13.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.313+0000 7fd38486e740 1 -- 172.21.15.44:0/176532166 shutdown_connections 2024-08-20T03:09:13.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.313+0000 7fd38486e740 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561ca40ad770 0x561ca40afc30 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.313+0000 7fd38486e740 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca409d130 0x561ca409abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.313+0000 7fd38486e740 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca40911a0 0x561ca409a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.313+0000 7fd38486e740 1 --2- 172.21.15.44:0/176532166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca4090880 0x561ca409a140 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.313+0000 7fd38486e740 1 -- 172.21.15.44:0/176532166 >> 172.21.15.44:0/176532166 conn(0x561ca408e6b0 msgr2=0x561ca4094250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.313+0000 7fd35b7fe640 0 client.16872 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.313+0000 7fd38486e740 1 -- 172.21.15.44:0/176532166 shutdown_connections 2024-08-20T03:09:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.313+0000 7fd38486e740 1 -- 172.21.15.44:0/176532166 wait complete. 2024-08-20T03:09:13.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.314+0000 7fd38486e740 20 client.16872 trim_cache size 0 max 16384 2024-08-20T03:09:13.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.339+0000 7fd38486e740 1 Processor -- start 2024-08-20T03:09:13.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.340+0000 7fd38486e740 1 -- start start 2024-08-20T03:09:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.340+0000 7fd38486e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca4090880 0x561ca409a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.340+0000 7fd38486e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca40911a0 0x561ca409ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.340+0000 7fd38486e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca409d130 0x561ca40b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.340+0000 7fd38486e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd36800e840 con 0x561ca4090880 2024-08-20T03:09:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.340+0000 7fd38486e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd3680104a0 con 0x561ca40911a0 2024-08-20T03:09:13.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.341+0000 7fd38486e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd36800ef10 con 0x561ca409d130 2024-08-20T03:09:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.342+0000 7fd380a12640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca409d130 0x561ca40b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.342+0000 7fd380a12640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca409d130 0x561ca40b5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41020/0 (socket says 172.21.15.44:41020) 2024-08-20T03:09:13.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.342+0000 7fd380a12640 1 -- 172.21.15.44:0/886077034 learned_addr learned my addr 172.21.15.44:0/886077034 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.343+0000 7fd37bfff640 1 --2- 172.21.15.44:0/886077034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca4090880 0x561ca409a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.343+0000 7fd380a12640 1 -- 172.21.15.44:0/886077034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca40911a0 msgr2=0x561ca409ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.343+0000 7fd37b7fe640 1 --2- 172.21.15.44:0/886077034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca40911a0 0x561ca409ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.343+0000 7fd380a12640 1 --2- 172.21.15.44:0/886077034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca40911a0 0x561ca409ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.343+0000 7fd380a12640 1 -- 172.21.15.44:0/886077034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca4090880 msgr2=0x561ca409a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.343+0000 7fd380a12640 1 --2- 172.21.15.44:0/886077034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca4090880 0x561ca409a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.343+0000 7fd380a12640 1 -- 172.21.15.44:0/886077034 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ca40abc10 con 0x561ca409d130 2024-08-20T03:09:13.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.344+0000 7fd380a12640 1 --2- 172.21.15.44:0/886077034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca409d130 0x561ca40b5500 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd3740cadf0 tx=0x7fd374009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93e85cafa89afc7d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:13.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.344+0000 7fd361ffb640 1 -- 172.21.15.44:0/886077034 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3740e4020 con 0x561ca409d130 2024-08-20T03:09:13.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.345+0000 7fd361ffb640 1 -- 172.21.15.44:0/886077034 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3740cb560 con 0x561ca409d130 2024-08-20T03:09:13.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.345+0000 7fd361ffb640 1 -- 172.21.15.44:0/886077034 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3740d3a80 con 0x561ca409d130 2024-08-20T03:09:13.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.346+0000 7fd38486e740 1 -- 172.21.15.44:0/886077034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca409d130 msgr2=0x561ca40b5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:13.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.346+0000 7fd38486e740 1 --2- 172.21.15.44:0/886077034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca409d130 0x561ca40b5500 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd3740cadf0 tx=0x7fd374009960 comp rx=0 tx=0).stop 2024-08-20T03:09:13.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.346+0000 7fd38486e740 1 -- 172.21.15.44:0/886077034 shutdown_connections 2024-08-20T03:09:13.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.346+0000 7fd38486e740 1 --2- 172.21.15.44:0/886077034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca409d130 0x561ca40b5500 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.346+0000 7fd38486e740 1 --2- 172.21.15.44:0/886077034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca40911a0 0x561ca409ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.346+0000 7fd38486e740 1 --2- 172.21.15.44:0/886077034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca4090880 0x561ca409a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.346+0000 7fd38486e740 1 -- 172.21.15.44:0/886077034 >> 172.21.15.44:0/886077034 conn(0x561ca408e6b0 msgr2=0x561ca409bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:13.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.350+0000 7fd38486e740 1 -- 172.21.15.44:0/886077034 shutdown_connections 2024-08-20T03:09:13.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.350+0000 7fd38486e740 1 -- 172.21.15.44:0/886077034 wait complete. 2024-08-20T03:09:13.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.351+0000 7fd38486e740 1 -- start start 2024-08-20T03:09:13.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.351+0000 7fd38486e740 1 Processor -- start 2024-08-20T03:09:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.352+0000 7fd38486e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca40911a0 0x561ca40a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.352+0000 7fd38486e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca409d130 0x561ca40a0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.352+0000 7fd38486e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40a0820 0x561ca41164c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.352+0000 7fd38486e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd36800fd10 con 0x561ca40911a0 2024-08-20T03:09:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.352+0000 7fd38486e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd36800f5e0 con 0x561ca409d130 2024-08-20T03:09:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.352+0000 7fd38486e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd36800c840 con 0x561ca40a0820 2024-08-20T03:09:13.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.353+0000 7fd38486e740 10 client.0 _ll_register_callbacks cb 0x7fd37000a660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.353+0000 7fd380a12640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40a0820 0x561ca41164c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.353+0000 7fd380a12640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40a0820 0x561ca41164c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41030/0 (socket says 172.21.15.44:41030) 2024-08-20T03:09:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.353+0000 7fd380a12640 1 -- 172.21.15.44:0/2596600196 learned_addr learned my addr 172.21.15.44:0/2596600196 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.353+0000 7fd37bfff640 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca40911a0 0x561ca40a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.354+0000 7fd37b7fe640 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca409d130 0x561ca40a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.354+0000 7fd37bfff640 1 -- 172.21.15.44:0/2596600196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40a0820 msgr2=0x561ca41164c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.354+0000 7fd37bfff640 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40a0820 0x561ca41164c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.354+0000 7fd37bfff640 1 -- 172.21.15.44:0/2596600196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca409d130 msgr2=0x561ca40a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.354+0000 7fd37bfff640 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca409d130 0x561ca40a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.354+0000 7fd37bfff640 1 -- 172.21.15.44:0/2596600196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ca4116a00 con 0x561ca40911a0 2024-08-20T03:09:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.355+0000 7fd37bfff640 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca40911a0 0x561ca40a5a20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd370018750 tx=0x7fd370005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8eabde77313cdf server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.355+0000 7fd35b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.355+0000 7fd35b7fe640 1 -- 172.21.15.44:0/2596600196 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37002f060 con 0x561ca40911a0 2024-08-20T03:09:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.355+0000 7fd35b7fe640 1 -- 172.21.15.44:0/2596600196 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7fd370018d00 con 0x561ca40911a0 2024-08-20T03:09:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.355+0000 7fd38486e740 1 -- 172.21.15.44:0/2596600196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3740cb560 con 0x561ca40911a0 2024-08-20T03:09:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.356+0000 7fd35b7fe640 1 -- 172.21.15.44:0/2596600196 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd370005c00 con 0x561ca40911a0 2024-08-20T03:09:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.356+0000 7fd38486e740 10 client.16878 fetch_fsmap learned FSMap version 33 2024-08-20T03:09:13.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.356+0000 7fd38486e740 1 -- 172.21.15.44:0/2596600196 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561ca4117ad0 con 0x561ca40911a0 2024-08-20T03:09:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.356+0000 7fd35b7fe640 1 -- 172.21.15.44:0/2596600196 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3700054f0 con 0x561ca40911a0 2024-08-20T03:09:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.356+0000 7fd35b7fe640 1 -- 172.21.15.44:0/2596600196 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7fd370004c80 con 0x561ca40911a0 2024-08-20T03:09:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.356+0000 7fd38486e740 10 client.16878 fetch_fsmap finished waiting for FSMap version 33 2024-08-20T03:09:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.357+0000 7fd38486e740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.357+0000 7fd38486e740 20 client.16878 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:13.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.357+0000 7fd38486e740 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561ca4118290 0x561ca411a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:13.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.357+0000 7fd38486e740 4 client.16878 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:09:13.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.357+0000 7fd38486e740 1 -- 172.21.15.44:0/2596600196 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd3740cb560 con 0x561ca4118290 2024-08-20T03:09:13.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.361+0000 7fd37b7fe640 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561ca4118290 0x561ca411a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:13.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.368+0000 7fd37b7fe640 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561ca4118290 0x561ca411a6b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d157f250884ece server_cookie=feec84feb870c656 in_seq=0 out_seq=0 2024-08-20T03:09:13.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.369+0000 7fd35b7fe640 10 client.16878 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:13.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.369+0000 7fd35b7fe640 1 -- 172.21.15.44:0/2596600196 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd3740cb560 con 0x561ca4118290 2024-08-20T03:09:13.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.369+0000 7fd35b7fe640 10 client.16878 handle_command_reply: tid=0 2024-08-20T03:09:13.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.369+0000 7fd38486e740 1 client.16878 shutdown 2024-08-20T03:09:13.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.370+0000 7fd38486e740 1 -- 172.21.15.44:0/2596600196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca40911a0 msgr2=0x561ca40a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:13.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.370+0000 7fd38486e740 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca40911a0 0x561ca40a5a20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd370018750 tx=0x7fd370005a80 comp rx=0 tx=0).stop 2024-08-20T03:09:13.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.371+0000 7fd38486e740 1 -- 172.21.15.44:0/2596600196 shutdown_connections 2024-08-20T03:09:13.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.371+0000 7fd38486e740 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561ca4118290 0x561ca411a6b0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.371+0000 7fd38486e740 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561ca40a0820 0x561ca41164c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.371+0000 7fd35b7fe640 0 client.16878 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:13.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.371+0000 7fd38486e740 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ca409d130 0x561ca40a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.371+0000 7fd38486e740 1 --2- 172.21.15.44:0/2596600196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca40911a0 0x561ca40a5a20 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.371+0000 7fd38486e740 1 -- 172.21.15.44:0/2596600196 >> 172.21.15.44:0/2596600196 conn(0x561ca408e6b0 msgr2=0x561ca408cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:13.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.372+0000 7fd38486e740 1 -- 172.21.15.44:0/2596600196 shutdown_connections 2024-08-20T03:09:13.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.372+0000 7fd38486e740 1 -- 172.21.15.44:0/2596600196 wait complete. 2024-08-20T03:09:13.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.372+0000 7fd38486e740 20 client.16878 trim_cache size 0 max 16384 2024-08-20T03:09:13.375 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:09:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.374+0000 7fd382475640 1 -- 172.21.15.44:0/3219868760 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd3640b3ed0 msgr2=0x7fd3640b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.374+0000 7fd382475640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd3640b3ed0 0x7fd3640b6390 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd36c023000 tx=0x7fd36c020000 comp rx=0 tx=0).stop 2024-08-20T03:09:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.374+0000 7fd382475640 1 -- 172.21.15.44:0/3219868760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd37c11bdd0 msgr2=0x7fd37c1ad650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.374+0000 7fd382475640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd37c11bdd0 0x7fd37c1ad650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd374009b30 tx=0x7fd374003c10 comp rx=0 tx=0).stop 2024-08-20T03:09:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.374+0000 7fd382475640 1 -- 172.21.15.44:0/3219868760 shutdown_connections 2024-08-20T03:09:13.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.374+0000 7fd382475640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd3640b3ed0 0x7fd3640b6390 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.374+0000 7fd382475640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd37c11bdd0 0x7fd37c1ad650 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.374+0000 7fd382475640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd37c10a260 0x7fd37c1a82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.374+0000 7fd382475640 1 --2- 172.21.15.44:0/3219868760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd37c109e80 0x7fd37c1a7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:13.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.374+0000 7fd382475640 1 -- 172.21.15.44:0/3219868760 >> 172.21.15.44:0/3219868760 conn(0x7fd37c06dcc0 msgr2=0x7fd37c11c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:13.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.375+0000 7fd382475640 1 -- 172.21.15.44:0/3219868760 shutdown_connections 2024-08-20T03:09:13.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:13.375+0000 7fd382475640 1 -- 172.21.15.44:0/3219868760 wait complete. 2024-08-20T03:09:14.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:14 smithi044 ceph-mon[18790]: pgmap v523: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 843 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 408 op/s 2024-08-20T03:09:14.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:14 smithi165 ceph-mon[22357]: pgmap v523: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 843 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 408 op/s 2024-08-20T03:09:14.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:14 smithi099 ceph-mon[21984]: pgmap v523: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 843 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 408 op/s 2024-08-20T03:09:15.148 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T03:09:15.148 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T03:09:15.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:15 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:15.556 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:15.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:15 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:15.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:15 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:16.414 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:16 smithi044 ceph-mon[18790]: pgmap v524: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 836 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 356 op/s 2024-08-20T03:09:16.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.411+0000 7f951da75640 1 -- 172.21.15.44:0/4246169139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95100948a0 msgr2=0x7f9510094ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:16.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.411+0000 7f951da75640 1 --2- 172.21.15.44:0/4246169139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95100948a0 0x7f9510094ce0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f950c009900 tx=0x7f950c02a480 comp rx=0 tx=0).stop 2024-08-20T03:09:16.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.411+0000 7f951da75640 1 -- 172.21.15.44:0/4246169139 shutdown_connections 2024-08-20T03:09:16.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.411+0000 7f951da75640 1 --2- 172.21.15.44:0/4246169139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510095220 0x7f95100a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.411+0000 7f951da75640 1 --2- 172.21.15.44:0/4246169139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95100948a0 0x7f9510094ce0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.411+0000 7f951da75640 1 --2- 172.21.15.44:0/4246169139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9510097270 0x7f9510097650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.411+0000 7f951da75640 1 -- 172.21.15.44:0/4246169139 >> 172.21.15.44:0/4246169139 conn(0x7f951000a150 msgr2=0x7f951000c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:16.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.412+0000 7f951da75640 1 -- 172.21.15.44:0/4246169139 shutdown_connections 2024-08-20T03:09:16.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.412+0000 7f951da75640 1 -- 172.21.15.44:0/4246169139 wait complete. 2024-08-20T03:09:16.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.412+0000 7f951da75640 1 Processor -- start 2024-08-20T03:09:16.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.413+0000 7f951da75640 1 -- start start 2024-08-20T03:09:16.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.413+0000 7f951da75640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95100948a0 0x7f95100a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.413+0000 7f951da75640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510095220 0x7f95100a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.413+0000 7f951da75640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9510097270 0x7f95100aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.413+0000 7f951da75640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95100a3690 con 0x7f9510097270 2024-08-20T03:09:16.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.413+0000 7f951da75640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95100a3510 con 0x7f95100948a0 2024-08-20T03:09:16.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.413+0000 7f951da75640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f95100a3810 con 0x7f9510095220 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.413+0000 7f951ca73640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95100948a0 0x7f95100a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.413+0000 7f951ca73640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95100948a0 0x7f95100a4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:46858/0 (socket says 172.21.15.44:46858) 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.413+0000 7f9517fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510095220 0x7f95100a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.413+0000 7f951ca73640 1 -- 172.21.15.44:0/599183753 learned_addr learned my addr 172.21.15.44:0/599183753 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.413+0000 7f951d274640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9510097270 0x7f95100aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.414+0000 7f9517fff640 1 -- 172.21.15.44:0/599183753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95100948a0 msgr2=0x7f95100a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.414+0000 7f9517fff640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95100948a0 0x7f95100a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.414+0000 7f9517fff640 1 -- 172.21.15.44:0/599183753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9510097270 msgr2=0x7f95100aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.414+0000 7f9517fff640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9510097270 0x7f95100aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.414+0000 7f9517fff640 1 -- 172.21.15.44:0/599183753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95100aae30 con 0x7f9510095220 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.414+0000 7f951d274640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9510097270 0x7f95100aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.414+0000 7f9517fff640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510095220 0x7f95100a5310 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f950c02a9a0 tx=0x7f950c003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:16.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.415+0000 7f9515ffb640 1 -- 172.21.15.44:0/599183753 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c03f070 con 0x7f9510095220 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.415+0000 7f951da75640 1 -- 172.21.15.44:0/599183753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95100ab0c0 con 0x7f9510095220 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.415+0000 7f9515ffb640 1 -- 172.21.15.44:0/599183753 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f950c03bd20 con 0x7f9510095220 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.415+0000 7f9515ffb640 1 -- 172.21.15.44:0/599183753 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c035030 con 0x7f9510095220 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.415+0000 7f951da75640 1 -- 172.21.15.44:0/599183753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95100af840 con 0x7f9510095220 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.419+0000 7f95210d0740 1 Processor -- start 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.420+0000 7f95210d0740 1 -- start start 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.420+0000 7f95210d0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309d530 0x55b54309d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.420+0000 7f95210d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b54309de50 0x55b5430a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.420+0000 7f95210d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5430a9de0 0x55b5430ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.420+0000 7f95210d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b543013940 con 0x55b5430a9de0 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.420+0000 7f95210d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b543002e40 con 0x55b54309de50 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.420+0000 7f95210d0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b543006e80 con 0x55b54309d530 2024-08-20T03:09:16.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.420+0000 7f951d274640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309d530 0x55b54309d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.420+0000 7f951d274640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309d530 0x55b54309d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41052/0 (socket says 172.21.15.44:41052) 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.420+0000 7f951d274640 1 -- 172.21.15.44:0/3119157275 learned_addr learned my addr 172.21.15.44:0/3119157275 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.420+0000 7f951ca73640 1 --2- 172.21.15.44:0/3119157275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b54309de50 0x55b5430a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.421+0000 7f951d274640 1 -- 172.21.15.44:0/3119157275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b54309de50 msgr2=0x55b5430a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.421+0000 7f951d274640 1 --2- 172.21.15.44:0/3119157275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b54309de50 0x55b5430a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.421+0000 7f9515ffb640 1 -- 172.21.15.44:0/599183753 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f950c042070 con 0x7f9510095220 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.421+0000 7f951d274640 1 -- 172.21.15.44:0/3119157275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5430a9de0 msgr2=0x55b5430ac1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.421+0000 7f951d274640 1 --2- 172.21.15.44:0/3119157275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5430a9de0 0x55b5430ac1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.421+0000 7f951d274640 1 -- 172.21.15.44:0/3119157275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b542f9f520 con 0x55b54309d530 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.424+0000 7f9515ffb640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95000b3f40 0x7f95000b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.424+0000 7f9515ffb640 1 -- 172.21.15.44:0/599183753 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f950c031a40 con 0x7f9510095220 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.424+0000 7f951ca73640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95000b3f40 0x7f95000b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.424+0000 7f951d274640 1 --2- 172.21.15.44:0/3119157275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309d530 0x55b54309d910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9508009e10 tx=0x7f950800e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2f23a046db87bab server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.425+0000 7f94feffd640 1 -- 172.21.15.44:0/3119157275 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508017070 con 0x55b54309d530 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.425+0000 7f94feffd640 1 -- 172.21.15.44:0/3119157275 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9508013920 con 0x55b54309d530 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.426+0000 7f94feffd640 1 -- 172.21.15.44:0/3119157275 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508012780 con 0x55b54309d530 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.426+0000 7f95210d0740 1 -- 172.21.15.44:0/3119157275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309d530 msgr2=0x55b54309d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.426+0000 7f95210d0740 1 --2- 172.21.15.44:0/3119157275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309d530 0x55b54309d910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9508009e10 tx=0x7f950800e5a0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.426+0000 7f95210d0740 1 -- 172.21.15.44:0/3119157275 shutdown_connections 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.426+0000 7f95210d0740 1 --2- 172.21.15.44:0/3119157275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5430a9de0 0x55b5430ac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.426+0000 7f95210d0740 1 --2- 172.21.15.44:0/3119157275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b54309de50 0x55b5430a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.426+0000 7f95210d0740 1 --2- 172.21.15.44:0/3119157275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309d530 0x55b54309d910 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.426+0000 7f95210d0740 1 -- 172.21.15.44:0/3119157275 >> 172.21.15.44:0/3119157275 conn(0x55b54309b360 msgr2=0x55b54309c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.427+0000 7f951ca73640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95000b3f40 0x7f95000b6400 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9518065ff0 tx=0x7f9518080040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.428+0000 7f95210d0740 1 -- 172.21.15.44:0/3119157275 shutdown_connections 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.428+0000 7f95210d0740 1 -- 172.21.15.44:0/3119157275 wait complete. 2024-08-20T03:09:16.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.429+0000 7f95210d0740 1 -- start start 2024-08-20T03:09:16.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.429+0000 7f95210d0740 1 Processor -- start 2024-08-20T03:09:16.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.430+0000 7f95210d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b54309d530 0x55b5430a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.430+0000 7f95210d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 0x55b5430a7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.430+0000 7f95210d0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5430a9de0 0x55b5430a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.430+0000 7f95210d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b542e65890 con 0x55b54309de50 2024-08-20T03:09:16.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.430+0000 7f95210d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b542f89700 con 0x55b54309d530 2024-08-20T03:09:16.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.430+0000 7f95210d0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b5430acbd0 con 0x55b5430a9de0 2024-08-20T03:09:16.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.430+0000 7f95210d0740 10 client.0 _ll_register_callbacks cb 0x55b542ffe6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:16.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.430+0000 7f9517fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 0x55b5430a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.430+0000 7f9517fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 0x55b5430a7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51530/0 (socket says 172.21.15.44:51530) 2024-08-20T03:09:16.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.430+0000 7f9517fff640 1 -- 172.21.15.44:0/2523040646 learned_addr learned my addr 172.21.15.44:0/2523040646 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:16.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.430+0000 7f951d274640 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5430a9de0 0x55b5430a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.430+0000 7f951ca73640 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b54309d530 0x55b5430a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.431+0000 7f951d274640 1 -- 172.21.15.44:0/2523040646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b54309d530 msgr2=0x55b5430a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.431+0000 7f951d274640 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b54309d530 0x55b5430a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.431+0000 7f951d274640 1 -- 172.21.15.44:0/2523040646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 msgr2=0x55b5430a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.431+0000 7f951d274640 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 0x55b5430a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.431+0000 7f951d274640 1 -- 172.21.15.44:0/2523040646 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b542d62930 con 0x55b5430a9de0 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.431+0000 7f951d274640 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5430a9de0 0x55b5430a77b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c00 tx=0x7f950800ef30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6712b7045f11a520 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.432+0000 7f94f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.433+0000 7f94f7fff640 1 -- 172.21.15.44:0/2523040646 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508017040 con 0x55b5430a9de0 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.433+0000 7f95210d0740 1 -- 172.21.15.44:0/2523040646 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9508013920 con 0x55b5430a9de0 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.433+0000 7f94f7fff640 1 -- 172.21.15.44:0/2523040646 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f9508012910 con 0x55b5430a9de0 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.434+0000 7f94f7fff640 1 -- 172.21.15.44:0/2523040646 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9508012b50 con 0x55b5430a9de0 2024-08-20T03:09:16.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.434+0000 7f95210d0740 10 client.26290 fetch_fsmap learned FSMap version 34 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.434+0000 7f95210d0740 1 -- 172.21.15.44:0/2523040646 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b5430b8760 con 0x55b5430a9de0 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.434+0000 7f94f7fff640 1 -- 172.21.15.44:0/2523040646 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9508013920 con 0x55b5430a9de0 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.434+0000 7f94f7fff640 1 -- 172.21.15.44:0/2523040646 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f9508023070 con 0x55b5430a9de0 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.435+0000 7f95210d0740 10 client.26290 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.435+0000 7f95210d0740 10 client.26290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.435+0000 7f95210d0740 20 client.26290 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.435+0000 7f95210d0740 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b5430b9ab0 0x55b5430bbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.435+0000 7f95210d0740 4 client.26290 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.435+0000 7f95210d0740 1 -- 172.21.15.44:0/2523040646 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b543015890 con 0x55b5430b9ab0 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.435+0000 7f951ca73640 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b5430b9ab0 0x55b5430bbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.439+0000 7f951ca73640 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b5430b9ab0 0x55b5430bbea0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a06a5ca1cc281ceb server_cookie=c6b278f3437eab8b in_seq=0 out_seq=0 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.440+0000 7f94f7fff640 10 client.26290 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.441+0000 7f94f7fff640 1 -- 172.21.15.44:0/2523040646 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b543015890 con 0x55b5430b9ab0 2024-08-20T03:09:16.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.441+0000 7f94f7fff640 10 client.26290 handle_command_reply: tid=0 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.443+0000 7f95210d0740 1 client.26290 shutdown 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.443+0000 7f95210d0740 1 -- 172.21.15.44:0/2523040646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5430a9de0 msgr2=0x55b5430a77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.443+0000 7f95210d0740 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5430a9de0 0x55b5430a77b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c00 tx=0x7f950800ef30 comp rx=0 tx=0).stop 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.443+0000 7f95210d0740 1 -- 172.21.15.44:0/2523040646 shutdown_connections 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.443+0000 7f95210d0740 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b5430b9ab0 0x55b5430bbea0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.444+0000 7f95210d0740 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5430a9de0 0x55b5430a77b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.444+0000 7f95210d0740 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 0x55b5430a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.444+0000 7f95210d0740 1 --2- 172.21.15.44:0/2523040646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b54309d530 0x55b5430a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.444+0000 7f95210d0740 1 -- 172.21.15.44:0/2523040646 >> 172.21.15.44:0/2523040646 conn(0x55b54309b360 msgr2=0x55b5430a0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.444+0000 7f94f7fff640 0 client.26290 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.444+0000 7f95210d0740 1 -- 172.21.15.44:0/2523040646 shutdown_connections 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.444+0000 7f95210d0740 1 -- 172.21.15.44:0/2523040646 wait complete. 2024-08-20T03:09:16.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.445+0000 7f95210d0740 20 client.26290 trim_cache size 0 max 16384 2024-08-20T03:09:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.477+0000 7f95210d0740 1 Processor -- start 2024-08-20T03:09:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.477+0000 7f95210d0740 1 -- start start 2024-08-20T03:09:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.478+0000 7f95210d0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309d530 0x55b54309a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.478+0000 7f95210d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 0x55b5430c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.478+0000 7f95210d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5430a9de0 0x55b5430c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.478+0000 7f95210d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f951014fee0 con 0x55b54309de50 2024-08-20T03:09:16.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.478+0000 7f95210d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9510151b70 con 0x55b5430a9de0 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.478+0000 7f95210d0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f95101505b0 con 0x55b54309d530 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.478+0000 7f9517fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 0x55b5430c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.478+0000 7f9517fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 0x55b5430c1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51532/0 (socket says 172.21.15.44:51532) 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.478+0000 7f9517fff640 1 -- 172.21.15.44:0/1928257661 learned_addr learned my addr 172.21.15.44:0/1928257661 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.479+0000 7f951d274640 1 --2- 172.21.15.44:0/1928257661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5430a9de0 0x55b5430c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.479+0000 7f951ca73640 1 --2- 172.21.15.44:0/1928257661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309d530 0x55b54309a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.479+0000 7f9517fff640 1 -- 172.21.15.44:0/1928257661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309d530 msgr2=0x55b54309a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.479+0000 7f9517fff640 1 --2- 172.21.15.44:0/1928257661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309d530 0x55b54309a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.479+0000 7f9517fff640 1 -- 172.21.15.44:0/1928257661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5430a9de0 msgr2=0x55b5430c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.479+0000 7f9517fff640 1 --2- 172.21.15.44:0/1928257661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5430a9de0 0x55b5430c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.479+0000 7f9517fff640 1 -- 172.21.15.44:0/1928257661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b54309bde0 con 0x55b54309de50 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.479+0000 7f951d274640 1 --2- 172.21.15.44:0/1928257661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5430a9de0 0x55b5430c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:09:16.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.480+0000 7f9517fff640 1 --2- 172.21.15.44:0/1928257661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 0x55b5430c1660 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f950c000c00 tx=0x7f950c003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5fb58bed91dfb2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:16.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.480+0000 7f94fe7fc640 1 -- 172.21.15.44:0/1928257661 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c0e5070 con 0x55b54309de50 2024-08-20T03:09:16.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.480+0000 7f94fe7fc640 1 -- 172.21.15.44:0/1928257661 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f950c0cb1e0 con 0x55b54309de50 2024-08-20T03:09:16.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.480+0000 7f94fe7fc640 1 -- 172.21.15.44:0/1928257661 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c049070 con 0x55b54309de50 2024-08-20T03:09:16.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.481+0000 7f95210d0740 1 -- 172.21.15.44:0/1928257661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 msgr2=0x55b5430c1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:16.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.481+0000 7f95210d0740 1 --2- 172.21.15.44:0/1928257661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 0x55b5430c1660 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f950c000c00 tx=0x7f950c003ed0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.481+0000 7f95210d0740 1 -- 172.21.15.44:0/1928257661 shutdown_connections 2024-08-20T03:09:16.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.481+0000 7f95210d0740 1 --2- 172.21.15.44:0/1928257661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5430a9de0 0x55b5430c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.481+0000 7f95210d0740 1 --2- 172.21.15.44:0/1928257661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309de50 0x55b5430c1660 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.481+0000 7f95210d0740 1 --2- 172.21.15.44:0/1928257661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309d530 0x55b54309a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.481+0000 7f95210d0740 1 -- 172.21.15.44:0/1928257661 >> 172.21.15.44:0/1928257661 conn(0x55b54309b360 msgr2=0x55b543099c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:16.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.482+0000 7f95210d0740 1 -- 172.21.15.44:0/1928257661 shutdown_connections 2024-08-20T03:09:16.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.483+0000 7f95210d0740 1 -- 172.21.15.44:0/1928257661 wait complete. 2024-08-20T03:09:16.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.483+0000 7f95210d0740 1 -- start start 2024-08-20T03:09:16.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.483+0000 7f95210d0740 1 Processor -- start 2024-08-20T03:09:16.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.484+0000 7f95210d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309d530 0x55b5430ad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.484+0000 7f95210d0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309de50 0x55b5430adc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.484+0000 7f95210d0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5430a9de0 0x55b543122620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.484+0000 7f95210d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f95101513e0 con 0x55b54309d530 2024-08-20T03:09:16.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f95210d0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9510150cb0 con 0x55b5430a9de0 2024-08-20T03:09:16.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f95210d0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f951014dee0 con 0x55b54309de50 2024-08-20T03:09:16.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f951ca73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309d530 0x55b5430ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f95210d0740 10 client.0 _ll_register_callbacks cb 0x7f9508009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:16.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f951ca73640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309d530 0x55b5430ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51546/0 (socket says 172.21.15.44:51546) 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f951ca73640 1 -- 172.21.15.44:0/3514296112 learned_addr learned my addr 172.21.15.44:0/3514296112 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f951d274640 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5430a9de0 0x55b543122620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f9517fff640 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309de50 0x55b5430adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f951ca73640 1 -- 172.21.15.44:0/3514296112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309de50 msgr2=0x55b5430adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f951ca73640 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309de50 0x55b5430adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f951ca73640 1 -- 172.21.15.44:0/3514296112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5430a9de0 msgr2=0x55b543122620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f951ca73640 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5430a9de0 0x55b543122620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f951ca73640 1 -- 172.21.15.44:0/3514296112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b543122d60 con 0x55b54309d530 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.485+0000 7f9517fff640 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309de50 0x55b5430adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.486+0000 7f951ca73640 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309d530 0x55b5430ad660 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9518050ee0 tx=0x7f9518068db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbbbec240e3bf62 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.486+0000 7f94f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.486+0000 7f94f7fff640 1 -- 172.21.15.44:0/3514296112 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951808a4f0 con 0x55b54309d530 2024-08-20T03:09:16.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.486+0000 7f94f7fff640 1 -- 172.21.15.44:0/3514296112 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7f9518093040 con 0x55b54309d530 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.486+0000 7f95210d0740 1 -- 172.21.15.44:0/3514296112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f950c0cb1e0 con 0x55b54309d530 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.486+0000 7f94f7fff640 1 -- 172.21.15.44:0/3514296112 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f951808a690 con 0x55b54309d530 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.486+0000 7f95210d0740 10 client.16893 fetch_fsmap learned FSMap version 34 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.486+0000 7f95210d0740 1 -- 172.21.15.44:0/3514296112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b543124010 con 0x55b54309d530 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.487+0000 7f94f7fff640 1 -- 172.21.15.44:0/3514296112 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f950c0cb1e0 con 0x55b54309d530 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.487+0000 7f94f7fff640 1 -- 172.21.15.44:0/3514296112 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f951807e8d0 con 0x55b54309d530 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.487+0000 7f95210d0740 10 client.16893 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.487+0000 7f95210d0740 10 client.16893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.487+0000 7f95210d0740 20 client.16893 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.487+0000 7f95210d0740 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b543124850 0x55b543126c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.487+0000 7f95210d0740 4 client.16893 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.487+0000 7f95210d0740 1 -- 172.21.15.44:0/3514296112 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b543011330 con 0x55b543124850 2024-08-20T03:09:16.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.488+0000 7f9517fff640 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b543124850 0x55b543126c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.488+0000 7f9517fff640 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b543124850 0x55b543126c70 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90850b06db1c24cf server_cookie=2c6eb3627dcbf2d9 in_seq=0 out_seq=0 2024-08-20T03:09:16.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.488+0000 7f94f7fff640 10 client.16893 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:16.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.494+0000 7f94f7fff640 1 -- 172.21.15.44:0/3514296112 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b543011330 con 0x55b543124850 2024-08-20T03:09:16.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.494+0000 7f94f7fff640 10 client.16893 handle_command_reply: tid=0 2024-08-20T03:09:16.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f95210d0740 1 client.16893 shutdown 2024-08-20T03:09:16.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f95210d0740 1 -- 172.21.15.44:0/3514296112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309d530 msgr2=0x55b5430ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:16.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f95210d0740 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309d530 0x55b5430ad660 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9518050ee0 tx=0x7f9518068db0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f95210d0740 1 -- 172.21.15.44:0/3514296112 shutdown_connections 2024-08-20T03:09:16.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f95210d0740 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b543124850 0x55b543126c70 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f95210d0740 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5430a9de0 0x55b543122620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f95210d0740 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b54309de50 0x55b5430adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f95210d0740 1 --2- 172.21.15.44:0/3514296112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b54309d530 0x55b5430ad660 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f94f7fff640 0 client.16893 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:16.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f95210d0740 1 -- 172.21.15.44:0/3514296112 >> 172.21.15.44:0/3514296112 conn(0x55b54309b360 msgr2=0x55b54309a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:16.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f95210d0740 1 -- 172.21.15.44:0/3514296112 shutdown_connections 2024-08-20T03:09:16.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f95210d0740 1 -- 172.21.15.44:0/3514296112 wait complete. 2024-08-20T03:09:16.499 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.495+0000 7f95210d0740 20 client.16893 trim_cache size 0 max 16384 2024-08-20T03:09:16.499 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:09:16.499 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-08-20T03:09:16.499 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "2ce35429-4f01-40b5-b6d0-1fe8f28e438c", 2024-08-20T03:09:16.499 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-08-20T03:09:16.499 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:09:16.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.496+0000 7f951da75640 1 -- 172.21.15.44:0/599183753 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95000b3f40 msgr2=0x7f95000b6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:16.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.496+0000 7f951da75640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95000b3f40 0x7f95000b6400 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9518065ff0 tx=0x7f9518080040 comp rx=0 tx=0).stop 2024-08-20T03:09:16.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.496+0000 7f951da75640 1 -- 172.21.15.44:0/599183753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510095220 msgr2=0x7f95100a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:16.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.496+0000 7f951da75640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510095220 0x7f95100a5310 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f950c02a9a0 tx=0x7f950c003c20 comp rx=0 tx=0).stop 2024-08-20T03:09:16.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.496+0000 7f951da75640 1 -- 172.21.15.44:0/599183753 shutdown_connections 2024-08-20T03:09:16.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.496+0000 7f951da75640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f95000b3f40 0x7f95000b6400 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.496+0000 7f951da75640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9510097270 0x7f95100aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.497+0000 7f951da75640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9510095220 0x7f95100a5310 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.497+0000 7f951da75640 1 --2- 172.21.15.44:0/599183753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95100948a0 0x7f95100a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:16.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.497+0000 7f951da75640 1 -- 172.21.15.44:0/599183753 >> 172.21.15.44:0/599183753 conn(0x7f951000a150 msgr2=0x7f9510093890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:16.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.497+0000 7f951da75640 1 -- 172.21.15.44:0/599183753 shutdown_connections 2024-08-20T03:09:16.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:16.497+0000 7f951da75640 1 -- 172.21.15.44:0/599183753 wait complete. 2024-08-20T03:09:16.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:16 smithi165 ceph-mon[22357]: pgmap v524: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 836 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 356 op/s 2024-08-20T03:09:16.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:16 smithi099 ceph-mon[21984]: pgmap v524: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 836 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 356 op/s 2024-08-20T03:09:17.270 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:09:16 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:09:16.951+0000 7f9a16f7f640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000b0f2 [...a,head] /volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-163/ auth{2=1,4=1} fragtree_t(*^3) v252684 ap=2 RANDEPHEMERALPIN f(v5 m2024-08-20T03:08:06.621416+0000 123=123+0) n(v8 rc2024-08-20T03:08:06.621416+0000 b1270865 124=123+1) old_inodes=2 (inest sync->mix g=2,4) caps={15411=pAsLsXsFs/pAsLsXsFsx@137} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55e4647eab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000b0f2:[<0x10000000399/blog-163 v240566>,<0x10000000208/blogtest_in v1625>,<0x10000000201/src v2398>,<0x10000000006/blogbench-1.0 v2121>,<0x10000000005/tmp v399>,<0x100000001fb/client.0 v393>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v388>,<0x10000000001/sv_1 v411>,<0x10000000000/qa v390>,<0x1/volumes v384>]//[]","memoryvalue":"(2)0x1000000b0f2:[<0x10000000399/blog-163 v252684>,<0x10000000208/blogtest_in v2534>,<0x10000000201/src v2704>,<0x10000000006/blogbench-1.0 v2349>,<0x10000000005/tmp v619>,<0x100000001fb/client.0 v609>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v606>,<0x10000000001/sv_1 v609>,<0x10000000000/qa v597>,<0x1/volumes v577>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-20T03:07:53.564619+0000 122=122+0)","ondisk_value.rstat":"n(v0 rc2024-08-20T03:08:06.621416+0000 b1270865 124=123+1)","memory_value.dirstat":"f(v5 m2024-08-20T03:08:06.621416+0000 123=123+0)","memory_value.rstat":"n(v8 rc2024-08-20T03:08:06.621416+0000 b1270865 124=123+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-20T03:09:17.271 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:09:16 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:09:16.994+0000 7f9a16f7f640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000037db [...a,head] /volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-66/ auth{2=1,4=1} fragtree_t(*^3) v252686 ap=2 RANDEPHEMERALPIN f(v8 m2024-08-20T03:08:04.456450+0000 127=127+0) n(v14 rc2024-08-20T03:08:04.456450+0000 b3093058 128=127+1) old_inodes=5 (inest sync->mix g=2,4) caps={15411=pAsLsXsFs/pAsLsXsFsx@337} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55e45a360000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000037db:[<0x10000000399/blog-66 v123307>,<0x10000000208/blogtest_in v633>,<0x10000000201/src v2106>,<0x10000000006/blogbench-1.0 v1848>,<0x10000000005/tmp v133>,<0x100000001fb/client.0 v129>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v140>,<0x10000000001/sv_1 v166>,<0x10000000000/qa v157>,<0x1/volumes v148>]//[]","memoryvalue":"(2)0x100000037db:[<0x10000000399/blog-66 v252686>,<0x10000000208/blogtest_in v2534>,<0x10000000201/src v2704>,<0x10000000006/blogbench-1.0 v2349>,<0x10000000005/tmp v619>,<0x100000001fb/client.0 v609>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v606>,<0x10000000001/sv_1 v609>,<0x10000000000/qa v597>,<0x1/volumes v577>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-20T03:07:45.486744+0000 126=126+0)","ondisk_value.rstat":"n(v0 rc2024-08-20T03:08:04.456450+0000 b3093058 128=127+1)","memory_value.dirstat":"f(v8 m2024-08-20T03:08:04.456450+0000 127=127+0)","memory_value.rstat":"n(v14 rc2024-08-20T03:08:04.456450+0000 b3093058 128=127+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-20T03:09:17.357 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:09:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:17 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:17 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:17 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:17 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:17 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:17.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:17 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:17 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:17.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:17 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:17 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:17.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:17 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.591 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:17.591 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:17.591 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.591 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.592 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:17.592 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.592 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:17.592 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.592 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:17.592 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.592 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:09:17.514+0000 7f9a16f7f640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000a595 [...a,head] /volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-152/ auth{2=1,3=1,4=1} fragtree_t(*^3) v270239 ap=2 RANDEPHEMERALPIN f(v7 m2024-08-20T03:08:27.223097+0000 161=161+0) n(v11 rc2024-08-20T03:08:27.223097+0000 b3089660 162=161+1) old_inodes=3 (inest sync->mix g=2,3,4) caps={15411=pAsLsXsFs/pAsLsXsFsx@239} mcw={3=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55e461c2e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000a595:[<0x10000000399/blog-152 v238726>,<0x10000000208/blogtest_in v1506>,<0x10000000201/src v2365>,<0x10000000006/blogbench-1.0 v2093>,<0x10000000005/tmp v370>,<0x100000001fb/client.0 v365>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v359>,<0x10000000001/sv_1 v385>,<0x10000000000/qa v362>,<0x1/volumes v357>]//[]","memoryvalue":"(2)0x1000000a595:[<0x10000000399/blog-152 v270239>,<0x10000000208/blogtest_in v2534>,<0x10000000201/src v2704>,<0x10000000006/blogbench-1.0 v2349>,<0x10000000005/tmp v619>,<0x100000001fb/client.0 v609>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v606>,<0x10000000001/sv_1 v609>,<0x10000000000/qa v597>,<0x1/volumes v577>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-20T03:08:24.223144+0000 160=160+0)","ondisk_value.rstat":"n(v0 rc2024-08-20T03:08:27.223097+0000 b3089660 162=161+1)","memory_value.dirstat":"f(v7 m2024-08-20T03:08:27.223097+0000 161=161+0)","memory_value.rstat":"n(v11 rc2024-08-20T03:08:27.223097+0000 b3089660 162=161+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-20T03:09:17.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:17 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:17.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:17 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:17.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:17 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:17 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:17 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:17.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:17 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:17 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:17.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:17 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:17 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:17.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:17 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:17.751 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:17.844 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:09:17.588+0000 7f9a16f7f640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000a926 [...a,head] /volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-155/ auth{2=1,3=1,4=1} fragtree_t(*^3) v256363 ap=2 RANDEPHEMERALPIN f(v2 m2024-08-20T03:08:00.721508+0000 116=116+0) n(v6 rc2024-08-20T03:08:00.721508+0000 b3911151 117=116+1) old_inodes=3 (inest sync->mix g=2,3,4) caps={15411=pAsLsXsFs/pAsLsXsFsx@263} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55e456f87700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000a926:[<0x10000000399/blog-155 v238308>,<0x10000000208/blogtest_in v1506>,<0x10000000201/src v2365>,<0x10000000006/blogbench-1.0 v2093>,<0x10000000005/tmp v370>,<0x100000001fb/client.0 v365>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v359>,<0x10000000001/sv_1 v385>,<0x10000000000/qa v362>,<0x1/volumes v357>]//[]","memoryvalue":"(2)0x1000000a926:[<0x10000000399/blog-155 v256363>,<0x10000000208/blogtest_in v2534>,<0x10000000201/src v2704>,<0x10000000006/blogbench-1.0 v2349>,<0x10000000005/tmp v619>,<0x100000001fb/client.0 v609>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v606>,<0x10000000001/sv_1 v609>,<0x10000000000/qa v597>,<0x1/volumes v577>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-20T03:07:43.373776+0000 115=115+0)","ondisk_value.rstat":"n(v0 rc2024-08-20T03:08:00.721508+0000 b3911151 117=116+1)","memory_value.dirstat":"f(v2 m2024-08-20T03:08:00.721508+0000 116=116+0)","memory_value.rstat":"n(v6 rc2024-08-20T03:08:00.721508+0000 b3911151 117=116+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-20T03:09:17.844 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:09:17 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:09:17.840+0000 7f9a16f7f640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000ad34 [...a,head] /volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-160/ auth{2=1,3=1,4=1} fragtree_t(*^3) v256365 ap=2 RANDEPHEMERALPIN f(v3 m2024-08-20T03:08:12.053332+0000 170=170+0) n(v7 rc2024-08-20T03:08:12.053332+0000 b2756566 171=170+1) old_inodes=3 (inest sync->mix g=2,3,4) caps={15411=pAsLsXsFs/pAsLsXsFsx@197} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55e457eda100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000ad34:[<0x10000000399/blog-160 v240579>,<0x10000000208/blogtest_in v1625>,<0x10000000201/src v2398>,<0x10000000006/blogbench-1.0 v2121>,<0x10000000005/tmp v399>,<0x100000001fb/client.0 v393>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v388>,<0x10000000001/sv_1 v411>,<0x10000000000/qa v390>,<0x1/volumes v384>]//[]","memoryvalue":"(2)0x1000000ad34:[<0x10000000399/blog-160 v256365>,<0x10000000208/blogtest_in v2536>,<0x10000000201/src v2706>,<0x10000000006/blogbench-1.0 v2351>,<0x10000000005/tmp v621>,<0x100000001fb/client.0 v611>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v608>,<0x10000000001/sv_1 v611>,<0x10000000000/qa v599>,<0x1/volumes v579>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 100=100+0)","ondisk_value.rstat":"n(v0 rc2024-08-20T03:08:12.053332+0000 b2756566 171=170+1)","memory_value.dirstat":"f(v3 m2024-08-20T03:08:12.053332+0000 170=170+0)","memory_value.rstat":"n(v7 rc2024-08-20T03:08:12.053332+0000 b2756566 171=170+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-20T03:09:18.519 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub queued for path: / 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: pgmap v525: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 837 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 337 op/s 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: Scrub error on inode 0x1000000b0f2 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-163) see mds.i log and `damage ls` output for details 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: Scrub error on inode 0x100000037db (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-66) see mds.i log and `damage ls` output for details 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:18.520 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.521 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:18.521 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.521 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:18.521 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:18.521 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:18 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.529+0000 7fbd0959a640 1 -- 172.21.15.44:0/211272375 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc035020 con 0x7fbd0c1090d0 2024-08-20T03:09:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.530+0000 7fbd10e40640 1 -- 172.21.15.44:0/211272375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c1090d0 msgr2=0x7fbd0c1094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.530+0000 7fbd10e40640 1 --2- 172.21.15.44:0/211272375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c1090d0 0x7fbd0c1094b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009900 tx=0x7fbcfc02a410 comp rx=0 tx=0).stop 2024-08-20T03:09:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.530+0000 7fbd10e40640 1 -- 172.21.15.44:0/211272375 shutdown_connections 2024-08-20T03:09:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.530+0000 7fbd10e40640 1 --2- 172.21.15.44:0/211272375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd0c10f460 0x7fbd0c115c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.530+0000 7fbd10e40640 1 --2- 172.21.15.44:0/211272375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd0c10eae0 0x7fbd0c10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.530+0000 7fbd10e40640 1 --2- 172.21.15.44:0/211272375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c1090d0 0x7fbd0c1094b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.530+0000 7fbd10e40640 1 -- 172.21.15.44:0/211272375 >> 172.21.15.44:0/211272375 conn(0x7fbd0c078800 msgr2=0x7fbd0c07ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.530+0000 7fbd10e40640 1 -- 172.21.15.44:0/211272375 shutdown_connections 2024-08-20T03:09:18.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.530+0000 7fbd10e40640 1 -- 172.21.15.44:0/211272375 wait complete. 2024-08-20T03:09:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.531+0000 7fbd10e40640 1 Processor -- start 2024-08-20T03:09:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.531+0000 7fbd10e40640 1 -- start start 2024-08-20T03:09:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd10e40640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c1090d0 0x7fbd0c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd10e40640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd0c10eae0 0x7fbd0c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd10e40640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd0c10f460 0x7fbd0c1a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd10e40640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1190c0 con 0x7fbd0c1090d0 2024-08-20T03:09:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd10e40640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c118f40 con 0x7fbd0c10f460 2024-08-20T03:09:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd10e40640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c119240 con 0x7fbd0c10eae0 2024-08-20T03:09:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd0a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c1090d0 0x7fbd0c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd0a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c1090d0 0x7fbd0c19f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41726/0 (socket says 172.21.15.44:41726) 2024-08-20T03:09:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd0a59c640 1 -- 172.21.15.44:0/868249922 learned_addr learned my addr 172.21.15.44:0/868249922 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd0ad9d640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd0c10f460 0x7fbd0c1a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd0c10eae0 0x7fbd0c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd0ad9d640 1 -- 172.21.15.44:0/868249922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd0c10eae0 msgr2=0x7fbd0c19fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd0ad9d640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd0c10eae0 0x7fbd0c19fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd0ad9d640 1 -- 172.21.15.44:0/868249922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c1090d0 msgr2=0x7fbd0c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.532+0000 7fbd0ad9d640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c1090d0 0x7fbd0c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.533+0000 7fbd0ad9d640 1 -- 172.21.15.44:0/868249922 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c1a5630 con 0x7fbd0c10f460 2024-08-20T03:09:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.533+0000 7fbd0a59c640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c1090d0 0x7fbd0c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.533+0000 7fbd0ad9d640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd0c10f460 0x7fbd0c1a4e60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd04005fd0 tx=0x7fbd0400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.533+0000 7fbcfb7fe640 1 -- 172.21.15.44:0/868249922 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400cd80 con 0x7fbd0c10f460 2024-08-20T03:09:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.533+0000 7fbcfb7fe640 1 -- 172.21.15.44:0/868249922 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd04005020 con 0x7fbd0c10f460 2024-08-20T03:09:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.534+0000 7fbcfb7fe640 1 -- 172.21.15.44:0/868249922 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04010690 con 0x7fbd0c10f460 2024-08-20T03:09:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.534+0000 7fbd10e40640 1 -- 172.21.15.44:0/868249922 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0c1a5920 con 0x7fbd0c10f460 2024-08-20T03:09:18.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.534+0000 7fbd10e40640 1 -- 172.21.15.44:0/868249922 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0c1abfe0 con 0x7fbd0c10f460 2024-08-20T03:09:18.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.537+0000 7fbcfb7fe640 1 -- 172.21.15.44:0/868249922 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbd040054d0 con 0x7fbd0c10f460 2024-08-20T03:09:18.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.538+0000 7fbd13239740 1 Processor -- start 2024-08-20T03:09:18.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.538+0000 7fbd13239740 1 -- start start 2024-08-20T03:09:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.538+0000 7fbd13239740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 0x563cc01bfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.538+0000 7fbd13239740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01c0150 0x563cc01cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.538+0000 7fbd13239740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01cc0e0 0x563cc01ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.538+0000 7fbd13239740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563cc01368e0 con 0x563cc01bf830 2024-08-20T03:09:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.538+0000 7fbd13239740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563cc0125de0 con 0x563cc01c0150 2024-08-20T03:09:18.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.538+0000 7fbd13239740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563cc0129e20 con 0x563cc01cc0e0 2024-08-20T03:09:18.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.539+0000 7fbd09d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 0x563cc01bfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.539+0000 7fbd09d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 0x563cc01bfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41738/0 (socket says 172.21.15.44:41738) 2024-08-20T03:09:18.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.539+0000 7fbd09d9b640 1 -- 172.21.15.44:0/1067453649 learned_addr learned my addr 172.21.15.44:0/1067453649 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:18.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.539+0000 7fbcfb7fe640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbcf40b3dc0 0x7fbcf40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.539+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/1067453649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01cc0e0 0x563cc01ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.539+0000 7fbd0ad9d640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbcf40b3dc0 0x7fbcf40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.539+0000 7fbcfb7fe640 1 -- 172.21.15.44:0/868249922 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7fbd040e4530 con 0x7fbd0c10f460 2024-08-20T03:09:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.540+0000 7fbd0a59c640 1 --2- 172.21.15.44:0/1067453649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01c0150 0x563cc01cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.540+0000 7fbd09d9b640 1 -- 172.21.15.44:0/1067453649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01cc0e0 msgr2=0x563cc01ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.540+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/1067453649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01cc0e0 0x563cc01ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.540+0000 7fbd09d9b640 1 -- 172.21.15.44:0/1067453649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01c0150 msgr2=0x563cc01cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.540+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/1067453649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01c0150 0x563cc01cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.540+0000 7fbd09d9b640 1 -- 172.21.15.44:0/1067453649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cbfe85650 con 0x563cc01bf830 2024-08-20T03:09:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.540+0000 7fbd0ad9d640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbcf40b3dc0 0x7fbcf40b6280 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbd040ab0f0 tx=0x7fbd040aaf30 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.540+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/1067453649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 0x563cc01bfc10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbd00012840 tx=0x7fbd0000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de5dbb143dfc351 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.540+0000 7fbcf8ff9640 1 -- 172.21.15.44:0/1067453649 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00012dc0 con 0x563cc01bf830 2024-08-20T03:09:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.540+0000 7fbcf8ff9640 1 -- 172.21.15.44:0/1067453649 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd00010c20 con 0x563cc01bf830 2024-08-20T03:09:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.540+0000 7fbcf8ff9640 1 -- 172.21.15.44:0/1067453649 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0001eac0 con 0x563cc01bf830 2024-08-20T03:09:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.541+0000 7fbd13239740 1 -- 172.21.15.44:0/1067453649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 msgr2=0x563cc01bfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.541+0000 7fbd13239740 1 --2- 172.21.15.44:0/1067453649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 0x563cc01bfc10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbd00012840 tx=0x7fbd0000b810 comp rx=0 tx=0).stop 2024-08-20T03:09:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.541+0000 7fbd13239740 1 -- 172.21.15.44:0/1067453649 shutdown_connections 2024-08-20T03:09:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.541+0000 7fbd13239740 1 --2- 172.21.15.44:0/1067453649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01cc0e0 0x563cc01ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.541+0000 7fbd13239740 1 --2- 172.21.15.44:0/1067453649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01c0150 0x563cc01cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.541+0000 7fbd13239740 1 --2- 172.21.15.44:0/1067453649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 0x563cc01bfc10 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.541+0000 7fbd13239740 1 -- 172.21.15.44:0/1067453649 >> 172.21.15.44:0/1067453649 conn(0x563cc01bd660 msgr2=0x563cc01be800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.542+0000 7fbd13239740 1 -- 172.21.15.44:0/1067453649 shutdown_connections 2024-08-20T03:09:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.542+0000 7fbd13239740 1 -- 172.21.15.44:0/1067453649 wait complete. 2024-08-20T03:09:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.542+0000 7fbd13239740 1 -- start start 2024-08-20T03:09:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.542+0000 7fbd13239740 1 Processor -- start 2024-08-20T03:09:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.542+0000 7fbd13239740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01bf830 0x563cc01cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.542+0000 7fbd13239740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 0x563cc01c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.542+0000 7fbd13239740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01cc0e0 0x563cc01c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.542+0000 7fbd13239740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563cbff88700 con 0x563cc01c0150 2024-08-20T03:09:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.542+0000 7fbd13239740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563cc00ac7c0 con 0x563cc01bf830 2024-08-20T03:09:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.542+0000 7fbd13239740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563cc01ced30 con 0x563cc01cc0e0 2024-08-20T03:09:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.542+0000 7fbd13239740 10 client.0 _ll_register_callbacks cb 0x563cc011f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:18.549 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbd09d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01bf830 0x563cc01cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbd0a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 0x563cc01c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbd09d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01bf830 0x563cc01cbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52202/0 (socket says 172.21.15.44:52202) 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbd09d9b640 1 -- 172.21.15.44:0/1867662813 learned_addr learned my addr 172.21.15.44:0/1867662813 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01cc0e0 0x563cc01c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbd09d9b640 1 -- 172.21.15.44:0/1867662813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01cc0e0 msgr2=0x563cc01c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01cc0e0 0x563cc01c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbd09d9b640 1 -- 172.21.15.44:0/1867662813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 msgr2=0x563cc01c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 0x563cc01c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbd09d9b640 1 -- 172.21.15.44:0/1867662813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cc01d8060 con 0x563cc01bf830 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbd0a59c640 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 0x563cc01c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01bf830 0x563cc01cbb70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbd00005d00 tx=0x7fbd0001e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e09c95070d925b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbcf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:09:18.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.543+0000 7fbcf1ffb640 1 -- 172.21.15.44:0/1867662813 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0001e5b0 con 0x563cc01bf830 2024-08-20T03:09:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbcf1ffb640 1 -- 172.21.15.44:0/1867662813 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7fbd0001ed50 con 0x563cc01bf830 2024-08-20T03:09:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbd13239740 1 -- 172.21.15.44:0/1867662813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563cc00d5100 con 0x563cc01bf830 2024-08-20T03:09:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbcf1ffb640 1 -- 172.21.15.44:0/1867662813 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbd00004020 con 0x563cc01bf830 2024-08-20T03:09:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbd13239740 10 client.26330 fetch_fsmap learned FSMap version 34 2024-08-20T03:09:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbd13239740 1 -- 172.21.15.44:0/1867662813 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563cc01da9e0 con 0x563cc01bf830 2024-08-20T03:09:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbcf1ffb640 1 -- 172.21.15.44:0/1867662813 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563cc00d5100 con 0x563cc01bf830 2024-08-20T03:09:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbcf1ffb640 1 -- 172.21.15.44:0/1867662813 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7fbd000035f0 con 0x563cc01bf830 2024-08-20T03:09:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbd13239740 10 client.26330 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T03:09:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbd13239740 10 client.26330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:18.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbd13239740 20 client.26330 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:18.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbd13239740 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563cc01dbe40 0x563cc01de230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbd13239740 4 client.26330 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:18.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.544+0000 7fbd13239740 1 -- 172.21.15.44:0/1867662813 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cc0138830 con 0x563cc01dbe40 2024-08-20T03:09:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.545+0000 7fbd0a59c640 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563cc01dbe40 0x563cc01de230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.546+0000 7fbd0a59c640 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563cc01dbe40 0x563cc01de230 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=362b69c0dfab0fa7 server_cookie=b81032217e586c2d in_seq=0 out_seq=0 2024-08-20T03:09:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.546+0000 7fbcf1ffb640 10 client.26330 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.549+0000 7fbcf1ffb640 1 -- 172.21.15.44:0/1867662813 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563cc0138830 con 0x563cc01dbe40 2024-08-20T03:09:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.549+0000 7fbcf1ffb640 10 client.26330 handle_command_reply: tid=0 2024-08-20T03:09:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.549+0000 7fbd13239740 1 client.26330 shutdown 2024-08-20T03:09:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.549+0000 7fbd13239740 1 -- 172.21.15.44:0/1867662813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01bf830 msgr2=0x563cc01cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:18.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.549+0000 7fbd13239740 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01bf830 0x563cc01cbb70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbd00005d00 tx=0x7fbd0001e7e0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.549+0000 7fbd13239740 1 -- 172.21.15.44:0/1867662813 shutdown_connections 2024-08-20T03:09:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.550+0000 7fbd13239740 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563cc01dbe40 0x563cc01de230 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.550+0000 7fbd13239740 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01cc0e0 0x563cc01c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.550+0000 7fbd13239740 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 0x563cc01c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.550+0000 7fbd13239740 1 --2- 172.21.15.44:0/1867662813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01bf830 0x563cc01cbb70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.550+0000 7fbcf1ffb640 0 client.26330 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.550+0000 7fbd13239740 1 -- 172.21.15.44:0/1867662813 >> 172.21.15.44:0/1867662813 conn(0x563cc01bd660 msgr2=0x563cc01c31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.550+0000 7fbd13239740 1 -- 172.21.15.44:0/1867662813 shutdown_connections 2024-08-20T03:09:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.550+0000 7fbd13239740 1 -- 172.21.15.44:0/1867662813 wait complete. 2024-08-20T03:09:18.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.550+0000 7fbd13239740 20 client.26330 trim_cache size 0 max 16384 2024-08-20T03:09:18.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub queued for path: / 2024-08-20T03:09:18.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:18.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: pgmap v525: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 837 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 337 op/s 2024-08-20T03:09:18.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:18.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: Scrub error on inode 0x1000000b0f2 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-163) see mds.i log and `damage ls` output for details 2024-08-20T03:09:18.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: Scrub error on inode 0x100000037db (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-66) see mds.i log and `damage ls` output for details 2024-08-20T03:09:18.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:18.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:18.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:18.566 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:18.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:18.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:18.567 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:18 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.573+0000 7fbd13239740 1 Processor -- start 2024-08-20T03:09:18.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.574+0000 7fbd13239740 1 -- start start 2024-08-20T03:09:18.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.574+0000 7fbd13239740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 0x563cc01c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.574+0000 7fbd13239740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01c0150 0x563cc01e3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.574+0000 7fbd13239740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01cc0e0 0x563cc01e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.574+0000 7fbd13239740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b48a0 con 0x563cc01bf830 2024-08-20T03:09:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.574+0000 7fbd13239740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b6530 con 0x563cc01cc0e0 2024-08-20T03:09:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.574+0000 7fbd13239740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b4f70 con 0x563cc01c0150 2024-08-20T03:09:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.574+0000 7fbd09d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 0x563cc01c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.574+0000 7fbd09d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 0x563cc01c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41754/0 (socket says 172.21.15.44:41754) 2024-08-20T03:09:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.574+0000 7fbd09d9b640 1 -- 172.21.15.44:0/4273648331 learned_addr learned my addr 172.21.15.44:0/4273648331 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.575+0000 7fbd0a59c640 1 --2- 172.21.15.44:0/4273648331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01c0150 0x563cc01e3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.575+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/4273648331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01cc0e0 0x563cc01e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.575+0000 7fbd09d9b640 1 -- 172.21.15.44:0/4273648331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01c0150 msgr2=0x563cc01e3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.575+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/4273648331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01c0150 0x563cc01e3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.575+0000 7fbd09d9b640 1 -- 172.21.15.44:0/4273648331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01cc0e0 msgr2=0x563cc01e3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.575+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/4273648331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01cc0e0 0x563cc01e3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.575+0000 7fbd09d9b640 1 -- 172.21.15.44:0/4273648331 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cc01be0e0 con 0x563cc01bf830 2024-08-20T03:09:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.575+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/4273648331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 0x563cc01c8850 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000bdf0 tx=0x7fbd0000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9325314e92dcd6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.576+0000 7fbcf3fff640 1 -- 172.21.15.44:0/4273648331 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0001ed50 con 0x563cc01bf830 2024-08-20T03:09:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.576+0000 7fbcf3fff640 1 -- 172.21.15.44:0/4273648331 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd00016070 con 0x563cc01bf830 2024-08-20T03:09:18.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.576+0000 7fbcf3fff640 1 -- 172.21.15.44:0/4273648331 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00010bf0 con 0x563cc01bf830 2024-08-20T03:09:18.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.577+0000 7fbd13239740 1 -- 172.21.15.44:0/4273648331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 msgr2=0x563cc01c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:18.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.577+0000 7fbd13239740 1 --2- 172.21.15.44:0/4273648331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 0x563cc01c8850 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000bdf0 tx=0x7fbd0000bd60 comp rx=0 tx=0).stop 2024-08-20T03:09:18.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.577+0000 7fbd13239740 1 -- 172.21.15.44:0/4273648331 shutdown_connections 2024-08-20T03:09:18.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.577+0000 7fbd13239740 1 --2- 172.21.15.44:0/4273648331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01cc0e0 0x563cc01e3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.577+0000 7fbd13239740 1 --2- 172.21.15.44:0/4273648331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01c0150 0x563cc01e3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.577+0000 7fbd13239740 1 --2- 172.21.15.44:0/4273648331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01bf830 0x563cc01c8850 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.577+0000 7fbd13239740 1 -- 172.21.15.44:0/4273648331 >> 172.21.15.44:0/4273648331 conn(0x563cc01bd660 msgr2=0x563cc01cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:18.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.577+0000 7fbd13239740 1 -- 172.21.15.44:0/4273648331 shutdown_connections 2024-08-20T03:09:18.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.577+0000 7fbd13239740 1 -- 172.21.15.44:0/4273648331 wait complete. 2024-08-20T03:09:18.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd13239740 1 -- start start 2024-08-20T03:09:18.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd13239740 1 Processor -- start 2024-08-20T03:09:18.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd13239740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01bf830 0x563cc01c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd13239740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 0x563cc01c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd13239740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01cc0e0 0x563cc01d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd0a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 0x563cc01c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd0a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 0x563cc01c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41760/0 (socket says 172.21.15.44:41760) 2024-08-20T03:09:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd09d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01bf830 0x563cc01c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd09d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01bf830 0x563cc01c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41140/0 (socket says 172.21.15.44:41140) 2024-08-20T03:09:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd0a59c640 1 -- 172.21.15.44:0/3874669865 learned_addr learned my addr 172.21.15.44:0/3874669865 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd13239740 1 -- 172.21.15.44:0/3874669865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b5da0 con 0x563cc01c0150 2024-08-20T03:09:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd13239740 1 -- 172.21.15.44:0/3874669865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b5670 con 0x563cc01cc0e0 2024-08-20T03:09:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.578+0000 7fbd13239740 1 -- 172.21.15.44:0/3874669865 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b28a0 con 0x563cc01bf830 2024-08-20T03:09:18.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.579+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/3874669865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01cc0e0 0x563cc01d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.579+0000 7fbd0a59c640 1 -- 172.21.15.44:0/3874669865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01bf830 msgr2=0x563cc01c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.579+0000 7fbd0a59c640 1 --2- 172.21.15.44:0/3874669865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01bf830 0x563cc01c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.579+0000 7fbd0a59c640 1 -- 172.21.15.44:0/3874669865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01cc0e0 msgr2=0x563cc01d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.579+0000 7fbd0a59c640 1 --2- 172.21.15.44:0/3874669865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01cc0e0 0x563cc01d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.579+0000 7fbd0a59c640 1 -- 172.21.15.44:0/3874669865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cc01d3500 con 0x563cc01c0150 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.579+0000 7fbd13239740 10 client.0 _ll_register_callbacks cb 0x7fbcfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.579+0000 7fbd13239740 1 -- 172.21.15.44:0/3874669865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd00016070 con 0x563cc01c0150 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.579+0000 7fbd0a59c640 1 --2- 172.21.15.44:0/3874669865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 0x563cc01c5b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc004650 tx=0x7fbcfc0052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c862394bafb9905b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.579+0000 7fbcf1ffb640 10 client.16911 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.580+0000 7fbcf1ffb640 1 -- 172.21.15.44:0/3874669865 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc035040 con 0x563cc01c0150 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.580+0000 7fbcf1ffb640 1 -- 172.21.15.44:0/3874669865 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7fbcfc004980 con 0x563cc01c0150 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.580+0000 7fbcf1ffb640 1 -- 172.21.15.44:0/3874669865 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbcfc004bc0 con 0x563cc01c0150 2024-08-20T03:09:18.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.580+0000 7fbd13239740 10 client.16911 fetch_fsmap learned FSMap version 34 2024-08-20T03:09:18.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.580+0000 7fbd13239740 1 -- 172.21.15.44:0/3874669865 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=34}) -- 0x563cc01d46c0 con 0x563cc01c0150 2024-08-20T03:09:18.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.580+0000 7fbcf1ffb640 1 -- 172.21.15.44:0/3874669865 <== mon.0 v2:172.21.15.44:3300/0 4 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7fbcfc0315c0 con 0x563cc01c0150 2024-08-20T03:09:18.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.580+0000 7fbd13239740 10 client.16911 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T03:09:18.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.580+0000 7fbd13239740 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:18.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.580+0000 7fbd13239740 20 client.16911 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:18.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.580+0000 7fbd13239740 1 --2- 172.21.15.44:0/3874669865 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563cc0246a50 0x563cc0248e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:18.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.581+0000 7fbd13239740 4 client.16911 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:09:18.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.581+0000 7fbd13239740 1 -- 172.21.15.44:0/3874669865 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563cc01342d0 con 0x563cc0246a50 2024-08-20T03:09:18.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.582+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/3874669865 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563cc0246a50 0x563cc0248e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:18.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.583+0000 7fbcf1ffb640 10 client.16911 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:18.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.583+0000 7fbd09d9b640 1 --2- 172.21.15.44:0/3874669865 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563cc0246a50 0x563cc0248e40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb52d2ee07e39022 server_cookie=5b4e79198ca87220 in_seq=0 out_seq=0 2024-08-20T03:09:18.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.588+0000 7fbcf1ffb640 1 -- 172.21.15.44:0/3874669865 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x563cc01342d0 con 0x563cc0246a50 2024-08-20T03:09:18.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.588+0000 7fbcf1ffb640 10 client.16911 handle_command_reply: tid=0 2024-08-20T03:09:18.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.588+0000 7fbd13239740 1 client.16911 shutdown 2024-08-20T03:09:18.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.588+0000 7fbd13239740 1 -- 172.21.15.44:0/3874669865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 msgr2=0x563cc01c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:18.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.588+0000 7fbd13239740 1 --2- 172.21.15.44:0/3874669865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 0x563cc01c5b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc004650 tx=0x7fbcfc0052c0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.588+0000 7fbd13239740 1 -- 172.21.15.44:0/3874669865 shutdown_connections 2024-08-20T03:09:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.588+0000 7fbd13239740 1 --2- 172.21.15.44:0/3874669865 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563cc0246a50 0x563cc0248e40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.588+0000 7fbcf1ffb640 0 client.16911 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.588+0000 7fbd13239740 1 --2- 172.21.15.44:0/3874669865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563cc01cc0e0 0x563cc01d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.588+0000 7fbd13239740 1 --2- 172.21.15.44:0/3874669865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563cc01c0150 0x563cc01c5b50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.588+0000 7fbd13239740 1 --2- 172.21.15.44:0/3874669865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563cc01bf830 0x563cc01c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.588+0000 7fbd13239740 1 -- 172.21.15.44:0/3874669865 >> 172.21.15.44:0/3874669865 conn(0x563cc01bd660 msgr2=0x563cc01ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.589+0000 7fbd13239740 1 -- 172.21.15.44:0/3874669865 shutdown_connections 2024-08-20T03:09:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.589+0000 7fbd13239740 1 -- 172.21.15.44:0/3874669865 wait complete. 2024-08-20T03:09:18.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.589+0000 7fbd13239740 20 client.16911 trim_cache size 0 max 16384 2024-08-20T03:09:18.592 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:09:18.593 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "scrub active (21046 inodes in the stack)", 2024-08-20T03:09:18.593 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-08-20T03:09:18.593 INFO:teuthology.orchestra.run.smithi044.stdout: "2ce35429-4f01-40b5-b6d0-1fe8f28e438c": { 2024-08-20T03:09:18.593 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-08-20T03:09:18.593 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "2ce35429-4f01-40b5-b6d0-1fe8f28e438c", 2024-08-20T03:09:18.593 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-08-20T03:09:18.593 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:09:18.593 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:09:18.593 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:09:18.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.589+0000 7fbd10e40640 1 -- 172.21.15.44:0/868249922 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbcf40b3dc0 msgr2=0x7fbcf40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:18.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.589+0000 7fbd10e40640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbcf40b3dc0 0x7fbcf40b6280 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbd040ab0f0 tx=0x7fbd040aaf30 comp rx=0 tx=0).stop 2024-08-20T03:09:18.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.589+0000 7fbd10e40640 1 -- 172.21.15.44:0/868249922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd0c10f460 msgr2=0x7fbd0c1a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:18.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.589+0000 7fbd10e40640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd0c10f460 0x7fbd0c1a4e60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd04005fd0 tx=0x7fbd0400ee60 comp rx=0 tx=0).stop 2024-08-20T03:09:18.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.590+0000 7fbd10e40640 1 -- 172.21.15.44:0/868249922 shutdown_connections 2024-08-20T03:09:18.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.590+0000 7fbd10e40640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbcf40b3dc0 0x7fbcf40b6280 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.590+0000 7fbd10e40640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd0c10f460 0x7fbd0c1a4e60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.590+0000 7fbd10e40640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd0c10eae0 0x7fbd0c19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.590+0000 7fbd10e40640 1 --2- 172.21.15.44:0/868249922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd0c1090d0 0x7fbd0c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:18.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.590+0000 7fbd10e40640 1 -- 172.21.15.44:0/868249922 >> 172.21.15.44:0/868249922 conn(0x7fbd0c078800 msgr2=0x7fbd0c079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:18.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.590+0000 7fbd10e40640 1 -- 172.21.15.44:0/868249922 shutdown_connections 2024-08-20T03:09:18.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:18.590+0000 7fbd10e40640 1 -- 172.21.15.44:0/868249922 wait complete. 2024-08-20T03:09:18.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub queued for path: / 2024-08-20T03:09:18.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:18.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: pgmap v525: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 837 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 337 op/s 2024-08-20T03:09:18.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:18.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: Scrub error on inode 0x1000000b0f2 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-163) see mds.i log and `damage ls` output for details 2024-08-20T03:09:18.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: Scrub error on inode 0x100000037db (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-66) see mds.i log and `damage ls` output for details 2024-08-20T03:09:18.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:18.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:18.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:18.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:18.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:18.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:18.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:18.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:18 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:19.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:19 smithi044 ceph-mon[18790]: Scrub error on inode 0x1000000a595 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-152) see mds.i log and `damage ls` output for details 2024-08-20T03:09:19.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:19 smithi044 ceph-mon[18790]: Scrub error on inode 0x1000000a926 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-155) see mds.i log and `damage ls` output for details 2024-08-20T03:09:19.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:19 smithi044 ceph-mon[18790]: Scrub error on inode 0x1000000ad34 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-160) see mds.i log and `damage ls` output for details 2024-08-20T03:09:19.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:19 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:19.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:19 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:09:19.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:19 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:19.582 INFO:tasks.cephfs.filesystem:scrub status for tag:2ce35429-4f01-40b5-b6d0-1fe8f28e438c - {'path': '/', 'tag': '2ce35429-4f01-40b5-b6d0-1fe8f28e438c', 'options': 'recursive,force'} 2024-08-20T03:09:19.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:19 smithi165 ceph-mon[22357]: Scrub error on inode 0x1000000a595 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-152) see mds.i log and `damage ls` output for details 2024-08-20T03:09:19.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:19 smithi165 ceph-mon[22357]: Scrub error on inode 0x1000000a926 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-155) see mds.i log and `damage ls` output for details 2024-08-20T03:09:19.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:19 smithi165 ceph-mon[22357]: Scrub error on inode 0x1000000ad34 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-160) see mds.i log and `damage ls` output for details 2024-08-20T03:09:19.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:19 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:19.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:19 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:09:19.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:19 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:19.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:19 smithi099 ceph-mon[21984]: Scrub error on inode 0x1000000a595 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-152) see mds.i log and `damage ls` output for details 2024-08-20T03:09:19.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:19 smithi099 ceph-mon[21984]: Scrub error on inode 0x1000000a926 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-155) see mds.i log and `damage ls` output for details 2024-08-20T03:09:19.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:19 smithi099 ceph-mon[21984]: Scrub error on inode 0x1000000ad34 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-160) see mds.i log and `damage ls` output for details 2024-08-20T03:09:19.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:19 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:19.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:19 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:09:19.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:19 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:20.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:20 smithi044 ceph-mon[18790]: pgmap v526: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 841 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 352 op/s 2024-08-20T03:09:20.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:20 smithi165 ceph-mon[22357]: pgmap v526: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 841 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 352 op/s 2024-08-20T03:09:20.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:20 smithi099 ceph-mon[21984]: pgmap v526: 97 pgs: 10 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 65 active+clean; 841 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 352 op/s 2024-08-20T03:09:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.381+0000 7fe0e0d43640 1 -- 172.21.15.44:0/2308898785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc101920 msgr2=0x7fe0dc101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:21.383 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.381+0000 7fe0e0d43640 1 --2- 172.21.15.44:0/2308898785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc101920 0x7fe0dc101d00 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009900 tx=0x7fe0cc02a410 comp rx=0 tx=0).stop 2024-08-20T03:09:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.381+0000 7fe0e0d43640 1 -- 172.21.15.44:0/2308898785 shutdown_connections 2024-08-20T03:09:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.381+0000 7fe0e0d43640 1 --2- 172.21.15.44:0/2308898785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc107c50 0x7fe0dc10e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.381+0000 7fe0e0d43640 1 --2- 172.21.15.44:0/2308898785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0dc1072d0 0x7fe0dc107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.381+0000 7fe0e0d43640 1 --2- 172.21.15.44:0/2308898785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc101920 0x7fe0dc101d00 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.381+0000 7fe0e0d43640 1 -- 172.21.15.44:0/2308898785 >> 172.21.15.44:0/2308898785 conn(0x7fe0dc078620 msgr2=0x7fe0dc0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.381+0000 7fe0e0d43640 1 -- 172.21.15.44:0/2308898785 shutdown_connections 2024-08-20T03:09:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.381+0000 7fe0e0d43640 1 -- 172.21.15.44:0/2308898785 wait complete. 2024-08-20T03:09:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0e0d43640 1 Processor -- start 2024-08-20T03:09:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0e0d43640 1 -- start start 2024-08-20T03:09:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0e0d43640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0dc101920 0x7fe0dc198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0e0d43640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc1072d0 0x7fe0dc199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0e0d43640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc107c50 0x7fe0dc19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0e0d43640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc111910 con 0x7fe0dc1072d0 2024-08-20T03:09:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0e0d43640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc111790 con 0x7fe0dc101920 2024-08-20T03:09:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0e0d43640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc111a90 con 0x7fe0dc107c50 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0d3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc1072d0 0x7fe0dc199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0d3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc1072d0 0x7fe0dc199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41766/0 (socket says 172.21.15.44:41766) 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0d3fff640 1 -- 172.21.15.44:0/1182108886 learned_addr learned my addr 172.21.15.44:0/1182108886 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0dad9d640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc107c50 0x7fe0dc19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0da59c640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0dc101920 0x7fe0dc198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0d3fff640 1 -- 172.21.15.44:0/1182108886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc107c50 msgr2=0x7fe0dc19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0d3fff640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc107c50 0x7fe0dc19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0d3fff640 1 -- 172.21.15.44:0/1182108886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0dc101920 msgr2=0x7fe0dc198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0d3fff640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0dc101920 0x7fe0dc198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.382+0000 7fe0d3fff640 1 -- 172.21.15.44:0/1182108886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc19efa0 con 0x7fe0dc1072d0 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.383+0000 7fe0d3fff640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc1072d0 0x7fe0dc199440 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400e9f0 tx=0x7fe0c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.383+0000 7fe0d37fe640 1 -- 172.21.15.44:0/1182108886 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c400cd80 con 0x7fe0dc1072d0 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.383+0000 7fe0d37fe640 1 -- 172.21.15.44:0/1182108886 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0c40108e0 con 0x7fe0dc1072d0 2024-08-20T03:09:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.383+0000 7fe0e0d43640 1 -- 172.21.15.44:0/1182108886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc19f290 con 0x7fe0dc1072d0 2024-08-20T03:09:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.383+0000 7fe0d37fe640 1 -- 172.21.15.44:0/1182108886 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4018980 con 0x7fe0dc1072d0 2024-08-20T03:09:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.383+0000 7fe0e0d43640 1 -- 172.21.15.44:0/1182108886 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc1a5950 con 0x7fe0dc1072d0 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.385+0000 7fe0d37fe640 1 -- 172.21.15.44:0/1182108886 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fe0c4010430 con 0x7fe0dc1072d0 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.385+0000 7fe0e3c0f740 1 Processor -- start 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0e3c0f740 1 -- start start 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0e3c0f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b661bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0e3c0f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6627cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0e3c0f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b662a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0e3c0f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5605b6591360 con 0x5605b661b950 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0e3c0f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5605b6580860 con 0x5605b661c270 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0e3c0f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5605b65848a0 con 0x5605b6628200 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0dad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b661bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0dad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b661bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41774/0 (socket says 172.21.15.44:41774) 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0dad9d640 1 -- 172.21.15.44:0/2959081530 learned_addr learned my addr 172.21.15.44:0/2959081530 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0d3fff640 1 --2- 172.21.15.44:0/2959081530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b662a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0da59c640 1 --2- 172.21.15.44:0/2959081530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6627cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0dad9d640 1 -- 172.21.15.44:0/2959081530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 msgr2=0x5605b662a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0dad9d640 1 --2- 172.21.15.44:0/2959081530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b662a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0dad9d640 1 -- 172.21.15.44:0/2959081530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 msgr2=0x5605b6627cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0dad9d640 1 --2- 172.21.15.44:0/2959081530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6627cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0dad9d640 1 -- 172.21.15.44:0/2959081530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b6527e40 con 0x5605b661b950 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0d3fff640 1 --2- 172.21.15.44:0/2959081530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b662a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0da59c640 1 --2- 172.21.15.44:0/2959081530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6627cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.386+0000 7fe0d37fe640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0c80b3d40 0x7fe0c80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.387+0000 7fe0dad9d640 1 --2- 172.21.15.44:0/2959081530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b661bd30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009fd0 tx=0x7fe0d40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3348bfd30d3123e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.387+0000 7fe0da59c640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0c80b3d40 0x7fe0c80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.387+0000 7fe0d37fe640 1 -- 172.21.15.44:0/1182108886 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7fe0c40e42f0 con 0x7fe0dc1072d0 2024-08-20T03:09:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.387+0000 7fe0d0ff9640 1 -- 172.21.15.44:0/2959081530 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4004d20 con 0x5605b661b950 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.387+0000 7fe0d0ff9640 1 -- 172.21.15.44:0/2959081530 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0d4004050 con 0x5605b661b950 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.387+0000 7fe0d0ff9640 1 -- 172.21.15.44:0/2959081530 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4005280 con 0x5605b661b950 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2959081530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 msgr2=0x5605b661bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0da59c640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0c80b3d40 0x7fe0c80b6200 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc031440 tx=0x7fe0cc005890 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2959081530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b661bd30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009fd0 tx=0x7fe0d40118d0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2959081530 shutdown_connections 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2959081530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b662a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2959081530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6627cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2959081530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b661bd30 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2959081530 >> 172.21.15.44:0/2959081530 conn(0x5605b6619730 msgr2=0x5605b661a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2959081530 shutdown_connections 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2959081530 wait complete. 2024-08-20T03:09:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 -- start start 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 Processor -- start 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b661b950 0x5605b6627ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661c270 0x5605b6624f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b6628200 0x5605b6625550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5605b64b9ab0 con 0x5605b661c270 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5605b6507780 con 0x5605b6628200 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.388+0000 7fe0e3c0f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5605b662aee0 con 0x5605b661b950 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0e3c0f740 10 client.0 _ll_register_callbacks cb 0x5605b628fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0d3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661c270 0x5605b6624f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0d3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661c270 0x5605b6624f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41790/0 (socket says 172.21.15.44:41790) 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0d3fff640 1 -- 172.21.15.44:0/2836758607 learned_addr learned my addr 172.21.15.44:0/2836758607 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0da59c640 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b661b950 0x5605b6627ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0dad9d640 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b6628200 0x5605b6625550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.396 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0d3fff640 1 -- 172.21.15.44:0/2836758607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b661b950 msgr2=0x5605b6627ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0d3fff640 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b661b950 0x5605b6627ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0d3fff640 1 -- 172.21.15.44:0/2836758607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b6628200 msgr2=0x5605b6625550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0d3fff640 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b6628200 0x5605b6625550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0d3fff640 1 -- 172.21.15.44:0/2836758607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b62e3790 con 0x5605b661c270 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0d3fff640 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661c270 0x5605b6624f20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40a8240 tx=0x7fe0c4011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84da78dbb30195c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0c1ffb640 1 -- 172.21.15.44:0/2836758607 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40aaae0 con 0x5605b661c270 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0c1ffb640 1 -- 172.21.15.44:0/2836758607 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7fe0c40a7260 con 0x5605b661c270 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2836758607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5605b652fe00 con 0x5605b661c270 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0c1ffb640 1 -- 172.21.15.44:0/2836758607 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe0c40aafd0 con 0x5605b661c270 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0e3c0f740 10 client.16923 fetch_fsmap learned FSMap version 34 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2836758607 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5605b635c620 con 0x5605b661c270 2024-08-20T03:09:21.397 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.389+0000 7fe0c1ffb640 1 -- 172.21.15.44:0/2836758607 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5605b652fe00 con 0x5605b661c270 2024-08-20T03:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.390+0000 7fe0c1ffb640 1 -- 172.21.15.44:0/2836758607 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7fe0c4018c40 con 0x5605b661c270 2024-08-20T03:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.390+0000 7fe0e3c0f740 10 client.16923 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T03:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.390+0000 7fe0e3c0f740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.390+0000 7fe0e3c0f740 20 client.16923 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.390+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5605b6637bf0 0x5605b663a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.390+0000 7fe0e3c0f740 4 client.16923 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.390+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2836758607 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605b65932b0 con 0x5605b6637bf0 2024-08-20T03:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.390+0000 7fe0da59c640 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5605b6637bf0 0x5605b663a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.390+0000 7fe0c1ffb640 10 client.16923 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:21.398 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.390+0000 7fe0da59c640 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5605b6637bf0 0x5605b663a000 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e9c1e35fcc5ce server_cookie=e354cfb81cd2ee82 in_seq=0 out_seq=0 2024-08-20T03:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.393+0000 7fe0c1ffb640 1 -- 172.21.15.44:0/2836758607 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605b65932b0 con 0x5605b6637bf0 2024-08-20T03:09:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.393+0000 7fe0c1ffb640 10 client.16923 handle_command_reply: tid=0 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.393+0000 7fe0e3c0f740 1 client.16923 shutdown 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.393+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2836758607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661c270 msgr2=0x5605b6624f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.393+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661c270 0x5605b6624f20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40a8240 tx=0x7fe0c4011f90 comp rx=0 tx=0).stop 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.393+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2836758607 shutdown_connections 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.393+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5605b6637bf0 0x5605b663a000 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.394+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b6628200 0x5605b6625550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.394+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661c270 0x5605b6624f20 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.394+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2836758607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b661b950 0x5605b6627ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.394+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2836758607 >> 172.21.15.44:0/2836758607 conn(0x5605b6619730 msgr2=0x5605b661f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.394+0000 7fe0c1ffb640 0 client.16923 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.394+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2836758607 shutdown_connections 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.394+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2836758607 wait complete. 2024-08-20T03:09:21.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.394+0000 7fe0e3c0f740 20 client.16923 trim_cache size 0 max 16384 2024-08-20T03:09:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.408+0000 7fe0e3c0f740 1 Processor -- start 2024-08-20T03:09:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.408+0000 7fe0e3c0f740 1 -- start start 2024-08-20T03:09:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.408+0000 7fe0e3c0f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b661af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.408+0000 7fe0e3c0f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6619040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.408+0000 7fe0e3c0f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b6624790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.408+0000 7fe0e3c0f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1ae360 con 0x5605b661b950 2024-08-20T03:09:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.408+0000 7fe0e3c0f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1afff0 con 0x5605b661c270 2024-08-20T03:09:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.408+0000 7fe0e3c0f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1aea30 con 0x5605b6628200 2024-08-20T03:09:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0da59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b661af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0da59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b661af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41806/0 (socket says 172.21.15.44:41806) 2024-08-20T03:09:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0da59c640 1 -- 172.21.15.44:0/569243989 learned_addr learned my addr 172.21.15.44:0/569243989 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0dad9d640 1 --2- 172.21.15.44:0/569243989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b6624790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0d3fff640 1 --2- 172.21.15.44:0/569243989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6619040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0da59c640 1 -- 172.21.15.44:0/569243989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 msgr2=0x5605b6624790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0da59c640 1 --2- 172.21.15.44:0/569243989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b6624790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0da59c640 1 -- 172.21.15.44:0/569243989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 msgr2=0x5605b6619040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0da59c640 1 --2- 172.21.15.44:0/569243989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6619040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0da59c640 1 -- 172.21.15.44:0/569243989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b661a1a0 con 0x5605b661b950 2024-08-20T03:09:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0da59c640 1 --2- 172.21.15.44:0/569243989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b661af30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009900 tx=0x7fe0cc03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b14630fe2dbc0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0c3fff640 1 -- 172.21.15.44:0/569243989 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc035040 con 0x5605b661b950 2024-08-20T03:09:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0c3fff640 1 -- 172.21.15.44:0/569243989 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0cc057070 con 0x5605b661b950 2024-08-20T03:09:21.413 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.409+0000 7fe0c3fff640 1 -- 172.21.15.44:0/569243989 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc004b40 con 0x5605b661b950 2024-08-20T03:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.410+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/569243989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 msgr2=0x5605b661af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.410+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/569243989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b661af30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009900 tx=0x7fe0cc03e000 comp rx=0 tx=0).stop 2024-08-20T03:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.410+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/569243989 shutdown_connections 2024-08-20T03:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.410+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/569243989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b6624790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.410+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/569243989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6619040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.416 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.410+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/569243989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b661af30 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.410+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/569243989 >> 172.21.15.44:0/569243989 conn(0x5605b6619730 msgr2=0x5605b661a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.410+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/569243989 shutdown_connections 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.410+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/569243989 wait complete. 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.410+0000 7fe0e3c0f740 1 -- start start 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.410+0000 7fe0e3c0f740 1 Processor -- start 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0e3c0f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b6629990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0e3c0f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6629ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0e3c0f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b66287e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0e3c0f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1af860 con 0x5605b661b950 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0e3c0f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1af130 con 0x5605b661c270 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0e3c0f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc1ac360 con 0x5605b6628200 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0e3c0f740 10 client.0 _ll_register_callbacks cb 0x7fe0c40a6e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0da59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b6629990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.417 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0da59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b6629990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41810/0 (socket says 172.21.15.44:41810) 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0da59c640 1 -- 172.21.15.44:0/2967502687 learned_addr learned my addr 172.21.15.44:0/2967502687 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0d3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6629ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0dad9d640 1 --2- 172.21.15.44:0/2967502687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b66287e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0da59c640 1 -- 172.21.15.44:0/2967502687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 msgr2=0x5605b66287e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0da59c640 1 --2- 172.21.15.44:0/2967502687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b66287e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0da59c640 1 -- 172.21.15.44:0/2967502687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 msgr2=0x5605b6629ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0da59c640 1 --2- 172.21.15.44:0/2967502687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6629ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0da59c640 1 -- 172.21.15.44:0/2967502687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b6629010 con 0x5605b661b950 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0da59c640 1 --2- 172.21.15.44:0/2967502687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b6629990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc000c00 tx=0x7fe0cc031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e45475793c53b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0c1ffb640 1 -- 172.21.15.44:0/2967502687 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc005040 con 0x5605b661b950 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0c1ffb640 1 -- 172.21.15.44:0/2967502687 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8493+0+0 (secure 0 0 0) 0x7fe0cc0051e0 con 0x5605b661b950 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.411+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2967502687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0cc057070 con 0x5605b661b950 2024-08-20T03:09:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.412+0000 7fe0c1ffb640 1 -- 172.21.15.44:0/2967502687 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe0cc06b050 con 0x5605b661b950 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.412+0000 7fe0e3c0f740 10 client.16929 fetch_fsmap learned FSMap version 34 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.412+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2967502687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5605b662feb0 con 0x5605b661b950 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.412+0000 7fe0c1ffb640 1 -- 172.21.15.44:0/2967502687 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0cc057070 con 0x5605b661b950 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.412+0000 7fe0c1ffb640 1 -- 172.21.15.44:0/2967502687 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7fe0cc003610 con 0x5605b661b950 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.412+0000 7fe0e3c0f740 10 client.16929 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.412+0000 7fe0e3c0f740 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.412+0000 7fe0e3c0f740 20 client.16929 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.412+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2967502687 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5605b6630770 0x5605b66a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.412+0000 7fe0e3c0f740 4 client.16929 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.412+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2967502687 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605b658ed50 con 0x5605b6630770 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.412+0000 7fe0d3fff640 1 --2- 172.21.15.44:0/2967502687 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5605b6630770 0x5605b66a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.413+0000 7fe0d3fff640 1 --2- 172.21.15.44:0/2967502687 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5605b6630770 0x5605b66a48f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14ac45de21b4e29 server_cookie=6ab3bbaa50fdf2d5 in_seq=0 out_seq=0 2024-08-20T03:09:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.413+0000 7fe0c1ffb640 10 client.16929 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0c1ffb640 1 -- 172.21.15.44:0/2967502687 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605b658ed50 con 0x5605b6630770 2024-08-20T03:09:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0c1ffb640 10 client.16929 handle_command_reply: tid=0 2024-08-20T03:09:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0e3c0f740 1 client.16929 shutdown 2024-08-20T03:09:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2967502687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 msgr2=0x5605b6629990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2967502687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b6629990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc000c00 tx=0x7fe0cc031630 comp rx=0 tx=0).stop 2024-08-20T03:09:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2967502687 shutdown_connections 2024-08-20T03:09:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2967502687 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5605b6630770 0x5605b66a48f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2967502687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5605b6628200 0x5605b66287e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2967502687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5605b661c270 0x5605b6629ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0e3c0f740 1 --2- 172.21.15.44:0/2967502687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5605b661b950 0x5605b6629990 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0c1ffb640 0 client.16929 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2967502687 >> 172.21.15.44:0/2967502687 conn(0x5605b6619730 msgr2=0x5605b6626550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2967502687 shutdown_connections 2024-08-20T03:09:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0e3c0f740 1 -- 172.21.15.44:0/2967502687 wait complete. 2024-08-20T03:09:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.954+0000 7fe0e3c0f740 20 client.16929 trim_cache size 0 max 16384 2024-08-20T03:09:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:09:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:09:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:09:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:09:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.955+0000 7fe0e0d43640 1 -- 172.21.15.44:0/1182108886 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0c80b3d40 msgr2=0x7fe0c80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.955+0000 7fe0e0d43640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0c80b3d40 0x7fe0c80b6200 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc031440 tx=0x7fe0cc005890 comp rx=0 tx=0).stop 2024-08-20T03:09:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.955+0000 7fe0e0d43640 1 -- 172.21.15.44:0/1182108886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc1072d0 msgr2=0x7fe0dc199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.955+0000 7fe0e0d43640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc1072d0 0x7fe0dc199440 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400e9f0 tx=0x7fe0c400eeb0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.955+0000 7fe0e0d43640 1 -- 172.21.15.44:0/1182108886 shutdown_connections 2024-08-20T03:09:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.955+0000 7fe0e0d43640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fe0c80b3d40 0x7fe0c80b6200 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.955+0000 7fe0e0d43640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0dc107c50 0x7fe0dc19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.955+0000 7fe0e0d43640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc1072d0 0x7fe0dc199440 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.955+0000 7fe0e0d43640 1 --2- 172.21.15.44:0/1182108886 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0dc101920 0x7fe0dc198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.955+0000 7fe0e0d43640 1 -- 172.21.15.44:0/1182108886 >> 172.21.15.44:0/1182108886 conn(0x7fe0dc078620 msgr2=0x7fe0dc1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.955+0000 7fe0e0d43640 1 -- 172.21.15.44:0/1182108886 shutdown_connections 2024-08-20T03:09:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:21.955+0000 7fe0e0d43640 1 -- 172.21.15.44:0/1182108886 wait complete. 2024-08-20T03:09:22.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:22 smithi044 ceph-mon[18790]: pgmap v527: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 841 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 295 op/s 2024-08-20T03:09:22.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:22 smithi165 ceph-mon[22357]: pgmap v527: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 841 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 295 op/s 2024-08-20T03:09:22.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:22 smithi099 ceph-mon[21984]: pgmap v527: 97 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 79 active+clean; 841 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 295 op/s 2024-08-20T03:09:24.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:24 smithi165 ceph-mon[22357]: pgmap v528: 97 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 81 active+clean; 843 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 412 op/s 2024-08-20T03:09:24.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:24 smithi099 ceph-mon[21984]: pgmap v528: 97 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 81 active+clean; 843 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 412 op/s 2024-08-20T03:09:24.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:24 smithi044 ceph-mon[18790]: pgmap v528: 97 pgs: 5 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 81 active+clean; 843 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 412 op/s 2024-08-20T03:09:25.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:25 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:25.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:25 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:25.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:25 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:26.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:26 smithi165 ceph-mon[22357]: pgmap v529: 97 pgs: 97 active+clean; 832 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 1.19k op/s 2024-08-20T03:09:26.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:26 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:26.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:26 smithi099 ceph-mon[21984]: pgmap v529: 97 pgs: 97 active+clean; 832 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 1.19k op/s 2024-08-20T03:09:26.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:26 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:26.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:26 smithi044 ceph-mon[18790]: pgmap v529: 97 pgs: 97 active+clean; 832 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 1.19k op/s 2024-08-20T03:09:26.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:26 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:27 smithi044 ceph-mon[18790]: pgmap v530: 97 pgs: 97 active+clean; 830 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 1.35k op/s 2024-08-20T03:09:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:27 smithi044 ceph-mon[18790]: osdmap e79: 12 total, 12 up, 12 in 2024-08-20T03:09:28.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:27 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:27 smithi165 ceph-mon[22357]: pgmap v530: 97 pgs: 97 active+clean; 830 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 1.35k op/s 2024-08-20T03:09:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:27 smithi165 ceph-mon[22357]: osdmap e79: 12 total, 12 up, 12 in 2024-08-20T03:09:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:27 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:27 smithi099 ceph-mon[21984]: pgmap v530: 97 pgs: 97 active+clean; 830 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 1.35k op/s 2024-08-20T03:09:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:27 smithi099 ceph-mon[21984]: osdmap e79: 12 total, 12 up, 12 in 2024-08-20T03:09:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:27 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:29.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:28 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:29.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:28 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:29.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:28 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:09:30.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:29 smithi044 ceph-mon[18790]: pgmap v532: 97 pgs: 97 active+clean; 834 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 2.27k op/s 2024-08-20T03:09:30.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:29 smithi165 ceph-mon[22357]: pgmap v532: 97 pgs: 97 active+clean; 834 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 2.27k op/s 2024-08-20T03:09:30.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:29 smithi099 ceph-mon[21984]: pgmap v532: 97 pgs: 97 active+clean; 834 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 2.27k op/s 2024-08-20T03:09:31.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:30 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:09:31.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:30 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:09:31.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:30 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:09:31.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:30 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:09:31.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:30 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:09:31.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:30 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:09:31.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:30 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:09:31.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:30 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:09:31.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:30 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:09:31.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:30 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:09:31.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:30 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:09:31.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:30 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:09:32.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:31 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:09:32.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:31 smithi044 ceph-mon[18790]: pgmap v533: 97 pgs: 97 active+clean; 835 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 2.45k op/s 2024-08-20T03:09:32.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:31 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:32.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:31 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:09:32.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:31 smithi165 ceph-mon[22357]: pgmap v533: 97 pgs: 97 active+clean; 835 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 2.45k op/s 2024-08-20T03:09:32.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:31 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:32.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:31 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:09:32.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:31 smithi099 ceph-mon[21984]: pgmap v533: 97 pgs: 97 active+clean; 835 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 2.45k op/s 2024-08-20T03:09:32.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:31 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:33.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:32 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:33.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:32 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:33.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:32 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:33 smithi165 ceph-mon[22357]: pgmap v534: 97 pgs: 97 active+clean; 842 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.5 MiB/s wr, 2.38k op/s 2024-08-20T03:09:34.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:33 smithi099 ceph-mon[21984]: pgmap v534: 97 pgs: 97 active+clean; 842 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.5 MiB/s wr, 2.38k op/s 2024-08-20T03:09:34.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:33 smithi044 ceph-mon[18790]: pgmap v534: 97 pgs: 97 active+clean; 842 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.5 MiB/s wr, 2.38k op/s 2024-08-20T03:09:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.067+0000 7f8833ba9640 1 -- 172.21.15.44:0/1683487792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f882c101920 msgr2=0x7f882c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.067+0000 7f8833ba9640 1 --2- 172.21.15.44:0/1683487792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f882c101920 0x7f882c101d00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8824009900 tx=0x7f882402a410 comp rx=0 tx=0).stop 2024-08-20T03:09:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.067+0000 7f8833ba9640 1 -- 172.21.15.44:0/1683487792 shutdown_connections 2024-08-20T03:09:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.067+0000 7f8833ba9640 1 --2- 172.21.15.44:0/1683487792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f882c107c50 0x7f882c10e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.067+0000 7f8833ba9640 1 --2- 172.21.15.44:0/1683487792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f882c1072d0 0x7f882c107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.067+0000 7f8833ba9640 1 --2- 172.21.15.44:0/1683487792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f882c101920 0x7f882c101d00 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8833ba9640 1 -- 172.21.15.44:0/1683487792 >> 172.21.15.44:0/1683487792 conn(0x7f882c078620 msgr2=0x7f882c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8833ba9640 1 -- 172.21.15.44:0/1683487792 shutdown_connections 2024-08-20T03:09:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8833ba9640 1 -- 172.21.15.44:0/1683487792 wait complete. 2024-08-20T03:09:35.072 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8833ba9640 1 Processor -- start 2024-08-20T03:09:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8833ba9640 1 -- start start 2024-08-20T03:09:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8833ba9640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f882c101920 0x7f882c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8833ba9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f882c1072d0 0x7f882c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8833ba9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f882c107c50 0x7f882c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8833ba9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f882c111910 con 0x7f882c1072d0 2024-08-20T03:09:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8833ba9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f882c111790 con 0x7f882c107c50 2024-08-20T03:09:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8833ba9640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f882c111a90 con 0x7f882c101920 2024-08-20T03:09:35.073 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8831144640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f882c1072d0 0x7f882c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8831144640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f882c1072d0 0x7f882c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41950/0 (socket says 172.21.15.44:41950) 2024-08-20T03:09:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.068+0000 7f8831144640 1 -- 172.21.15.44:0/4191744530 learned_addr learned my addr 172.21.15.44:0/4191744530 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f8831945640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f882c101920 0x7f882c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f8832146640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f882c107c50 0x7f882c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f8831144640 1 -- 172.21.15.44:0/4191744530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f882c101920 msgr2=0x7f882c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f8831144640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f882c101920 0x7f882c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f8831144640 1 -- 172.21.15.44:0/4191744530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f882c107c50 msgr2=0x7f882c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f8831144640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f882c107c50 0x7f882c19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f8831144640 1 -- 172.21.15.44:0/4191744530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882c19de90 con 0x7f882c1072d0 2024-08-20T03:09:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f8831144640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f882c1072d0 0x7f882c198330 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f882000e9f0 tx=0x7f882000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f882affd640 1 -- 172.21.15.44:0/4191744530 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882000cd80 con 0x7f882c1072d0 2024-08-20T03:09:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f882affd640 1 -- 172.21.15.44:0/4191744530 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88200108e0 con 0x7f882c1072d0 2024-08-20T03:09:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f8833ba9640 1 -- 172.21.15.44:0/4191744530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882c19e180 con 0x7f882c1072d0 2024-08-20T03:09:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f882affd640 1 -- 172.21.15.44:0/4191744530 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820018980 con 0x7f882c1072d0 2024-08-20T03:09:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.069+0000 7f8833ba9640 1 -- 172.21.15.44:0/4191744530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f882c1a4a20 con 0x7f882c1072d0 2024-08-20T03:09:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.071+0000 7f882affd640 1 -- 172.21.15.44:0/4191744530 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8820010430 con 0x7f882c1072d0 2024-08-20T03:09:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.072+0000 7f8836a6e740 1 Processor -- start 2024-08-20T03:09:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.072+0000 7f8836a6e740 1 -- start start 2024-08-20T03:09:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.072+0000 7f8836a6e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0c0b60 0x56480e0c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.072+0000 7f8836a6e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c1480 0x56480e0cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.072+0000 7f8836a6e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0cd410 0x56480e0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.072+0000 7f8836a6e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56480e034360 con 0x56480e0c1480 2024-08-20T03:09:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.072+0000 7f8836a6e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56480e023860 con 0x56480e0c0b60 2024-08-20T03:09:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.072+0000 7f8836a6e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56480e0278a0 con 0x56480e0cd410 2024-08-20T03:09:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.072+0000 7f8831945640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c1480 0x56480e0cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.072+0000 7f8831945640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c1480 0x56480e0cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41960/0 (socket says 172.21.15.44:41960) 2024-08-20T03:09:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.072+0000 7f8831945640 1 -- 172.21.15.44:0/2266257329 learned_addr learned my addr 172.21.15.44:0/2266257329 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.072+0000 7f8831144640 1 --2- 172.21.15.44:0/2266257329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0cd410 0x56480e0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f882affd640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f88140b3dc0 0x7f88140b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8832146640 1 --2- 172.21.15.44:0/2266257329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0c0b60 0x56480e0c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8831945640 1 -- 172.21.15.44:0/2266257329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0cd410 msgr2=0x56480e0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8831945640 1 --2- 172.21.15.44:0/2266257329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0cd410 0x56480e0cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8831945640 1 -- 172.21.15.44:0/2266257329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0c0b60 msgr2=0x56480e0c0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8831945640 1 --2- 172.21.15.44:0/2266257329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0c0b60 0x56480e0c0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8831945640 1 -- 172.21.15.44:0/2266257329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480dfcae40 con 0x56480e0c1480 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8832146640 1 --2- 172.21.15.44:0/2266257329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0c0b60 0x56480e0c0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8832146640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f88140b3dc0 0x7f88140b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f882affd640 1 -- 172.21.15.44:0/4191744530 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f88200e4260 con 0x7f882c1072d0 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8831945640 1 --2- 172.21.15.44:0/2266257329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c1480 0x56480e0cced0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8824009900 tx=0x7f8824032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144333b13ee11c3f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8813fff640 1 -- 172.21.15.44:0/2266257329 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824040070 con 0x56480e0c1480 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8813fff640 1 -- 172.21.15.44:0/2266257329 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8824004030 con 0x56480e0c1480 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8813fff640 1 -- 172.21.15.44:0/2266257329 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824035040 con 0x56480e0c1480 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.073+0000 7f8832146640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f88140b3dc0 0x7f88140b6280 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f881c005fd0 tx=0x7f881c014040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.074+0000 7f8836a6e740 1 -- 172.21.15.44:0/2266257329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c1480 msgr2=0x56480e0cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.074+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2266257329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c1480 0x56480e0cced0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8824009900 tx=0x7f8824032b20 comp rx=0 tx=0).stop 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.074+0000 7f8836a6e740 1 -- 172.21.15.44:0/2266257329 shutdown_connections 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.074+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2266257329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0cd410 0x56480e0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.074+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2266257329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c1480 0x56480e0cced0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.074+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2266257329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0c0b60 0x56480e0c0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.074+0000 7f8836a6e740 1 -- 172.21.15.44:0/2266257329 >> 172.21.15.44:0/2266257329 conn(0x56480e0be940 msgr2=0x56480e0bfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.074+0000 7f8836a6e740 1 -- 172.21.15.44:0/2266257329 shutdown_connections 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.074+0000 7f8836a6e740 1 -- 172.21.15.44:0/2266257329 wait complete. 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.074+0000 7f8836a6e740 1 -- start start 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.074+0000 7f8836a6e740 1 Processor -- start 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8836a6e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8836a6e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0c1480 0x56480e0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8836a6e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0cd410 0x56480e0ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8836a6e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56480df5cab0 con 0x56480e0c0b60 2024-08-20T03:09:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8836a6e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56480dfaa780 con 0x56480e0cd410 2024-08-20T03:09:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8836a6e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56480e0d0090 con 0x56480e0c1480 2024-08-20T03:09:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8836a6e740 10 client.0 _ll_register_callbacks cb 0x56480dd32a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8831945640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8831945640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41976/0 (socket says 172.21.15.44:41976) 2024-08-20T03:09:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8831945640 1 -- 172.21.15.44:0/2075893345 learned_addr learned my addr 172.21.15.44:0/2075893345 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8831144640 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0c1480 0x56480e0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8832146640 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0cd410 0x56480e0ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8831945640 1 -- 172.21.15.44:0/2075893345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0c1480 msgr2=0x56480e0ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8831945640 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0c1480 0x56480e0ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8831945640 1 -- 172.21.15.44:0/2075893345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0cd410 msgr2=0x56480e0ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8831945640 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0cd410 0x56480e0ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8831945640 1 -- 172.21.15.44:0/2075893345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56480dd86790 con 0x56480e0c0b60 2024-08-20T03:09:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8832146640 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0cd410 0x56480e0ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:09:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f8831945640 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0ccd90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f882402abf0 tx=0x7f8824031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121c0271915c316 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.075+0000 7f88117fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f88117fa640 1 -- 172.21.15.44:0/2075893345 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882404b020 con 0x56480e0c0b60 2024-08-20T03:09:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f88117fa640 1 -- 172.21.15.44:0/2075893345 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f88240314a0 con 0x56480e0c0b60 2024-08-20T03:09:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f8836a6e740 1 -- 172.21.15.44:0/2075893345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56480dfd2e00 con 0x56480e0c0b60 2024-08-20T03:09:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f88117fa640 1 -- 172.21.15.44:0/2075893345 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8824056050 con 0x56480e0c0b60 2024-08-20T03:09:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f8836a6e740 10 client.16941 fetch_fsmap learned FSMap version 39 2024-08-20T03:09:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f8836a6e740 1 -- 172.21.15.44:0/2075893345 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56480ddff620 con 0x56480e0c0b60 2024-08-20T03:09:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f88117fa640 1 -- 172.21.15.44:0/2075893345 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56480dfd2e00 con 0x56480e0c0b60 2024-08-20T03:09:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f88117fa640 1 -- 172.21.15.44:0/2075893345 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8406+0+0 (secure 0 0 0) 0x7f8824004b60 con 0x56480e0c0b60 2024-08-20T03:09:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f8836a6e740 10 client.16941 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T03:09:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f8836a6e740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f8836a6e740 20 client.16941 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56480e0dcbe0 0x56480e0defd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f8836a6e740 4 client.16941 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f8836a6e740 1 -- 172.21.15.44:0/2075893345 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56480e0362b0 con 0x56480e0dcbe0 2024-08-20T03:09:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.076+0000 7f8831144640 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56480e0dcbe0 0x56480e0defd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.077+0000 7f8831144640 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56480e0dcbe0 0x56480e0defd0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645c6c4dce24a617 server_cookie=9d61650254d9e733 in_seq=0 out_seq=0 2024-08-20T03:09:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.077+0000 7f88117fa640 10 client.16941 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f88117fa640 1 -- 172.21.15.44:0/2075893345 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56480e0362b0 con 0x56480e0dcbe0 2024-08-20T03:09:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f88117fa640 10 client.16941 handle_command_reply: tid=0 2024-08-20T03:09:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f8836a6e740 1 client.16941 shutdown 2024-08-20T03:09:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f8836a6e740 1 -- 172.21.15.44:0/2075893345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 msgr2=0x56480e0ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0ccd90 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f882402abf0 tx=0x7f8824031400 comp rx=0 tx=0).stop 2024-08-20T03:09:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f8836a6e740 1 -- 172.21.15.44:0/2075893345 shutdown_connections 2024-08-20T03:09:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56480e0dcbe0 0x56480e0defd0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0cd410 0x56480e0ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0c1480 0x56480e0ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2075893345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0ccd90 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f8836a6e740 1 -- 172.21.15.44:0/2075893345 >> 172.21.15.44:0/2075893345 conn(0x56480e0be940 msgr2=0x56480e0bb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f88117fa640 0 client.16941 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f8836a6e740 1 -- 172.21.15.44:0/2075893345 shutdown_connections 2024-08-20T03:09:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f8836a6e740 1 -- 172.21.15.44:0/2075893345 wait complete. 2024-08-20T03:09:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.079+0000 7f8836a6e740 20 client.16941 trim_cache size 0 max 16384 2024-08-20T03:09:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8836a6e740 1 Processor -- start 2024-08-20T03:09:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8836a6e740 1 -- start start 2024-08-20T03:09:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8836a6e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8836a6e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0c1480 0x56480e0bb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8836a6e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0cd410 0x56480e0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8836a6e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1abfb0 con 0x56480e0c0b60 2024-08-20T03:09:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8836a6e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1adcd0 con 0x56480e0c1480 2024-08-20T03:09:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8836a6e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1ac710 con 0x56480e0cd410 2024-08-20T03:09:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8831945640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8831945640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41992/0 (socket says 172.21.15.44:41992) 2024-08-20T03:09:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8831945640 1 -- 172.21.15.44:0/2684966020 learned_addr learned my addr 172.21.15.44:0/2684966020 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8832146640 1 --2- 172.21.15.44:0/2684966020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0cd410 0x56480e0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8831144640 1 --2- 172.21.15.44:0/2684966020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0c1480 0x56480e0bb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8831945640 1 -- 172.21.15.44:0/2684966020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0cd410 msgr2=0x56480e0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8831945640 1 --2- 172.21.15.44:0/2684966020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0cd410 0x56480e0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8831945640 1 -- 172.21.15.44:0/2684966020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0c1480 msgr2=0x56480e0bb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8831945640 1 --2- 172.21.15.44:0/2684966020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0c1480 0x56480e0bb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.094+0000 7f8831945640 1 -- 172.21.15.44:0/2684966020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480e0cac10 con 0x56480e0c0b60 2024-08-20T03:09:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f8831945640 1 --2- 172.21.15.44:0/2684966020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0c9910 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8824009800 tx=0x7f882402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8583e3e715ec3d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f88137fe640 1 -- 172.21.15.44:0/2684966020 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882405f070 con 0x56480e0c0b60 2024-08-20T03:09:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f88137fe640 1 -- 172.21.15.44:0/2684966020 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88240314a0 con 0x56480e0c0b60 2024-08-20T03:09:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f88137fe640 1 -- 172.21.15.44:0/2684966020 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882404b070 con 0x56480e0c0b60 2024-08-20T03:09:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f8836a6e740 1 -- 172.21.15.44:0/2684966020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 msgr2=0x56480e0c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2684966020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0c9910 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8824009800 tx=0x7f882402a410 comp rx=0 tx=0).stop 2024-08-20T03:09:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f8836a6e740 1 -- 172.21.15.44:0/2684966020 shutdown_connections 2024-08-20T03:09:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2684966020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0cd410 0x56480e0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2684966020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0c1480 0x56480e0bb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2684966020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0c9910 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f8836a6e740 1 -- 172.21.15.44:0/2684966020 >> 172.21.15.44:0/2684966020 conn(0x56480e0be940 msgr2=0x56480e0bb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f8836a6e740 1 -- 172.21.15.44:0/2684966020 shutdown_connections 2024-08-20T03:09:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.095+0000 7f8836a6e740 1 -- 172.21.15.44:0/2684966020 wait complete. 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8836a6e740 1 -- start start 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8836a6e740 1 Processor -- start 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8836a6e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8836a6e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0c1480 0x56480e0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8836a6e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0cd410 0x56480e0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8836a6e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1ad5a0 con 0x56480e0c0b60 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8836a6e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1ace70 con 0x56480e0cd410 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8836a6e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1a9fb0 con 0x56480e0c1480 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8836a6e740 10 client.0 _ll_register_callbacks cb 0x7f88200aad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8831945640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8831945640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41998/0 (socket says 172.21.15.44:41998) 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8831945640 1 -- 172.21.15.44:0/2391449360 learned_addr learned my addr 172.21.15.44:0/2391449360 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8831144640 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0c1480 0x56480e0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8832146640 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0cd410 0x56480e0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8831945640 1 -- 172.21.15.44:0/2391449360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0c1480 msgr2=0x56480e0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8831945640 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0c1480 0x56480e0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8831945640 1 -- 172.21.15.44:0/2391449360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0cd410 msgr2=0x56480e0cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.096+0000 7f8831945640 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0cd410 0x56480e0cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f8831945640 1 -- 172.21.15.44:0/2391449360 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56480e0d4320 con 0x56480e0c0b60 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f8831945640 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0cf030 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8824033000 tx=0x7f88240052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde171c560665743 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f88117fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f88117fa640 1 -- 172.21.15.44:0/2391449360 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882404b070 con 0x56480e0c0b60 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f88117fa640 1 -- 172.21.15.44:0/2391449360 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f8824031650 con 0x56480e0c0b60 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f8836a6e740 1 -- 172.21.15.44:0/2391449360 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88240314a0 con 0x56480e0c0b60 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f88117fa640 1 -- 172.21.15.44:0/2391449360 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f882405f070 con 0x56480e0c0b60 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f8836a6e740 10 client.16947 fetch_fsmap learned FSMap version 39 2024-08-20T03:09:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f8836a6e740 1 -- 172.21.15.44:0/2391449360 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56480e0d5630 con 0x56480e0c0b60 2024-08-20T03:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f88117fa640 1 -- 172.21.15.44:0/2391449360 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88240314a0 con 0x56480e0c0b60 2024-08-20T03:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f88117fa640 1 -- 172.21.15.44:0/2391449360 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 39) ==== 8406+0+0 (secure 0 0 0) 0x7f88240035f0 con 0x56480e0c0b60 2024-08-20T03:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f8836a6e740 10 client.16947 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T03:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f8836a6e740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f8836a6e740 20 client.16947 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56480e0d5d90 0x56480e149610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f8836a6e740 4 client.16947 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.097+0000 7f8836a6e740 1 -- 172.21.15.44:0/2391449360 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56480e031d50 con 0x56480e0d5d90 2024-08-20T03:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.098+0000 7f8831144640 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56480e0d5d90 0x56480e149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.098+0000 7f88117fa640 10 client.16947 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:35.098+0000 7f8831144640 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56480e0d5d90 0x56480e149610 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e9e5f2304771c3c server_cookie=85a38b9b51b9006d in_seq=0 out_seq=0 2024-08-20T03:09:36.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:35 smithi165 ceph-mon[22357]: pgmap v535: 97 pgs: 97 active+clean; 848 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 1.57k op/s 2024-08-20T03:09:36.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:35 smithi099 ceph-mon[21984]: pgmap v535: 97 pgs: 97 active+clean; 848 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 1.57k op/s 2024-08-20T03:09:36.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:35 smithi044 ceph-mon[18790]: pgmap v535: 97 pgs: 97 active+clean; 848 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 1.57k op/s 2024-08-20T03:09:37.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:09:37.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-08-20T03:09:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: Adjusting osd_memory_target on smithi165 to 176.0M 2024-08-20T03:09:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-08-20T03:09:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: Unable to set osd_memory_target on smithi165 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-20T03:09:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:09:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:09:37.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:09:37.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:09:37.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-08-20T03:09:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: Adjusting osd_memory_target on smithi165 to 176.0M 2024-08-20T03:09:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-08-20T03:09:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: Unable to set osd_memory_target on smithi165 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-20T03:09:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:09:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:09:37.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:09:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:09:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T03:09:37.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-08-20T03:09:37.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: Adjusting osd_memory_target on smithi165 to 176.0M 2024-08-20T03:09:37.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-08-20T03:09:37.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: Unable to set osd_memory_target on smithi165 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-20T03:09:37.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:09:37.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:09:37.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:09:38.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:38 smithi165 ceph-mon[22357]: pgmap v536: 97 pgs: 97 active+clean; 848 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 1.31k op/s 2024-08-20T03:09:38.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:38 smithi099 ceph-mon[21984]: pgmap v536: 97 pgs: 97 active+clean; 848 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 1.31k op/s 2024-08-20T03:09:38.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:38 smithi044 ceph-mon[18790]: pgmap v536: 97 pgs: 97 active+clean; 848 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 1.31k op/s 2024-08-20T03:09:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.954+0000 7f88117fa640 1 -- 172.21.15.44:0/2391449360 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56480e031d50 con 0x56480e0d5d90 2024-08-20T03:09:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.954+0000 7f88117fa640 10 client.16947 handle_command_reply: tid=0 2024-08-20T03:09:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.954+0000 7f8836a6e740 1 client.16947 shutdown 2024-08-20T03:09:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.954+0000 7f8836a6e740 1 -- 172.21.15.44:0/2391449360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 msgr2=0x56480e0cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.954+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0cf030 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8824033000 tx=0x7f88240052c0 comp rx=0 tx=0).stop 2024-08-20T03:09:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.954+0000 7f8836a6e740 1 -- 172.21.15.44:0/2391449360 shutdown_connections 2024-08-20T03:09:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.954+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56480e0d5d90 0x56480e149610 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.954+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56480e0cd410 0x56480e0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.954+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56480e0c1480 0x56480e0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.954+0000 7f8836a6e740 1 --2- 172.21.15.44:0/2391449360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56480e0c0b60 0x56480e0cf030 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.954+0000 7f8836a6e740 1 -- 172.21.15.44:0/2391449360 >> 172.21.15.44:0/2391449360 conn(0x56480e0be940 msgr2=0x56480e0bb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.954+0000 7f88117fa640 0 client.16947 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.955+0000 7f8836a6e740 1 -- 172.21.15.44:0/2391449360 shutdown_connections 2024-08-20T03:09:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.955+0000 7f8836a6e740 1 -- 172.21.15.44:0/2391449360 wait complete. 2024-08-20T03:09:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.955+0000 7f8836a6e740 20 client.16947 trim_cache size 0 max 16384 2024-08-20T03:09:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:09:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:09:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:09:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:09:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.956+0000 7f8833ba9640 1 -- 172.21.15.44:0/4191744530 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f88140b3dc0 msgr2=0x7f88140b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.956+0000 7f8833ba9640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f88140b3dc0 0x7f88140b6280 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f881c005fd0 tx=0x7f881c014040 comp rx=0 tx=0).stop 2024-08-20T03:09:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.956+0000 7f8833ba9640 1 -- 172.21.15.44:0/4191744530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f882c1072d0 msgr2=0x7f882c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.956+0000 7f8833ba9640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f882c1072d0 0x7f882c198330 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f882000e9f0 tx=0x7f882000eeb0 comp rx=0 tx=0).stop 2024-08-20T03:09:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.956+0000 7f8833ba9640 1 -- 172.21.15.44:0/4191744530 shutdown_connections 2024-08-20T03:09:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.956+0000 7f8833ba9640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f88140b3dc0 0x7f88140b6280 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.956+0000 7f8833ba9640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f882c107c50 0x7f882c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.956+0000 7f8833ba9640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f882c1072d0 0x7f882c198330 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.956+0000 7f8833ba9640 1 --2- 172.21.15.44:0/4191744530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f882c101920 0x7f882c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.956+0000 7f8833ba9640 1 -- 172.21.15.44:0/4191744530 >> 172.21.15.44:0/4191744530 conn(0x7f882c078620 msgr2=0x7f882c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.956+0000 7f8833ba9640 1 -- 172.21.15.44:0/4191744530 shutdown_connections 2024-08-20T03:09:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:39.956+0000 7f8833ba9640 1 -- 172.21.15.44:0/4191744530 wait complete. 2024-08-20T03:09:40.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:40 smithi165 ceph-mon[22357]: pgmap v537: 97 pgs: 97 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 1.22k op/s 2024-08-20T03:09:40.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:40 smithi099 ceph-mon[21984]: pgmap v537: 97 pgs: 97 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 1.22k op/s 2024-08-20T03:09:40.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:40 smithi044 ceph-mon[18790]: pgmap v537: 97 pgs: 97 active+clean; 867 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 1.22k op/s 2024-08-20T03:09:41.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:41 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:41.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:41 smithi165 ceph-mon[22357]: pgmap v538: 97 pgs: 97 active+clean; 871 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 632 op/s 2024-08-20T03:09:41.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:41 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:41.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:41 smithi099 ceph-mon[21984]: pgmap v538: 97 pgs: 97 active+clean; 871 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 632 op/s 2024-08-20T03:09:41.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:41 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:41.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:41 smithi044 ceph-mon[18790]: pgmap v538: 97 pgs: 97 active+clean; 871 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 632 op/s 2024-08-20T03:09:43.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:43 smithi165 ceph-mon[22357]: pgmap v539: 97 pgs: 97 active+clean; 883 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.1 MiB/s wr, 501 op/s 2024-08-20T03:09:43.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:43 smithi099 ceph-mon[21984]: pgmap v539: 97 pgs: 97 active+clean; 883 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.1 MiB/s wr, 501 op/s 2024-08-20T03:09:44.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:43 smithi044 ceph-mon[18790]: pgmap v539: 97 pgs: 97 active+clean; 883 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.1 MiB/s wr, 501 op/s 2024-08-20T03:09:45.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:45 smithi165 ceph-mon[22357]: pgmap v540: 97 pgs: 97 active+clean; 874 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.8 MiB/s wr, 451 op/s 2024-08-20T03:09:45.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:45 smithi099 ceph-mon[21984]: pgmap v540: 97 pgs: 97 active+clean; 874 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.8 MiB/s wr, 451 op/s 2024-08-20T03:09:46.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:45 smithi044 ceph-mon[18790]: pgmap v540: 97 pgs: 97 active+clean; 874 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.8 MiB/s wr, 451 op/s 2024-08-20T03:09:47.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:47 smithi165 ceph-mon[22357]: pgmap v541: 97 pgs: 97 active+clean; 883 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.0 MiB/s wr, 330 op/s 2024-08-20T03:09:47.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:47 smithi099 ceph-mon[21984]: pgmap v541: 97 pgs: 97 active+clean; 883 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.0 MiB/s wr, 330 op/s 2024-08-20T03:09:48.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:47 smithi044 ceph-mon[18790]: pgmap v541: 97 pgs: 97 active+clean; 883 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.0 MiB/s wr, 330 op/s 2024-08-20T03:09:49.583 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:09:49.855 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:49.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:49 smithi165 ceph-mon[22357]: pgmap v542: 97 pgs: 97 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.6 MiB/s wr, 382 op/s 2024-08-20T03:09:49.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:49 smithi099 ceph-mon[21984]: pgmap v542: 97 pgs: 97 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.6 MiB/s wr, 382 op/s 2024-08-20T03:09:49.980 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:49 smithi044 ceph-mon[18790]: pgmap v542: 97 pgs: 97 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.6 MiB/s wr, 382 op/s 2024-08-20T03:09:50.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.390+0000 7f8ccfb38640 1 -- 172.21.15.44:0/1543121947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc81042c0 msgr2=0x7f8cc810c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.390+0000 7f8ccfb38640 1 --2- 172.21.15.44:0/1543121947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc81042c0 0x7f8cc810c7b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009900 tx=0x7f8cb802a410 comp rx=0 tx=0).stop 2024-08-20T03:09:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.390+0000 7f8ccfb38640 1 -- 172.21.15.44:0/1543121947 shutdown_connections 2024-08-20T03:09:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.390+0000 7f8ccfb38640 1 --2- 172.21.15.44:0/1543121947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cc810ce00 0x7f8cc810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.390+0000 7f8ccfb38640 1 --2- 172.21.15.44:0/1543121947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc81042c0 0x7f8cc810c7b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.390+0000 7f8ccfb38640 1 --2- 172.21.15.44:0/1543121947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cc81039a0 0x7f8cc8103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.390+0000 7f8ccfb38640 1 -- 172.21.15.44:0/1543121947 >> 172.21.15.44:0/1543121947 conn(0x7f8cc80fd400 msgr2=0x7f8cc80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.390+0000 7f8ccfb38640 1 -- 172.21.15.44:0/1543121947 shutdown_connections 2024-08-20T03:09:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.390+0000 7f8ccfb38640 1 -- 172.21.15.44:0/1543121947 wait complete. 2024-08-20T03:09:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.390+0000 7f8ccfb38640 1 Processor -- start 2024-08-20T03:09:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.390+0000 7f8ccfb38640 1 -- start start 2024-08-20T03:09:50.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccfb38640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc81039a0 0x7f8cc8198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccfb38640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cc81042c0 0x7f8cc8199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccfb38640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cc810ce00 0x7f8cc819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccfb38640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8112ac0 con 0x7f8cc81039a0 2024-08-20T03:09:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccfb38640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8112940 con 0x7f8cc810ce00 2024-08-20T03:09:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccfb38640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8112c40 con 0x7f8cc81042c0 2024-08-20T03:09:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccd8d4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc81039a0 0x7f8cc8198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccd8d4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc81039a0 0x7f8cc8198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42690/0 (socket says 172.21.15.44:42690) 2024-08-20T03:09:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccd8d4640 1 -- 172.21.15.44:0/2943528948 learned_addr learned my addr 172.21.15.44:0/2943528948 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccd0d3640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cc81042c0 0x7f8cc8199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cc810ce00 0x7f8cc819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccd8d4640 1 -- 172.21.15.44:0/2943528948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cc81042c0 msgr2=0x7f8cc8199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:50.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccd8d4640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cc81042c0 0x7f8cc8199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccd8d4640 1 -- 172.21.15.44:0/2943528948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cc810ce00 msgr2=0x7f8cc819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccd8d4640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cc810ce00 0x7f8cc819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccd8d4640 1 -- 172.21.15.44:0/2943528948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc819f080 con 0x7f8cc81039a0 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccd8d4640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc81039a0 0x7f8cc8198fd0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400d950 tx=0x7f8cc400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8cbeffd640 1 -- 172.21.15.44:0/2943528948 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4017070 con 0x7f8cc81039a0 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.391+0000 7f8ccfb38640 1 -- 172.21.15.44:0/2943528948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc819f370 con 0x7f8cc81039a0 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.392+0000 7f8cbeffd640 1 -- 172.21.15.44:0/2943528948 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc4009c30 con 0x7f8cc81039a0 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.392+0000 7f8cbeffd640 1 -- 172.21.15.44:0/2943528948 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400f620 con 0x7f8cc81039a0 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.392+0000 7f8ccfb38640 1 -- 172.21.15.44:0/2943528948 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc81a5c00 con 0x7f8cc81039a0 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.394+0000 7f8cd1f31740 1 Processor -- start 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.394+0000 7f8cbeffd640 1 -- 172.21.15.44:0/2943528948 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8cc4010460 con 0x7f8cc81039a0 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.394+0000 7f8cd1f31740 1 -- start start 2024-08-20T03:09:50.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cd1f31740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec7830 0x5641cbec7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cd1f31740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec8150 0x5641cbed3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cd1f31740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbed40e0 0x5641cbed64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cd1f31740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5641cbe3e8e0 con 0x5641cbec7830 2024-08-20T03:09:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cd1f31740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5641cbe2dde0 con 0x5641cbed40e0 2024-08-20T03:09:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cd1f31740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641cbe31e20 con 0x5641cbec8150 2024-08-20T03:09:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cce0d5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec7830 0x5641cbec7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cce0d5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec7830 0x5641cbec7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42694/0 (socket says 172.21.15.44:42694) 2024-08-20T03:09:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cce0d5640 1 -- 172.21.15.44:0/3946099068 learned_addr learned my addr 172.21.15.44:0/3946099068 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8ccd8d4640 1 --2- 172.21.15.44:0/3946099068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec8150 0x5641cbed3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8ccd0d3640 1 --2- 172.21.15.44:0/3946099068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbed40e0 0x5641cbed64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cce0d5640 1 -- 172.21.15.44:0/3946099068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec8150 msgr2=0x5641cbed3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:50.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/3946099068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec8150 0x5641cbed3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cce0d5640 1 -- 172.21.15.44:0/3946099068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbed40e0 msgr2=0x5641cbed64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/3946099068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbed40e0 0x5641cbed64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cce0d5640 1 -- 172.21.15.44:0/3946099068 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641cbb8d650 con 0x5641cbec7830 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/3946099068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec7830 0x5641cbec7c10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0011440 tx=0x7f8cc0011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c51591d104b7cb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.395+0000 7f8cbeffd640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8cb40b3d40 0x7f8cb40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8ccd8d4640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8cb40b3d40 0x7f8cb40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8cb3fff640 1 -- 172.21.15.44:0/3946099068 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0004d20 con 0x5641cbec7830 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8cbeffd640 1 -- 172.21.15.44:0/2943528948 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f8cc40e3b50 con 0x7f8cc81039a0 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8cb3fff640 1 -- 172.21.15.44:0/3946099068 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc0004050 con 0x5641cbec7830 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8cb3fff640 1 -- 172.21.15.44:0/3946099068 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000f7a0 con 0x5641cbec7830 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8ccd8d4640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8cb40b3d40 0x7f8cb40b6200 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40e1390 tx=0x7f8cc4092040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8cd1f31740 1 -- 172.21.15.44:0/3946099068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec7830 msgr2=0x5641cbec7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/3946099068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec7830 0x5641cbec7c10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0011440 tx=0x7f8cc0011900 comp rx=0 tx=0).stop 2024-08-20T03:09:50.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8cd1f31740 1 -- 172.21.15.44:0/3946099068 shutdown_connections 2024-08-20T03:09:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/3946099068 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbed40e0 0x5641cbed64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/3946099068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec8150 0x5641cbed3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/3946099068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec7830 0x5641cbec7c10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.396+0000 7f8cd1f31740 1 -- 172.21.15.44:0/3946099068 >> 172.21.15.44:0/3946099068 conn(0x5641cbec5660 msgr2=0x5641cbec6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.397+0000 7f8cd1f31740 1 -- 172.21.15.44:0/3946099068 shutdown_connections 2024-08-20T03:09:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.397+0000 7f8cd1f31740 1 -- 172.21.15.44:0/3946099068 wait complete. 2024-08-20T03:09:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.397+0000 7f8cd1f31740 1 -- start start 2024-08-20T03:09:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.397+0000 7f8cd1f31740 1 Processor -- start 2024-08-20T03:09:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.397+0000 7f8cd1f31740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec7830 0x5641cbed1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.397+0000 7f8cd1f31740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbec8150 0x5641cbed15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.397+0000 7f8cd1f31740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 0x5641cbed1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.397+0000 7f8cd1f31740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5641cbc90700 con 0x5641cbed40e0 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.397+0000 7f8cd1f31740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5641cbdb47c0 con 0x5641cbec8150 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.397+0000 7f8cd1f31740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5641cbed6d90 con 0x5641cbec7830 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.397+0000 7f8cd1f31740 10 client.0 _ll_register_callbacks cb 0x5641cbe276b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.397+0000 7f8cce0d5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 0x5641cbed1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cce0d5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 0x5641cbed1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42702/0 (socket says 172.21.15.44:42702) 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cce0d5640 1 -- 172.21.15.44:0/4041621666 learned_addr learned my addr 172.21.15.44:0/4041621666 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec7830 0x5641cbed1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8ccd0d3640 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbec8150 0x5641cbed15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cce0d5640 1 -- 172.21.15.44:0/4041621666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec7830 msgr2=0x5641cbed1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec7830 0x5641cbed1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cce0d5640 1 -- 172.21.15.44:0/4041621666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbec8150 msgr2=0x5641cbed15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbec8150 0x5641cbed15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cce0d5640 1 -- 172.21.15.44:0/4041621666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641cbee0140 con 0x5641cbed40e0 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 0x5641cbed1b00 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009270 tx=0x7f8cc0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ab0f7e60629d2f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:50.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cb17fa640 1 -- 172.21.15.44:0/4041621666 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc001d070 con 0x5641cbed40e0 2024-08-20T03:09:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cb17fa640 1 -- 172.21.15.44:0/4041621666 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f8cc0004460 con 0x5641cbed40e0 2024-08-20T03:09:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cd1f31740 1 -- 172.21.15.44:0/4041621666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5641cbddd100 con 0x5641cbed40e0 2024-08-20T03:09:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cb17fa640 1 -- 172.21.15.44:0/4041621666 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8cc0036050 con 0x5641cbed40e0 2024-08-20T03:09:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cd1f31740 10 client.16959 fetch_fsmap learned FSMap version 40 2024-08-20T03:09:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.398+0000 7f8cd1f31740 1 -- 172.21.15.44:0/4041621666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5641cbee2af0 con 0x5641cbed40e0 2024-08-20T03:09:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.399+0000 7f8cb17fa640 1 -- 172.21.15.44:0/4041621666 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5641cbddd100 con 0x5641cbed40e0 2024-08-20T03:09:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.399+0000 7f8cb17fa640 1 -- 172.21.15.44:0/4041621666 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8402+0+0 (secure 0 0 0) 0x7f8cc0019430 con 0x5641cbed40e0 2024-08-20T03:09:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.399+0000 7f8cd1f31740 10 client.16959 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T03:09:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.399+0000 7f8cd1f31740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:50.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.399+0000 7f8cd1f31740 20 client.16959 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.399+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641cbee3ef0 0x5641cbee62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.399+0000 7f8cd1f31740 4 client.16959 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.399+0000 7f8cd1f31740 1 -- 172.21.15.44:0/4041621666 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641cbe40830 con 0x5641cbee3ef0 2024-08-20T03:09:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.399+0000 7f8ccd0d3640 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641cbee3ef0 0x5641cbee62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.399+0000 7f8ccd0d3640 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641cbee3ef0 0x5641cbee62e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e12952873b4a0c5 server_cookie=cb7c459b698dd11 in_seq=0 out_seq=0 2024-08-20T03:09:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.399+0000 7f8cb17fa640 10 client.16959 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cb17fa640 1 -- 172.21.15.44:0/4041621666 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641cbe40830 con 0x5641cbee3ef0 2024-08-20T03:09:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cb17fa640 10 client.16959 handle_command_reply: tid=0 2024-08-20T03:09:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cd1f31740 1 client.16959 shutdown 2024-08-20T03:09:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cd1f31740 1 -- 172.21.15.44:0/4041621666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 msgr2=0x5641cbed1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 0x5641cbed1b00 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009270 tx=0x7f8cc0009500 comp rx=0 tx=0).stop 2024-08-20T03:09:50.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cd1f31740 1 -- 172.21.15.44:0/4041621666 shutdown_connections 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641cbee3ef0 0x5641cbee62e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 0x5641cbed1b00 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbec8150 0x5641cbed15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/4041621666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec7830 0x5641cbed1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cb17fa640 0 client.16959 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cd1f31740 1 -- 172.21.15.44:0/4041621666 >> 172.21.15.44:0/4041621666 conn(0x5641cbec5660 msgr2=0x5641cbecb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.401+0000 7f8cd1f31740 1 -- 172.21.15.44:0/4041621666 shutdown_connections 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.402+0000 7f8cd1f31740 1 -- 172.21.15.44:0/4041621666 wait complete. 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.402+0000 7f8cd1f31740 20 client.16959 trim_cache size 0 max 16384 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.419+0000 7f8cd1f31740 1 Processor -- start 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.419+0000 7f8cd1f31740 1 -- start start 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.419+0000 7f8cd1f31740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbec7830 0x5641cbed3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.419+0000 7f8cd1f31740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec8150 0x5641cbed0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cd1f31740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 0x5641cbeeba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cd1f31740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81ae330 con 0x5641cbed40e0 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cd1f31740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81b0020 con 0x5641cbec7830 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cd1f31740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81aea90 con 0x5641cbec8150 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cce0d5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 0x5641cbeeba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cce0d5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 0x5641cbeeba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42710/0 (socket says 172.21.15.44:42710) 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cce0d5640 1 -- 172.21.15.44:0/1753846513 learned_addr learned my addr 172.21.15.44:0/1753846513 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8ccd0d3640 1 --2- 172.21.15.44:0/1753846513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec8150 0x5641cbed0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/1753846513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbec7830 0x5641cbed3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cce0d5640 1 -- 172.21.15.44:0/1753846513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec8150 msgr2=0x5641cbed0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/1753846513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec8150 0x5641cbed0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cce0d5640 1 -- 172.21.15.44:0/1753846513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbec7830 msgr2=0x5641cbed3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:50.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/1753846513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbec7830 0x5641cbed3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cce0d5640 1 -- 172.21.15.44:0/1753846513 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641cbec60e0 con 0x5641cbed40e0 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8ccd0d3640 1 --2- 172.21.15.44:0/1753846513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec8150 0x5641cbed0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/1753846513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 0x5641cbeeba70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0004980 tx=0x7f8cc0011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bd12c4c945038e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cb37fe640 1 -- 172.21.15.44:0/1753846513 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0041070 con 0x5641cbed40e0 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cb37fe640 1 -- 172.21.15.44:0/1753846513 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc002d070 con 0x5641cbed40e0 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.420+0000 7f8cb37fe640 1 -- 172.21.15.44:0/1753846513 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc001d070 con 0x5641cbed40e0 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.421+0000 7f8cd1f31740 1 -- 172.21.15.44:0/1753846513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 msgr2=0x5641cbeeba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.421+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/1753846513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 0x5641cbeeba70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0004980 tx=0x7f8cc0011900 comp rx=0 tx=0).stop 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.421+0000 7f8cd1f31740 1 -- 172.21.15.44:0/1753846513 shutdown_connections 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.421+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/1753846513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbed40e0 0x5641cbeeba70 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.421+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/1753846513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec8150 0x5641cbed0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.421+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/1753846513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbec7830 0x5641cbed3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.421+0000 7f8cd1f31740 1 -- 172.21.15.44:0/1753846513 >> 172.21.15.44:0/1753846513 conn(0x5641cbec5660 msgr2=0x5641cbed1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.421+0000 7f8cd1f31740 1 -- 172.21.15.44:0/1753846513 shutdown_connections 2024-08-20T03:09:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.421+0000 7f8cd1f31740 1 -- 172.21.15.44:0/1753846513 wait complete. 2024-08-20T03:09:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8cd1f31740 1 -- start start 2024-08-20T03:09:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8cd1f31740 1 Processor -- start 2024-08-20T03:09:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8cd1f31740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec7830 0x5641cbf46be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8cd1f31740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec8150 0x5641cbf47180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8cd1f31740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbed40e0 0x5641cbedbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8cd1f31740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81af920 con 0x5641cbec8150 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8cd1f31740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81af1f0 con 0x5641cbed40e0 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8cd1f31740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81ac330 con 0x5641cbec7830 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8cd1f31740 10 client.0 _ll_register_callbacks cb 0x7f8cc00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8ccd0d3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec8150 0x5641cbf47180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8ccd0d3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec8150 0x5641cbf47180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42716/0 (socket says 172.21.15.44:42716) 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8cce0d5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec7830 0x5641cbf46be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8ccd0d3640 1 -- 172.21.15.44:0/2200719530 learned_addr learned my addr 172.21.15.44:0/2200719530 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbed40e0 0x5641cbedbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8ccd0d3640 1 -- 172.21.15.44:0/2200719530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec7830 msgr2=0x5641cbf46be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8ccd0d3640 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec7830 0x5641cbf46be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.422+0000 7f8ccd0d3640 1 -- 172.21.15.44:0/2200719530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbed40e0 msgr2=0x5641cbedbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.423+0000 7f8ccd0d3640 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbed40e0 0x5641cbedbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.423+0000 7f8ccd0d3640 1 -- 172.21.15.44:0/2200719530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641cbed1f20 con 0x5641cbec8150 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.423+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec7830 0x5641cbf46be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.423+0000 7f8ccd0d3640 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec8150 0x5641cbf47180 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8005ab0 tx=0x7f8cb80044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6808a8ce00e3e881 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.423+0000 7f8cb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.423+0000 7f8cb17fa640 1 -- 172.21.15.44:0/2200719530 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8035040 con 0x5641cbec8150 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.423+0000 7f8cb17fa640 1 -- 172.21.15.44:0/2200719530 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f8cb8005510 con 0x5641cbec8150 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.423+0000 7f8cd1f31740 1 -- 172.21.15.44:0/2200719530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc002d070 con 0x5641cbec8150 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.423+0000 7f8cb17fa640 1 -- 172.21.15.44:0/2200719530 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8cb8004750 con 0x5641cbec8150 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.423+0000 7f8cd1f31740 10 client.16965 fetch_fsmap learned FSMap version 40 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.423+0000 7f8cd1f31740 1 -- 172.21.15.44:0/2200719530 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5641cbf4e400 con 0x5641cbec8150 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.423+0000 7f8cb17fa640 1 -- 172.21.15.44:0/2200719530 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc002d070 con 0x5641cbec8150 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.424+0000 7f8cb17fa640 1 -- 172.21.15.44:0/2200719530 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 40) ==== 8402+0+0 (secure 0 0 0) 0x7f8cb8003b20 con 0x5641cbec8150 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.424+0000 7f8cd1f31740 10 client.16965 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T03:09:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.424+0000 7f8cd1f31740 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.424+0000 7f8cd1f31740 20 client.16965 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.424+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641cbf4eb50 0x5641cbf50f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.424+0000 7f8cd1f31740 4 client.16965 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.424+0000 7f8cd1f31740 1 -- 172.21.15.44:0/2200719530 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5641cbe3c2d0 con 0x5641cbf4eb50 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.424+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641cbf4eb50 0x5641cbf50f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.424+0000 7f8cce0d5640 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641cbf4eb50 0x5641cbf50f70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d2e22468ae05b76 server_cookie=a5a3fb6fcb1373d3 in_seq=0 out_seq=0 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.424+0000 7f8cb17fa640 10 client.16965 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.424+0000 7f8cb17fa640 1 -- 172.21.15.44:0/2200719530 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5641cbe3c2d0 con 0x5641cbf4eb50 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.424+0000 7f8cb17fa640 10 client.16965 handle_command_reply: tid=0 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cd1f31740 1 client.16965 shutdown 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cd1f31740 1 -- 172.21.15.44:0/2200719530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec8150 msgr2=0x5641cbf47180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec8150 0x5641cbf47180 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8005ab0 tx=0x7f8cb80044d0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cd1f31740 1 -- 172.21.15.44:0/2200719530 shutdown_connections 2024-08-20T03:09:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5641cbf4eb50 0x5641cbf50f70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641cbed40e0 0x5641cbedbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5641cbec8150 0x5641cbf47180 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cd1f31740 1 --2- 172.21.15.44:0/2200719530 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5641cbec7830 0x5641cbf46be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cd1f31740 1 -- 172.21.15.44:0/2200719530 >> 172.21.15.44:0/2200719530 conn(0x5641cbec5660 msgr2=0x5641cbeca8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cb17fa640 0 client.16965 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cd1f31740 1 -- 172.21.15.44:0/2200719530 shutdown_connections 2024-08-20T03:09:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cd1f31740 1 -- 172.21.15.44:0/2200719530 wait complete. 2024-08-20T03:09:50.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.425+0000 7f8cd1f31740 20 client.16965 trim_cache size 0 max 16384 2024-08-20T03:09:50.443 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:09:50.443 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-08-20T03:09:50.444 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-08-20T03:09:50.444 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:09:50.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.426+0000 7f8ccfb38640 1 -- 172.21.15.44:0/2943528948 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8cb40b3d40 msgr2=0x7f8cb40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:50.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.426+0000 7f8ccfb38640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8cb40b3d40 0x7f8cb40b6200 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40e1390 tx=0x7f8cc4092040 comp rx=0 tx=0).stop 2024-08-20T03:09:50.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.426+0000 7f8ccfb38640 1 -- 172.21.15.44:0/2943528948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc81039a0 msgr2=0x7f8cc8198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:50.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.426+0000 7f8ccfb38640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc81039a0 0x7f8cc8198fd0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400d950 tx=0x7f8cc400de10 comp rx=0 tx=0).stop 2024-08-20T03:09:50.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.426+0000 7f8ccfb38640 1 -- 172.21.15.44:0/2943528948 shutdown_connections 2024-08-20T03:09:50.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.426+0000 7f8ccfb38640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8cb40b3d40 0x7f8cb40b6200 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.426+0000 7f8ccfb38640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cc810ce00 0x7f8cc819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.426+0000 7f8ccfb38640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cc81042c0 0x7f8cc8199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.426+0000 7f8ccfb38640 1 --2- 172.21.15.44:0/2943528948 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cc81039a0 0x7f8cc8198fd0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.426+0000 7f8ccfb38640 1 -- 172.21.15.44:0/2943528948 >> 172.21.15.44:0/2943528948 conn(0x7f8cc80fd400 msgr2=0x7f8cc80fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.426+0000 7f8ccfb38640 1 -- 172.21.15.44:0/2943528948 shutdown_connections 2024-08-20T03:09:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:50.426+0000 7f8ccfb38640 1 -- 172.21.15.44:0/2943528948 wait complete. 2024-08-20T03:09:51.350 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T03:09:51.351 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:09:51.608 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:51.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:51 smithi165 ceph-mon[22357]: pgmap v543: 97 pgs: 97 active+clean; 891 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.0 MiB/s wr, 311 op/s 2024-08-20T03:09:51.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:51 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:51.890 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:51 smithi044 ceph-mon[18790]: pgmap v543: 97 pgs: 97 active+clean; 891 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.0 MiB/s wr, 311 op/s 2024-08-20T03:09:51.890 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:51 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:51.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:51 smithi099 ceph-mon[21984]: pgmap v543: 97 pgs: 97 active+clean; 891 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.0 MiB/s wr, 311 op/s 2024-08-20T03:09:51.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:51 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:09:52.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.111+0000 7f47c1d0a640 1 -- 172.21.15.44:0/2260069170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47bc101800 msgr2=0x7f47bc101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:52.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.111+0000 7f47c1d0a640 1 --2- 172.21.15.44:0/2260069170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47bc101800 0x7f47bc101be0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009900 tx=0x7f47b002a410 comp rx=0 tx=0).stop 2024-08-20T03:09:52.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.111+0000 7f47c1d0a640 1 -- 172.21.15.44:0/2260069170 shutdown_connections 2024-08-20T03:09:52.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.111+0000 7f47c1d0a640 1 --2- 172.21.15.44:0/2260069170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47bc107b90 0x7f47bc10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:52.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.111+0000 7f47c1d0a640 1 --2- 172.21.15.44:0/2260069170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47bc107210 0x7f47bc107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:52.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.111+0000 7f47c1d0a640 1 --2- 172.21.15.44:0/2260069170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47bc101800 0x7f47bc101be0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:52.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.111+0000 7f47c1d0a640 1 -- 172.21.15.44:0/2260069170 >> 172.21.15.44:0/2260069170 conn(0x7f47bc0fd2d0 msgr2=0x7f47bc0ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:52.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.112+0000 7f47c1d0a640 1 -- 172.21.15.44:0/2260069170 shutdown_connections 2024-08-20T03:09:52.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.112+0000 7f47c1d0a640 1 -- 172.21.15.44:0/2260069170 wait complete. 2024-08-20T03:09:52.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.113+0000 7f47c1d0a640 1 Processor -- start 2024-08-20T03:09:52.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.113+0000 7f47c1d0a640 1 -- start start 2024-08-20T03:09:52.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.113+0000 7f47c1d0a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47bc101800 0x7f47bc197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:52.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.113+0000 7f47c1d0a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47bc107210 0x7f47bc198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:52.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.113+0000 7f47c1d0a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47bc107b90 0x7f47bc19d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:52.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.113+0000 7f47c1d0a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc111810 con 0x7f47bc107210 2024-08-20T03:09:52.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.113+0000 7f47c1d0a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc111690 con 0x7f47bc101800 2024-08-20T03:09:52.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.113+0000 7f47c1d0a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc111990 con 0x7f47bc107b90 2024-08-20T03:09:52.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.114+0000 7f47c1509640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47bc107b90 0x7f47bc19d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:52.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.114+0000 7f47c1509640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47bc107b90 0x7f47bc19d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41730/0 (socket says 172.21.15.44:41730) 2024-08-20T03:09:52.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.114+0000 7f47c1509640 1 -- 172.21.15.44:0/1867667997 learned_addr learned my addr 172.21.15.44:0/1867667997 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:52.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.115+0000 7f47c1509640 1 -- 172.21.15.44:0/1867667997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47bc101800 msgr2=0x7f47bc197c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T03:09:52.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.115+0000 7f47c1509640 1 --2- 172.21.15.44:0/1867667997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47bc101800 0x7f47bc197c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:52.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.115+0000 7f47c1509640 1 -- 172.21.15.44:0/1867667997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47bc107210 msgr2=0x7f47bc198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:52.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.115+0000 7f47c1509640 1 --2- 172.21.15.44:0/1867667997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47bc107210 0x7f47bc198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:52.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.115+0000 7f47c1509640 1 -- 172.21.15.44:0/1867667997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc19dcd0 con 0x7f47bc107b90 2024-08-20T03:09:52.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.115+0000 7f47c1509640 1 --2- 172.21.15.44:0/1867667997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47bc107b90 0x7f47bc19d500 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009f60 tx=0x7f47b4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:52.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.116+0000 7f47b9ffb640 1 -- 172.21.15.44:0/1867667997 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4019070 con 0x7f47bc107b90 2024-08-20T03:09:52.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.116+0000 7f47b9ffb640 1 -- 172.21.15.44:0/1867667997 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47b400e2d0 con 0x7f47bc107b90 2024-08-20T03:09:52.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.116+0000 7f47b9ffb640 1 -- 172.21.15.44:0/1867667997 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4015b70 con 0x7f47bc107b90 2024-08-20T03:09:52.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.116+0000 7f47c1d0a640 1 -- 172.21.15.44:0/1867667997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47bc19dfc0 con 0x7f47bc107b90 2024-08-20T03:09:52.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.116+0000 7f47c1d0a640 1 -- 172.21.15.44:0/1867667997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47bc06a7b0 con 0x7f47bc107b90 2024-08-20T03:09:52.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.118+0000 7f47c1d0a640 1 -- 172.21.15.44:0/1867667997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47bc101be0 con 0x7f47bc107b90 2024-08-20T03:09:52.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.120+0000 7f47b9ffb640 1 -- 172.21.15.44:0/1867667997 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f47b4015d10 con 0x7f47bc107b90 2024-08-20T03:09:52.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.121+0000 7f47b9ffb640 1 --2- 172.21.15.44:0/1867667997 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f47ac0b3ee0 0x7f47ac0b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:52.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.121+0000 7f47b9ffb640 1 -- 172.21.15.44:0/1867667997 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f47b40e7d20 con 0x7f47bc107b90 2024-08-20T03:09:52.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.122+0000 7f47c0d08640 1 --2- 172.21.15.44:0/1867667997 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f47ac0b3ee0 0x7f47ac0b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:52.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.122+0000 7f47c0d08640 1 --2- 172.21.15.44:0/1867667997 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f47ac0b3ee0 0x7f47ac0b63a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f47b002a930 tx=0x7f47b0000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:52.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.126+0000 7f47b9ffb640 1 -- 172.21.15.44:0/1867667997 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f47b40af010 con 0x7f47bc107b90 2024-08-20T03:09:52.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.420+0000 7f47c1d0a640 1 -- 172.21.15.44:0/1867667997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47bc198d50 con 0x7f47bc107b90 2024-08-20T03:09:52.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.421+0000 7f47b9ffb640 1 -- 172.21.15.44:0/1867667997 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11979 (secure 0 0 0) 0x7f47b40ae910 con 0x7f47bc107b90 2024-08-20T03:09:52.424 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:09:52.425 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":41,"btime":"2024-08-20T03:09:50:618324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:09:49.873550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24604,24589,24596,24583,24569]},"id":1}]} 2024-08-20T03:09:52.425 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 41 2024-08-20T03:09:52.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.423+0000 7f47c1d0a640 1 -- 172.21.15.44:0/1867667997 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f47ac0b3ee0 msgr2=0x7f47ac0b63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:52.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.423+0000 7f47c1d0a640 1 --2- 172.21.15.44:0/1867667997 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f47ac0b3ee0 0x7f47ac0b63a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f47b002a930 tx=0x7f47b0000bc0 comp rx=0 tx=0).stop 2024-08-20T03:09:52.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.423+0000 7f47c1d0a640 1 -- 172.21.15.44:0/1867667997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47bc107b90 msgr2=0x7f47bc19d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:52.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.423+0000 7f47c1d0a640 1 --2- 172.21.15.44:0/1867667997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47bc107b90 0x7f47bc19d500 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009f60 tx=0x7f47b4009f90 comp rx=0 tx=0).stop 2024-08-20T03:09:52.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.423+0000 7f47c1d0a640 1 -- 172.21.15.44:0/1867667997 shutdown_connections 2024-08-20T03:09:52.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.423+0000 7f47c1d0a640 1 --2- 172.21.15.44:0/1867667997 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f47ac0b3ee0 0x7f47ac0b63a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:52.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.423+0000 7f47c1d0a640 1 --2- 172.21.15.44:0/1867667997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47bc107b90 0x7f47bc19d500 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:52.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.423+0000 7f47c1d0a640 1 --2- 172.21.15.44:0/1867667997 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47bc107210 0x7f47bc198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:52.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.423+0000 7f47c1d0a640 1 --2- 172.21.15.44:0/1867667997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f47bc101800 0x7f47bc197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:52.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.423+0000 7f47c1d0a640 1 -- 172.21.15.44:0/1867667997 >> 172.21.15.44:0/1867667997 conn(0x7f47bc0fd2d0 msgr2=0x7f47bc0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:52.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.423+0000 7f47c1d0a640 1 -- 172.21.15.44:0/1867667997 shutdown_connections 2024-08-20T03:09:52.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:52.423+0000 7f47c1d0a640 1 -- 172.21.15.44:0/1867667997 wait complete. 2024-08-20T03:09:52.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:52 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/1867667997' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:09:52.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:52 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/1867667997' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:09:52.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:52 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/1867667997' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:09:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.082+0000 7f55f273c640 1 -- 172.21.15.44:0/1029434739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ec1014f0 msgr2=0x7f55ec1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.082+0000 7f55f273c640 1 --2- 172.21.15.44:0/1029434739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ec1014f0 0x7f55ec1018d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009900 tx=0x7f55dc02a410 comp rx=0 tx=0).stop 2024-08-20T03:09:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.082+0000 7f55f273c640 1 -- 172.21.15.44:0/1029434739 shutdown_connections 2024-08-20T03:09:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.082+0000 7f55f273c640 1 --2- 172.21.15.44:0/1029434739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55ec1073c0 0x7f55ec10dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.082+0000 7f55f273c640 1 --2- 172.21.15.44:0/1029434739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec106a40 0x7f55ec106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.082+0000 7f55f273c640 1 --2- 172.21.15.44:0/1029434739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ec1014f0 0x7f55ec1018d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.082+0000 7f55f273c640 1 -- 172.21.15.44:0/1029434739 >> 172.21.15.44:0/1029434739 conn(0x7f55ec076b10 msgr2=0x7f55ec076f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.082+0000 7f55f273c640 1 -- 172.21.15.44:0/1029434739 shutdown_connections 2024-08-20T03:09:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.082+0000 7f55f273c640 1 -- 172.21.15.44:0/1029434739 wait complete. 2024-08-20T03:09:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.083+0000 7f55f273c640 1 Processor -- start 2024-08-20T03:09:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.083+0000 7f55f273c640 1 -- start start 2024-08-20T03:09:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.083+0000 7f55f273c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ec1014f0 0x7f55ec10cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.087 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.083+0000 7f55f273c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec106a40 0x7f55ec10d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.083+0000 7f55f273c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55ec1073c0 0x7f55ec1a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.083+0000 7f55f273c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec112ab0 con 0x7f55ec106a40 2024-08-20T03:09:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.083+0000 7f55f273c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec112930 con 0x7f55ec1014f0 2024-08-20T03:09:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.083+0000 7f55f273c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec112c30 con 0x7f55ec1073c0 2024-08-20T03:09:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.083+0000 7f55e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec106a40 0x7f55ec10d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.083+0000 7f55e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec106a40 0x7f55ec10d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42752/0 (socket says 172.21.15.44:42752) 2024-08-20T03:09:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.083+0000 7f55e3fff640 1 -- 172.21.15.44:0/230259650 learned_addr learned my addr 172.21.15.44:0/230259650 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.084+0000 7f55ebfff640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ec1014f0 0x7f55ec10cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.084+0000 7f55e3fff640 1 -- 172.21.15.44:0/230259650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55ec1073c0 msgr2=0x7f55ec1a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.084+0000 7f55e3fff640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55ec1073c0 0x7f55ec1a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.084+0000 7f55e3fff640 1 -- 172.21.15.44:0/230259650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ec1014f0 msgr2=0x7f55ec10cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.084+0000 7f55e3fff640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ec1014f0 0x7f55ec10cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.084+0000 7f55e3fff640 1 -- 172.21.15.44:0/230259650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ec1a23f0 con 0x7f55ec106a40 2024-08-20T03:09:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.084+0000 7f55e3fff640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec106a40 0x7f55ec10d490 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f55d400e9f0 tx=0x7f55d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.084+0000 7f55e9ffb640 1 -- 172.21.15.44:0/230259650 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d400cd80 con 0x7f55ec106a40 2024-08-20T03:09:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.084+0000 7f55f273c640 1 -- 172.21.15.44:0/230259650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ec1a26e0 con 0x7f55ec106a40 2024-08-20T03:09:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.084+0000 7f55e9ffb640 1 -- 172.21.15.44:0/230259650 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d40108e0 con 0x7f55ec106a40 2024-08-20T03:09:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.084+0000 7f55f273c640 1 -- 172.21.15.44:0/230259650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ec1ac1e0 con 0x7f55ec106a40 2024-08-20T03:09:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.085+0000 7f55e9ffb640 1 -- 172.21.15.44:0/230259650 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4026060 con 0x7f55ec106a40 2024-08-20T03:09:53.091 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.088+0000 7f55f5607740 1 Processor -- start 2024-08-20T03:09:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.089+0000 7f55e9ffb640 1 -- 172.21.15.44:0/230259650 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f55d4010430 con 0x7f55ec106a40 2024-08-20T03:09:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.089+0000 7f55f5607740 1 -- start start 2024-08-20T03:09:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.090+0000 7f55f5607740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 0x55890fab8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.090+0000 7f55f5607740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fab9510 0x55890fac4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.090+0000 7f55f5607740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fac54a0 0x55890fac7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.090+0000 7f55f5607740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55890fa2e360 con 0x55890fac54a0 2024-08-20T03:09:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.090+0000 7f55f5607740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55890fa1d860 con 0x55890fab8bf0 2024-08-20T03:09:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.090+0000 7f55f5607740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55890fa218a0 con 0x55890fab9510 2024-08-20T03:09:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.090+0000 7f55e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fac54a0 0x55890fac7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.090+0000 7f55e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fac54a0 0x55890fac7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42768/0 (socket says 172.21.15.44:42768) 2024-08-20T03:09:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.090+0000 7f55e3fff640 1 -- 172.21.15.44:0/2136981736 learned_addr learned my addr 172.21.15.44:0/2136981736 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.090+0000 7f55e9ffb640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55d80b3d40 0x7f55d80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.091+0000 7f55e9ffb640 1 -- 172.21.15.44:0/230259650 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f55d40e43e0 con 0x7f55ec106a40 2024-08-20T03:09:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.091+0000 7f55ebfff640 1 --2- 172.21.15.44:0/2136981736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fab9510 0x55890fac4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.091+0000 7f55ebfff640 1 -- 172.21.15.44:0/2136981736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 msgr2=0x55890fab8fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:09:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.091+0000 7f55ebfff640 1 --2- 172.21.15.44:0/2136981736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 0x55890fab8fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.091+0000 7f55ebfff640 1 -- 172.21.15.44:0/2136981736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fac54a0 msgr2=0x55890fac7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.091+0000 7f55ebfff640 1 --2- 172.21.15.44:0/2136981736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fac54a0 0x55890fac7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.091+0000 7f55ebfff640 1 -- 172.21.15.44:0/2136981736 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890f9c4e40 con 0x55890fab9510 2024-08-20T03:09:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.091+0000 7f55e3fff640 1 --2- 172.21.15.44:0/2136981736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fac54a0 0x55890fac7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.092+0000 7f55ebfff640 1 --2- 172.21.15.44:0/2136981736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fab9510 0x55890fac4f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f55dc02aa00 tx=0x7f55dc005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b59cd6b34136b1de server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.092+0000 7f55f0cd9640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55d80b3d40 0x7f55d80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.093+0000 7f55e27fc640 1 -- 172.21.15.44:0/2136981736 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc035040 con 0x55890fab9510 2024-08-20T03:09:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.093+0000 7f55f0cd9640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55d80b3d40 0x7f55d80b6200 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f55e4005c40 tx=0x7f55e4005bd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.093+0000 7f55e27fc640 1 -- 172.21.15.44:0/2136981736 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55dc0054f0 con 0x55890fab9510 2024-08-20T03:09:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.093+0000 7f55e27fc640 1 -- 172.21.15.44:0/2136981736 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc035040 con 0x55890fab9510 2024-08-20T03:09:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.094+0000 7f55f5607740 1 -- 172.21.15.44:0/2136981736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fab9510 msgr2=0x55890fac4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.094+0000 7f55f5607740 1 --2- 172.21.15.44:0/2136981736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fab9510 0x55890fac4f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f55dc02aa00 tx=0x7f55dc005c00 comp rx=0 tx=0).stop 2024-08-20T03:09:53.098 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.094+0000 7f55f5607740 1 -- 172.21.15.44:0/2136981736 shutdown_connections 2024-08-20T03:09:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.094+0000 7f55f5607740 1 --2- 172.21.15.44:0/2136981736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fac54a0 0x55890fac7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.094+0000 7f55f5607740 1 --2- 172.21.15.44:0/2136981736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fab9510 0x55890fac4f60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.094+0000 7f55f5607740 1 --2- 172.21.15.44:0/2136981736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 0x55890fab8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.094+0000 7f55f5607740 1 -- 172.21.15.44:0/2136981736 >> 172.21.15.44:0/2136981736 conn(0x55890fab6a20 msgr2=0x55890fab7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:53.099 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.094+0000 7f55f5607740 1 -- 172.21.15.44:0/2136981736 shutdown_connections 2024-08-20T03:09:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.094+0000 7f55f5607740 1 -- 172.21.15.44:0/2136981736 wait complete. 2024-08-20T03:09:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.094+0000 7f55f5607740 1 -- start start 2024-08-20T03:09:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.095+0000 7f55f5607740 1 Processor -- start 2024-08-20T03:09:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.095+0000 7f55f5607740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 0x55890fac4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.095+0000 7f55f5607740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 0x55890fac2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.095+0000 7f55f5607740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890fac2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.095+0000 7f55f5607740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55890f956ab0 con 0x55890fab9510 2024-08-20T03:09:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.095+0000 7f55f5607740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55890f9a4780 con 0x55890fab8bf0 2024-08-20T03:09:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.095+0000 7f55f5607740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55890fac8120 con 0x55890fac54a0 2024-08-20T03:09:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.095+0000 7f55f5607740 10 client.0 _ll_register_callbacks cb 0x55890f72ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:53.102 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.095+0000 7f55e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 0x55890fac2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.095+0000 7f55e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 0x55890fac2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42776/0 (socket says 172.21.15.44:42776) 2024-08-20T03:09:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.095+0000 7f55e3fff640 1 -- 172.21.15.44:0/1286618053 learned_addr learned my addr 172.21.15.44:0/1286618053 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.096+0000 7f55ebfff640 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 0x55890fac4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.096+0000 7f55f0cd9640 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890fac2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.096+0000 7f55e3fff640 1 -- 172.21.15.44:0/1286618053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 msgr2=0x55890fac2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.096+0000 7f55e3fff640 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890fac2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.096+0000 7f55e3fff640 1 -- 172.21.15.44:0/1286618053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 msgr2=0x55890fac4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.096+0000 7f55e3fff640 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 0x55890fac4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.096+0000 7f55e3fff640 1 -- 172.21.15.44:0/1286618053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890f780790 con 0x55890fab9510 2024-08-20T03:09:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.096+0000 7f55f0cd9640 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890fac2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:09:53.103 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.096+0000 7f55e3fff640 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 0x55890fac2280 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c6a0 tx=0x7f55d4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f821903f66880183 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.096+0000 7f55cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.097+0000 7f55cb7fe640 1 -- 172.21.15.44:0/1286618053 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d40e2000 con 0x55890fab9510 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.097+0000 7f55f5607740 1 -- 172.21.15.44:0/1286618053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55890f9cce00 con 0x55890fab9510 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.097+0000 7f55cb7fe640 1 -- 172.21.15.44:0/1286618053 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f55d40aa290 con 0x55890fab9510 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.097+0000 7f55cb7fe640 1 -- 172.21.15.44:0/1286618053 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f55d4010720 con 0x55890fab9510 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.097+0000 7f55f5607740 10 client.16980 fetch_fsmap learned FSMap version 41 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.097+0000 7f55f5607740 1 -- 172.21.15.44:0/1286618053 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55890f7f9620 con 0x55890fab9510 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.097+0000 7f55cb7fe640 1 -- 172.21.15.44:0/1286618053 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55890f9cce00 con 0x55890fab9510 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.097+0000 7f55cb7fe640 1 -- 172.21.15.44:0/1286618053 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f55d40a69a0 con 0x55890fab9510 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.097+0000 7f55f5607740 10 client.16980 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.097+0000 7f55f5607740 10 client.16980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.097+0000 7f55f5607740 20 client.16980 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.098+0000 7f55f5607740 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55890fad4e50 0x55890fad7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.104 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.098+0000 7f55f5607740 4 client.16980 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:53.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.098+0000 7f55f5607740 1 -- 172.21.15.44:0/1286618053 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55890fa302b0 con 0x55890fad4e50 2024-08-20T03:09:53.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.098+0000 7f55ebfff640 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55890fad4e50 0x55890fad7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.098+0000 7f55ebfff640 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55890fad4e50 0x55890fad7240 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73710b15b75b3505 server_cookie=61aaeb9ab1f9d696 in_seq=0 out_seq=0 2024-08-20T03:09:53.105 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.098+0000 7f55cb7fe640 10 client.16980 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.100+0000 7f55cb7fe640 1 -- 172.21.15.44:0/1286618053 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55890fa302b0 con 0x55890fad4e50 2024-08-20T03:09:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.100+0000 7f55cb7fe640 10 client.16980 handle_command_reply: tid=0 2024-08-20T03:09:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.100+0000 7f55f5607740 1 client.16980 shutdown 2024-08-20T03:09:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.100+0000 7f55f5607740 1 -- 172.21.15.44:0/1286618053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 msgr2=0x55890fac2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.100+0000 7f55f5607740 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 0x55890fac2280 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c6a0 tx=0x7f55d4009310 comp rx=0 tx=0).stop 2024-08-20T03:09:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.101+0000 7f55f5607740 1 -- 172.21.15.44:0/1286618053 shutdown_connections 2024-08-20T03:09:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.101+0000 7f55f5607740 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55890fad4e50 0x55890fad7240 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.101+0000 7f55f5607740 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890fac2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.101+0000 7f55f5607740 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 0x55890fac2280 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.106 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.101+0000 7f55f5607740 1 --2- 172.21.15.44:0/1286618053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 0x55890fac4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.101+0000 7f55f5607740 1 -- 172.21.15.44:0/1286618053 >> 172.21.15.44:0/1286618053 conn(0x55890fab6a20 msgr2=0x55890fabc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.101+0000 7f55cb7fe640 0 client.16980 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.101+0000 7f55f5607740 1 -- 172.21.15.44:0/1286618053 shutdown_connections 2024-08-20T03:09:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.101+0000 7f55f5607740 1 -- 172.21.15.44:0/1286618053 wait complete. 2024-08-20T03:09:53.107 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.101+0000 7f55f5607740 20 client.16980 trim_cache size 0 max 16384 2024-08-20T03:09:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55f5607740 1 Processor -- start 2024-08-20T03:09:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55f5607740 1 -- start start 2024-08-20T03:09:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55f5607740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 0x55890fad1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55f5607740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 0x55890fac6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55f5607740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890fac6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55f5607740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec1b4940 con 0x55890fab9510 2024-08-20T03:09:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55f5607740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec1b6600 con 0x55890fab8bf0 2024-08-20T03:09:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55f5607740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec1b50a0 con 0x55890fac54a0 2024-08-20T03:09:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 0x55890fac6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 0x55890fac6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42778/0 (socket says 172.21.15.44:42778) 2024-08-20T03:09:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55e3fff640 1 -- 172.21.15.44:0/4129521110 learned_addr learned my addr 172.21.15.44:0/4129521110 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55f0cd9640 1 --2- 172.21.15.44:0/4129521110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890fac6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55ebfff640 1 --2- 172.21.15.44:0/4129521110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 0x55890fad1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55e3fff640 1 -- 172.21.15.44:0/4129521110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 msgr2=0x55890fac6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55e3fff640 1 --2- 172.21.15.44:0/4129521110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890fac6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55e3fff640 1 -- 172.21.15.44:0/4129521110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 msgr2=0x55890fad1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55e3fff640 1 --2- 172.21.15.44:0/4129521110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 0x55890fad1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55e3fff640 1 -- 172.21.15.44:0/4129521110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55890fad3a40 con 0x55890fab9510 2024-08-20T03:09:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55ebfff640 1 --2- 172.21.15.44:0/4129521110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 0x55890fad1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:09:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.120+0000 7f55e3fff640 1 --2- 172.21.15.44:0/4129521110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 0x55890fac6890 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f55d40a75d0 tx=0x7f55d400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7aa035b9700eae server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.121+0000 7f55e1ffb640 1 -- 172.21.15.44:0/4129521110 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d40aab40 con 0x55890fab9510 2024-08-20T03:09:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.121+0000 7f55e1ffb640 1 -- 172.21.15.44:0/4129521110 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d40ab730 con 0x55890fab9510 2024-08-20T03:09:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.121+0000 7f55e1ffb640 1 -- 172.21.15.44:0/4129521110 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d40a9d00 con 0x55890fab9510 2024-08-20T03:09:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.121+0000 7f55f5607740 1 -- 172.21.15.44:0/4129521110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 msgr2=0x55890fac6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.121+0000 7f55f5607740 1 --2- 172.21.15.44:0/4129521110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 0x55890fac6890 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f55d40a75d0 tx=0x7f55d400c7f0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.121+0000 7f55f5607740 1 -- 172.21.15.44:0/4129521110 shutdown_connections 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.121+0000 7f55f5607740 1 --2- 172.21.15.44:0/4129521110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890fac6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.121+0000 7f55f5607740 1 --2- 172.21.15.44:0/4129521110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab9510 0x55890fac6890 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.121+0000 7f55f5607740 1 --2- 172.21.15.44:0/4129521110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab8bf0 0x55890fad1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.121+0000 7f55f5607740 1 -- 172.21.15.44:0/4129521110 >> 172.21.15.44:0/4129521110 conn(0x55890fab6a20 msgr2=0x55890fac0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.121+0000 7f55f5607740 1 -- 172.21.15.44:0/4129521110 shutdown_connections 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.121+0000 7f55f5607740 1 -- 172.21.15.44:0/4129521110 wait complete. 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55f5607740 1 -- start start 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55f5607740 1 Processor -- start 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55f5607740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab8bf0 0x55890fac5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55f5607740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab9510 0x55890fac5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55f5607740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890facc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55f5607740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec1b5f30 con 0x55890fab8bf0 2024-08-20T03:09:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55f5607740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec1b5800 con 0x55890fab9510 2024-08-20T03:09:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55f5607740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec1b2940 con 0x55890fac54a0 2024-08-20T03:09:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55f5607740 10 client.0 _ll_register_callbacks cb 0x7f55d400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55ebfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab8bf0 0x55890fac5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55ebfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab8bf0 0x55890fac5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42794/0 (socket says 172.21.15.44:42794) 2024-08-20T03:09:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55ebfff640 1 -- 172.21.15.44:0/3887632514 learned_addr learned my addr 172.21.15.44:0/3887632514 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55f0cd9640 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890facc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.122+0000 7f55e3fff640 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab9510 0x55890fac5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55ebfff640 1 -- 172.21.15.44:0/3887632514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 msgr2=0x55890facc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55ebfff640 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890facc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55ebfff640 1 -- 172.21.15.44:0/3887632514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab9510 msgr2=0x55890fac5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55ebfff640 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab9510 0x55890fac5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55ebfff640 1 -- 172.21.15.44:0/3887632514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55890facc8f0 con 0x55890fab8bf0 2024-08-20T03:09:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55e3fff640 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab9510 0x55890fac5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:09:53.133 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55ebfff640 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab8bf0 0x55890fac5ab0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009900 tx=0x7f55dc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9d712b606dc22b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55cb7fe640 1 -- 172.21.15.44:0/3887632514 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc055020 con 0x55890fab8bf0 2024-08-20T03:09:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55cb7fe640 1 -- 172.21.15.44:0/3887632514 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f55dc003940 con 0x55890fab8bf0 2024-08-20T03:09:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55f5607740 1 -- 172.21.15.44:0/3887632514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d40ab730 con 0x55890fab8bf0 2024-08-20T03:09:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55cb7fe640 1 -- 172.21.15.44:0/3887632514 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f55dc060050 con 0x55890fab8bf0 2024-08-20T03:09:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55f5607740 10 client.16986 fetch_fsmap learned FSMap version 41 2024-08-20T03:09:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55f5607740 1 -- 172.21.15.44:0/3887632514 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55890facdc00 con 0x55890fab8bf0 2024-08-20T03:09:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.123+0000 7f55cb7fe640 1 -- 172.21.15.44:0/3887632514 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d40ab730 con 0x55890fab8bf0 2024-08-20T03:09:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.124+0000 7f55cb7fe640 1 -- 172.21.15.44:0/3887632514 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f55dc031c40 con 0x55890fab8bf0 2024-08-20T03:09:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.124+0000 7f55f5607740 10 client.16986 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:09:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.124+0000 7f55f5607740 10 client.16986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:09:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.124+0000 7f55f5607740 20 client.16986 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.124+0000 7f55f5607740 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55890fb3fa90 0x55890fb41e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.124+0000 7f55f5607740 4 client.16986 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:09:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.124+0000 7f55f5607740 1 -- 172.21.15.44:0/3887632514 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55890fa2bd50 con 0x55890fb3fa90 2024-08-20T03:09:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.124+0000 7f55e3fff640 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55890fb3fa90 0x55890fb41e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.125+0000 7f55e3fff640 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55890fb3fa90 0x55890fb41e80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c0f7871c750fd2 server_cookie=884e9d55213e50ee in_seq=0 out_seq=0 2024-08-20T03:09:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:53.125+0000 7f55cb7fe640 10 client.16986 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:53.422 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-20T03:09:53.713 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:53.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:53 smithi044 ceph-mon[18790]: pgmap v544: 97 pgs: 97 active+clean; 893 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 292 op/s 2024-08-20T03:09:53.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:53 smithi165 ceph-mon[22357]: pgmap v544: 97 pgs: 97 active+clean; 893 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 292 op/s 2024-08-20T03:09:53.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:53 smithi099 ceph-mon[21984]: pgmap v544: 97 pgs: 97 active+clean; 893 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 292 op/s 2024-08-20T03:09:54.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.247+0000 7fa833e9d640 1 -- 172.21.15.44:0/3185346267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82c107d20 msgr2=0x7fa82c10e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:54.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.247+0000 7fa833e9d640 1 --2- 172.21.15.44:0/3185346267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82c107d20 0x7fa82c10e520 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa824009ca0 tx=0x7fa82402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.247+0000 7fa833e9d640 1 -- 172.21.15.44:0/3185346267 shutdown_connections 2024-08-20T03:09:54.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.247+0000 7fa833e9d640 1 --2- 172.21.15.44:0/3185346267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82c107d20 0x7fa82c10e520 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.247+0000 7fa833e9d640 1 --2- 172.21.15.44:0/3185346267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c1073a0 0x7fa82c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.247+0000 7fa833e9d640 1 --2- 172.21.15.44:0/3185346267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa82c101990 0x7fa82c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.247+0000 7fa833e9d640 1 -- 172.21.15.44:0/3185346267 >> 172.21.15.44:0/3185346267 conn(0x7fa82c0fd400 msgr2=0x7fa82c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:54.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.248+0000 7fa833e9d640 1 -- 172.21.15.44:0/3185346267 shutdown_connections 2024-08-20T03:09:54.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.248+0000 7fa833e9d640 1 -- 172.21.15.44:0/3185346267 wait complete. 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.248+0000 7fa833e9d640 1 Processor -- start 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.248+0000 7fa833e9d640 1 -- start start 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.248+0000 7fa833e9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82c101990 0x7fa82c199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.248+0000 7fa833e9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa82c1073a0 0x7fa82c199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.248+0000 7fa833e9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c107d20 0x7fa82c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.248+0000 7fa833e9d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1119a0 con 0x7fa82c101990 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.248+0000 7fa833e9d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c111820 con 0x7fa82c107d20 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.248+0000 7fa833e9d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c111b20 con 0x7fa82c1073a0 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.248+0000 7fa831c39640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82c101990 0x7fa82c199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa831c39640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82c101990 0x7fa82c199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42812/0 (socket says 172.21.15.44:42812) 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa831c39640 1 -- 172.21.15.44:0/3173012581 learned_addr learned my addr 172.21.15.44:0/3173012581 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa831438640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa82c1073a0 0x7fa82c199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa83243a640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c107d20 0x7fa82c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa831c39640 1 -- 172.21.15.44:0/3173012581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa82c1073a0 msgr2=0x7fa82c199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa831c39640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa82c1073a0 0x7fa82c199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa831c39640 1 -- 172.21.15.44:0/3173012581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c107d20 msgr2=0x7fa82c19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa831c39640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c107d20 0x7fa82c19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa831c39640 1 -- 172.21.15.44:0/3173012581 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c19f0c0 con 0x7fa82c101990 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa83243a640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c107d20 0x7fa82c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa831c39640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82c101990 0x7fa82c199010 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa82800d950 tx=0x7fa82800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa822ffd640 1 -- 172.21.15.44:0/3173012581 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828017070 con 0x7fa82c101990 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa833e9d640 1 -- 172.21.15.44:0/3173012581 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82c19f3b0 con 0x7fa82c101990 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa822ffd640 1 -- 172.21.15.44:0/3173012581 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa828009c30 con 0x7fa82c101990 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa822ffd640 1 -- 172.21.15.44:0/3173012581 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82800f620 con 0x7fa82c101990 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.249+0000 7fa833e9d640 1 -- 172.21.15.44:0/3173012581 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82c1a5c40 con 0x7fa82c101990 2024-08-20T03:09:54.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.252+0000 7fa822ffd640 1 -- 172.21.15.44:0/3173012581 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fa828010460 con 0x7fa82c101990 2024-08-20T03:09:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.252+0000 7fa836296740 1 Processor -- start 2024-08-20T03:09:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.252+0000 7fa836296740 1 -- start start 2024-08-20T03:09:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.252+0000 7fa836296740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ed3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.252+0000 7fa836296740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 0x55b5a2edf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.252+0000 7fa836296740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 0x55b5a2ee1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa836296740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b5a2e45930 con 0x55b5a2ed2c90 2024-08-20T03:09:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa836296740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b5a2e34e30 con 0x55b5a2ed35b0 2024-08-20T03:09:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa836296740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b5a2e38e70 con 0x55b5a2edf540 2024-08-20T03:09:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa83243a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ed3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa83243a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ed3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42820/0 (socket says 172.21.15.44:42820) 2024-08-20T03:09:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa831c39640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 0x55b5a2edf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa83243a640 1 -- 172.21.15.44:0/1406001971 learned_addr learned my addr 172.21.15.44:0/1406001971 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa831438640 1 --2- 172.21.15.44:0/1406001971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 0x55b5a2ee1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa822ffd640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8180b3d40 0x7fa8180b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa83243a640 1 -- 172.21.15.44:0/1406001971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 msgr2=0x55b5a2ee1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa83243a640 1 --2- 172.21.15.44:0/1406001971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 0x55b5a2ee1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa83243a640 1 -- 172.21.15.44:0/1406001971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 msgr2=0x55b5a2edf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa83243a640 1 --2- 172.21.15.44:0/1406001971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 0x55b5a2edf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa83243a640 1 -- 172.21.15.44:0/1406001971 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a2b94650 con 0x55b5a2ed2c90 2024-08-20T03:09:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa83243a640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8180b3d40 0x7fa8180b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa822ffd640 1 -- 172.21.15.44:0/3173012581 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fa8280e3b50 con 0x7fa82c101990 2024-08-20T03:09:54.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa83243a640 1 --2- 172.21.15.44:0/1406001971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ed3070 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa824033040 tx=0x7fa82402f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5cba335c199ee3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.253+0000 7fa817fff640 1 -- 172.21.15.44:0/1406001971 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824043040 con 0x55b5a2ed2c90 2024-08-20T03:09:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.254+0000 7fa817fff640 1 -- 172.21.15.44:0/1406001971 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa824036040 con 0x55b5a2ed2c90 2024-08-20T03:09:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.254+0000 7fa817fff640 1 -- 172.21.15.44:0/1406001971 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82402f8c0 con 0x55b5a2ed2c90 2024-08-20T03:09:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.254+0000 7fa83243a640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8180b3d40 0x7fa8180b6200 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa82404e000 tx=0x7fa82404f000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.254+0000 7fa836296740 1 -- 172.21.15.44:0/1406001971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 msgr2=0x55b5a2ed3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.254+0000 7fa836296740 1 --2- 172.21.15.44:0/1406001971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ed3070 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa824033040 tx=0x7fa82402f490 comp rx=0 tx=0).stop 2024-08-20T03:09:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.254+0000 7fa836296740 1 -- 172.21.15.44:0/1406001971 shutdown_connections 2024-08-20T03:09:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.254+0000 7fa836296740 1 --2- 172.21.15.44:0/1406001971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 0x55b5a2ee1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.254+0000 7fa836296740 1 --2- 172.21.15.44:0/1406001971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 0x55b5a2edf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.254+0000 7fa836296740 1 --2- 172.21.15.44:0/1406001971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ed3070 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.254+0000 7fa836296740 1 -- 172.21.15.44:0/1406001971 >> 172.21.15.44:0/1406001971 conn(0x55b5a2ed0ac0 msgr2=0x55b5a2ed1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:54.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.254+0000 7fa836296740 1 -- 172.21.15.44:0/1406001971 shutdown_connections 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.254+0000 7fa836296740 1 -- 172.21.15.44:0/1406001971 wait complete. 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa836296740 1 -- start start 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa836296740 1 Processor -- start 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa836296740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 0x55b5a2edf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa836296740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 0x55b5a2edc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa836296740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 0x55b5a2edcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa836296740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b5a2c97700 con 0x55b5a2ed2c90 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa836296740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b5a2dbb810 con 0x55b5a2ed35b0 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa836296740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b5a2ee21c0 con 0x55b5a2edf540 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa836296740 10 client.0 _ll_register_callbacks cb 0x55b5a2e2e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa831c39640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 0x55b5a2edf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa831c39640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 0x55b5a2edf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42828/0 (socket says 172.21.15.44:42828) 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa831c39640 1 -- 172.21.15.44:0/1606862259 learned_addr learned my addr 172.21.15.44:0/1606862259 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:54.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa83243a640 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 0x55b5a2edcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.255+0000 7fa831438640 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 0x55b5a2edc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa831c39640 1 -- 172.21.15.44:0/1606862259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 msgr2=0x55b5a2edcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa831c39640 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 0x55b5a2edcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa831c39640 1 -- 172.21.15.44:0/1606862259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 msgr2=0x55b5a2edc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa831c39640 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 0x55b5a2edc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa831c39640 1 -- 172.21.15.44:0/1606862259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a2eeb4f0 con 0x55b5a2ed2c90 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa831c39640 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 0x55b5a2edf000 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa8280a68d0 tx=0x7fa828005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc0933c51034594 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa8157fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa8157fa640 1 -- 172.21.15.44:0/1606862259 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828020070 con 0x55b5a2ed2c90 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa8157fa640 1 -- 172.21.15.44:0/1606862259 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fa8280cd2b0 con 0x55b5a2ed2c90 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa836296740 1 -- 172.21.15.44:0/1606862259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5a2de4150 con 0x55b5a2ed2c90 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa8157fa640 1 -- 172.21.15.44:0/1606862259 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa8280aa1a0 con 0x55b5a2ed2c90 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa836296740 10 client.16998 fetch_fsmap learned FSMap version 41 2024-08-20T03:09:54.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa836296740 1 -- 172.21.15.44:0/1606862259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b5a2eedf00 con 0x55b5a2ed2c90 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.256+0000 7fa8157fa640 1 -- 172.21.15.44:0/1606862259 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b5a2de4150 con 0x55b5a2ed2c90 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.257+0000 7fa8157fa640 1 -- 172.21.15.44:0/1606862259 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7fa8280a61c0 con 0x55b5a2ed2c90 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.257+0000 7fa836296740 10 client.16998 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.257+0000 7fa836296740 10 client.16998 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.257+0000 7fa836296740 20 client.16998 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.257+0000 7fa836296740 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55b5a2eef3c0 0x55b5a2ef17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.257+0000 7fa836296740 4 client.16998 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.257+0000 7fa836296740 1 -- 172.21.15.44:0/1606862259 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5a2e47880 con 0x55b5a2eef3c0 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.257+0000 7fa831438640 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55b5a2eef3c0 0x55b5a2ef17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.257+0000 7fa831438640 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55b5a2eef3c0 0x55b5a2ef17b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=551bffadedef504 server_cookie=7643ef36f30e0c74 in_seq=0 out_seq=0 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.257+0000 7fa8157fa640 10 client.16998 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa8157fa640 1 -- 172.21.15.44:0/1606862259 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5a2e47880 con 0x55b5a2eef3c0 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa8157fa640 10 client.16998 handle_command_reply: tid=0 2024-08-20T03:09:54.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa836296740 1 client.16998 shutdown 2024-08-20T03:09:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa836296740 1 -- 172.21.15.44:0/1606862259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 msgr2=0x55b5a2edf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa836296740 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 0x55b5a2edf000 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa8280a68d0 tx=0x7fa828005f70 comp rx=0 tx=0).stop 2024-08-20T03:09:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa836296740 1 -- 172.21.15.44:0/1606862259 shutdown_connections 2024-08-20T03:09:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa836296740 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 0x55b5a2edcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa836296740 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55b5a2eef3c0 0x55b5a2ef17b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa836296740 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 0x55b5a2edc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa836296740 1 --2- 172.21.15.44:0/1606862259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed2c90 0x55b5a2edf000 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa8157fa640 0 client.16998 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:09:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa836296740 1 -- 172.21.15.44:0/1606862259 >> 172.21.15.44:0/1606862259 conn(0x55b5a2ed0ac0 msgr2=0x55b5a2ed6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa836296740 1 -- 172.21.15.44:0/1606862259 shutdown_connections 2024-08-20T03:09:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa836296740 1 -- 172.21.15.44:0/1606862259 wait complete. 2024-08-20T03:09:54.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.258+0000 7fa836296740 20 client.16998 trim_cache size 0 max 16384 2024-08-20T03:09:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa836296740 1 Processor -- start 2024-08-20T03:09:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa836296740 1 -- start start 2024-08-20T03:09:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa836296740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ef17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa836296740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 0x55b5a2edbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa836296740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2edf540 0x55b5a2ede190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa836296740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1ac110 con 0x55b5a2edf540 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa836296740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1addd0 con 0x55b5a2ed35b0 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa836296740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1ac870 con 0x55b5a2ed2c90 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa83243a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2edf540 0x55b5a2ede190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa831c39640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ef17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa831c39640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ef17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41824/0 (socket says 172.21.15.44:41824) 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa831c39640 1 -- 172.21.15.44:0/765061282 learned_addr learned my addr 172.21.15.44:0/765061282 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.272+0000 7fa831438640 1 --2- 172.21.15.44:0/765061282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 0x55b5a2edbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.273+0000 7fa831c39640 1 -- 172.21.15.44:0/765061282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 msgr2=0x55b5a2edbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.273+0000 7fa831c39640 1 --2- 172.21.15.44:0/765061282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 0x55b5a2edbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.273+0000 7fa831c39640 1 -- 172.21.15.44:0/765061282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2edf540 msgr2=0x55b5a2ede190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.273+0000 7fa831c39640 1 --2- 172.21.15.44:0/765061282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2edf540 0x55b5a2ede190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.273+0000 7fa831c39640 1 -- 172.21.15.44:0/765061282 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a2edcf90 con 0x55b5a2ed2c90 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.273+0000 7fa83243a640 1 --2- 172.21.15.44:0/765061282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2edf540 0x55b5a2ede190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.273+0000 7fa831c39640 1 --2- 172.21.15.44:0/765061282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ef17b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa8280e18e0 tx=0x7fa8280e1820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a65bba6486e80662 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:54.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.273+0000 7fa8177fe640 1 -- 172.21.15.44:0/765061282 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828020070 con 0x55b5a2ed2c90 2024-08-20T03:09:54.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.273+0000 7fa8177fe640 1 -- 172.21.15.44:0/765061282 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa828009de0 con 0x55b5a2ed2c90 2024-08-20T03:09:54.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.273+0000 7fa8177fe640 1 -- 172.21.15.44:0/765061282 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8280a9fc0 con 0x55b5a2ed2c90 2024-08-20T03:09:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.274+0000 7fa836296740 1 -- 172.21.15.44:0/765061282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2ed2c90 msgr2=0x55b5a2ef17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.274+0000 7fa836296740 1 --2- 172.21.15.44:0/765061282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ef17b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa8280e18e0 tx=0x7fa8280e1820 comp rx=0 tx=0).stop 2024-08-20T03:09:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.274+0000 7fa836296740 1 -- 172.21.15.44:0/765061282 shutdown_connections 2024-08-20T03:09:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.274+0000 7fa836296740 1 --2- 172.21.15.44:0/765061282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2edf540 0x55b5a2ede190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.274+0000 7fa836296740 1 --2- 172.21.15.44:0/765061282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed35b0 0x55b5a2edbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.274+0000 7fa836296740 1 --2- 172.21.15.44:0/765061282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ef17b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.274+0000 7fa836296740 1 -- 172.21.15.44:0/765061282 >> 172.21.15.44:0/765061282 conn(0x55b5a2ed0ac0 msgr2=0x55b5a2edda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.274+0000 7fa836296740 1 -- 172.21.15.44:0/765061282 shutdown_connections 2024-08-20T03:09:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.274+0000 7fa836296740 1 -- 172.21.15.44:0/765061282 wait complete. 2024-08-20T03:09:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.274+0000 7fa836296740 1 -- start start 2024-08-20T03:09:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.274+0000 7fa836296740 1 Processor -- start 2024-08-20T03:09:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa836296740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ee7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa836296740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed35b0 0x55b5a2ee7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa836296740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 0x55b5a2ed8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa836296740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1ad700 con 0x55b5a2ed35b0 2024-08-20T03:09:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa836296740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1acfd0 con 0x55b5a2ed2c90 2024-08-20T03:09:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa836296740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c1aa110 con 0x55b5a2edf540 2024-08-20T03:09:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa836296740 10 client.0 _ll_register_callbacks cb 0x7fa828005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:54.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa831438640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed35b0 0x55b5a2ee7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa831c39640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ee7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa831438640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed35b0 0x55b5a2ee7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42834/0 (socket says 172.21.15.44:42834) 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa83243a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 0x55b5a2ed8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa831438640 1 -- 172.21.15.44:0/3664742194 learned_addr learned my addr 172.21.15.44:0/3664742194 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa831c39640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ee7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:44774/0 (socket says 172.21.15.44:44774) 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa831438640 1 -- 172.21.15.44:0/3664742194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 msgr2=0x55b5a2ed8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa831438640 1 --2- 172.21.15.44:0/3664742194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 0x55b5a2ed8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa831438640 1 -- 172.21.15.44:0/3664742194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed2c90 msgr2=0x55b5a2ee7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa831438640 1 --2- 172.21.15.44:0/3664742194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ee7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa831438640 1 -- 172.21.15.44:0/3664742194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a2edf370 con 0x55b5a2ed35b0 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa831c39640 1 --2- 172.21.15.44:0/3664742194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ee7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa831438640 1 --2- 172.21.15.44:0/3664742194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed35b0 0x55b5a2ee7990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa81c004700 tx=0x7fa81c010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a714bc5de87ff13 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa8157fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.275+0000 7fa8157fa640 1 -- 172.21.15.44:0/3664742194 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c005020 con 0x55b5a2ed35b0 2024-08-20T03:09:54.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa8157fa640 1 -- 172.21.15.44:0/3664742194 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fa81c0051c0 con 0x55b5a2ed35b0 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa836296740 1 -- 172.21.15.44:0/3664742194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa828009de0 con 0x55b5a2ed35b0 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa8157fa640 1 -- 172.21.15.44:0/3664742194 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa81c0053c0 con 0x55b5a2ed35b0 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa836296740 10 client.17004 fetch_fsmap learned FSMap version 41 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa836296740 1 -- 172.21.15.44:0/3664742194 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b5a2f59560 con 0x55b5a2ed35b0 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa8157fa640 1 -- 172.21.15.44:0/3664742194 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa828009de0 con 0x55b5a2ed35b0 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa8157fa640 1 -- 172.21.15.44:0/3664742194 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7fa81c004a70 con 0x55b5a2ed35b0 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa836296740 10 client.17004 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa836296740 10 client.17004 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa836296740 20 client.17004 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa836296740 1 --2- 172.21.15.44:0/3664742194 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55b5a2f59d50 0x55b5a2f5c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa836296740 4 client.17004 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa836296740 1 -- 172.21.15.44:0/3664742194 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5a2e43320 con 0x55b5a2f59d50 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.276+0000 7fa831c39640 1 --2- 172.21.15.44:0/3664742194 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55b5a2f59d50 0x55b5a2f5c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa831c39640 1 --2- 172.21.15.44:0/3664742194 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55b5a2f59d50 0x55b5a2f5c170 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4546980da6382a0f server_cookie=8fc791f8af64ba5b in_seq=0 out_seq=0 2024-08-20T03:09:54.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa8157fa640 10 client.17004 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa8157fa640 1 -- 172.21.15.44:0/3664742194 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5a2e43320 con 0x55b5a2f59d50 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa8157fa640 10 client.17004 handle_command_reply: tid=0 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa836296740 1 client.17004 shutdown 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa836296740 1 -- 172.21.15.44:0/3664742194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed35b0 msgr2=0x55b5a2ee7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa836296740 1 --2- 172.21.15.44:0/3664742194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed35b0 0x55b5a2ee7990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa81c004700 tx=0x7fa81c010af0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa836296740 1 -- 172.21.15.44:0/3664742194 shutdown_connections 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa836296740 1 --2- 172.21.15.44:0/3664742194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5a2edf540 0x55b5a2ed8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa836296740 1 --2- 172.21.15.44:0/3664742194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b5a2ed35b0 0x55b5a2ee7990 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa836296740 1 --2- 172.21.15.44:0/3664742194 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55b5a2f59d50 0x55b5a2f5c170 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa836296740 1 --2- 172.21.15.44:0/3664742194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b5a2ed2c90 0x55b5a2ee7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa8157fa640 0 client.17004 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa836296740 1 -- 172.21.15.44:0/3664742194 >> 172.21.15.44:0/3664742194 conn(0x55b5a2ed0ac0 msgr2=0x55b5a2ede2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa836296740 1 -- 172.21.15.44:0/3664742194 shutdown_connections 2024-08-20T03:09:54.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa836296740 1 -- 172.21.15.44:0/3664742194 wait complete. 2024-08-20T03:09:54.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.277+0000 7fa836296740 20 client.17004 trim_cache size 0 max 16384 2024-08-20T03:09:54.286 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:09:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.278+0000 7fa833e9d640 1 -- 172.21.15.44:0/3173012581 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8180b3d40 msgr2=0x7fa8180b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.278+0000 7fa833e9d640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8180b3d40 0x7fa8180b6200 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa82404e000 tx=0x7fa82404f000 comp rx=0 tx=0).stop 2024-08-20T03:09:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.278+0000 7fa833e9d640 1 -- 172.21.15.44:0/3173012581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82c101990 msgr2=0x7fa82c199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.278+0000 7fa833e9d640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82c101990 0x7fa82c199010 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa82800d950 tx=0x7fa82800de10 comp rx=0 tx=0).stop 2024-08-20T03:09:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.278+0000 7fa833e9d640 1 -- 172.21.15.44:0/3173012581 shutdown_connections 2024-08-20T03:09:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.278+0000 7fa833e9d640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa8180b3d40 0x7fa8180b6200 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.278+0000 7fa833e9d640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c107d20 0x7fa82c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.278+0000 7fa833e9d640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa82c1073a0 0x7fa82c199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.278+0000 7fa833e9d640 1 --2- 172.21.15.44:0/3173012581 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa82c101990 0x7fa82c199010 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.278+0000 7fa833e9d640 1 -- 172.21.15.44:0/3173012581 >> 172.21.15.44:0/3173012581 conn(0x7fa82c0fd400 msgr2=0x7fa82c1064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.278+0000 7fa833e9d640 1 -- 172.21.15.44:0/3173012581 shutdown_connections 2024-08-20T03:09:54.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:54.278+0000 7fa833e9d640 1 -- 172.21.15.44:0/3173012581 wait complete. 2024-08-20T03:09:54.497 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55cb7fe640 1 -- 172.21.15.44:0/3887632514 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55890fa2bd50 con 0x55890fb3fa90 2024-08-20T03:09:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55cb7fe640 10 client.16986 handle_command_reply: tid=0 2024-08-20T03:09:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55f5607740 1 client.16986 shutdown 2024-08-20T03:09:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55f5607740 1 -- 172.21.15.44:0/3887632514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab8bf0 msgr2=0x55890fac5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55f5607740 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab8bf0 0x55890fac5ab0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009900 tx=0x7f55dc02a410 comp rx=0 tx=0).stop 2024-08-20T03:09:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55f5607740 1 -- 172.21.15.44:0/3887632514 shutdown_connections 2024-08-20T03:09:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55f5607740 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55890fb3fa90 0x55890fb41e80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55f5607740 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55890fac54a0 0x55890facc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55f5607740 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55890fab9510 0x55890fac5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55f5607740 1 --2- 172.21.15.44:0/3887632514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55890fab8bf0 0x55890fac5ab0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55f5607740 1 -- 172.21.15.44:0/3887632514 >> 172.21.15.44:0/3887632514 conn(0x55890fab6a20 msgr2=0x55890fac40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:54.498 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55cb7fe640 0 client.16986 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:09:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55f5607740 1 -- 172.21.15.44:0/3887632514 shutdown_connections 2024-08-20T03:09:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55f5607740 1 -- 172.21.15.44:0/3887632514 wait complete. 2024-08-20T03:09:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.494+0000 7f55f5607740 20 client.16986 trim_cache size 0 max 16384 2024-08-20T03:09:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:09:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:09:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:09:54.499 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:09:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.495+0000 7f55f273c640 1 -- 172.21.15.44:0/230259650 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55d80b3d40 msgr2=0x7f55d80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.495+0000 7f55f273c640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55d80b3d40 0x7f55d80b6200 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f55e4005c40 tx=0x7f55e4005bd0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.495+0000 7f55f273c640 1 -- 172.21.15.44:0/230259650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec106a40 msgr2=0x7f55ec10d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.495+0000 7f55f273c640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec106a40 0x7f55ec10d490 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f55d400e9f0 tx=0x7f55d400eeb0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.495+0000 7f55f273c640 1 -- 172.21.15.44:0/230259650 shutdown_connections 2024-08-20T03:09:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.495+0000 7f55f273c640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f55d80b3d40 0x7f55d80b6200 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.495+0000 7f55f273c640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55ec1073c0 0x7f55ec1a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.495+0000 7f55f273c640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55ec106a40 0x7f55ec10d490 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.495+0000 7f55f273c640 1 --2- 172.21.15.44:0/230259650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ec1014f0 0x7f55ec10cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.496+0000 7f55f273c640 1 -- 172.21.15.44:0/230259650 >> 172.21.15.44:0/230259650 conn(0x7f55ec076b10 msgr2=0x7f55ec102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.496+0000 7f55f273c640 1 -- 172.21.15.44:0/230259650 shutdown_connections 2024-08-20T03:09:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:09:54.496+0000 7f55f273c640 1 -- 172.21.15.44:0/230259650 wait complete. 2024-08-20T03:09:54.939 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-20T03:09:55.216 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:55.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.805+0000 7f0de678c640 1 -- 172.21.15.44:0/716252314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de01084c0 msgr2=0x7f0de0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:55.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.805+0000 7f0de678c640 1 --2- 172.21.15.44:0/716252314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de01084c0 0x7f0de0108900 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009900 tx=0x7f0dd402a410 comp rx=0 tx=0).stop 2024-08-20T03:09:55.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.805+0000 7f0de678c640 1 -- 172.21.15.44:0/716252314 shutdown_connections 2024-08-20T03:09:55.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.805+0000 7f0de678c640 1 --2- 172.21.15.44:0/716252314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0de0108e40 0x7f0de010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.805+0000 7f0de678c640 1 --2- 172.21.15.44:0/716252314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de01084c0 0x7f0de0108900 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.805+0000 7f0de678c640 1 --2- 172.21.15.44:0/716252314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0de0102ab0 0x7f0de0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.805+0000 7f0de678c640 1 -- 172.21.15.44:0/716252314 >> 172.21.15.44:0/716252314 conn(0x7f0de00fe500 msgr2=0x7f0de0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:55.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.805+0000 7f0de678c640 1 -- 172.21.15.44:0/716252314 shutdown_connections 2024-08-20T03:09:55.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.805+0000 7f0de678c640 1 -- 172.21.15.44:0/716252314 wait complete. 2024-08-20T03:09:55.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.806+0000 7f0de678c640 1 Processor -- start 2024-08-20T03:09:55.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.806+0000 7f0de678c640 1 -- start start 2024-08-20T03:09:55.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0de678c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0de0102ab0 0x7f0de0198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0de678c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0de01084c0 0x7f0de01994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0de678c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de0108e40 0x7f0de019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0de678c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0de0112b10 con 0x7f0de0108e40 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0de678c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0de0112990 con 0x7f0de01084c0 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0de678c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0de0112c90 con 0x7f0de0102ab0 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0de0102ab0 0x7f0de0198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0de0102ab0 0x7f0de0198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41864/0 (socket says 172.21.15.44:41864) 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0de4d29640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de0108e40 0x7f0de019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddffff640 1 -- 172.21.15.44:0/3634530625 learned_addr learned my addr 172.21.15.44:0/3634530625 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddf7fe640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0de01084c0 0x7f0de01994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddffff640 1 -- 172.21.15.44:0/3634530625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0de01084c0 msgr2=0x7f0de01994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddffff640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0de01084c0 0x7f0de01994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddffff640 1 -- 172.21.15.44:0/3634530625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de0108e40 msgr2=0x7f0de019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddffff640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de0108e40 0x7f0de019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddffff640 1 -- 172.21.15.44:0/3634530625 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de019f010 con 0x7f0de0102ab0 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddffff640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0de0102ab0 0x7f0de0198f60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000e9f0 tx=0x7f0dd000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddd7fa640 1 -- 172.21.15.44:0/3634530625 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd000cd80 con 0x7f0de0102ab0 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0de678c640 1 -- 172.21.15.44:0/3634530625 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de019f300 con 0x7f0de0102ab0 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddd7fa640 1 -- 172.21.15.44:0/3634530625 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd00108e0 con 0x7f0de0102ab0 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.807+0000 7f0ddd7fa640 1 -- 172.21.15.44:0/3634530625 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0018980 con 0x7f0de0102ab0 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.808+0000 7f0de678c640 1 -- 172.21.15.44:0/3634530625 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de01a5b90 con 0x7f0de0102ab0 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0de8b85740 1 Processor -- start 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0de8b85740 1 -- start start 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0ddd7fa640 1 -- 172.21.15.44:0/3634530625 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0dd0010430 con 0x7f0de0102ab0 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0de8b85740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579c3730 0x5623579c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0de8b85740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c4050 0x5623579d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0de8b85740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579d43f0 0x5623579d67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0de8b85740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56235793a930 con 0x5623579c3730 2024-08-20T03:09:55.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0de8b85740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562357929e30 con 0x5623579d43f0 2024-08-20T03:09:55.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0de8b85740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56235792de70 con 0x5623579c4050 2024-08-20T03:09:55.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0de4d29640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579c3730 0x5623579c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0de4d29640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579c3730 0x5623579c3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42858/0 (socket says 172.21.15.44:42858) 2024-08-20T03:09:55.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0ddffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c4050 0x5623579d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0de4d29640 1 -- 172.21.15.44:0/720910446 learned_addr learned my addr 172.21.15.44:0/720910446 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:55.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.810+0000 7f0ddf7fe640 1 --2- 172.21.15.44:0/720910446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579d43f0 0x5623579d67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.811+0000 7f0de4d29640 1 -- 172.21.15.44:0/720910446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c4050 msgr2=0x5623579d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:55.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.811+0000 7f0de4d29640 1 --2- 172.21.15.44:0/720910446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c4050 0x5623579d3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.811+0000 7f0de4d29640 1 -- 172.21.15.44:0/720910446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579d43f0 msgr2=0x5623579d67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:55.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.811+0000 7f0de4d29640 1 --2- 172.21.15.44:0/720910446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579d43f0 0x5623579d67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.811+0000 7f0de4d29640 1 -- 172.21.15.44:0/720910446 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562357689650 con 0x5623579c3730 2024-08-20T03:09:55.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.811+0000 7f0de4d29640 1 --2- 172.21.15.44:0/720910446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579c3730 0x5623579c3b10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8011440 tx=0x7f0dd8011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1cfe6bea9425f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:55.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.811+0000 7f0dc67fc640 1 -- 172.21.15.44:0/720910446 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8004d20 con 0x5623579c3730 2024-08-20T03:09:55.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.811+0000 7f0dc67fc640 1 -- 172.21.15.44:0/720910446 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd8004050 con 0x5623579c3730 2024-08-20T03:09:55.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.811+0000 7f0dc67fc640 1 -- 172.21.15.44:0/720910446 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd800f7a0 con 0x5623579c3730 2024-08-20T03:09:55.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.811+0000 7f0ddd7fa640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0dc80b3d40 0x7f0dc80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.811+0000 7f0ddffff640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0dc80b3d40 0x7f0dc80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.811+0000 7f0ddd7fa640 1 -- 172.21.15.44:0/3634530625 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f0dd00e4330 con 0x7f0de0102ab0 2024-08-20T03:09:55.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.812+0000 7f0de8b85740 1 -- 172.21.15.44:0/720910446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579c3730 msgr2=0x5623579c3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:55.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.812+0000 7f0de8b85740 1 --2- 172.21.15.44:0/720910446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579c3730 0x5623579c3b10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8011440 tx=0x7f0dd8011900 comp rx=0 tx=0).stop 2024-08-20T03:09:55.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.812+0000 7f0ddffff640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0dc80b3d40 0x7f0dc80b6200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x5623579c68b0 tx=0x7f0dd00e7de0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.812+0000 7f0de8b85740 1 -- 172.21.15.44:0/720910446 shutdown_connections 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.812+0000 7f0de8b85740 1 --2- 172.21.15.44:0/720910446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579d43f0 0x5623579d67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.812+0000 7f0de8b85740 1 --2- 172.21.15.44:0/720910446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c4050 0x5623579d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.812+0000 7f0de8b85740 1 --2- 172.21.15.44:0/720910446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579c3730 0x5623579c3b10 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.812+0000 7f0de8b85740 1 -- 172.21.15.44:0/720910446 >> 172.21.15.44:0/720910446 conn(0x5623579c1560 msgr2=0x5623579c2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.812+0000 7f0de8b85740 1 -- 172.21.15.44:0/720910446 shutdown_connections 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.812+0000 7f0de8b85740 1 -- 172.21.15.44:0/720910446 wait complete. 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.812+0000 7f0de8b85740 1 -- start start 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.812+0000 7f0de8b85740 1 Processor -- start 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de8b85740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c3730 0x5623579d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de8b85740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c4050 0x5623579d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de8b85740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.831 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de8b85740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56235778c700 con 0x5623579d43f0 2024-08-20T03:09:55.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de8b85740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5623578b0810 con 0x5623579c4050 2024-08-20T03:09:55.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de8b85740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5623579d70a0 con 0x5623579c3730 2024-08-20T03:09:55.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de8b85740 10 client.0 _ll_register_callbacks cb 0x562357923700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:55.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de4d29640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0ddf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c4050 0x5623579d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de4d29640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579d1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42872/0 (socket says 172.21.15.44:42872) 2024-08-20T03:09:55.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de4d29640 1 -- 172.21.15.44:0/2673876126 learned_addr learned my addr 172.21.15.44:0/2673876126 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:55.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de4d29640 1 --2- 172.21.15.44:0/2673876126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c3730 0x5623579d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de4d29640 1 -- 172.21.15.44:0/2673876126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c3730 msgr2=0x5623579d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de4d29640 1 --2- 172.21.15.44:0/2673876126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c3730 0x5623579d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de4d29640 1 -- 172.21.15.44:0/2673876126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c4050 msgr2=0x5623579d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de4d29640 1 --2- 172.21.15.44:0/2673876126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c4050 0x5623579d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de4d29640 1 -- 172.21.15.44:0/2673876126 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623579e04d0 con 0x5623579d43f0 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de4d29640 1 --2- 172.21.15.44:0/2673876126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579d1e70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009270 tx=0x7f0dd8003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b144cc7374fa03 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0dbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0dbf7fe640 1 -- 172.21.15.44:0/2673876126 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd801d070 con 0x5623579d43f0 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0dbf7fe640 1 -- 172.21.15.44:0/2673876126 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f0dd8004460 con 0x5623579d43f0 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.813+0000 7f0de8b85740 1 -- 172.21.15.44:0/2673876126 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5623578d9150 con 0x5623579d43f0 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.814+0000 7f0dbf7fe640 1 -- 172.21.15.44:0/2673876126 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0dd8036050 con 0x5623579d43f0 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.814+0000 7f0de8b85740 10 client.17016 fetch_fsmap learned FSMap version 41 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.814+0000 7f0de8b85740 1 -- 172.21.15.44:0/2673876126 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5623579e2e60 con 0x5623579d43f0 2024-08-20T03:09:55.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.814+0000 7f0dbf7fe640 1 -- 172.21.15.44:0/2673876126 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5623578d9150 con 0x5623579d43f0 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.814+0000 7f0dbf7fe640 1 -- 172.21.15.44:0/2673876126 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f0dd8019430 con 0x5623579d43f0 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.814+0000 7f0de8b85740 10 client.17016 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.814+0000 7f0de8b85740 10 client.17016 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.814+0000 7f0de8b85740 20 client.17016 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.814+0000 7f0de8b85740 1 --2- 172.21.15.44:0/2673876126 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5623579e41b0 0x5623579e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.814+0000 7f0de8b85740 4 client.17016 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.814+0000 7f0de8b85740 1 -- 172.21.15.44:0/2673876126 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56235793c880 con 0x5623579e41b0 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.814+0000 7f0ddf7fe640 1 --2- 172.21.15.44:0/2673876126 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5623579e41b0 0x5623579e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.815+0000 7f0ddf7fe640 1 --2- 172.21.15.44:0/2673876126 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5623579e41b0 0x5623579e65a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd7e024f619ebcc1 server_cookie=63cd12cac9e6bca6 in_seq=0 out_seq=0 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.815+0000 7f0dbf7fe640 10 client.17016 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.815+0000 7f0dbf7fe640 1 -- 172.21.15.44:0/2673876126 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56235793c880 con 0x5623579e41b0 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.815+0000 7f0dbf7fe640 10 client.17016 handle_command_reply: tid=0 2024-08-20T03:09:55.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.815+0000 7f0de8b85740 1 client.17016 shutdown 2024-08-20T03:09:55.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.815+0000 7f0de8b85740 1 -- 172.21.15.44:0/2673876126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 msgr2=0x5623579d1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:55.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.815+0000 7f0de8b85740 1 --2- 172.21.15.44:0/2673876126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579d1e70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009270 tx=0x7f0dd8003f50 comp rx=0 tx=0).stop 2024-08-20T03:09:55.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.816+0000 7f0de8b85740 1 -- 172.21.15.44:0/2673876126 shutdown_connections 2024-08-20T03:09:55.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.816+0000 7f0de8b85740 1 --2- 172.21.15.44:0/2673876126 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5623579e41b0 0x5623579e65a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.816+0000 7f0de8b85740 1 --2- 172.21.15.44:0/2673876126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579d1e70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.816+0000 7f0de8b85740 1 --2- 172.21.15.44:0/2673876126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c4050 0x5623579d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.816+0000 7f0de8b85740 1 --2- 172.21.15.44:0/2673876126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c3730 0x5623579d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.816+0000 7f0de8b85740 1 -- 172.21.15.44:0/2673876126 >> 172.21.15.44:0/2673876126 conn(0x5623579c1560 msgr2=0x5623579c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:55.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.816+0000 7f0dbf7fe640 0 client.17016 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:09:55.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.816+0000 7f0de8b85740 1 -- 172.21.15.44:0/2673876126 shutdown_connections 2024-08-20T03:09:55.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.816+0000 7f0de8b85740 1 -- 172.21.15.44:0/2673876126 wait complete. 2024-08-20T03:09:55.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.816+0000 7f0de8b85740 20 client.17016 trim_cache size 0 max 16384 2024-08-20T03:09:55.840 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0de8b85740 1 Processor -- start 2024-08-20T03:09:55.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0de8b85740 1 -- start start 2024-08-20T03:09:55.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0de8b85740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c3730 0x5623579cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0de8b85740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c4050 0x5623579c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0de8b85740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0de8b85740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0de01ac0d0 con 0x5623579d43f0 2024-08-20T03:09:55.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0de8b85740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0de01add90 con 0x5623579c3730 2024-08-20T03:09:55.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0de8b85740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0de01ac830 con 0x5623579c4050 2024-08-20T03:09:55.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0de4d29640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0de4d29640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42884/0 (socket says 172.21.15.44:42884) 2024-08-20T03:09:55.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0de4d29640 1 -- 172.21.15.44:0/3449185387 learned_addr learned my addr 172.21.15.44:0/3449185387 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:55.841 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0ddf7fe640 1 --2- 172.21.15.44:0/3449185387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c4050 0x5623579c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.829+0000 7f0de4d29640 1 --2- 172.21.15.44:0/3449185387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c3730 0x5623579cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.830+0000 7f0de4d29640 1 -- 172.21.15.44:0/3449185387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c4050 msgr2=0x5623579c0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.830+0000 7f0de4d29640 1 --2- 172.21.15.44:0/3449185387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c4050 0x5623579c0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.830+0000 7f0de4d29640 1 -- 172.21.15.44:0/3449185387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c3730 msgr2=0x5623579cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.830+0000 7f0de4d29640 1 --2- 172.21.15.44:0/3449185387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c3730 0x5623579cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.830+0000 7f0de4d29640 1 -- 172.21.15.44:0/3449185387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623579e6b30 con 0x5623579d43f0 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.830+0000 7f0de4d29640 1 --2- 172.21.15.44:0/3449185387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579d0920 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8005d70 tx=0x7f0dd8011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fec310a176d4b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.830+0000 7f0dc5ffb640 1 -- 172.21.15.44:0/3449185387 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd801d070 con 0x5623579d43f0 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.830+0000 7f0dc5ffb640 1 -- 172.21.15.44:0/3449185387 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd802d070 con 0x5623579d43f0 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.830+0000 7f0dc5ffb640 1 -- 172.21.15.44:0/3449185387 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8019a70 con 0x5623579d43f0 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.831+0000 7f0de8b85740 1 -- 172.21.15.44:0/3449185387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 msgr2=0x5623579d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.831+0000 7f0de8b85740 1 --2- 172.21.15.44:0/3449185387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579d0920 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8005d70 tx=0x7f0dd8011900 comp rx=0 tx=0).stop 2024-08-20T03:09:55.842 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.831+0000 7f0de8b85740 1 -- 172.21.15.44:0/3449185387 shutdown_connections 2024-08-20T03:09:55.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.831+0000 7f0de8b85740 1 --2- 172.21.15.44:0/3449185387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579d0920 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.831+0000 7f0de8b85740 1 --2- 172.21.15.44:0/3449185387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c4050 0x5623579c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.831+0000 7f0de8b85740 1 --2- 172.21.15.44:0/3449185387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c3730 0x5623579cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.831+0000 7f0de8b85740 1 -- 172.21.15.44:0/3449185387 >> 172.21.15.44:0/3449185387 conn(0x5623579c1560 msgr2=0x5623579d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:55.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.831+0000 7f0de8b85740 1 -- 172.21.15.44:0/3449185387 shutdown_connections 2024-08-20T03:09:55.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.831+0000 7f0de8b85740 1 -- 172.21.15.44:0/3449185387 wait complete. 2024-08-20T03:09:55.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.831+0000 7f0de8b85740 1 -- start start 2024-08-20T03:09:55.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.831+0000 7f0de8b85740 1 Processor -- start 2024-08-20T03:09:55.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0de8b85740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c3730 0x5623579dc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0de8b85740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c4050 0x5623579dc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0de8b85740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.843 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0de8b85740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0de01ad6c0 con 0x5623579d43f0 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0de8b85740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0de01acf90 con 0x5623579c4050 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0de8b85740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0de01aa0b0 con 0x5623579c3730 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0de8b85740 10 client.0 _ll_register_callbacks cb 0x7f0dd80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0de4d29640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0ddf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c4050 0x5623579dc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0de4d29640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c3730 0x5623579dc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0ddf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c4050 0x5623579dc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:44824/0 (socket says 172.21.15.44:44824) 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0de4d29640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c3730 0x5623579dc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:41900/0 (socket says 172.21.15.44:41900) 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0ddf7fe640 1 -- 172.21.15.44:0/832621086 learned_addr learned my addr 172.21.15.44:0/832621086 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0ddf7fe640 1 -- 172.21.15.44:0/832621086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c3730 msgr2=0x5623579dc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0ddf7fe640 1 --2- 172.21.15.44:0/832621086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c3730 0x5623579dc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0ddf7fe640 1 -- 172.21.15.44:0/832621086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 msgr2=0x5623579c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:55.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0ddf7fe640 1 --2- 172.21.15.44:0/832621086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0ddf7fe640 1 -- 172.21.15.44:0/832621086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623579dcf10 con 0x5623579c4050 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0de4d29640 1 --2- 172.21.15.44:0/832621086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c3730 0x5623579dc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0ddf7fe640 1 --2- 172.21.15.44:0/832621086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c4050 0x5623579dc870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4005ab0 tx=0x7f0dd4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c511b2a505b997 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.832+0000 7f0dbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0dbf7fe640 1 -- 172.21.15.44:0/832621086 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4035040 con 0x5623579c4050 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0de8b85740 1 -- 172.21.15.44:0/832621086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dd802d070 con 0x5623579c4050 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0dbf7fe640 1 -- 172.21.15.44:0/832621086 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f0dd4005510 con 0x5623579c4050 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0dbf7fe640 1 -- 172.21.15.44:0/832621086 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0dd4004770 con 0x5623579c4050 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0de8b85740 10 client.26441 fetch_fsmap learned FSMap version 41 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0de8b85740 1 -- 172.21.15.44:0/832621086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5623579eb4a0 con 0x5623579c4050 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0dbf7fe640 1 -- 172.21.15.44:0/832621086 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dd802d070 con 0x5623579c4050 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0dbf7fe640 1 -- 172.21.15.44:0/832621086 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f0dd4003a00 con 0x5623579c4050 2024-08-20T03:09:55.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0de8b85740 10 client.26441 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:09:55.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0de8b85740 10 client.26441 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:09:55.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0de8b85740 20 client.26441 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:55.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0de8b85740 1 --2- 172.21.15.44:0/832621086 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x562357a4ed30 0x562357a51120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:55.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0de8b85740 4 client.26441 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:09:55.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0de8b85740 1 -- 172.21.15.44:0/832621086 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562357938320 con 0x562357a4ed30 2024-08-20T03:09:55.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.833+0000 7f0de4d29640 1 --2- 172.21.15.44:0/832621086 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x562357a4ed30 0x562357a51120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:55.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de4d29640 1 --2- 172.21.15.44:0/832621086 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x562357a4ed30 0x562357a51120 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d0f8b5d02cf164f7 server_cookie=c8fccca675cfba48 in_seq=0 out_seq=0 2024-08-20T03:09:55.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0dbf7fe640 10 client.26441 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:09:55.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0dbf7fe640 1 -- 172.21.15.44:0/832621086 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562357938320 con 0x562357a4ed30 2024-08-20T03:09:55.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0dbf7fe640 10 client.26441 handle_command_reply: tid=0 2024-08-20T03:09:55.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de8b85740 1 client.26441 shutdown 2024-08-20T03:09:55.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de8b85740 1 -- 172.21.15.44:0/832621086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c4050 msgr2=0x5623579dc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:55.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de8b85740 1 --2- 172.21.15.44:0/832621086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c4050 0x5623579dc870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4005ab0 tx=0x7f0dd4005c00 comp rx=0 tx=0).stop 2024-08-20T03:09:55.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de8b85740 1 -- 172.21.15.44:0/832621086 shutdown_connections 2024-08-20T03:09:55.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de8b85740 1 --2- 172.21.15.44:0/832621086 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x562357a4ed30 0x562357a51120 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de8b85740 1 --2- 172.21.15.44:0/832621086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5623579d43f0 0x5623579c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de8b85740 1 --2- 172.21.15.44:0/832621086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5623579c4050 0x5623579dc870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0dbf7fe640 0 client.26441 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:09:55.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de8b85740 1 --2- 172.21.15.44:0/832621086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5623579c3730 0x5623579dc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de8b85740 1 -- 172.21.15.44:0/832621086 >> 172.21.15.44:0/832621086 conn(0x5623579c1560 msgr2=0x5623579d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:55.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de8b85740 1 -- 172.21.15.44:0/832621086 shutdown_connections 2024-08-20T03:09:55.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de8b85740 1 -- 172.21.15.44:0/832621086 wait complete. 2024-08-20T03:09:55.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.834+0000 7f0de8b85740 20 client.26441 trim_cache size 0 max 16384 2024-08-20T03:09:55.848 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:09:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.835+0000 7f0de678c640 1 -- 172.21.15.44:0/3634530625 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0dc80b3d40 msgr2=0x7f0dc80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.835+0000 7f0de678c640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0dc80b3d40 0x7f0dc80b6200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x5623579c68b0 tx=0x7f0dd00e7de0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.835+0000 7f0de678c640 1 -- 172.21.15.44:0/3634530625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0de0102ab0 msgr2=0x7f0de0198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.835+0000 7f0de678c640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0de0102ab0 0x7f0de0198f60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000e9f0 tx=0x7f0dd000eeb0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.835+0000 7f0de678c640 1 -- 172.21.15.44:0/3634530625 shutdown_connections 2024-08-20T03:09:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.835+0000 7f0de678c640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0dc80b3d40 0x7f0dc80b6200 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.835+0000 7f0de678c640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0de0108e40 0x7f0de019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.835+0000 7f0de678c640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0de01084c0 0x7f0de01994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.835+0000 7f0de678c640 1 --2- 172.21.15.44:0/3634530625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0de0102ab0 0x7f0de0198f60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:55.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.835+0000 7f0de678c640 1 -- 172.21.15.44:0/3634530625 >> 172.21.15.44:0/3634530625 conn(0x7f0de00fe500 msgr2=0x7f0de01008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:55.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.835+0000 7f0de678c640 1 -- 172.21.15.44:0/3634530625 shutdown_connections 2024-08-20T03:09:55.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:55.835+0000 7f0de678c640 1 -- 172.21.15.44:0/3634530625 wait complete. 2024-08-20T03:09:55.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:55 smithi165 ceph-mon[22357]: pgmap v545: 97 pgs: 97 active+clean; 900 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 383 op/s 2024-08-20T03:09:55.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:55 smithi099 ceph-mon[21984]: pgmap v545: 97 pgs: 97 active+clean; 900 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 383 op/s 2024-08-20T03:09:56.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:55 smithi044 ceph-mon[18790]: pgmap v545: 97 pgs: 97 active+clean; 900 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.6 MiB/s wr, 383 op/s 2024-08-20T03:09:56.533 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T03:09:56.789 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:57.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.325+0000 7effdf0e7640 1 -- 172.21.15.44:0/2910217666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd81084c0 msgr2=0x7effd8108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:57.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.325+0000 7effdf0e7640 1 --2- 172.21.15.44:0/2910217666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd81084c0 0x7effd8108900 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7effc4009900 tx=0x7effc402a410 comp rx=0 tx=0).stop 2024-08-20T03:09:57.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.325+0000 7effdf0e7640 1 -- 172.21.15.44:0/2910217666 shutdown_connections 2024-08-20T03:09:57.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.325+0000 7effdf0e7640 1 --2- 172.21.15.44:0/2910217666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd8108e40 0x7effd810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.325+0000 7effdf0e7640 1 --2- 172.21.15.44:0/2910217666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd81084c0 0x7effd8108900 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.325+0000 7effdf0e7640 1 --2- 172.21.15.44:0/2910217666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effd8102ab0 0x7effd8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.325+0000 7effdf0e7640 1 -- 172.21.15.44:0/2910217666 >> 172.21.15.44:0/2910217666 conn(0x7effd80fe500 msgr2=0x7effd8100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:57.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.325+0000 7effdf0e7640 1 -- 172.21.15.44:0/2910217666 shutdown_connections 2024-08-20T03:09:57.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.325+0000 7effdf0e7640 1 -- 172.21.15.44:0/2910217666 wait complete. 2024-08-20T03:09:57.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdf0e7640 1 Processor -- start 2024-08-20T03:09:57.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdf0e7640 1 -- start start 2024-08-20T03:09:57.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdf0e7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd8102ab0 0x7effd8198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdf0e7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd81084c0 0x7effd81994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdf0e7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effd8108e40 0x7effd819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdf0e7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7effd8112a90 con 0x7effd8102ab0 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdf0e7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7effd8112910 con 0x7effd81084c0 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdf0e7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7effd8112c10 con 0x7effd8108e40 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdce83640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd8102ab0 0x7effd8198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdce83640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd8102ab0 0x7effd8198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42922/0 (socket says 172.21.15.44:42922) 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdce83640 1 -- 172.21.15.44:0/280815747 learned_addr learned my addr 172.21.15.44:0/280815747 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effd7fff640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd81084c0 0x7effd81994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdd684640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effd8108e40 0x7effd819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdce83640 1 -- 172.21.15.44:0/280815747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effd8108e40 msgr2=0x7effd819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdce83640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effd8108e40 0x7effd819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdce83640 1 -- 172.21.15.44:0/280815747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd81084c0 msgr2=0x7effd81994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdce83640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd81084c0 0x7effd81994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.326+0000 7effdce83640 1 -- 172.21.15.44:0/280815747 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd819f020 con 0x7effd8102ab0 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.327+0000 7effdce83640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd8102ab0 0x7effd8198f70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7effcc00d950 tx=0x7effcc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.327+0000 7effd5ffb640 1 -- 172.21.15.44:0/280815747 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc017030 con 0x7effd8102ab0 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.327+0000 7effd5ffb640 1 -- 172.21.15.44:0/280815747 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc004020 con 0x7effd8102ab0 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.327+0000 7effdf0e7640 1 -- 172.21.15.44:0/280815747 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd819f310 con 0x7effd8102ab0 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.327+0000 7effd5ffb640 1 -- 172.21.15.44:0/280815747 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0047a0 con 0x7effd8102ab0 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.327+0000 7effdf0e7640 1 -- 172.21.15.44:0/280815747 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd81a59d0 con 0x7effd8102ab0 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effe14e0740 1 Processor -- start 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effd5ffb640 1 -- 172.21.15.44:0/280815747 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7effcc009a30 con 0x7effd8102ab0 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effe14e0740 1 -- start start 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effe14e0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 0x558942b11c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effe14e0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b121a0 0x558942b1dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effe14e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b1e130 0x558942b20520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effe14e0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558942a88930 con 0x558942b11880 2024-08-20T03:09:57.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effe14e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558942a77e30 con 0x558942b1e130 2024-08-20T03:09:57.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effe14e0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558942a7be70 con 0x558942b121a0 2024-08-20T03:09:57.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effdd684640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 0x558942b11c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effdd684640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 0x558942b11c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42934/0 (socket says 172.21.15.44:42934) 2024-08-20T03:09:57.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effdd684640 1 -- 172.21.15.44:0/4143570259 learned_addr learned my addr 172.21.15.44:0/4143570259 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:57.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effdce83640 1 --2- 172.21.15.44:0/4143570259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b121a0 0x558942b1dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.330+0000 7effd7fff640 1 --2- 172.21.15.44:0/4143570259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b1e130 0x558942b20520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.331+0000 7effdd684640 1 -- 172.21.15.44:0/4143570259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b121a0 msgr2=0x558942b1dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:57.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.331+0000 7effdd684640 1 --2- 172.21.15.44:0/4143570259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b121a0 0x558942b1dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.331+0000 7effdd684640 1 -- 172.21.15.44:0/4143570259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b1e130 msgr2=0x558942b20520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:57.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.331+0000 7effdd684640 1 --2- 172.21.15.44:0/4143570259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b1e130 0x558942b20520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.331+0000 7effdd684640 1 -- 172.21.15.44:0/4143570259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589427d7650 con 0x558942b11880 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.331+0000 7effd5ffb640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7effc80b3d40 0x7effc80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.331+0000 7effdce83640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7effc80b3d40 0x7effc80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.331+0000 7effdd684640 1 --2- 172.21.15.44:0/4143570259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 0x558942b11c60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7effd0009fa0 tx=0x7effd0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a9908067f6930d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.331+0000 7effd5ffb640 1 -- 172.21.15.44:0/280815747 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7effcc0e3a60 con 0x7effd8102ab0 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.331+0000 7effc2ffd640 1 -- 172.21.15.44:0/4143570259 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0016070 con 0x558942b11880 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.331+0000 7effc2ffd640 1 -- 172.21.15.44:0/4143570259 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd0004d20 con 0x558942b11880 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.331+0000 7effc2ffd640 1 -- 172.21.15.44:0/4143570259 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0012850 con 0x558942b11880 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.332+0000 7effe14e0740 1 -- 172.21.15.44:0/4143570259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 msgr2=0x558942b11c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.332+0000 7effdce83640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7effc80b3d40 0x7effc80b6200 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x558942b14a00 tx=0x7effcc0b3040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.332+0000 7effe14e0740 1 --2- 172.21.15.44:0/4143570259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 0x558942b11c60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7effd0009fa0 tx=0x7effd0010800 comp rx=0 tx=0).stop 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.332+0000 7effe14e0740 1 -- 172.21.15.44:0/4143570259 shutdown_connections 2024-08-20T03:09:57.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.332+0000 7effe14e0740 1 --2- 172.21.15.44:0/4143570259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b1e130 0x558942b20520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.332+0000 7effe14e0740 1 --2- 172.21.15.44:0/4143570259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b121a0 0x558942b1dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.332+0000 7effe14e0740 1 --2- 172.21.15.44:0/4143570259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 0x558942b11c60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.332+0000 7effe14e0740 1 -- 172.21.15.44:0/4143570259 >> 172.21.15.44:0/4143570259 conn(0x558942b0f6b0 msgr2=0x558942b10850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.332+0000 7effe14e0740 1 -- 172.21.15.44:0/4143570259 shutdown_connections 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.332+0000 7effe14e0740 1 -- 172.21.15.44:0/4143570259 wait complete. 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.332+0000 7effe14e0740 1 -- start start 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.332+0000 7effe14e0740 1 Processor -- start 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effe14e0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b11880 0x558942b1b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effe14e0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b121a0 0x558942b1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effe14e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b1e130 0x558942b1bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effe14e0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5589428da700 con 0x558942b121a0 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effe14e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5589429fe810 con 0x558942b1e130 2024-08-20T03:09:57.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effe14e0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558942b20de0 con 0x558942b11880 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effe14e0740 10 client.0 _ll_register_callbacks cb 0x558942a71700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effd7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b121a0 0x558942b1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effdd684640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b11880 0x558942b1b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effd7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b121a0 0x558942b1b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42938/0 (socket says 172.21.15.44:42938) 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effd7fff640 1 -- 172.21.15.44:0/2695891810 learned_addr learned my addr 172.21.15.44:0/2695891810 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effdd684640 1 --2- 172.21.15.44:0/2695891810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b1e130 0x558942b1bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effd7fff640 1 -- 172.21.15.44:0/2695891810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b11880 msgr2=0x558942b1b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effd7fff640 1 --2- 172.21.15.44:0/2695891810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b11880 0x558942b1b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effd7fff640 1 -- 172.21.15.44:0/2695891810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b1e130 msgr2=0x558942b1bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effd7fff640 1 --2- 172.21.15.44:0/2695891810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b1e130 0x558942b1bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effd7fff640 1 -- 172.21.15.44:0/2695891810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558942b2a190 con 0x558942b121a0 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effd7fff640 1 --2- 172.21.15.44:0/2695891810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b121a0 0x558942b1b600 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7effc4033000 tx=0x7effc4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506813aa802747dc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:57.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effb7fff640 1 -- 172.21.15.44:0/2695891810 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4035040 con 0x558942b121a0 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.333+0000 7effb7fff640 1 -- 172.21.15.44:0/2695891810 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7effc4004050 con 0x558942b121a0 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effe14e0740 1 -- 172.21.15.44:0/2695891810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558942a27150 con 0x558942b121a0 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effb7fff640 1 -- 172.21.15.44:0/2695891810 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7effc4004250 con 0x558942b121a0 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effe14e0740 10 client.17034 fetch_fsmap learned FSMap version 41 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effe14e0740 1 -- 172.21.15.44:0/2695891810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558942b2cb40 con 0x558942b121a0 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effb7fff640 1 -- 172.21.15.44:0/2695891810 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558942a27150 con 0x558942b121a0 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effb7fff640 1 -- 172.21.15.44:0/2695891810 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7effc40048e0 con 0x558942b121a0 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effe14e0740 10 client.17034 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effe14e0740 10 client.17034 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effe14e0740 20 client.17034 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effe14e0740 1 --2- 172.21.15.44:0/2695891810 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558942b2e120 0x558942b30540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effe14e0740 4 client.17034 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:57.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effe14e0740 1 -- 172.21.15.44:0/2695891810 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558942a8a880 con 0x558942b2e120 2024-08-20T03:09:57.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.334+0000 7effdd684640 1 --2- 172.21.15.44:0/2695891810 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558942b2e120 0x558942b30540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.335+0000 7effdd684640 1 --2- 172.21.15.44:0/2695891810 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558942b2e120 0x558942b30540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12013e0e571dafd0 server_cookie=5601fefce2108af4 in_seq=0 out_seq=0 2024-08-20T03:09:57.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.335+0000 7effb7fff640 10 client.17034 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:09:57.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effb7fff640 1 -- 172.21.15.44:0/2695891810 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558942a8a880 con 0x558942b2e120 2024-08-20T03:09:57.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effb7fff640 10 client.17034 handle_command_reply: tid=0 2024-08-20T03:09:57.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effe14e0740 1 client.17034 shutdown 2024-08-20T03:09:57.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effe14e0740 1 -- 172.21.15.44:0/2695891810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b121a0 msgr2=0x558942b1b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:57.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effe14e0740 1 --2- 172.21.15.44:0/2695891810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b121a0 0x558942b1b600 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7effc4033000 tx=0x7effc4032f20 comp rx=0 tx=0).stop 2024-08-20T03:09:57.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effe14e0740 1 -- 172.21.15.44:0/2695891810 shutdown_connections 2024-08-20T03:09:57.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effe14e0740 1 --2- 172.21.15.44:0/2695891810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b1e130 0x558942b1bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effe14e0740 1 --2- 172.21.15.44:0/2695891810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b121a0 0x558942b1b600 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effe14e0740 1 --2- 172.21.15.44:0/2695891810 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558942b2e120 0x558942b30540 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effe14e0740 1 --2- 172.21.15.44:0/2695891810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b11880 0x558942b1b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effe14e0740 1 -- 172.21.15.44:0/2695891810 >> 172.21.15.44:0/2695891810 conn(0x558942b0f6b0 msgr2=0x558942b15250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:57.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effb7fff640 0 client.17034 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:09:57.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effe14e0740 1 -- 172.21.15.44:0/2695891810 shutdown_connections 2024-08-20T03:09:57.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effe14e0740 1 -- 172.21.15.44:0/2695891810 wait complete. 2024-08-20T03:09:57.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.336+0000 7effe14e0740 20 client.17034 trim_cache size 0 max 16384 2024-08-20T03:09:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effe14e0740 1 Processor -- start 2024-08-20T03:09:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effe14e0740 1 -- start start 2024-08-20T03:09:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effe14e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b11880 0x558942b1a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effe14e0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b121a0 0x558942b15140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effe14e0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b1e130 0x558942b35e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effe14e0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7effd81ae390 con 0x558942b1e130 2024-08-20T03:09:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effe14e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7effd81b0050 con 0x558942b11880 2024-08-20T03:09:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effe14e0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7effd81aeaf0 con 0x558942b121a0 2024-08-20T03:09:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effdd684640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b11880 0x558942b1a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effdd684640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b11880 0x558942b1a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:44884/0 (socket says 172.21.15.44:44884) 2024-08-20T03:09:57.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effdd684640 1 -- 172.21.15.44:0/2404165484 learned_addr learned my addr 172.21.15.44:0/2404165484 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effd7fff640 1 --2- 172.21.15.44:0/2404165484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b121a0 0x558942b15140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effdd684640 1 --2- 172.21.15.44:0/2404165484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b1e130 0x558942b35e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effdd684640 1 -- 172.21.15.44:0/2404165484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b121a0 msgr2=0x558942b15140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effdd684640 1 --2- 172.21.15.44:0/2404165484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b121a0 0x558942b15140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effdd684640 1 -- 172.21.15.44:0/2404165484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b1e130 msgr2=0x558942b35e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effdd684640 1 --2- 172.21.15.44:0/2404165484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b1e130 0x558942b35e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.351+0000 7effdd684640 1 -- 172.21.15.44:0/2404165484 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558942b10ea0 con 0x558942b11880 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.352+0000 7effdd684640 1 --2- 172.21.15.44:0/2404165484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b11880 0x558942b1a820 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7effd0009fd0 tx=0x7effd0012450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f30099fb8d01a4e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.352+0000 7effc27fc640 1 -- 172.21.15.44:0/2404165484 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0016040 con 0x558942b11880 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.352+0000 7effc27fc640 1 -- 172.21.15.44:0/2404165484 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd0013c50 con 0x558942b11880 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.352+0000 7effc27fc640 1 -- 172.21.15.44:0/2404165484 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0023040 con 0x558942b11880 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.353+0000 7effe14e0740 1 -- 172.21.15.44:0/2404165484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b11880 msgr2=0x558942b1a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:57.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.353+0000 7effe14e0740 1 --2- 172.21.15.44:0/2404165484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b11880 0x558942b1a820 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7effd0009fd0 tx=0x7effd0012450 comp rx=0 tx=0).stop 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.353+0000 7effe14e0740 1 -- 172.21.15.44:0/2404165484 shutdown_connections 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.353+0000 7effe14e0740 1 --2- 172.21.15.44:0/2404165484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b1e130 0x558942b35e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.353+0000 7effe14e0740 1 --2- 172.21.15.44:0/2404165484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b121a0 0x558942b15140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.353+0000 7effe14e0740 1 --2- 172.21.15.44:0/2404165484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b11880 0x558942b1a820 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.353+0000 7effe14e0740 1 -- 172.21.15.44:0/2404165484 >> 172.21.15.44:0/2404165484 conn(0x558942b0f6b0 msgr2=0x558942b14a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.353+0000 7effe14e0740 1 -- 172.21.15.44:0/2404165484 shutdown_connections 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.353+0000 7effe14e0740 1 -- 172.21.15.44:0/2404165484 wait complete. 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.353+0000 7effe14e0740 1 -- start start 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.353+0000 7effe14e0740 1 Processor -- start 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effe14e0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 0x558942b21820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effe14e0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b121a0 0x558942b21d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effe14e0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b1e130 0x558942b96dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effe14e0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7effd81af980 con 0x558942b11880 2024-08-20T03:09:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effe14e0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7effd81af250 con 0x558942b121a0 2024-08-20T03:09:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effe14e0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7effd81ac390 con 0x558942b1e130 2024-08-20T03:09:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effe14e0740 10 client.0 _ll_register_callbacks cb 0x7effc4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effdd684640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 0x558942b21820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effd7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b121a0 0x558942b21d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effd7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b121a0 0x558942b21d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:44892/0 (socket says 172.21.15.44:44892) 2024-08-20T03:09:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effdd684640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 0x558942b21820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42954/0 (socket says 172.21.15.44:42954) 2024-08-20T03:09:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effd7fff640 1 -- 172.21.15.44:0/173375236 learned_addr learned my addr 172.21.15.44:0/173375236 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effdd684640 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b1e130 0x558942b96dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effd7fff640 1 -- 172.21.15.44:0/173375236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b1e130 msgr2=0x558942b96dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effd7fff640 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b1e130 0x558942b96dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effd7fff640 1 -- 172.21.15.44:0/173375236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 msgr2=0x558942b21820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effd7fff640 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 0x558942b21820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effd7fff640 1 -- 172.21.15.44:0/173375236 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558942b97470 con 0x558942b121a0 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effdd684640 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 0x558942b21820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effd7fff640 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b121a0 0x558942b21d60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7effc4004cb0 tx=0x7effc4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9da323ad8c6cc78a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effb7fff640 1 -- 172.21.15.44:0/173375236 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4051040 con 0x558942b121a0 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effb7fff640 1 -- 172.21.15.44:0/173375236 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7effc40035f0 con 0x558942b121a0 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.354+0000 7effe14e0740 1 -- 172.21.15.44:0/173375236 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effd0013c50 con 0x558942b121a0 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.355+0000 7effb7fff640 1 -- 172.21.15.44:0/173375236 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7effc4035040 con 0x558942b121a0 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.355+0000 7effe14e0740 10 client.26459 fetch_fsmap learned FSMap version 41 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.355+0000 7effe14e0740 1 -- 172.21.15.44:0/173375236 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558942b987a0 con 0x558942b121a0 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.355+0000 7effb7fff640 1 -- 172.21.15.44:0/173375236 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd0013c50 con 0x558942b121a0 2024-08-20T03:09:57.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.355+0000 7effb7fff640 1 -- 172.21.15.44:0/173375236 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7effc4003b90 con 0x558942b121a0 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.355+0000 7effe14e0740 10 client.26459 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.355+0000 7effe14e0740 10 client.26459 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.355+0000 7effe14e0740 20 client.26459 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.355+0000 7effe14e0740 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558942b99030 0x558942b9b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.355+0000 7effe14e0740 4 client.26459 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.355+0000 7effe14e0740 1 -- 172.21.15.44:0/173375236 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558942a86320 con 0x558942b99030 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.355+0000 7effdd684640 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558942b99030 0x558942b9b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effdd684640 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558942b99030 0x558942b9b450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9be699161136866 server_cookie=684a6462b6e11f31 in_seq=0 out_seq=0 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effb7fff640 10 client.26459 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effb7fff640 1 -- 172.21.15.44:0/173375236 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558942a86320 con 0x558942b99030 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effb7fff640 10 client.26459 handle_command_reply: tid=0 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effe14e0740 1 client.26459 shutdown 2024-08-20T03:09:57.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effe14e0740 1 -- 172.21.15.44:0/173375236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b121a0 msgr2=0x558942b21d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effe14e0740 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b121a0 0x558942b21d60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7effc4004cb0 tx=0x7effc4005fd0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effe14e0740 1 -- 172.21.15.44:0/173375236 shutdown_connections 2024-08-20T03:09:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effe14e0740 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558942b99030 0x558942b9b450 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effe14e0740 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558942b1e130 0x558942b96dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effe14e0740 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558942b121a0 0x558942b21d60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effb7fff640 0 client.26459 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:09:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effe14e0740 1 --2- 172.21.15.44:0/173375236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558942b11880 0x558942b21820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effe14e0740 1 -- 172.21.15.44:0/173375236 >> 172.21.15.44:0/173375236 conn(0x558942b0f6b0 msgr2=0x558942b1cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effe14e0740 1 -- 172.21.15.44:0/173375236 shutdown_connections 2024-08-20T03:09:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effe14e0740 1 -- 172.21.15.44:0/173375236 wait complete. 2024-08-20T03:09:57.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.356+0000 7effe14e0740 20 client.26459 trim_cache size 0 max 16384 2024-08-20T03:09:57.367 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:09:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.357+0000 7effdf0e7640 1 -- 172.21.15.44:0/280815747 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7effc80b3d40 msgr2=0x7effc80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.357+0000 7effdf0e7640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7effc80b3d40 0x7effc80b6200 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x558942b14a00 tx=0x7effcc0b3040 comp rx=0 tx=0).stop 2024-08-20T03:09:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.357+0000 7effdf0e7640 1 -- 172.21.15.44:0/280815747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd8102ab0 msgr2=0x7effd8198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:57.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.357+0000 7effdf0e7640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd8102ab0 0x7effd8198f70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7effcc00d950 tx=0x7effcc00de10 comp rx=0 tx=0).stop 2024-08-20T03:09:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.357+0000 7effdf0e7640 1 -- 172.21.15.44:0/280815747 shutdown_connections 2024-08-20T03:09:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.357+0000 7effdf0e7640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7effc80b3d40 0x7effc80b6200 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.357+0000 7effdf0e7640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7effd8108e40 0x7effd819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.357+0000 7effdf0e7640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd81084c0 0x7effd81994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.357+0000 7effdf0e7640 1 --2- 172.21.15.44:0/280815747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7effd8102ab0 0x7effd8198f70 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.357+0000 7effdf0e7640 1 -- 172.21.15.44:0/280815747 >> 172.21.15.44:0/280815747 conn(0x7effd80fe500 msgr2=0x7effd81008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.357+0000 7effdf0e7640 1 -- 172.21.15.44:0/280815747 shutdown_connections 2024-08-20T03:09:57.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:57.357+0000 7effdf0e7640 1 -- 172.21.15.44:0/280815747 wait complete. 2024-08-20T03:09:57.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:57 smithi099 ceph-mon[21984]: pgmap v546: 97 pgs: 97 active+clean; 885 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 442 op/s 2024-08-20T03:09:58.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:57 smithi044 ceph-mon[18790]: pgmap v546: 97 pgs: 97 active+clean; 885 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 442 op/s 2024-08-20T03:09:58.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:57 smithi165 ceph-mon[22357]: pgmap v546: 97 pgs: 97 active+clean; 885 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 442 op/s 2024-08-20T03:09:58.921 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T03:09:59.224 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:09:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.695+0000 7fc4a4505640 1 -- 172.21.15.44:0/3468067794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc49c1039a0 msgr2=0x7fc49c103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.695+0000 7fc4a4505640 1 --2- 172.21.15.44:0/3468067794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc49c1039a0 0x7fc49c103d80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc498009900 tx=0x7fc49802a410 comp rx=0 tx=0).stop 2024-08-20T03:09:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.695+0000 7fc4a4505640 1 -- 172.21.15.44:0/3468067794 shutdown_connections 2024-08-20T03:09:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.695+0000 7fc4a4505640 1 --2- 172.21.15.44:0/3468067794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc49c10ce00 0x7fc49c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.695+0000 7fc4a4505640 1 --2- 172.21.15.44:0/3468067794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c1042c0 0x7fc49c10c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.695+0000 7fc4a4505640 1 --2- 172.21.15.44:0/3468067794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc49c1039a0 0x7fc49c103d80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.695+0000 7fc4a4505640 1 -- 172.21.15.44:0/3468067794 >> 172.21.15.44:0/3468067794 conn(0x7fc49c0fd400 msgr2=0x7fc49c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.695+0000 7fc4a4505640 1 -- 172.21.15.44:0/3468067794 shutdown_connections 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.695+0000 7fc4a4505640 1 -- 172.21.15.44:0/3468067794 wait complete. 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a4505640 1 Processor -- start 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a4505640 1 -- start start 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a4505640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc49c1039a0 0x7fc49c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a4505640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c1042c0 0x7fc49c1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a4505640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc49c10ce00 0x7fc49c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a4505640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c112a70 con 0x7fc49c10ce00 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a4505640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1128f0 con 0x7fc49c1039a0 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a4505640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c112bf0 con 0x7fc49c1042c0 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a2aa2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc49c10ce00 0x7fc49c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a22a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc49c1039a0 0x7fc49c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a22a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc49c1039a0 0x7fc49c198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:38000/0 (socket says 172.21.15.44:38000) 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a22a1640 1 -- 172.21.15.44:0/2794120217 learned_addr learned my addr 172.21.15.44:0/2794120217 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a2aa2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc49c10ce00 0x7fc49c19e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36034/0 (socket says 172.21.15.44:36034) 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a1aa0640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c1042c0 0x7fc49c1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a22a1640 1 -- 172.21.15.44:0/2794120217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c1042c0 msgr2=0x7fc49c1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a22a1640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c1042c0 0x7fc49c1994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a22a1640 1 -- 172.21.15.44:0/2794120217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc49c10ce00 msgr2=0x7fc49c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a22a1640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc49c10ce00 0x7fc49c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a22a1640 1 -- 172.21.15.44:0/2794120217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c19f050 con 0x7fc49c1039a0 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.696+0000 7fc4a2aa2640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc49c10ce00 0x7fc49c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.697+0000 7fc4a22a1640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc49c1039a0 0x7fc49c198fa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc498005940 tx=0x7fc4980040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.697+0000 7fc4937fe640 1 -- 172.21.15.44:0/2794120217 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49803f030 con 0x7fc49c1039a0 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.697+0000 7fc4a4505640 1 -- 172.21.15.44:0/2794120217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c19f2e0 con 0x7fc49c1039a0 2024-08-20T03:09:59.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.697+0000 7fc4937fe640 1 -- 172.21.15.44:0/2794120217 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4980044c0 con 0x7fc49c1039a0 2024-08-20T03:09:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.697+0000 7fc4937fe640 1 -- 172.21.15.44:0/2794120217 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498035070 con 0x7fc49c1039a0 2024-08-20T03:09:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.697+0000 7fc4a4505640 1 -- 172.21.15.44:0/2794120217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c1a5a00 con 0x7fc49c1039a0 2024-08-20T03:09:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.699+0000 7fc4a68fe740 1 Processor -- start 2024-08-20T03:09:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a68fe740 1 -- start start 2024-08-20T03:09:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a68fe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 0x5649a8cbdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a68fe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbe1a0 0x5649a8cc9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a68fe740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cca130 0x5649a8ccc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a68fe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5649a8c34930 con 0x5649a8cbd880 2024-08-20T03:09:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a68fe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5649a8c23e30 con 0x5649a8cbe1a0 2024-08-20T03:09:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a68fe740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5649a8c27e70 con 0x5649a8cca130 2024-08-20T03:09:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4937fe640 1 -- 172.21.15.44:0/2794120217 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc498042030 con 0x7fc49c1039a0 2024-08-20T03:09:59.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a2aa2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 0x5649a8cbdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a2aa2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 0x5649a8cbdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36040/0 (socket says 172.21.15.44:36040) 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a2aa2640 1 -- 172.21.15.44:0/718056483 learned_addr learned my addr 172.21.15.44:0/718056483 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a1aa0640 1 --2- 172.21.15.44:0/718056483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cca130 0x5649a8ccc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a22a1640 1 --2- 172.21.15.44:0/718056483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbe1a0 0x5649a8cc9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a2aa2640 1 -- 172.21.15.44:0/718056483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cca130 msgr2=0x5649a8ccc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a2aa2640 1 --2- 172.21.15.44:0/718056483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cca130 0x5649a8ccc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a2aa2640 1 -- 172.21.15.44:0/718056483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbe1a0 msgr2=0x5649a8cc9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a2aa2640 1 --2- 172.21.15.44:0/718056483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbe1a0 0x5649a8cc9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a2aa2640 1 -- 172.21.15.44:0/718056483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a8983650 con 0x5649a8cbd880 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.700+0000 7fc4a1aa0640 1 --2- 172.21.15.44:0/718056483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cca130 0x5649a8ccc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.701+0000 7fc4a2aa2640 1 --2- 172.21.15.44:0/718056483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 0x5649a8cbdc60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc494011810 tx=0x7fc494011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec216d695cdc213 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.701+0000 7fc490ff9640 1 -- 172.21.15.44:0/718056483 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494011310 con 0x5649a8cbd880 2024-08-20T03:09:59.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.701+0000 7fc490ff9640 1 -- 172.21.15.44:0/718056483 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc494003e20 con 0x5649a8cbd880 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.701+0000 7fc490ff9640 1 -- 172.21.15.44:0/718056483 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494004490 con 0x5649a8cbd880 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.701+0000 7fc4937fe640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc4880b3d40 0x7fc4880b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.701+0000 7fc4a22a1640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc4880b3d40 0x7fc4880b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.701+0000 7fc4937fe640 1 -- 172.21.15.44:0/2794120217 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fc4981042e0 con 0x7fc49c1039a0 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.701+0000 7fc4a68fe740 1 -- 172.21.15.44:0/718056483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 msgr2=0x5649a8cbdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.701+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/718056483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 0x5649a8cbdc60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc494011810 tx=0x7fc494011cd0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.701+0000 7fc4a22a1640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc4880b3d40 0x7fc4880b6200 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc498101b20 tx=0x7fc49802a410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 -- 172.21.15.44:0/718056483 shutdown_connections 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/718056483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cca130 0x5649a8ccc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/718056483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbe1a0 0x5649a8cc9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/718056483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 0x5649a8cbdc60 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 -- 172.21.15.44:0/718056483 >> 172.21.15.44:0/718056483 conn(0x5649a8cbb6b0 msgr2=0x5649a8cbc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 -- 172.21.15.44:0/718056483 shutdown_connections 2024-08-20T03:09:59.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 -- 172.21.15.44:0/718056483 wait complete. 2024-08-20T03:09:59.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 -- start start 2024-08-20T03:09:59.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 Processor -- start 2024-08-20T03:09:59.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 0x5649a8cc7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cbe1a0 0x5649a8cc7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cca130 0x5649a8cc7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5649a8a86700 con 0x5649a8cbd880 2024-08-20T03:09:59.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5649a8baa810 con 0x5649a8cca130 2024-08-20T03:09:59.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5649a8cccde0 con 0x5649a8cbe1a0 2024-08-20T03:09:59.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a2aa2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 0x5649a8cc7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a2aa2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 0x5649a8cc7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36054/0 (socket says 172.21.15.44:36054) 2024-08-20T03:09:59.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a2aa2640 1 -- 172.21.15.44:0/1364498612 learned_addr learned my addr 172.21.15.44:0/1364498612 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:59.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a68fe740 10 client.0 _ll_register_callbacks cb 0x5649a8c1d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:59.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.702+0000 7fc4a1aa0640 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cbe1a0 0x5649a8cc7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc4a2aa2640 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cca130 0x5649a8cc7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc4a2aa2640 1 -- 172.21.15.44:0/1364498612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cbe1a0 msgr2=0x5649a8cc7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:59.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc4a2aa2640 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cbe1a0 0x5649a8cc7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc4a2aa2640 1 -- 172.21.15.44:0/1364498612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cca130 msgr2=0x5649a8cc7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:59.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc4a2aa2640 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cca130 0x5649a8cc7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc4a2aa2640 1 -- 172.21.15.44:0/1364498612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a8cd6210 con 0x5649a8cbd880 2024-08-20T03:09:59.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc4a2aa2640 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 0x5649a8cc7130 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc494004110 tx=0x7fc494005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47834df8dcdc0348 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:59.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:59.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc485ffb640 1 -- 172.21.15.44:0/1364498612 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494023040 con 0x5649a8cbd880 2024-08-20T03:09:59.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc485ffb640 1 -- 172.21.15.44:0/1364498612 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fc49400fba0 con 0x5649a8cbd880 2024-08-20T03:09:59.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc4a68fe740 1 -- 172.21.15.44:0/1364498612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5649a8bd3150 con 0x5649a8cbd880 2024-08-20T03:09:59.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc485ffb640 1 -- 172.21.15.44:0/1364498612 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc49400fda0 con 0x5649a8cbd880 2024-08-20T03:09:59.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc4a68fe740 10 client.17052 fetch_fsmap learned FSMap version 41 2024-08-20T03:09:59.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.703+0000 7fc4a68fe740 1 -- 172.21.15.44:0/1364498612 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5649a8cd8b80 con 0x5649a8cbd880 2024-08-20T03:09:59.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.704+0000 7fc485ffb640 1 -- 172.21.15.44:0/1364498612 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5649a8bd3150 con 0x5649a8cbd880 2024-08-20T03:09:59.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.704+0000 7fc485ffb640 1 -- 172.21.15.44:0/1364498612 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7fc494018020 con 0x5649a8cbd880 2024-08-20T03:09:59.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.704+0000 7fc4a68fe740 10 client.17052 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:09:59.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.704+0000 7fc4a68fe740 10 client.17052 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:09:59.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.704+0000 7fc4a68fe740 20 client.17052 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:59.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.704+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5649a8cd9ed0 0x5649a8cdc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.704+0000 7fc4a68fe740 4 client.17052 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:09:59.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.704+0000 7fc4a68fe740 1 -- 172.21.15.44:0/1364498612 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649a8c36880 con 0x5649a8cd9ed0 2024-08-20T03:09:59.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.704+0000 7fc4a1aa0640 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5649a8cd9ed0 0x5649a8cdc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.704+0000 7fc4a1aa0640 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5649a8cd9ed0 0x5649a8cdc2c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7fb785cad48558ee server_cookie=fc7bccbc5591791c in_seq=0 out_seq=0 2024-08-20T03:09:59.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.704+0000 7fc485ffb640 10 client.17052 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:09:59.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc485ffb640 1 -- 172.21.15.44:0/1364498612 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649a8c36880 con 0x5649a8cd9ed0 2024-08-20T03:09:59.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc485ffb640 10 client.17052 handle_command_reply: tid=0 2024-08-20T03:09:59.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc4a68fe740 1 client.17052 shutdown 2024-08-20T03:09:59.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc4a68fe740 1 -- 172.21.15.44:0/1364498612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 msgr2=0x5649a8cc7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:59.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 0x5649a8cc7130 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc494004110 tx=0x7fc494005a50 comp rx=0 tx=0).stop 2024-08-20T03:09:59.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc4a68fe740 1 -- 172.21.15.44:0/1364498612 shutdown_connections 2024-08-20T03:09:59.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5649a8cd9ed0 0x5649a8cdc2c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cca130 0x5649a8cc7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cbe1a0 0x5649a8cc7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/1364498612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbd880 0x5649a8cc7130 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc485ffb640 0 client.17052 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:09:59.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc4a68fe740 1 -- 172.21.15.44:0/1364498612 >> 172.21.15.44:0/1364498612 conn(0x5649a8cbb6b0 msgr2=0x5649a8cc1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:59.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc4a68fe740 1 -- 172.21.15.44:0/1364498612 shutdown_connections 2024-08-20T03:09:59.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc4a68fe740 1 -- 172.21.15.44:0/1364498612 wait complete. 2024-08-20T03:09:59.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.705+0000 7fc4a68fe740 20 client.17052 trim_cache size 0 max 16384 2024-08-20T03:09:59.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.719+0000 7fc4a68fe740 1 Processor -- start 2024-08-20T03:09:59.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a68fe740 1 -- start start 2024-08-20T03:09:59.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a68fe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbd880 0x5649a8cc69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a68fe740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cbe1a0 0x5649a8ce1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a68fe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cca130 0x5649a8ce2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a68fe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1ae320 con 0x5649a8cca130 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a68fe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1affe0 con 0x5649a8cbd880 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a68fe740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1aea80 con 0x5649a8cbe1a0 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a1aa0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cbe1a0 0x5649a8ce1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a1aa0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cbe1a0 0x5649a8ce1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:57496/0 (socket says 172.21.15.44:57496) 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a2aa2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cca130 0x5649a8ce2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a1aa0640 1 -- 172.21.15.44:0/141086203 learned_addr learned my addr 172.21.15.44:0/141086203 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a2aa2640 1 --2- 172.21.15.44:0/141086203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbd880 0x5649a8cc69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a1aa0640 1 -- 172.21.15.44:0/141086203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbd880 msgr2=0x5649a8cc69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a1aa0640 1 --2- 172.21.15.44:0/141086203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbd880 0x5649a8cc69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a1aa0640 1 -- 172.21.15.44:0/141086203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cca130 msgr2=0x5649a8ce2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a1aa0640 1 --2- 172.21.15.44:0/141086203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cca130 0x5649a8ce2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a1aa0640 1 -- 172.21.15.44:0/141086203 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649a8cbc130 con 0x5649a8cbe1a0 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.720+0000 7fc4a1aa0640 1 --2- 172.21.15.44:0/141086203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cbe1a0 0x5649a8ce1af0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x5649a8cbd830 tx=0x7fc48c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81983b6e5c701dc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.721+0000 7fc487fff640 1 -- 172.21.15.44:0/141086203 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c005220 con 0x5649a8cbe1a0 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.721+0000 7fc487fff640 1 -- 172.21.15.44:0/141086203 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc48c004c50 con 0x5649a8cbe1a0 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.721+0000 7fc487fff640 1 -- 172.21.15.44:0/141086203 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c01e8c0 con 0x5649a8cbe1a0 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.721+0000 7fc4a68fe740 1 -- 172.21.15.44:0/141086203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cbe1a0 msgr2=0x5649a8ce1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.721+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/141086203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cbe1a0 0x5649a8ce1af0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x5649a8cbd830 tx=0x7fc48c005f90 comp rx=0 tx=0).stop 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.721+0000 7fc4a68fe740 1 -- 172.21.15.44:0/141086203 shutdown_connections 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.721+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/141086203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cca130 0x5649a8ce2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.721+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/141086203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cbe1a0 0x5649a8ce1af0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.721+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/141086203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbd880 0x5649a8cc69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.721+0000 7fc4a68fe740 1 -- 172.21.15.44:0/141086203 >> 172.21.15.44:0/141086203 conn(0x5649a8cbb6b0 msgr2=0x5649a8cc8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.721+0000 7fc4a68fe740 1 -- 172.21.15.44:0/141086203 shutdown_connections 2024-08-20T03:09:59.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.721+0000 7fc4a68fe740 1 -- 172.21.15.44:0/141086203 wait complete. 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a68fe740 1 -- start start 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a68fe740 1 Processor -- start 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a68fe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbd880 0x5649a8cd2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a68fe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbe1a0 0x5649a8cd25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a68fe740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cca130 0x5649a8d3faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a68fe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1af910 con 0x5649a8cbe1a0 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a68fe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1af1e0 con 0x5649a8cbd880 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a68fe740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c1ac320 con 0x5649a8cca130 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a68fe740 10 client.0 _ll_register_callbacks cb 0x7fc48c009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a1aa0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbe1a0 0x5649a8cd25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a1aa0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbe1a0 0x5649a8cd25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36064/0 (socket says 172.21.15.44:36064) 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a1aa0640 1 -- 172.21.15.44:0/2784458127 learned_addr learned my addr 172.21.15.44:0/2784458127 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:09:59.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a2aa2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbd880 0x5649a8cd2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.722+0000 7fc4a2aa2640 1 --2- 172.21.15.44:0/2784458127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cca130 0x5649a8d3faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc4a1aa0640 1 -- 172.21.15.44:0/2784458127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cca130 msgr2=0x5649a8d3faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc4a1aa0640 1 --2- 172.21.15.44:0/2784458127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cca130 0x5649a8d3faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc4a1aa0640 1 -- 172.21.15.44:0/2784458127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbd880 msgr2=0x5649a8cd2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:09:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc4a1aa0640 1 --2- 172.21.15.44:0/2784458127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbd880 0x5649a8cd2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc4a1aa0640 1 -- 172.21.15.44:0/2784458127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649a8cbd260 con 0x5649a8cbe1a0 2024-08-20T03:09:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc4a1aa0640 1 --2- 172.21.15.44:0/2784458127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbe1a0 0x5649a8cd25b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc48c005e00 tx=0x7fc48c009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6ec25a7b7bded2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:09:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:09:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc485ffb640 1 -- 172.21.15.44:0/2784458127 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c005020 con 0x5649a8cbe1a0 2024-08-20T03:09:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc485ffb640 1 -- 172.21.15.44:0/2784458127 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7fc48c0051c0 con 0x5649a8cbe1a0 2024-08-20T03:09:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc4a68fe740 1 -- 172.21.15.44:0/2784458127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc48c004c50 con 0x5649a8cbe1a0 2024-08-20T03:09:59.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc485ffb640 1 -- 172.21.15.44:0/2784458127 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc48c0053c0 con 0x5649a8cbe1a0 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc4a68fe740 10 client.17058 fetch_fsmap learned FSMap version 41 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc4a68fe740 1 -- 172.21.15.44:0/2784458127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5649a8cc4060 con 0x5649a8cbe1a0 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc485ffb640 1 -- 172.21.15.44:0/2784458127 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc48c004c50 con 0x5649a8cbe1a0 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.723+0000 7fc485ffb640 1 -- 172.21.15.44:0/2784458127 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7fc48c025030 con 0x5649a8cbe1a0 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 10 client.17058 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 10 client.17058 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 20 client.17058 populate_metadata read hostname 'smithi044' 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/2784458127 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5649a8d48ed0 0x5649a8d4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 4 client.17058 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 1 -- 172.21.15.44:0/2784458127 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649a8c32320 con 0x5649a8d48ed0 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a2aa2640 1 --2- 172.21.15.44:0/2784458127 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5649a8d48ed0 0x5649a8d4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a2aa2640 1 --2- 172.21.15.44:0/2784458127 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5649a8d48ed0 0x5649a8d4b2c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d973530ededd32b server_cookie=cff80fef8ddf8f8b in_seq=0 out_seq=0 2024-08-20T03:09:59.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc485ffb640 10 client.17058 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc485ffb640 1 -- 172.21.15.44:0/2784458127 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649a8c32320 con 0x5649a8d48ed0 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc485ffb640 10 client.17058 handle_command_reply: tid=0 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 1 client.17058 shutdown 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 1 -- 172.21.15.44:0/2784458127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbe1a0 msgr2=0x5649a8cd25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/2784458127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbe1a0 0x5649a8cd25b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc48c005e00 tx=0x7fc48c009b70 comp rx=0 tx=0).stop 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 1 -- 172.21.15.44:0/2784458127 shutdown_connections 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/2784458127 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5649a8d48ed0 0x5649a8d4b2c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/2784458127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5649a8cca130 0x5649a8d3faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/2784458127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649a8cbe1a0 0x5649a8cd25b0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc485ffb640 0 client.17058 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 1 --2- 172.21.15.44:0/2784458127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5649a8cbd880 0x5649a8cd2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.724+0000 7fc4a68fe740 1 -- 172.21.15.44:0/2784458127 >> 172.21.15.44:0/2784458127 conn(0x5649a8cbb6b0 msgr2=0x5649a8cc7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:59.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a68fe740 1 -- 172.21.15.44:0/2784458127 shutdown_connections 2024-08-20T03:09:59.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a68fe740 1 -- 172.21.15.44:0/2784458127 wait complete. 2024-08-20T03:09:59.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a68fe740 20 client.17058 trim_cache size 0 max 16384 2024-08-20T03:09:59.737 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:09:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a4505640 1 -- 172.21.15.44:0/2794120217 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc4880b3d40 msgr2=0x7fc4880b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a4505640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc4880b3d40 0x7fc4880b6200 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc498101b20 tx=0x7fc49802a410 comp rx=0 tx=0).stop 2024-08-20T03:09:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a4505640 1 -- 172.21.15.44:0/2794120217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc49c1039a0 msgr2=0x7fc49c198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:09:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a4505640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc49c1039a0 0x7fc49c198fa0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc498005940 tx=0x7fc4980040c0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a4505640 1 -- 172.21.15.44:0/2794120217 shutdown_connections 2024-08-20T03:09:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a4505640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc4880b3d40 0x7fc4880b6200 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a4505640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc49c10ce00 0x7fc49c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a4505640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc49c1042c0 0x7fc49c1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a4505640 1 --2- 172.21.15.44:0/2794120217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc49c1039a0 0x7fc49c198fa0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:09:59.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a4505640 1 -- 172.21.15.44:0/2794120217 >> 172.21.15.44:0/2794120217 conn(0x7fc49c0fd400 msgr2=0x7fc49c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:09:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a4505640 1 -- 172.21.15.44:0/2794120217 shutdown_connections 2024-08-20T03:09:59.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:09:59.725+0000 7fc4a4505640 1 -- 172.21.15.44:0/2794120217 wait complete. 2024-08-20T03:09:59.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:09:59 smithi099 ceph-mon[21984]: pgmap v547: 97 pgs: 97 active+clean; 868 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 603 op/s 2024-08-20T03:10:00.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:09:59 smithi044 ceph-mon[18790]: pgmap v547: 97 pgs: 97 active+clean; 868 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 603 op/s 2024-08-20T03:10:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:09:59 smithi165 ceph-mon[22357]: pgmap v547: 97 pgs: 97 active+clean; 868 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 603 op/s 2024-08-20T03:10:00.564 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T03:10:00.819 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:10:00.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:00 smithi099 ceph-mon[21984]: overall HEALTH_OK 2024-08-20T03:10:00.940 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:00 smithi044 ceph-mon[18790]: overall HEALTH_OK 2024-08-20T03:10:01.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:00 smithi165 ceph-mon[22357]: overall HEALTH_OK 2024-08-20T03:10:01.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.432+0000 7f864d364640 1 -- 172.21.15.44:0/1363069915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86481019d0 msgr2=0x7f8648101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:01.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.432+0000 7f864d364640 1 --2- 172.21.15.44:0/1363069915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86481019d0 0x7f8648101db0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8638009900 tx=0x7f863802a410 comp rx=0 tx=0).stop 2024-08-20T03:10:01.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.432+0000 7f864d364640 1 -- 172.21.15.44:0/1363069915 shutdown_connections 2024-08-20T03:10:01.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.432+0000 7f864d364640 1 --2- 172.21.15.44:0/1363069915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8648107d60 0x7f864810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.432+0000 7f864d364640 1 --2- 172.21.15.44:0/1363069915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86481073e0 0x7f8648107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.432+0000 7f864d364640 1 --2- 172.21.15.44:0/1363069915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86481019d0 0x7f8648101db0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.432+0000 7f864d364640 1 -- 172.21.15.44:0/1363069915 >> 172.21.15.44:0/1363069915 conn(0x7f86480fd440 msgr2=0x7f86480ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.432+0000 7f864d364640 1 -- 172.21.15.44:0/1363069915 shutdown_connections 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.432+0000 7f864d364640 1 -- 172.21.15.44:0/1363069915 wait complete. 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.433+0000 7f864d364640 1 Processor -- start 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.433+0000 7f864d364640 1 -- start start 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.433+0000 7f864d364640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86481019d0 0x7f8648199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.433+0000 7f864d364640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86481073e0 0x7f864819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.433+0000 7f864d364640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8648107d60 0x7f86481a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.433+0000 7f864d364640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8648111a20 con 0x7f8648107d60 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.433+0000 7f864d364640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86481118a0 con 0x7f86481019d0 2024-08-20T03:10:01.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.433+0000 7f864d364640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8648111ba0 con 0x7f86481073e0 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f86477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8648107d60 0x7f86481a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f8646ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86481019d0 0x7f8648199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f8646ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86481019d0 0x7f8648199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:38054/0 (socket says 172.21.15.44:38054) 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f8646ffd640 1 -- 172.21.15.44:0/1593171597 learned_addr learned my addr 172.21.15.44:0/1593171597 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f863ffff640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86481073e0 0x7f864819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f86477fe640 1 -- 172.21.15.44:0/1593171597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86481073e0 msgr2=0x7f864819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f86477fe640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86481073e0 0x7f864819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f86477fe640 1 -- 172.21.15.44:0/1593171597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86481019d0 msgr2=0x7f8648199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f86477fe640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86481019d0 0x7f8648199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f86477fe640 1 -- 172.21.15.44:0/1593171597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86481a5350 con 0x7f8648107d60 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f863ffff640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86481073e0 0x7f864819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f8646ffd640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86481019d0 0x7f8648199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f86477fe640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8648107d60 0x7f86481a4e10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f864000be20 tx=0x7f864000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:01.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f8644ff9640 1 -- 172.21.15.44:0/1593171597 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640019070 con 0x7f8648107d60 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f8644ff9640 1 -- 172.21.15.44:0/1593171597 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86400148f0 con 0x7f8648107d60 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f864d364640 1 -- 172.21.15.44:0/1593171597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86481a5520 con 0x7f8648107d60 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f8644ff9640 1 -- 172.21.15.44:0/1593171597 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640012840 con 0x7f8648107d60 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.434+0000 7f864d364640 1 -- 172.21.15.44:0/1593171597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86481a5a80 con 0x7f8648107d60 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.437+0000 7f864f75d740 1 Processor -- start 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.437+0000 7f8644ff9640 1 -- 172.21.15.44:0/1593171597 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f864001c070 con 0x7f8648107d60 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.437+0000 7f864f75d740 1 -- start start 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.437+0000 7f864f75d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 0x5633b64d7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.437+0000 7f864f75d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64e8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.437+0000 7f864f75d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64e8540 0x5633b64ea930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.437+0000 7f864f75d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5633b644e930 con 0x5633b64d7880 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.437+0000 7f864f75d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5633b643de30 con 0x5633b64e8540 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.437+0000 7f864f75d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5633b6441e70 con 0x5633b64d81a0 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 0x5633b64d7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 0x5633b64d7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36106/0 (socket says 172.21.15.44:36106) 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863ffff640 1 -- 172.21.15.44:0/2612695069 learned_addr learned my addr 172.21.15.44:0/2612695069 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f8646ffd640 1 --2- 172.21.15.44:0/2612695069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863ffff640 1 --2- 172.21.15.44:0/2612695069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64e8540 0x5633b64ea930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863ffff640 1 -- 172.21.15.44:0/2612695069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 msgr2=0x5633b64e8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863ffff640 1 --2- 172.21.15.44:0/2612695069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64e8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863ffff640 1 -- 172.21.15.44:0/2612695069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64e8540 msgr2=0x5633b64ea930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863ffff640 1 --2- 172.21.15.44:0/2612695069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64e8540 0x5633b64ea930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863ffff640 1 -- 172.21.15.44:0/2612695069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633b619d650 con 0x5633b64d7880 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f8644ff9640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86340b3dc0 0x7f86340b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863ffff640 1 --2- 172.21.15.44:0/2612695069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 0x5633b64d7c60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8630013640 tx=0x7f86300097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b9510a4a0a2fc1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f8646ffd640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86340b3dc0 0x7f86340b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f8644ff9640 1 -- 172.21.15.44:0/1593171597 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f86400e35f0 con 0x7f8648107d60 2024-08-20T03:10:01.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863d7fa640 1 -- 172.21.15.44:0/2612695069 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863000f520 con 0x5633b64d7880 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863d7fa640 1 -- 172.21.15.44:0/2612695069 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f863000fb70 con 0x5633b64d7880 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.438+0000 7f863d7fa640 1 -- 172.21.15.44:0/2612695069 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86300109c0 con 0x5633b64d7880 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.439+0000 7f864f75d740 1 -- 172.21.15.44:0/2612695069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 msgr2=0x5633b64d7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.439+0000 7f8646ffd640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86340b3dc0 0x7f86340b6280 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8638031440 tx=0x7f86380058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.439+0000 7f864f75d740 1 --2- 172.21.15.44:0/2612695069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 0x5633b64d7c60 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8630013640 tx=0x7f86300097d0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.439+0000 7f864f75d740 1 -- 172.21.15.44:0/2612695069 shutdown_connections 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.439+0000 7f864f75d740 1 --2- 172.21.15.44:0/2612695069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64e8540 0x5633b64ea930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.439+0000 7f864f75d740 1 --2- 172.21.15.44:0/2612695069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.439+0000 7f864f75d740 1 --2- 172.21.15.44:0/2612695069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 0x5633b64d7c60 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.439+0000 7f864f75d740 1 -- 172.21.15.44:0/2612695069 >> 172.21.15.44:0/2612695069 conn(0x5633b64d56b0 msgr2=0x5633b64d6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.439+0000 7f864f75d740 1 -- 172.21.15.44:0/2612695069 shutdown_connections 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.439+0000 7f864f75d740 1 -- 172.21.15.44:0/2612695069 wait complete. 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.439+0000 7f864f75d740 1 -- start start 2024-08-20T03:10:01.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f864f75d740 1 Processor -- start 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f864f75d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64d7880 0x5633b64e5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f864f75d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64e5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f864f75d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64e8540 0x5633b64e5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f864f75d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5633b62a0700 con 0x5633b64e8540 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f864f75d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5633b63c4810 con 0x5633b64d7880 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f864f75d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5633b64eb1f0 con 0x5633b64d81a0 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f864f75d740 10 client.0 _ll_register_callbacks cb 0x5633b6437700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f86477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64e8540 0x5633b64e5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f86477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64e8540 0x5633b64e5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36120/0 (socket says 172.21.15.44:36120) 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f86477fe640 1 -- 172.21.15.44:0/539205725 learned_addr learned my addr 172.21.15.44:0/539205725 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f8646ffd640 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64d7880 0x5633b64e5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f863ffff640 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64e5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f86477fe640 1 -- 172.21.15.44:0/539205725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 msgr2=0x5633b64e5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:01.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f86477fe640 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64e5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f86477fe640 1 -- 172.21.15.44:0/539205725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64d7880 msgr2=0x5633b64e5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:01.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f86477fe640 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64d7880 0x5633b64e5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f86477fe640 1 -- 172.21.15.44:0/539205725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633b64f45d0 con 0x5633b64e8540 2024-08-20T03:10:01.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f863ffff640 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64e5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:10:01.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f86477fe640 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64e8540 0x5633b64e5fe0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f86400aa390 tx=0x7f86400aa400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3515ae1d71a103 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:01.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.440+0000 7f862e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:01.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f862e7fc640 1 -- 172.21.15.44:0/539205725 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86400b3400 con 0x5633b64e8540 2024-08-20T03:10:01.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f862e7fc640 1 -- 172.21.15.44:0/539205725 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f86400b2f70 con 0x5633b64e8540 2024-08-20T03:10:01.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f864f75d740 1 -- 172.21.15.44:0/539205725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5633b63ed150 con 0x5633b64e8540 2024-08-20T03:10:01.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f862e7fc640 1 -- 172.21.15.44:0/539205725 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f86400b3170 con 0x5633b64e8540 2024-08-20T03:10:01.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f864f75d740 10 client.17070 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f864f75d740 1 -- 172.21.15.44:0/539205725 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5633b64f6fe0 con 0x5633b64e8540 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f862e7fc640 1 -- 172.21.15.44:0/539205725 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5633b63ed150 con 0x5633b64e8540 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f862e7fc640 1 -- 172.21.15.44:0/539205725 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f86400e32a0 con 0x5633b64e8540 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f864f75d740 10 client.17070 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f864f75d740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f864f75d740 20 client.17070 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f864f75d740 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5633b64f8540 0x5633b64fa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f864f75d740 4 client.17070 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.441+0000 7f864f75d740 1 -- 172.21.15.44:0/539205725 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633b6450880 con 0x5633b64f8540 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.442+0000 7f8646ffd640 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5633b64f8540 0x5633b64fa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.442+0000 7f862e7fc640 10 client.17070 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.442+0000 7f8646ffd640 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5633b64f8540 0x5633b64fa960 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa4412552d4cd4a server_cookie=2d9ce47bf84631ee in_seq=0 out_seq=0 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f862e7fc640 1 -- 172.21.15.44:0/539205725 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633b6450880 con 0x5633b64f8540 2024-08-20T03:10:01.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f862e7fc640 10 client.17070 handle_command_reply: tid=0 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f864f75d740 1 client.17070 shutdown 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f864f75d740 1 -- 172.21.15.44:0/539205725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64e8540 msgr2=0x5633b64e5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f864f75d740 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64e8540 0x5633b64e5fe0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f86400aa390 tx=0x7f86400aa400 comp rx=0 tx=0).stop 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f864f75d740 1 -- 172.21.15.44:0/539205725 shutdown_connections 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f864f75d740 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5633b64f8540 0x5633b64fa960 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f864f75d740 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64e8540 0x5633b64e5fe0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f864f75d740 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64e5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f864f75d740 1 --2- 172.21.15.44:0/539205725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64d7880 0x5633b64e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f862e7fc640 0 client.17070 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f864f75d740 1 -- 172.21.15.44:0/539205725 >> 172.21.15.44:0/539205725 conn(0x5633b64d56b0 msgr2=0x5633b64db220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f864f75d740 1 -- 172.21.15.44:0/539205725 shutdown_connections 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f864f75d740 1 -- 172.21.15.44:0/539205725 wait complete. 2024-08-20T03:10:01.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.443+0000 7f864f75d740 20 client.17070 trim_cache size 0 max 16384 2024-08-20T03:10:01.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.456+0000 7f864f75d740 1 Processor -- start 2024-08-20T03:10:01.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.456+0000 7f864f75d740 1 -- start start 2024-08-20T03:10:01.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.456+0000 7f864f75d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64d7880 0x5633b64e2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.456+0000 7f864f75d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d81a0 0x5633b64e7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.456+0000 7f864f75d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64e8540 0x5633b64e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.456+0000 7f864f75d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86481ac0d0 con 0x5633b64d81a0 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.456+0000 7f864f75d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86481add90 con 0x5633b64d7880 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.456+0000 7f864f75d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86481ac830 con 0x5633b64e8540 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f863ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d81a0 0x5633b64e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f863ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d81a0 0x5633b64e7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36128/0 (socket says 172.21.15.44:36128) 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f863ffff640 1 -- 172.21.15.44:0/4235500453 learned_addr learned my addr 172.21.15.44:0/4235500453 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f8646ffd640 1 --2- 172.21.15.44:0/4235500453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64d7880 0x5633b64e2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f86477fe640 1 --2- 172.21.15.44:0/4235500453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64e8540 0x5633b64e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f863ffff640 1 -- 172.21.15.44:0/4235500453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64e8540 msgr2=0x5633b64e5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f863ffff640 1 --2- 172.21.15.44:0/4235500453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64e8540 0x5633b64e5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f863ffff640 1 -- 172.21.15.44:0/4235500453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64d7880 msgr2=0x5633b64e2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f863ffff640 1 --2- 172.21.15.44:0/4235500453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64d7880 0x5633b64e2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f863ffff640 1 -- 172.21.15.44:0/4235500453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633b64e9160 con 0x5633b64d81a0 2024-08-20T03:10:01.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f863ffff640 1 --2- 172.21.15.44:0/4235500453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d81a0 0x5633b64e7250 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8630005d00 tx=0x7f8630005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5dc760d02609b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f863cff9640 1 -- 172.21.15.44:0/4235500453 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86300130c0 con 0x5633b64d81a0 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f863cff9640 1 -- 172.21.15.44:0/4235500453 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86300054f0 con 0x5633b64d81a0 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.457+0000 7f863cff9640 1 -- 172.21.15.44:0/4235500453 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630003e20 con 0x5633b64d81a0 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.458+0000 7f864f75d740 1 -- 172.21.15.44:0/4235500453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d81a0 msgr2=0x5633b64e7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.458+0000 7f864f75d740 1 --2- 172.21.15.44:0/4235500453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d81a0 0x5633b64e7250 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8630005d00 tx=0x7f8630005a70 comp rx=0 tx=0).stop 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.458+0000 7f864f75d740 1 -- 172.21.15.44:0/4235500453 shutdown_connections 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.458+0000 7f864f75d740 1 --2- 172.21.15.44:0/4235500453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64e8540 0x5633b64e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.458+0000 7f864f75d740 1 --2- 172.21.15.44:0/4235500453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d81a0 0x5633b64e7250 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.458+0000 7f864f75d740 1 --2- 172.21.15.44:0/4235500453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64d7880 0x5633b64e2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.458+0000 7f864f75d740 1 -- 172.21.15.44:0/4235500453 >> 172.21.15.44:0/4235500453 conn(0x5633b64d56b0 msgr2=0x5633b64e6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.458+0000 7f864f75d740 1 -- 172.21.15.44:0/4235500453 shutdown_connections 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.458+0000 7f864f75d740 1 -- 172.21.15.44:0/4235500453 wait complete. 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f864f75d740 1 -- start start 2024-08-20T03:10:01.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f864f75d740 1 Processor -- start 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f864f75d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 0x5633b64f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f864f75d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f864f75d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64e8540 0x5633b6500340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f864f75d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f86481ad6c0 con 0x5633b64d7880 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f864f75d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86481acf90 con 0x5633b64e8540 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f864f75d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f86481aa0d0 con 0x5633b64d81a0 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f864f75d740 10 client.0 _ll_register_callbacks cb 0x7f86400a62c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f8646ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 0x5633b64f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f8646ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 0x5633b64f0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36138/0 (socket says 172.21.15.44:36138) 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f8646ffd640 1 -- 172.21.15.44:0/337234247 learned_addr learned my addr 172.21.15.44:0/337234247 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f86477fe640 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64e8540 0x5633b6500340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.459+0000 7f863ffff640 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f8646ffd640 1 -- 172.21.15.44:0/337234247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 msgr2=0x5633b64ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f8646ffd640 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f8646ffd640 1 -- 172.21.15.44:0/337234247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64e8540 msgr2=0x5633b6500340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f8646ffd640 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64e8540 0x5633b6500340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f8646ffd640 1 -- 172.21.15.44:0/337234247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633b6500b40 con 0x5633b64d7880 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f86477fe640 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64e8540 0x5633b6500340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f8646ffd640 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 0x5633b64f0c20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8638005c40 tx=0x7f8638009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08e9953c1b1b4b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f862e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f862e7fc640 1 -- 172.21.15.44:0/337234247 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8638043070 con 0x5633b64d7880 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f862e7fc640 1 -- 172.21.15.44:0/337234247 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f8638005220 con 0x5633b64d7880 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f864f75d740 1 -- 172.21.15.44:0/337234247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86300054f0 con 0x5633b64d7880 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f862e7fc640 1 -- 172.21.15.44:0/337234247 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8638035040 con 0x5633b64d7880 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f864f75d740 10 client.17076 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f864f75d740 1 -- 172.21.15.44:0/337234247 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5633b6563400 con 0x5633b64d7880 2024-08-20T03:10:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.460+0000 7f862e7fc640 1 -- 172.21.15.44:0/337234247 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86300054f0 con 0x5633b64d7880 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.461+0000 7f862e7fc640 1 -- 172.21.15.44:0/337234247 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f8638003890 con 0x5633b64d7880 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.461+0000 7f864f75d740 10 client.17076 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.461+0000 7f864f75d740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.461+0000 7f864f75d740 20 client.17076 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.461+0000 7f864f75d740 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5633b6563b10 0x5633b6565f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.461+0000 7f864f75d740 4 client.17076 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.461+0000 7f864f75d740 1 -- 172.21.15.44:0/337234247 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633b644c320 con 0x5633b6563b10 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.461+0000 7f863ffff640 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5633b6563b10 0x5633b6565f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.461+0000 7f862e7fc640 10 client.17076 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.461+0000 7f863ffff640 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5633b6563b10 0x5633b6565f30 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a55e66971e2125 server_cookie=6b53c9366630c397 in_seq=0 out_seq=0 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f862e7fc640 1 -- 172.21.15.44:0/337234247 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633b644c320 con 0x5633b6563b10 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f862e7fc640 10 client.17076 handle_command_reply: tid=0 2024-08-20T03:10:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f864f75d740 1 client.17076 shutdown 2024-08-20T03:10:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f864f75d740 1 -- 172.21.15.44:0/337234247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 msgr2=0x5633b64f0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f864f75d740 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 0x5633b64f0c20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8638005c40 tx=0x7f8638009260 comp rx=0 tx=0).stop 2024-08-20T03:10:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f864f75d740 1 -- 172.21.15.44:0/337234247 shutdown_connections 2024-08-20T03:10:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f864f75d740 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5633b6563b10 0x5633b6565f30 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f864f75d740 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5633b64e8540 0x5633b6500340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f864f75d740 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633b64d81a0 0x5633b64ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f864f75d740 1 --2- 172.21.15.44:0/337234247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5633b64d7880 0x5633b64f0c20 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f864f75d740 1 -- 172.21.15.44:0/337234247 >> 172.21.15.44:0/337234247 conn(0x5633b64d56b0 msgr2=0x5633b64e7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f862e7fc640 0 client.17076 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f864f75d740 1 -- 172.21.15.44:0/337234247 shutdown_connections 2024-08-20T03:10:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f864f75d740 1 -- 172.21.15.44:0/337234247 wait complete. 2024-08-20T03:10:01.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.462+0000 7f864f75d740 20 client.17076 trim_cache size 0 max 16384 2024-08-20T03:10:01.472 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:10:01.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.463+0000 7f864d364640 1 -- 172.21.15.44:0/1593171597 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86340b3dc0 msgr2=0x7f86340b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:01.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.463+0000 7f864d364640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86340b3dc0 0x7f86340b6280 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8638031440 tx=0x7f86380058d0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.463+0000 7f864d364640 1 -- 172.21.15.44:0/1593171597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8648107d60 msgr2=0x7f86481a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:01.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.463+0000 7f864d364640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8648107d60 0x7f86481a4e10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f864000be20 tx=0x7f864000bf10 comp rx=0 tx=0).stop 2024-08-20T03:10:01.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.463+0000 7f864d364640 1 -- 172.21.15.44:0/1593171597 shutdown_connections 2024-08-20T03:10:01.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.463+0000 7f864d364640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f86340b3dc0 0x7f86340b6280 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.463+0000 7f864d364640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8648107d60 0x7f86481a4e10 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.463+0000 7f864d364640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86481073e0 0x7f864819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.463+0000 7f864d364640 1 --2- 172.21.15.44:0/1593171597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86481019d0 0x7f8648199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:01.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.463+0000 7f864d364640 1 -- 172.21.15.44:0/1593171597 >> 172.21.15.44:0/1593171597 conn(0x7f86480fd440 msgr2=0x7f86480ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:01.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.463+0000 7f864d364640 1 -- 172.21.15.44:0/1593171597 shutdown_connections 2024-08-20T03:10:01.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:01.463+0000 7f864d364640 1 -- 172.21.15.44:0/1593171597 wait complete. 2024-08-20T03:10:01.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:01 smithi044 ceph-mon[18790]: pgmap v548: 97 pgs: 97 active+clean; 865 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 581 op/s 2024-08-20T03:10:01.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:01 smithi099 ceph-mon[21984]: pgmap v548: 97 pgs: 97 active+clean; 865 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 581 op/s 2024-08-20T03:10:02.081 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:01 smithi165 ceph-mon[22357]: pgmap v548: 97 pgs: 97 active+clean; 865 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 581 op/s 2024-08-20T03:10:03.133 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T03:10:03.133 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T03:10:03.401 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:10:03.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:03 smithi044 ceph-mon[18790]: pgmap v549: 97 pgs: 97 active+clean; 867 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 579 op/s 2024-08-20T03:10:03.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:03 smithi099 ceph-mon[21984]: pgmap v549: 97 pgs: 97 active+clean; 867 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 579 op/s 2024-08-20T03:10:03.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.971+0000 7f7eeec5b640 1 -- 172.21.15.44:0/1344337697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee8102ab0 msgr2=0x7f7ee8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:03.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.971+0000 7f7eeec5b640 1 --2- 172.21.15.44:0/1344337697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee8102ab0 0x7f7ee8102e90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009ca0 tx=0x7f7ed802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.971+0000 7f7eeec5b640 1 -- 172.21.15.44:0/1344337697 shutdown_connections 2024-08-20T03:10:03.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.971+0000 7f7eeec5b640 1 --2- 172.21.15.44:0/1344337697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ee8108e40 0x7f7ee810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.971+0000 7f7eeec5b640 1 --2- 172.21.15.44:0/1344337697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ee81084c0 0x7f7ee8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.971+0000 7f7eeec5b640 1 --2- 172.21.15.44:0/1344337697 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee8102ab0 0x7f7ee8102e90 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.971+0000 7f7eeec5b640 1 -- 172.21.15.44:0/1344337697 >> 172.21.15.44:0/1344337697 conn(0x7f7ee80fe500 msgr2=0x7f7ee8100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:03.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.971+0000 7f7eeec5b640 1 -- 172.21.15.44:0/1344337697 shutdown_connections 2024-08-20T03:10:03.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.971+0000 7f7eeec5b640 1 -- 172.21.15.44:0/1344337697 wait complete. 2024-08-20T03:10:03.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eeec5b640 1 Processor -- start 2024-08-20T03:10:03.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eeec5b640 1 -- start start 2024-08-20T03:10:03.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eeec5b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ee8102ab0 0x7f7ee8198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:03.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eeec5b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ee81084c0 0x7f7ee81994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:03.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eeec5b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee8108e40 0x7f7ee819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:03.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eeec5b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8112b00 con 0x7f7ee8108e40 2024-08-20T03:10:03.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eeec5b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8112980 con 0x7f7ee81084c0 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eeec5b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8112c80 con 0x7f7ee8102ab0 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eed1f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee8108e40 0x7f7ee819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eed1f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee8108e40 0x7f7ee819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36148/0 (socket says 172.21.15.44:36148) 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eed1f8640 1 -- 172.21.15.44:0/267492135 learned_addr learned my addr 172.21.15.44:0/267492135 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ee81084c0 0x7f7ee81994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eec9f7640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ee8102ab0 0x7f7ee8198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eed1f8640 1 -- 172.21.15.44:0/267492135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ee8102ab0 msgr2=0x7f7ee8198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eed1f8640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ee8102ab0 0x7f7ee8198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eed1f8640 1 -- 172.21.15.44:0/267492135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ee81084c0 msgr2=0x7f7ee81994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eed1f8640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ee81084c0 0x7f7ee81994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.972+0000 7f7eed1f8640 1 -- 172.21.15.44:0/267492135 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee819f040 con 0x7f7ee8108e40 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.973+0000 7f7eed1f8640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee8108e40 0x7f7ee819e870 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000d750 tx=0x7f7ee000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:03.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.973+0000 7f7ee5ffb640 1 -- 172.21.15.44:0/267492135 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0017070 con 0x7f7ee8108e40 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.973+0000 7f7ee5ffb640 1 -- 172.21.15.44:0/267492135 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee00054d0 con 0x7f7ee8108e40 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.973+0000 7f7eeec5b640 1 -- 172.21.15.44:0/267492135 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee819f330 con 0x7f7ee8108e40 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.973+0000 7f7ee5ffb640 1 -- 172.21.15.44:0/267492135 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0004c60 con 0x7f7ee8108e40 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.973+0000 7f7eeec5b640 1 -- 172.21.15.44:0/267492135 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee81a59f0 con 0x7f7ee8108e40 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.975+0000 7f7ee5ffb640 1 -- 172.21.15.44:0/267492135 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7ee0009bd0 con 0x7f7ee8108e40 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ef1054740 1 Processor -- start 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ef1054740 1 -- start start 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ef1054740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bd3b0 0x55d7f28bd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ef1054740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bdcd0 0x55d7f28c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ef1054740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 0x55d7f28cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ef1054740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d7f2833940 con 0x55d7f28c9c60 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ef1054740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d7f2822e40 con 0x55d7f28bd3b0 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ef1054740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d7f2826e80 con 0x55d7f28bdcd0 2024-08-20T03:10:03.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 0x55d7f28cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:03.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 0x55d7f28cc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36162/0 (socket says 172.21.15.44:36162) 2024-08-20T03:10:03.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ee7fff640 1 -- 172.21.15.44:0/4237674636 learned_addr learned my addr 172.21.15.44:0/4237674636 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:03.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/4237674636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bd3b0 0x55d7f28bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:03.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7eec9f7640 1 --2- 172.21.15.44:0/4237674636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bdcd0 0x55d7f28c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:03.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ee5ffb640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7ed40b3d40 0x7f7ed40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:03.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ee7fff640 1 -- 172.21.15.44:0/4237674636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bdcd0 msgr2=0x55d7f28c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:03.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/4237674636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bdcd0 0x55d7f28c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ee7fff640 1 -- 172.21.15.44:0/4237674636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bd3b0 msgr2=0x55d7f28bd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:03.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/4237674636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bd3b0 0x55d7f28bd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7ee7fff640 1 -- 172.21.15.44:0/4237674636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f27bf520 con 0x55d7f28c9c60 2024-08-20T03:10:03.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.976+0000 7f7eed1f8640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7ed40b3d40 0x7f7ed40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:03.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.977+0000 7f7ee5ffb640 1 -- 172.21.15.44:0/267492135 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f7ee00e3930 con 0x7f7ee8108e40 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.977+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/4237674636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 0x55d7f28cc050 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00127f0 tx=0x7f7ed000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c06fdd9ae008fb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.977+0000 7f7edeffd640 1 -- 172.21.15.44:0/4237674636 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0012e30 con 0x55d7f28c9c60 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.977+0000 7f7edeffd640 1 -- 172.21.15.44:0/4237674636 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed0010c20 con 0x55d7f28c9c60 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.977+0000 7f7edeffd640 1 -- 172.21.15.44:0/4237674636 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed001ea70 con 0x55d7f28c9c60 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 -- 172.21.15.44:0/4237674636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 msgr2=0x55d7f28cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7eed1f8640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7ed40b3d40 0x7f7ed40b6200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00e1170 tx=0x7f7ee00cd040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 --2- 172.21.15.44:0/4237674636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 0x55d7f28cc050 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00127f0 tx=0x7f7ed000b7c0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 -- 172.21.15.44:0/4237674636 shutdown_connections 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 --2- 172.21.15.44:0/4237674636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 0x55d7f28cc050 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 --2- 172.21.15.44:0/4237674636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bdcd0 0x55d7f28c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 --2- 172.21.15.44:0/4237674636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bd3b0 0x55d7f28bd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 -- 172.21.15.44:0/4237674636 >> 172.21.15.44:0/4237674636 conn(0x55d7f28bb1e0 msgr2=0x55d7f28bc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:03.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 -- 172.21.15.44:0/4237674636 shutdown_connections 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 -- 172.21.15.44:0/4237674636 wait complete. 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 -- start start 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 Processor -- start 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bd3b0 0x55d7f28c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bdcd0 0x55d7f28c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 0x55d7f28c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d7f2685890 con 0x55d7f28c9c60 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d7f27a9700 con 0x55d7f28bd3b0 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.978+0000 7f7ef1054740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d7f28cc9b0 con 0x55d7f28bdcd0 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ef1054740 10 client.0 _ll_register_callbacks cb 0x55d7f281e6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 0x55d7f28c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 0x55d7f28c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36170/0 (socket says 172.21.15.44:36170) 2024-08-20T03:10:03.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ee7fff640 1 -- 172.21.15.44:0/1390834060 learned_addr learned my addr 172.21.15.44:0/1390834060 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bd3b0 0x55d7f28c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7eec9f7640 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bdcd0 0x55d7f28c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ee7fff640 1 -- 172.21.15.44:0/1390834060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bdcd0 msgr2=0x55d7f28c7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bdcd0 0x55d7f28c7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ee7fff640 1 -- 172.21.15.44:0/1390834060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bd3b0 msgr2=0x55d7f28c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bd3b0 0x55d7f28c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ee7fff640 1 -- 172.21.15.44:0/1390834060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f2582930 con 0x55d7f28c9c60 2024-08-20T03:10:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 0x55d7f28c7680 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000b440 tx=0x7f7ed001e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bf32d4ce4813d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ec7fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ec7fff640 1 -- 172.21.15.44:0/1390834060 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0030060 con 0x55d7f28c9c60 2024-08-20T03:10:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ec7fff640 1 -- 172.21.15.44:0/1390834060 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f7ed0010970 con 0x55d7f28c9c60 2024-08-20T03:10:03.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ef1054740 1 -- 172.21.15.44:0/1390834060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed0010c20 con 0x55d7f28c9c60 2024-08-20T03:10:03.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.979+0000 7f7ec7fff640 1 -- 172.21.15.44:0/1390834060 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7ed0024040 con 0x55d7f28c9c60 2024-08-20T03:10:03.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.980+0000 7f7ef1054740 10 client.17088 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:03.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.980+0000 7f7ef1054740 1 -- 172.21.15.44:0/1390834060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d7f28d8540 con 0x55d7f28c9c60 2024-08-20T03:10:03.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.980+0000 7f7ec7fff640 1 -- 172.21.15.44:0/1390834060 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed0010c20 con 0x55d7f28c9c60 2024-08-20T03:10:03.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.980+0000 7f7ec7fff640 1 -- 172.21.15.44:0/1390834060 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f7ed002a030 con 0x55d7f28c9c60 2024-08-20T03:10:03.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.980+0000 7f7ef1054740 10 client.17088 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:03.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.980+0000 7f7ef1054740 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:03.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.980+0000 7f7ef1054740 20 client.17088 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:03.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.980+0000 7f7ef1054740 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f28d99a0 0x55d7f28dbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:03.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.980+0000 7f7ef1054740 4 client.17088 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.980+0000 7f7ef1054740 1 -- 172.21.15.44:0/1390834060 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7f2835890 con 0x55d7f28d99a0 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.980+0000 7f7eec9f7640 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f28d99a0 0x55d7f28dbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.981+0000 7f7eec9f7640 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f28d99a0 0x55d7f28dbd90 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b9a3646045a3242 server_cookie=4e0fb6f21b2a0e9f in_seq=0 out_seq=0 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.981+0000 7f7ec7fff640 10 client.17088 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.981+0000 7f7ec7fff640 1 -- 172.21.15.44:0/1390834060 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7f2835890 con 0x55d7f28d99a0 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.981+0000 7f7ec7fff640 10 client.17088 handle_command_reply: tid=0 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.981+0000 7f7ef1054740 1 client.17088 shutdown 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.981+0000 7f7ef1054740 1 -- 172.21.15.44:0/1390834060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 msgr2=0x55d7f28c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.981+0000 7f7ef1054740 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 0x55d7f28c7680 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000b440 tx=0x7f7ed001e5b0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.981+0000 7f7ef1054740 1 -- 172.21.15.44:0/1390834060 shutdown_connections 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.981+0000 7f7ef1054740 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f28d99a0 0x55d7f28dbd90 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.982+0000 7f7ef1054740 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28c9c60 0x55d7f28c7680 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.982+0000 7f7ef1054740 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bdcd0 0x55d7f28c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.982+0000 7f7ec7fff640 0 client.17088 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.982+0000 7f7ef1054740 1 --2- 172.21.15.44:0/1390834060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bd3b0 0x55d7f28c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.982+0000 7f7ef1054740 1 -- 172.21.15.44:0/1390834060 >> 172.21.15.44:0/1390834060 conn(0x55d7f28bb1e0 msgr2=0x55d7f28c0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.982+0000 7f7ef1054740 1 -- 172.21.15.44:0/1390834060 shutdown_connections 2024-08-20T03:10:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.982+0000 7f7ef1054740 1 -- 172.21.15.44:0/1390834060 wait complete. 2024-08-20T03:10:03.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.982+0000 7f7ef1054740 20 client.17088 trim_cache size 0 max 16384 2024-08-20T03:10:04.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.999+0000 7f7ef1054740 1 Processor -- start 2024-08-20T03:10:04.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.999+0000 7f7ef1054740 1 -- start start 2024-08-20T03:10:04.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.999+0000 7f7ef1054740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bd3b0 0x55d7f28e1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:04.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.999+0000 7f7ef1054740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bdcd0 0x55d7f28e1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:04.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.999+0000 7f7ef1054740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28c9c60 0x55d7f28c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:04.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.999+0000 7f7ef1054740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81ae3e0 con 0x55d7f28bdcd0 2024-08-20T03:10:04.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.999+0000 7f7ef1054740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81b00a0 con 0x55d7f28c9c60 2024-08-20T03:10:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.999+0000 7f7ef1054740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81aeb40 con 0x55d7f28bd3b0 2024-08-20T03:10:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.999+0000 7f7eec9f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bdcd0 0x55d7f28e1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.999+0000 7f7eec9f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bdcd0 0x55d7f28e1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36178/0 (socket says 172.21.15.44:36178) 2024-08-20T03:10:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:03.999+0000 7f7eec9f7640 1 -- 172.21.15.44:0/3258331500 learned_addr learned my addr 172.21.15.44:0/3258331500 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.000+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/3258331500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28c9c60 0x55d7f28c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.000+0000 7f7eec9f7640 1 -- 172.21.15.44:0/3258331500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bd3b0 msgr2=0x55d7f28e1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.000+0000 7f7eec9f7640 1 --2- 172.21.15.44:0/3258331500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bd3b0 0x55d7f28e1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.000+0000 7f7eec9f7640 1 -- 172.21.15.44:0/3258331500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28c9c60 msgr2=0x55d7f28c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.000+0000 7f7eec9f7640 1 --2- 172.21.15.44:0/3258331500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28c9c60 0x55d7f28c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.000+0000 7f7eec9f7640 1 -- 172.21.15.44:0/3258331500 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f28bbc60 con 0x55d7f28bdcd0 2024-08-20T03:10:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.000+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/3258331500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28c9c60 0x55d7f28c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:10:04.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.000+0000 7f7eec9f7640 1 --2- 172.21.15.44:0/3258331500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bdcd0 0x55d7f28e1a40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009ca0 tx=0x7f7ed8031e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7406fb331c917500 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.000+0000 7f7ede7fc640 1 -- 172.21.15.44:0/3258331500 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8031650 con 0x55d7f28bdcd0 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.000+0000 7f7ede7fc640 1 -- 172.21.15.44:0/3258331500 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed8036040 con 0x55d7f28bdcd0 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.000+0000 7f7ede7fc640 1 -- 172.21.15.44:0/3258331500 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8004470 con 0x55d7f28bdcd0 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.001+0000 7f7ef1054740 1 -- 172.21.15.44:0/3258331500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bdcd0 msgr2=0x55d7f28e1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.001+0000 7f7ef1054740 1 --2- 172.21.15.44:0/3258331500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bdcd0 0x55d7f28e1a40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009ca0 tx=0x7f7ed8031e30 comp rx=0 tx=0).stop 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.001+0000 7f7ef1054740 1 -- 172.21.15.44:0/3258331500 shutdown_connections 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.001+0000 7f7ef1054740 1 --2- 172.21.15.44:0/3258331500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28c9c60 0x55d7f28c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.001+0000 7f7ef1054740 1 --2- 172.21.15.44:0/3258331500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bdcd0 0x55d7f28e1a40 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.001+0000 7f7ef1054740 1 --2- 172.21.15.44:0/3258331500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28bd3b0 0x55d7f28e1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.001+0000 7f7ef1054740 1 -- 172.21.15.44:0/3258331500 >> 172.21.15.44:0/3258331500 conn(0x55d7f28bb1e0 msgr2=0x55d7f28c7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.001+0000 7f7ef1054740 1 -- 172.21.15.44:0/3258331500 shutdown_connections 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.001+0000 7f7ef1054740 1 -- 172.21.15.44:0/3258331500 wait complete. 2024-08-20T03:10:04.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.001+0000 7f7ef1054740 1 -- start start 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.001+0000 7f7ef1054740 1 Processor -- start 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ef1054740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bd3b0 0x55d7f28d2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ef1054740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bdcd0 0x55d7f28cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ef1054740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28c9c60 0x55d7f28cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ef1054740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81af9d0 con 0x55d7f28bd3b0 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ef1054740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81af2a0 con 0x55d7f28bdcd0 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ef1054740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81ac3e0 con 0x55d7f28c9c60 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ef1054740 10 client.0 _ll_register_callbacks cb 0x7f7ed000be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bd3b0 0x55d7f28d2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ee7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bd3b0 0x55d7f28d2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36194/0 (socket says 172.21.15.44:36194) 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ee7fff640 1 -- 172.21.15.44:0/3358809088 learned_addr learned my addr 172.21.15.44:0/3358809088 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7eec9f7640 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bdcd0 0x55d7f28cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:04.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28c9c60 0x55d7f28cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ee7fff640 1 -- 172.21.15.44:0/3358809088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28c9c60 msgr2=0x55d7f28cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28c9c60 0x55d7f28cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ee7fff640 1 -- 172.21.15.44:0/3358809088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bdcd0 msgr2=0x55d7f28cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bdcd0 0x55d7f28cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ee7fff640 1 -- 172.21.15.44:0/3358809088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f28c7aa0 con 0x55d7f28bd3b0 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7eec9f7640 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bdcd0 0x55d7f28cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ee7fff640 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bd3b0 0x55d7f28d2420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7ed001f6f0 tx=0x7f7ed001f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d619d67d1dffb2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ec7fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.002+0000 7f7ec7fff640 1 -- 172.21.15.44:0/3358809088 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0030040 con 0x55d7f28bd3b0 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ec7fff640 1 -- 172.21.15.44:0/3358809088 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8449+0+0 (secure 0 0 0) 0x7f7ed001fae0 con 0x55d7f28bd3b0 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ef1054740 1 -- 172.21.15.44:0/3358809088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed8036040 con 0x55d7f28bd3b0 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ec7fff640 1 -- 172.21.15.44:0/3358809088 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7ed0042050 con 0x55d7f28bd3b0 2024-08-20T03:10:04.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ef1054740 10 client.17094 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:04.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ef1054740 1 -- 172.21.15.44:0/3358809088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d7f2944450 con 0x55d7f28bd3b0 2024-08-20T03:10:04.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ec7fff640 1 -- 172.21.15.44:0/3358809088 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed8036040 con 0x55d7f28bd3b0 2024-08-20T03:10:04.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ec7fff640 1 -- 172.21.15.44:0/3358809088 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f7ed002a040 con 0x55d7f28bd3b0 2024-08-20T03:10:04.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ef1054740 10 client.17094 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:04.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ef1054740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:04.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ef1054740 20 client.17094 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:04.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ef1054740 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f2944bd0 0x55d7f2946ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:04.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ef1054740 4 client.17094 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T03:10:04.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.003+0000 7f7ef1054740 1 -- 172.21.15.44:0/3358809088 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d7f2831330 con 0x55d7f2944bd0 2024-08-20T03:10:04.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.004+0000 7f7eec9f7640 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f2944bd0 0x55d7f2946ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:04.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.004+0000 7f7eec9f7640 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f2944bd0 0x55d7f2946ff0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=722d1620935cc441 server_cookie=3da5d1eaac5ab177 in_seq=0 out_seq=0 2024-08-20T03:10:04.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.004+0000 7f7ec7fff640 10 client.17094 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ec7fff640 1 -- 172.21.15.44:0/3358809088 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d7f2831330 con 0x55d7f2944bd0 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ec7fff640 10 client.17094 handle_command_reply: tid=0 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ef1054740 1 client.17094 shutdown 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ef1054740 1 -- 172.21.15.44:0/3358809088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bd3b0 msgr2=0x55d7f28d2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ef1054740 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bd3b0 0x55d7f28d2420 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7ed001f6f0 tx=0x7f7ed001f860 comp rx=0 tx=0).stop 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ef1054740 1 -- 172.21.15.44:0/3358809088 shutdown_connections 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ef1054740 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f2944bd0 0x55d7f2946ff0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ef1054740 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f28c9c60 0x55d7f28cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ef1054740 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f28bdcd0 0x55d7f28cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ef1054740 1 --2- 172.21.15.44:0/3358809088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f28bd3b0 0x55d7f28d2420 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ef1054740 1 -- 172.21.15.44:0/3358809088 >> 172.21.15.44:0/3358809088 conn(0x55d7f28bb1e0 msgr2=0x55d7f28b9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ec7fff640 0 client.17094 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:04.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ef1054740 1 -- 172.21.15.44:0/3358809088 shutdown_connections 2024-08-20T03:10:04.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.005+0000 7f7ef1054740 1 -- 172.21.15.44:0/3358809088 wait complete. 2024-08-20T03:10:04.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7ef1054740 20 client.17094 trim_cache size 0 max 16384 2024-08-20T03:10:04.013 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:10:04.013 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-08-20T03:10:04.013 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "e9d47d3a-94d2-4bd1-944d-da7d1e8a6465", 2024-08-20T03:10:04.013 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-08-20T03:10:04.013 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:10:04.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7eeec5b640 1 -- 172.21.15.44:0/267492135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7ed40b3d40 msgr2=0x7f7ed40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:04.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7eeec5b640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7ed40b3d40 0x7f7ed40b6200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00e1170 tx=0x7f7ee00cd040 comp rx=0 tx=0).stop 2024-08-20T03:10:04.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7eeec5b640 1 -- 172.21.15.44:0/267492135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee8108e40 msgr2=0x7f7ee819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:04.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7eeec5b640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee8108e40 0x7f7ee819e870 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000d750 tx=0x7f7ee000dc10 comp rx=0 tx=0).stop 2024-08-20T03:10:04.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7eeec5b640 1 -- 172.21.15.44:0/267492135 shutdown_connections 2024-08-20T03:10:04.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7eeec5b640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7ed40b3d40 0x7f7ed40b6200 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7eeec5b640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ee8108e40 0x7f7ee819e870 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7eeec5b640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ee81084c0 0x7f7ee81994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7eeec5b640 1 --2- 172.21.15.44:0/267492135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ee8102ab0 0x7f7ee8198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:04.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7eeec5b640 1 -- 172.21.15.44:0/267492135 >> 172.21.15.44:0/267492135 conn(0x7f7ee80fe500 msgr2=0x7f7ee81008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:04.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7eeec5b640 1 -- 172.21.15.44:0/267492135 shutdown_connections 2024-08-20T03:10:04.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:04.006+0000 7f7eeec5b640 1 -- 172.21.15.44:0/267492135 wait complete. 2024-08-20T03:10:04.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:03 smithi165 ceph-mon[22357]: pgmap v549: 97 pgs: 97 active+clean; 867 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 579 op/s 2024-08-20T03:10:04.636 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:10:04 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:10:04.184+0000 7f9a16f7f640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001b13 [...a,head] /volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-35/ auth{2=1,4=1} fragtree_t(*^3) v270241 ap=2 RANDEPHEMERALPIN f(v18 m2024-08-20T03:08:05.082440+0000 211=211+0) n(v24 rc2024-08-20T03:08:05.082440+0000 b3421242 212=211+1) old_inodes=6 (inest sync->mix g=2,4) caps={15411=pAsLsXsFs/pAsLsXsFsx@301} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 randepin 0x55e456b02000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000001b13:[<0x10000000399/blog-35 v71849>,<0x10000000208/blogtest_in v472>,<0x10000000201/src v2054>,<0x10000000006/blogbench-1.0 v1800>,<0x10000000005/tmp v86>,<0x100000001fb/client.0 v81>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v95>,<0x10000000001/sv_1 v117>,<0x10000000000/qa v107>,<0x1/volumes v98>]//[]","memoryvalue":"(2)0x10000001b13:[<0x10000000399/blog-35 v270241>,<0x10000000208/blogtest_in v2711>,<0x10000000201/src v2840>,<0x10000000006/blogbench-1.0 v2477>,<0x10000000005/tmp v697>,<0x100000001fb/client.0 v685>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v682>,<0x10000000001/sv_1 v685>,<0x10000000000/qa v673>,<0x1/volumes v650>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-20T03:07:37.887861+0000 210=210+0)","ondisk_value.rstat":"n(v0 rc2024-08-20T03:08:05.082440+0000 b3421242 212=211+1)","memory_value.dirstat":"f(v18 m2024-08-20T03:08:05.082440+0000 211=211+0)","memory_value.rstat":"n(v24 rc2024-08-20T03:08:05.082440+0000 b3421242 212=211+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-20T03:10:04.636 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:10:04 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:10:04.203+0000 7f9a16f7f640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000a595 [...a,head] /volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-152/ auth{3=1,4=1} fragtree_t(*^3) v270239 ap=2 RANDEPHEMERALPIN f(v7 m2024-08-20T03:08:27.223097+0000 161=161+0) n(v11 rc2024-08-20T03:08:27.223097+0000 b3089660 162=161+1) old_inodes=3 (inest sync->mix g=3,4) caps={15411=pAsLsXsFs/pAsLsXsFsx@239} mcw={3=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55e461c2e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000a595:[<0x10000000399/blog-152 v238726>,<0x10000000208/blogtest_in v1506>,<0x10000000201/src v2365>,<0x10000000006/blogbench-1.0 v2093>,<0x10000000005/tmp v370>,<0x100000001fb/client.0 v365>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v359>,<0x10000000001/sv_1 v385>,<0x10000000000/qa v362>,<0x1/volumes v357>]//[]","memoryvalue":"(2)0x1000000a595:[<0x10000000399/blog-152 v270239>,<0x10000000208/blogtest_in v2711>,<0x10000000201/src v2840>,<0x10000000006/blogbench-1.0 v2477>,<0x10000000005/tmp v697>,<0x100000001fb/client.0 v685>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v682>,<0x10000000001/sv_1 v685>,<0x10000000000/qa v673>,<0x1/volumes v650>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-20T03:08:24.223144+0000 160=160+0)","ondisk_value.rstat":"n(v0 rc2024-08-20T03:08:27.223097+0000 b3089660 162=161+1)","memory_value.dirstat":"f(v7 m2024-08-20T03:08:27.223097+0000 161=161+0)","memory_value.rstat":"n(v11 rc2024-08-20T03:08:27.223097+0000 b3089660 162=161+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-20T03:10:04.636 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:10:04 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:10:04.276+0000 7f9a16f7f640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000a926 [...a,head] /volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-155/ auth{3=1} fragtree_t(*^3) v256363 ap=2 RANDEPHEMERALPIN f(v2 m2024-08-20T03:08:00.721508+0000 116=116+0) n(v6 rc2024-08-20T03:08:00.721508+0000 b3911151 117=116+1) old_inodes=3 (inest sync->mix g=3) caps={15411=pAsLsXsFs/pAsLsXsFsx@263} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55e456f87700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000a926:[<0x10000000399/blog-155 v238308>,<0x10000000208/blogtest_in v1506>,<0x10000000201/src v2365>,<0x10000000006/blogbench-1.0 v2093>,<0x10000000005/tmp v370>,<0x100000001fb/client.0 v365>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v359>,<0x10000000001/sv_1 v385>,<0x10000000000/qa v362>,<0x1/volumes v357>]//[]","memoryvalue":"(2)0x1000000a926:[<0x10000000399/blog-155 v256363>,<0x10000000208/blogtest_in v2713>,<0x10000000201/src v2842>,<0x10000000006/blogbench-1.0 v2479>,<0x10000000005/tmp v699>,<0x100000001fb/client.0 v687>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v684>,<0x10000000001/sv_1 v687>,<0x10000000000/qa v675>,<0x1/volumes v653>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-08-20T03:07:43.373776+0000 115=115+0)","ondisk_value.rstat":"n(v0 rc2024-08-20T03:08:00.721508+0000 b3911151 117=116+1)","memory_value.dirstat":"f(v2 m2024-08-20T03:08:00.721508+0000 116=116+0)","memory_value.rstat":"n(v6 rc2024-08-20T03:08:00.721508+0000 b3911151 117=116+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-20T03:10:04.636 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:10:04 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:10:04.299+0000 7f9a16f7f640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000ad34 [...a,head] /volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-160/ auth{3=1,4=1} fragtree_t(*^3) v256365 ap=2 RANDEPHEMERALPIN f(v3 m2024-08-20T03:08:12.053332+0000 170=170+0) n(v7 rc2024-08-20T03:08:12.053332+0000 b2756566 171=170+1) old_inodes=3 (inest sync->mix g=3,4) caps={15411=pAsLsXsFs/pAsLsXsFsx@197} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55e457eda100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000ad34:[<0x10000000399/blog-160 v240579>,<0x10000000208/blogtest_in v1625>,<0x10000000201/src v2398>,<0x10000000006/blogbench-1.0 v2121>,<0x10000000005/tmp v399>,<0x100000001fb/client.0 v393>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v388>,<0x10000000001/sv_1 v411>,<0x10000000000/qa v390>,<0x1/volumes v384>]//[]","memoryvalue":"(2)0x1000000ad34:[<0x10000000399/blog-160 v256365>,<0x10000000208/blogtest_in v2713>,<0x10000000201/src v2842>,<0x10000000006/blogbench-1.0 v2479>,<0x10000000005/tmp v699>,<0x100000001fb/client.0 v687>,<0x100000001fa/015b80c1-9189-4015-8515-87e37dc378f2 v684>,<0x10000000001/sv_1 v687>,<0x10000000000/qa v675>,<0x1/volumes v653>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 100=100+0)","ondisk_value.rstat":"n(v0 rc2024-08-20T03:08:12.053332+0000 b2756566 171=170+1)","memory_value.dirstat":"f(v3 m2024-08-20T03:08:12.053332+0000 170=170+0)","memory_value.rstat":"n(v7 rc2024-08-20T03:08:12.053332+0000 b2756566 171=170+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-08-20T03:10:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:04 smithi044 ceph-mon[18790]: scrub queued for path: / 2024-08-20T03:10:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:04 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:04 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:10:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:04 smithi044 ceph-mon[18790]: Scrub error on inode 0x10000001b13 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-35) see mds.i log and `damage ls` output for details 2024-08-20T03:10:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:04 smithi044 ceph-mon[18790]: Scrub error on inode 0x1000000a595 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-152) see mds.i log and `damage ls` output for details 2024-08-20T03:10:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:04 smithi044 ceph-mon[18790]: Scrub error on inode 0x1000000a926 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-155) see mds.i log and `damage ls` output for details 2024-08-20T03:10:04.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:04 smithi044 ceph-mon[18790]: Scrub error on inode 0x1000000ad34 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-160) see mds.i log and `damage ls` output for details 2024-08-20T03:10:04.838 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:10:04.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:04 smithi099 ceph-mon[21984]: scrub queued for path: / 2024-08-20T03:10:04.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:04 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:04.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:04 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:10:04.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:04 smithi099 ceph-mon[21984]: Scrub error on inode 0x10000001b13 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-35) see mds.i log and `damage ls` output for details 2024-08-20T03:10:04.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:04 smithi099 ceph-mon[21984]: Scrub error on inode 0x1000000a595 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-152) see mds.i log and `damage ls` output for details 2024-08-20T03:10:04.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:04 smithi099 ceph-mon[21984]: Scrub error on inode 0x1000000a926 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-155) see mds.i log and `damage ls` output for details 2024-08-20T03:10:04.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:04 smithi099 ceph-mon[21984]: Scrub error on inode 0x1000000ad34 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-160) see mds.i log and `damage ls` output for details 2024-08-20T03:10:05.103 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:10:05.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:04 smithi165 ceph-mon[22357]: scrub queued for path: / 2024-08-20T03:10:05.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:04 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:04 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:10:05.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:04 smithi165 ceph-mon[22357]: Scrub error on inode 0x10000001b13 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-35) see mds.i log and `damage ls` output for details 2024-08-20T03:10:05.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:04 smithi165 ceph-mon[22357]: Scrub error on inode 0x1000000a595 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-152) see mds.i log and `damage ls` output for details 2024-08-20T03:10:05.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:04 smithi165 ceph-mon[22357]: Scrub error on inode 0x1000000a926 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-155) see mds.i log and `damage ls` output for details 2024-08-20T03:10:05.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:04 smithi165 ceph-mon[22357]: Scrub error on inode 0x1000000ad34 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-160) see mds.i log and `damage ls` output for details 2024-08-20T03:10:05.908 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:10:05.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:10:05.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:10:05.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:10:05.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:10:05.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:10:05.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:10:05.909 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.910 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.910 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:05 smithi044 ceph-mon[18790]: pgmap v550: 97 pgs: 97 active+clean; 873 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 617 op/s 2024-08-20T03:10:05.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:10:05.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:10:05.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:10:05.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:10:05.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:10:05.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:10:05.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:10:05.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:05.915 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:05 smithi099 ceph-mon[21984]: pgmap v550: 97 pgs: 97 active+clean; 873 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 617 op/s 2024-08-20T03:10:05.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.938+0000 7f12ca577640 1 -- 172.21.15.44:0/1982385643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c4109e80 msgr2=0x7f12c4110ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:05.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.938+0000 7f12ca577640 1 --2- 172.21.15.44:0/1982385643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c4109e80 0x7f12c4110ee0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f12b401a910 tx=0x7f12b403b500 comp rx=0 tx=0).stop 2024-08-20T03:10:05.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.942+0000 7f12ca577640 1 -- 172.21.15.44:0/1982385643 shutdown_connections 2024-08-20T03:10:05.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.942+0000 7f12ca577640 1 --2- 172.21.15.44:0/1982385643 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12c411bdd0 0x7f12c411e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.942+0000 7f12ca577640 1 --2- 172.21.15.44:0/1982385643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12c410a260 0x7f12c410a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.942+0000 7f12ca577640 1 --2- 172.21.15.44:0/1982385643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c4109e80 0x7f12c4110ee0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.942+0000 7f12ca577640 1 -- 172.21.15.44:0/1982385643 >> 172.21.15.44:0/1982385643 conn(0x7f12c406dcc0 msgr2=0x7f12c406e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.942+0000 7f12ca577640 1 -- 172.21.15.44:0/1982385643 shutdown_connections 2024-08-20T03:10:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.942+0000 7f12ca577640 1 -- 172.21.15.44:0/1982385643 wait complete. 2024-08-20T03:10:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.943+0000 7f12ca577640 1 Processor -- start 2024-08-20T03:10:05.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.943+0000 7f12ca577640 1 -- start start 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.944+0000 7f12ca577640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12c4109e80 0x7f12c411b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.944+0000 7f12ca577640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12c410a260 0x7f12c411b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.944+0000 7f12ca577640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c411bdd0 0x7f12c4117810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.944+0000 7f12ca577640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12c4113ed0 con 0x7f12c4109e80 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.944+0000 7f12ca577640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12c4113d50 con 0x7f12c411bdd0 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.944+0000 7f12ca577640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f12c4114050 con 0x7f12c410a260 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.944+0000 7f12c3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12c4109e80 0x7f12c411b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.944+0000 7f12c3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12c4109e80 0x7f12c411b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36216/0 (socket says 172.21.15.44:36216) 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.944+0000 7f12c3fff640 1 -- 172.21.15.44:0/4209478512 learned_addr learned my addr 172.21.15.44:0/4209478512 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.945+0000 7f12c3fff640 1 -- 172.21.15.44:0/4209478512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12c410a260 msgr2=0x7f12c411b890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.945+0000 7f12c3fff640 1 --2- 172.21.15.44:0/4209478512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12c410a260 0x7f12c411b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.945+0000 7f12c3fff640 1 -- 172.21.15.44:0/4209478512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c411bdd0 msgr2=0x7f12c4117810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.945+0000 7f12c3fff640 1 --2- 172.21.15.44:0/4209478512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c411bdd0 0x7f12c4117810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.945+0000 7f12c3fff640 1 -- 172.21.15.44:0/4209478512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c41180d0 con 0x7f12c4109e80 2024-08-20T03:10:05.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.945+0000 7f12c3fff640 1 --2- 172.21.15.44:0/4209478512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12c4109e80 0x7f12c411b350 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f12b4006010 tx=0x7f12b4004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:05.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.945+0000 7f12c17fa640 1 -- 172.21.15.44:0/4209478512 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4046070 con 0x7f12c4109e80 2024-08-20T03:10:05.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.945+0000 7f12c17fa640 1 -- 172.21.15.44:0/4209478512 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b4013070 con 0x7f12c4109e80 2024-08-20T03:10:05.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.945+0000 7f12c17fa640 1 -- 172.21.15.44:0/4209478512 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4040870 con 0x7f12c4109e80 2024-08-20T03:10:05.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.947+0000 7f12ca577640 1 -- 172.21.15.44:0/4209478512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c4118360 con 0x7f12c4109e80 2024-08-20T03:10:05.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.947+0000 7f12ca577640 1 -- 172.21.15.44:0/4209478512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c41cdfe0 con 0x7f12c4109e80 2024-08-20T03:10:05.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.953+0000 7f12c17fa640 1 -- 172.21.15.44:0/4209478512 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f12b400c040 con 0x7f12c4109e80 2024-08-20T03:10:05.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.955+0000 7f12c17fa640 1 --2- 172.21.15.44:0/4209478512 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f12a80b3f50 0x7f12a80b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:05.958 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.955+0000 7f12c17fa640 1 -- 172.21.15.44:0/4209478512 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8493+0+0 (secure 0 0 0) 0x7f12b4104150 con 0x7f12c4109e80 2024-08-20T03:10:05.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.963+0000 7f12c37fe640 1 --2- 172.21.15.44:0/4209478512 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f12a80b3f50 0x7f12a80b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:05.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.964+0000 7f12cc970740 1 Processor -- start 2024-08-20T03:10:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.966+0000 7f12cc970740 1 -- start start 2024-08-20T03:10:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.966+0000 7f12c37fe640 1 --2- 172.21.15.44:0/4209478512 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f12a80b3f50 0x7f12a80b6410 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f12c411b0a0 tx=0x7f12b0006bd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.966+0000 7f12cc970740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f356830 0x55a18f356c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.967+0000 7f12cc970740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f357150 0x55a18f362ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.967+0000 7f12cc970740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f3630e0 0x55a18f3654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.967+0000 7f12cc970740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a18f2cd8e0 con 0x55a18f3630e0 2024-08-20T03:10:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.967+0000 7f12cc970740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a18f2bcde0 con 0x55a18f356830 2024-08-20T03:10:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.967+0000 7f12cc970740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a18f2c0e20 con 0x55a18f357150 2024-08-20T03:10:05.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.971+0000 7f12c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f356830 0x55a18f356c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:05.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.971+0000 7f12c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f356830 0x55a18f356c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:38158/0 (socket says 172.21.15.44:38158) 2024-08-20T03:10:05.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.971+0000 7f12c3fff640 1 -- 172.21.15.44:0/2312942018 learned_addr learned my addr 172.21.15.44:0/2312942018 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.972+0000 7f12c3fff640 1 -- 172.21.15.44:0/2312942018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f357150 msgr2=0x55a18f362ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.972+0000 7f12c3fff640 1 --2- 172.21.15.44:0/2312942018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f357150 0x55a18f362ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.972+0000 7f12c3fff640 1 -- 172.21.15.44:0/2312942018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f3630e0 msgr2=0x55a18f3654d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:10:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.972+0000 7f12c3fff640 1 --2- 172.21.15.44:0/2312942018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f3630e0 0x55a18f3654d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.972+0000 7f12c3fff640 1 -- 172.21.15.44:0/2312942018 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18f01c650 con 0x55a18f356830 2024-08-20T03:10:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.972+0000 7f12c3fff640 1 --2- 172.21.15.44:0/2312942018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f356830 0x55a18f356c10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f12b40c6e30 tx=0x7f12b40ca5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82bdc634e74a7a53 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.972+0000 7f12ba7fc640 1 -- 172.21.15.44:0/2312942018 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b40cadb0 con 0x55a18f356830 2024-08-20T03:10:05.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.972+0000 7f12ba7fc640 1 -- 172.21.15.44:0/2312942018 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b40cf3b0 con 0x55a18f356830 2024-08-20T03:10:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.973+0000 7f12cc970740 1 -- 172.21.15.44:0/2312942018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f356830 msgr2=0x55a18f356c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:05.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.973+0000 7f12cc970740 1 --2- 172.21.15.44:0/2312942018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f356830 0x55a18f356c10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f12b40c6e30 tx=0x7f12b40ca5e0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.978+0000 7f12cc970740 1 -- 172.21.15.44:0/2312942018 shutdown_connections 2024-08-20T03:10:05.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.978+0000 7f12cc970740 1 --2- 172.21.15.44:0/2312942018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f3630e0 0x55a18f3654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.978+0000 7f12cc970740 1 --2- 172.21.15.44:0/2312942018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f357150 0x55a18f362ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.978+0000 7f12cc970740 1 --2- 172.21.15.44:0/2312942018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f356830 0x55a18f356c10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.978+0000 7f12cc970740 1 -- 172.21.15.44:0/2312942018 >> 172.21.15.44:0/2312942018 conn(0x55a18f354660 msgr2=0x55a18f355800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:05.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.982+0000 7f12cc970740 1 -- 172.21.15.44:0/2312942018 shutdown_connections 2024-08-20T03:10:05.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.982+0000 7f12cc970740 1 -- 172.21.15.44:0/2312942018 wait complete. 2024-08-20T03:10:05.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.983+0000 7f12cc970740 1 -- start start 2024-08-20T03:10:05.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.983+0000 7f12cc970740 1 Processor -- start 2024-08-20T03:10:05.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.984+0000 7f12cc970740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f356830 0x55a18f362b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:05.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.984+0000 7f12cc970740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f357150 0x55a18f360020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:05.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.984+0000 7f12cc970740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f3630e0 0x55a18f3607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:05.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.984+0000 7f12cc970740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a18f11f700 con 0x55a18f3630e0 2024-08-20T03:10:05.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.984+0000 7f12cc970740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a18f2437c0 con 0x55a18f357150 2024-08-20T03:10:05.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.984+0000 7f12cc970740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a18f365b50 con 0x55a18f356830 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.984+0000 7f12cc970740 10 client.0 _ll_register_callbacks cb 0x55a18f2b66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.984+0000 7f12c8b14640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f3630e0 0x55a18f3607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.984+0000 7f12c8b14640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f3630e0 0x55a18f3607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36228/0 (socket says 172.21.15.44:36228) 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.984+0000 7f12c8b14640 1 -- 172.21.15.44:0/2615870663 learned_addr learned my addr 172.21.15.44:0/2615870663 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.984+0000 7f12c37fe640 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f357150 0x55a18f360020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.984+0000 7f12c3fff640 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f356830 0x55a18f362b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.985+0000 7f12c8b14640 1 -- 172.21.15.44:0/2615870663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f356830 msgr2=0x55a18f362b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.985+0000 7f12c8b14640 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f356830 0x55a18f362b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.985+0000 7f12c8b14640 1 -- 172.21.15.44:0/2615870663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f357150 msgr2=0x55a18f360020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.985+0000 7f12c8b14640 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f357150 0x55a18f360020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.985+0000 7f12c8b14640 1 -- 172.21.15.44:0/2615870663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18f36eee0 con 0x55a18f3630e0 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.985+0000 7f12c8b14640 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f3630e0 0x55a18f3607b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00b470 tx=0x7f12bc00b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd2af5ec4a4be17 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.985+0000 7f12a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:05.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.985+0000 7f12a77fe640 1 -- 172.21.15.44:0/2615870663 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc015020 con 0x55a18f3630e0 2024-08-20T03:10:05.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.985+0000 7f12a77fe640 1 -- 172.21.15.44:0/2615870663 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8493+0+0 (secure 0 0 0) 0x7f12bc011800 con 0x55a18f3630e0 2024-08-20T03:10:05.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.986+0000 7f12cc970740 1 -- 172.21.15.44:0/2615870663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a18f26c100 con 0x55a18f3630e0 2024-08-20T03:10:05.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.986+0000 7f12a77fe640 1 -- 172.21.15.44:0/2615870663 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f12bc020050 con 0x55a18f3630e0 2024-08-20T03:10:05.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.989+0000 7f12cc970740 10 client.17100 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:05.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.989+0000 7f12cc970740 1 -- 172.21.15.44:0/2615870663 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a18f3718f0 con 0x55a18f3630e0 2024-08-20T03:10:05.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.990+0000 7f12a77fe640 1 -- 172.21.15.44:0/2615870663 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a18f26c100 con 0x55a18f3630e0 2024-08-20T03:10:05.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.990+0000 7f12a77fe640 1 -- 172.21.15.44:0/2615870663 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f12bc019d00 con 0x55a18f3630e0 2024-08-20T03:10:05.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.990+0000 7f12cc970740 10 client.17100 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:05.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.990+0000 7f12cc970740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:05.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.990+0000 7f12cc970740 20 client.17100 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:05.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.990+0000 7f12cc970740 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a18f372e00 0x55a18f375240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:05.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.990+0000 7f12cc970740 4 client.17100 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:05.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.990+0000 7f12cc970740 1 -- 172.21.15.44:0/2615870663 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a18f2cf830 con 0x55a18f372e00 2024-08-20T03:10:05.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.994+0000 7f12c3fff640 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a18f372e00 0x55a18f375240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:06.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.998+0000 7f12c3fff640 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a18f372e00 0x55a18f375240 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68dc95fdb085c795 server_cookie=a8e254c9a2d2592b in_seq=0 out_seq=0 2024-08-20T03:10:06.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:05.998+0000 7f12a77fe640 10 client.17100 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:06.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.001+0000 7f12a77fe640 1 -- 172.21.15.44:0/2615870663 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a18f2cf830 con 0x55a18f372e00 2024-08-20T03:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.001+0000 7f12a77fe640 10 client.17100 handle_command_reply: tid=0 2024-08-20T03:10:06.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.001+0000 7f12cc970740 1 client.17100 shutdown 2024-08-20T03:10:06.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.007+0000 7f12cc970740 1 -- 172.21.15.44:0/2615870663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f3630e0 msgr2=0x55a18f3607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:06.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.007+0000 7f12cc970740 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f3630e0 0x55a18f3607b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00b470 tx=0x7f12bc00b930 comp rx=0 tx=0).stop 2024-08-20T03:10:06.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.007+0000 7f12cc970740 1 -- 172.21.15.44:0/2615870663 shutdown_connections 2024-08-20T03:10:06.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.007+0000 7f12cc970740 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a18f372e00 0x55a18f375240 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.007+0000 7f12cc970740 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f3630e0 0x55a18f3607b0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.007+0000 7f12cc970740 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f357150 0x55a18f360020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.007+0000 7f12cc970740 1 --2- 172.21.15.44:0/2615870663 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f356830 0x55a18f362b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.007+0000 7f12cc970740 1 -- 172.21.15.44:0/2615870663 >> 172.21.15.44:0/2615870663 conn(0x55a18f354660 msgr2=0x55a18f35a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:06.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.007+0000 7f12a77fe640 0 client.17100 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:06.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.008+0000 7f12cc970740 1 -- 172.21.15.44:0/2615870663 shutdown_connections 2024-08-20T03:10:06.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.008+0000 7f12cc970740 1 -- 172.21.15.44:0/2615870663 wait complete. 2024-08-20T03:10:06.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.009+0000 7f12cc970740 20 client.17100 trim_cache size 0 max 16384 2024-08-20T03:10:06.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.042+0000 7f12cc970740 1 Processor -- start 2024-08-20T03:10:06.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.042+0000 7f12cc970740 1 -- start start 2024-08-20T03:10:06.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.043+0000 7f12cc970740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f356830 0x55a18f374e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:06.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.043+0000 7f12cc970740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f357150 0x55a18f35f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:06.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.043+0000 7f12cc970740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f3630e0 0x55a18f37ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:06.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.043+0000 7f12cc970740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12c41d6b30 con 0x55a18f357150 2024-08-20T03:10:06.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.043+0000 7f12cc970740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12c41d87f0 con 0x55a18f3630e0 2024-08-20T03:10:06.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.043+0000 7f12cc970740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f12c41d7290 con 0x55a18f356830 2024-08-20T03:10:06.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.044+0000 7f12c8b14640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f3630e0 0x55a18f37ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:06.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.044+0000 7f12c8b14640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f3630e0 0x55a18f37ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:38172/0 (socket says 172.21.15.44:38172) 2024-08-20T03:10:06.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.044+0000 7f12c8b14640 1 -- 172.21.15.44:0/1819805414 learned_addr learned my addr 172.21.15.44:0/1819805414 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:06.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.044+0000 7f12c3fff640 1 --2- 172.21.15.44:0/1819805414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f356830 0x55a18f374e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:06.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.047+0000 7f12c37fe640 1 --2- 172.21.15.44:0/1819805414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f357150 0x55a18f35f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.047+0000 7f12c8b14640 1 -- 172.21.15.44:0/1819805414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f356830 msgr2=0x55a18f374e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.047+0000 7f12c8b14640 1 --2- 172.21.15.44:0/1819805414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f356830 0x55a18f374e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.047+0000 7f12c8b14640 1 -- 172.21.15.44:0/1819805414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f357150 msgr2=0x55a18f35f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.047+0000 7f12c8b14640 1 --2- 172.21.15.44:0/1819805414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f357150 0x55a18f35f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.047+0000 7f12c8b14640 1 -- 172.21.15.44:0/1819805414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18f356f80 con 0x55a18f3630e0 2024-08-20T03:10:06.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.050+0000 7f12c8b14640 1 --2- 172.21.15.44:0/1819805414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f3630e0 0x55a18f37ab10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f12bc002230 tx=0x7f12bc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85fdd2719e113e4b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:06.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.050+0000 7f12b9ffb640 1 -- 172.21.15.44:0/1819805414 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc028070 con 0x55a18f3630e0 2024-08-20T03:10:06.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.051+0000 7f12b9ffb640 1 -- 172.21.15.44:0/1819805414 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12bc010c10 con 0x55a18f3630e0 2024-08-20T03:10:06.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.051+0000 7f12b9ffb640 1 -- 172.21.15.44:0/1819805414 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc015070 con 0x55a18f3630e0 2024-08-20T03:10:06.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.052+0000 7f12cc970740 1 -- 172.21.15.44:0/1819805414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f3630e0 msgr2=0x55a18f37ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:06.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.052+0000 7f12cc970740 1 --2- 172.21.15.44:0/1819805414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f3630e0 0x55a18f37ab10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f12bc002230 tx=0x7f12bc004a40 comp rx=0 tx=0).stop 2024-08-20T03:10:06.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.055+0000 7f12cc970740 1 -- 172.21.15.44:0/1819805414 shutdown_connections 2024-08-20T03:10:06.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.055+0000 7f12cc970740 1 --2- 172.21.15.44:0/1819805414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f3630e0 0x55a18f37ab10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.055+0000 7f12cc970740 1 --2- 172.21.15.44:0/1819805414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f357150 0x55a18f35f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.055+0000 7f12cc970740 1 --2- 172.21.15.44:0/1819805414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f356830 0x55a18f374e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.055+0000 7f12cc970740 1 -- 172.21.15.44:0/1819805414 >> 172.21.15.44:0/1819805414 conn(0x55a18f354660 msgr2=0x55a18f361e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:06.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.060+0000 7f12cc970740 1 -- 172.21.15.44:0/1819805414 shutdown_connections 2024-08-20T03:10:06.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.060+0000 7f12cc970740 1 -- 172.21.15.44:0/1819805414 wait complete. 2024-08-20T03:10:06.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.061+0000 7f12cc970740 1 -- start start 2024-08-20T03:10:06.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.061+0000 7f12cc970740 1 Processor -- start 2024-08-20T03:10:06.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.062+0000 7f12cc970740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f356830 0x55a18f36b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.062+0000 7f12cc970740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f357150 0x55a18f3668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.062+0000 7f12cc970740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f3630e0 0x55a18f366de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.062+0000 7f12c37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f357150 0x55a18f3668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.062+0000 7f12c37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f357150 0x55a18f3668a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:38174/0 (socket says 172.21.15.44:38174) 2024-08-20T03:10:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.062+0000 7f12c37fe640 1 -- 172.21.15.44:0/2957703316 learned_addr learned my addr 172.21.15.44:0/2957703316 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.062+0000 7f12c8b14640 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f3630e0 0x55a18f366de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.062+0000 7f12cc970740 1 -- 172.21.15.44:0/2957703316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f12c41d8120 con 0x55a18f356830 2024-08-20T03:10:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.062+0000 7f12cc970740 1 -- 172.21.15.44:0/2957703316 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12c41d79f0 con 0x55a18f357150 2024-08-20T03:10:06.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.062+0000 7f12cc970740 1 -- 172.21.15.44:0/2957703316 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f12c41d4b50 con 0x55a18f3630e0 2024-08-20T03:10:06.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.063+0000 7f12cc970740 10 client.0 _ll_register_callbacks cb 0x7f12bc009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:06.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.063+0000 7f12c37fe640 1 -- 172.21.15.44:0/2957703316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f3630e0 msgr2=0x55a18f366de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:06.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.063+0000 7f12c37fe640 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f3630e0 0x55a18f366de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.063+0000 7f12c37fe640 1 -- 172.21.15.44:0/2957703316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f356830 msgr2=0x55a18f36b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:06.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.063+0000 7f12c3fff640 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f356830 0x55a18f36b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:06.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.063+0000 7f12c37fe640 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f356830 0x55a18f36b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.063+0000 7f12c37fe640 1 -- 172.21.15.44:0/2957703316 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18f367520 con 0x55a18f357150 2024-08-20T03:10:06.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.063+0000 7f12c3fff640 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f356830 0x55a18f36b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:10:06.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.063+0000 7f12cc970740 1 -- 172.21.15.44:0/2957703316 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12bc010c10 con 0x55a18f357150 2024-08-20T03:10:06.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.067+0000 7f12c37fe640 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f357150 0x55a18f3668a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f12b0013f00 tx=0x7f12b00046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0bf13c57e76810 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:06.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.068+0000 7f12a77fe640 10 client.26525 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:10:06.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.068+0000 7f12a77fe640 1 -- 172.21.15.44:0/2957703316 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0024020 con 0x55a18f357150 2024-08-20T03:10:06.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.068+0000 7f12a77fe640 1 -- 172.21.15.44:0/2957703316 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8493+0+0 (secure 0 0 0) 0x7f12b0020450 con 0x55a18f357150 2024-08-20T03:10:06.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.068+0000 7f12a77fe640 1 -- 172.21.15.44:0/2957703316 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f12b0020670 con 0x55a18f357150 2024-08-20T03:10:06.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.070+0000 7f12cc970740 10 client.26525 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:06.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.070+0000 7f12cc970740 1 -- 172.21.15.44:0/2957703316 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=41}) -- 0x55a18f3dd9f0 con 0x55a18f357150 2024-08-20T03:10:06.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.071+0000 7f12a77fe640 1 -- 172.21.15.44:0/2957703316 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f12b0031020 con 0x55a18f357150 2024-08-20T03:10:06.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.074+0000 7f12cc970740 10 client.26525 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:06.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.074+0000 7f12cc970740 10 client.26525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:06.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.074+0000 7f12cc970740 20 client.26525 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:06.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.074+0000 7f12cc970740 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a18f3de010 0x55a18f3e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:06.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.074+0000 7f12cc970740 4 client.26525 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:10:06.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.074+0000 7f12cc970740 1 -- 172.21.15.44:0/2957703316 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a18f2cb2d0 con 0x55a18f3de010 2024-08-20T03:10:06.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.078+0000 7f12c3fff640 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a18f3de010 0x55a18f3e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:06.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.080+0000 7f12a77fe640 10 client.26525 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:06.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.080+0000 7f12c3fff640 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a18f3de010 0x55a18f3e0430 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ff0f740ea585dd server_cookie=67568b6d285ec6c0 in_seq=0 out_seq=0 2024-08-20T03:10:06.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.088+0000 7f12a77fe640 1 -- 172.21.15.44:0/2957703316 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x55a18f2cb2d0 con 0x55a18f3de010 2024-08-20T03:10:06.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.088+0000 7f12a77fe640 10 client.26525 handle_command_reply: tid=0 2024-08-20T03:10:06.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.088+0000 7f12cc970740 1 client.26525 shutdown 2024-08-20T03:10:06.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.090+0000 7f12cc970740 1 -- 172.21.15.44:0/2957703316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f357150 msgr2=0x55a18f3668a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:06.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.090+0000 7f12cc970740 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f357150 0x55a18f3668a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f12b0013f00 tx=0x7f12b00046e0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.092+0000 7f12cc970740 1 -- 172.21.15.44:0/2957703316 shutdown_connections 2024-08-20T03:10:06.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.092+0000 7f12cc970740 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a18f3de010 0x55a18f3e0430 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.092+0000 7f12cc970740 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a18f3630e0 0x55a18f366de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.092+0000 7f12cc970740 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a18f357150 0x55a18f3668a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.092+0000 7f12cc970740 1 --2- 172.21.15.44:0/2957703316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a18f356830 0x55a18f36b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.092+0000 7f12a77fe640 0 client.26525 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:06.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.092+0000 7f12cc970740 1 -- 172.21.15.44:0/2957703316 >> 172.21.15.44:0/2957703316 conn(0x55a18f354660 msgr2=0x55a18f3599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:06.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.093+0000 7f12cc970740 1 -- 172.21.15.44:0/2957703316 shutdown_connections 2024-08-20T03:10:06.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.093+0000 7f12cc970740 1 -- 172.21.15.44:0/2957703316 wait complete. 2024-08-20T03:10:06.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.093+0000 7f12cc970740 20 client.26525 trim_cache size 0 max 16384 2024-08-20T03:10:06.096 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:10:06.096 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "scrub active (10234 inodes in the stack)", 2024-08-20T03:10:06.096 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-08-20T03:10:06.096 INFO:teuthology.orchestra.run.smithi044.stdout: "e9d47d3a-94d2-4bd1-944d-da7d1e8a6465": { 2024-08-20T03:10:06.097 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-08-20T03:10:06.097 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "e9d47d3a-94d2-4bd1-944d-da7d1e8a6465", 2024-08-20T03:10:06.097 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-08-20T03:10:06.097 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:10:06.097 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:10:06.097 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:10:06.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.094+0000 7f12ca577640 1 -- 172.21.15.44:0/4209478512 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f12a80b3f50 msgr2=0x7f12a80b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:06.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.094+0000 7f12ca577640 1 --2- 172.21.15.44:0/4209478512 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f12a80b3f50 0x7f12a80b6410 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f12c411b0a0 tx=0x7f12b0006bd0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.094+0000 7f12ca577640 1 -- 172.21.15.44:0/4209478512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12c4109e80 msgr2=0x7f12c411b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:06.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.094+0000 7f12ca577640 1 --2- 172.21.15.44:0/4209478512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12c4109e80 0x7f12c411b350 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f12b4006010 tx=0x7f12b4004da0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.094+0000 7f12ca577640 1 -- 172.21.15.44:0/4209478512 shutdown_connections 2024-08-20T03:10:06.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.094+0000 7f12ca577640 1 --2- 172.21.15.44:0/4209478512 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f12a80b3f50 0x7f12a80b6410 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.094+0000 7f12ca577640 1 --2- 172.21.15.44:0/4209478512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c411bdd0 0x7f12c4117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.094+0000 7f12ca577640 1 --2- 172.21.15.44:0/4209478512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12c410a260 0x7f12c411b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.094+0000 7f12ca577640 1 --2- 172.21.15.44:0/4209478512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f12c4109e80 0x7f12c411b350 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:06.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.094+0000 7f12ca577640 1 -- 172.21.15.44:0/4209478512 >> 172.21.15.44:0/4209478512 conn(0x7f12c406dcc0 msgr2=0x7f12c40728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:06.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.094+0000 7f12ca577640 1 -- 172.21.15.44:0/4209478512 shutdown_connections 2024-08-20T03:10:06.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:06.094+0000 7f12ca577640 1 -- 172.21.15.44:0/4209478512 wait complete. 2024-08-20T03:10:06.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:10:06.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:10:06.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:06.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:06.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:10:06.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:06.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:10:06.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:06.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:10:06.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:06.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:10:06.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:10:06.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:06.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:06.137 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:05 smithi165 ceph-mon[22357]: pgmap v550: 97 pgs: 97 active+clean; 873 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 617 op/s 2024-08-20T03:10:06.793 INFO:tasks.cephfs.filesystem:scrub status for tag:e9d47d3a-94d2-4bd1-944d-da7d1e8a6465 - {'path': '/', 'tag': 'e9d47d3a-94d2-4bd1-944d-da7d1e8a6465', 'options': 'recursive,force'} 2024-08-20T03:10:07.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:06 smithi044 ceph-mon[18790]: osdmap e80: 12 total, 12 up, 12 in 2024-08-20T03:10:07.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:06 smithi165 ceph-mon[22357]: osdmap e80: 12 total, 12 up, 12 in 2024-08-20T03:10:07.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:06 smithi099 ceph-mon[21984]: osdmap e80: 12 total, 12 up, 12 in 2024-08-20T03:10:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.648+0000 7fb50e3e6640 1 -- 172.21.15.44:0/4140593258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5000963f0 msgr2=0x7fb500004740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:07.650 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.648+0000 7fb50e3e6640 1 --2- 172.21.15.44:0/4140593258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5000963f0 0x7fb500004740 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009860 tx=0x7fb4f802a360 comp rx=0 tx=0).stop 2024-08-20T03:10:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.649+0000 7fb50e3e6640 1 -- 172.21.15.44:0/4140593258 shutdown_connections 2024-08-20T03:10:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.649+0000 7fb50e3e6640 1 --2- 172.21.15.44:0/4140593258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb50009aa90 0x7fb5000a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.649+0000 7fb50e3e6640 1 --2- 172.21.15.44:0/4140593258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb500004d60 0x7fb5000051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.649+0000 7fb50e3e6640 1 --2- 172.21.15.44:0/4140593258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5000963f0 0x7fb500004740 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.649+0000 7fb50e3e6640 1 -- 172.21.15.44:0/4140593258 >> 172.21.15.44:0/4140593258 conn(0x7fb50000a180 msgr2=0x7fb50000c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:07.655 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.653+0000 7fb50e3e6640 1 -- 172.21.15.44:0/4140593258 shutdown_connections 2024-08-20T03:10:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.656+0000 7fb50e3e6640 1 -- 172.21.15.44:0/4140593258 wait complete. 2024-08-20T03:10:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.656+0000 7fb50e3e6640 1 Processor -- start 2024-08-20T03:10:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.657+0000 7fb50e3e6640 1 -- start start 2024-08-20T03:10:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.657+0000 7fb50e3e6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb500004d60 0x7fb5000a8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.657+0000 7fb50e3e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5000963f0 0x7fb5000a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.657+0000 7fb50e3e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb50009aa90 0x7fb5000a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.657+0000 7fb50e3e6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb50009e7f0 con 0x7fb5000963f0 2024-08-20T03:10:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.657+0000 7fb50e3e6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb50009e670 con 0x7fb50009aa90 2024-08-20T03:10:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.657+0000 7fb50e3e6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb50009e970 con 0x7fb500004d60 2024-08-20T03:10:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.657+0000 7fb50dbe5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb50009aa90 0x7fb5000a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.657+0000 7fb50dbe5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb50009aa90 0x7fb5000a2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:39822/0 (socket says 172.21.15.44:39822) 2024-08-20T03:10:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.657+0000 7fb50dbe5640 1 -- 172.21.15.44:0/3746993397 learned_addr learned my addr 172.21.15.44:0/3746993397 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.658+0000 7fb50cbe3640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5000963f0 0x7fb5000a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.658+0000 7fb50dbe5640 1 -- 172.21.15.44:0/3746993397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb500004d60 msgr2=0x7fb5000a8200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-20T03:10:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.658+0000 7fb50dbe5640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb500004d60 0x7fb5000a8200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.658+0000 7fb50dbe5640 1 -- 172.21.15.44:0/3746993397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5000963f0 msgr2=0x7fb5000a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.658+0000 7fb50dbe5640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5000963f0 0x7fb5000a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.658+0000 7fb50dbe5640 1 -- 172.21.15.44:0/3746993397 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5000a2f90 con 0x7fb50009aa90 2024-08-20T03:10:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.659+0000 7fb50dbe5640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb50009aa90 0x7fb5000a2950 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb5080687d0 tx=0x7fb50806c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.660+0000 7fb4fe7fc640 1 -- 172.21.15.44:0/3746993397 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50807cba0 con 0x7fb50009aa90 2024-08-20T03:10:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.660+0000 7fb4fe7fc640 1 -- 172.21.15.44:0/3746993397 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb50807cd40 con 0x7fb50009aa90 2024-08-20T03:10:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.660+0000 7fb4fe7fc640 1 -- 172.21.15.44:0/3746993397 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508085630 con 0x7fb50009aa90 2024-08-20T03:10:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.662+0000 7fb50e3e6640 1 -- 172.21.15.44:0/3746993397 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb500147350 con 0x7fb50009aa90 2024-08-20T03:10:07.666 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.662+0000 7fb50e3e6640 1 -- 172.21.15.44:0/3746993397 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb500147700 con 0x7fb50009aa90 2024-08-20T03:10:07.667 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.665+0000 7fb51243b740 1 Processor -- start 2024-08-20T03:10:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.666+0000 7fb51243b740 1 -- start start 2024-08-20T03:10:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.666+0000 7fb51243b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0a3d40 0x5567eb0a4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.666+0000 7fb51243b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a4660 0x5567eb0b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.666+0000 7fb51243b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0b05f0 0x5567eb0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.666+0000 7fb51243b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5567eb019360 con 0x5567eb0a3d40 2024-08-20T03:10:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.666+0000 7fb51243b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5567eb008860 con 0x5567eb0b05f0 2024-08-20T03:10:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.666+0000 7fb51243b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5567eb00c8a0 con 0x5567eb0a4660 2024-08-20T03:10:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.667+0000 7fb50d3e4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a4660 0x5567eb0b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.667+0000 7fb50d3e4640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a4660 0x5567eb0b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:42542/0 (socket says 172.21.15.44:42542) 2024-08-20T03:10:07.669 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.667+0000 7fb50d3e4640 1 -- 172.21.15.44:0/1404703660 learned_addr learned my addr 172.21.15.44:0/1404703660 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.668+0000 7fb50d3e4640 1 -- 172.21.15.44:0/1404703660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0b05f0 msgr2=0x5567eb0b29e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:10:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.668+0000 7fb50d3e4640 1 --2- 172.21.15.44:0/1404703660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0b05f0 0x5567eb0b29e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.668+0000 7fb50d3e4640 1 -- 172.21.15.44:0/1404703660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0a3d40 msgr2=0x5567eb0a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.668+0000 7fb50d3e4640 1 --2- 172.21.15.44:0/1404703660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0a3d40 0x5567eb0a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.668+0000 7fb50d3e4640 1 -- 172.21.15.44:0/1404703660 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567eafafe40 con 0x5567eb0a4660 2024-08-20T03:10:07.670 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.668+0000 7fb4fe7fc640 1 -- 172.21.15.44:0/3746993397 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb5080857d0 con 0x7fb50009aa90 2024-08-20T03:10:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.669+0000 7fb4fe7fc640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb4f40b3f50 0x7fb4f40b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.669+0000 7fb50d3e4640 1 --2- 172.21.15.44:0/1404703660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a4660 0x5567eb0b00b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb4f802a8d0 tx=0x7fb4f8030360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96e49a0518e77ac4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.670+0000 7fb4fe7fc640 1 -- 172.21.15.44:0/3746993397 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8493+0+0 (secure 0 0 0) 0x7fb508148c90 con 0x7fb50009aa90 2024-08-20T03:10:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.671+0000 7fb4f37fe640 1 -- 172.21.15.44:0/1404703660 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8040020 con 0x5567eb0a4660 2024-08-20T03:10:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.671+0000 7fb4f37fe640 1 -- 172.21.15.44:0/1404703660 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f8038040 con 0x5567eb0a4660 2024-08-20T03:10:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.671+0000 7fb50cbe3640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb4f40b3f50 0x7fb4f40b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.671+0000 7fb4f37fe640 1 -- 172.21.15.44:0/1404703660 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8037070 con 0x5567eb0a4660 2024-08-20T03:10:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.672+0000 7fb51243b740 1 -- 172.21.15.44:0/1404703660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a4660 msgr2=0x5567eb0b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.672+0000 7fb51243b740 1 --2- 172.21.15.44:0/1404703660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a4660 0x5567eb0b00b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb4f802a8d0 tx=0x7fb4f8030360 comp rx=0 tx=0).stop 2024-08-20T03:10:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.673+0000 7fb51243b740 1 -- 172.21.15.44:0/1404703660 shutdown_connections 2024-08-20T03:10:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.673+0000 7fb51243b740 1 --2- 172.21.15.44:0/1404703660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0b05f0 0x5567eb0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.673+0000 7fb51243b740 1 --2- 172.21.15.44:0/1404703660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a4660 0x5567eb0b00b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.673+0000 7fb51243b740 1 --2- 172.21.15.44:0/1404703660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0a3d40 0x5567eb0a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.673+0000 7fb51243b740 1 -- 172.21.15.44:0/1404703660 >> 172.21.15.44:0/1404703660 conn(0x5567eb0a1b70 msgr2=0x5567eb0a2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.675+0000 7fb51243b740 1 -- 172.21.15.44:0/1404703660 shutdown_connections 2024-08-20T03:10:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.675+0000 7fb50cbe3640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb4f40b3f50 0x7fb4f40b6410 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb504006dc0 tx=0x7fb504008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.675+0000 7fb51243b740 1 -- 172.21.15.44:0/1404703660 wait complete. 2024-08-20T03:10:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.675+0000 7fb51243b740 1 -- start start 2024-08-20T03:10:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.676+0000 7fb51243b740 1 Processor -- start 2024-08-20T03:10:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.676+0000 7fb51243b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a3d40 0x5567eb0ad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.676+0000 7fb51243b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0b05f0 0x5567eb0adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.676+0000 7fb51243b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.677+0000 7fb51243b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5567eaf41ab0 con 0x5567eb0b05f0 2024-08-20T03:10:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.677+0000 7fb51243b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5567eaf8f780 con 0x5567eb0ae010 2024-08-20T03:10:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.677+0000 7fb51243b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5567eb0c42d0 con 0x5567eb0a3d40 2024-08-20T03:10:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.677+0000 7fb51243b740 10 client.0 _ll_register_callbacks cb 0x5567ead17a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.677+0000 7fb50dbe5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.677+0000 7fb50dbe5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:39836/0 (socket says 172.21.15.44:39836) 2024-08-20T03:10:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.677+0000 7fb50dbe5640 1 -- 172.21.15.44:0/2724646892 learned_addr learned my addr 172.21.15.44:0/2724646892 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.677+0000 7fb50d3e4640 1 --2- 172.21.15.44:0/2724646892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a3d40 0x5567eb0ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.678+0000 7fb50d3e4640 1 -- 172.21.15.44:0/2724646892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 msgr2=0x5567eb0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.678+0000 7fb50d3e4640 1 --2- 172.21.15.44:0/2724646892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.678+0000 7fb50d3e4640 1 -- 172.21.15.44:0/2724646892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0b05f0 msgr2=0x5567eb0adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.678+0000 7fb50d3e4640 1 --2- 172.21.15.44:0/2724646892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0b05f0 0x5567eb0adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.678+0000 7fb50d3e4640 1 -- 172.21.15.44:0/2724646892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567ead6b790 con 0x5567eb0a3d40 2024-08-20T03:10:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.679+0000 7fb50d3e4640 1 --2- 172.21.15.44:0/2724646892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a3d40 0x5567eb0ad590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5567eb0a2850 tx=0x7fb4f8030a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b45392c2e46651e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:07.681 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.679+0000 7fb4f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:10:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.680+0000 7fb4f0ff9640 1 -- 172.21.15.44:0/2724646892 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f804a060 con 0x5567eb0a3d40 2024-08-20T03:10:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.680+0000 7fb51243b740 1 -- 172.21.15.44:0/2724646892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5567eafb7e00 con 0x5567eb0a3d40 2024-08-20T03:10:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.681+0000 7fb4f0ff9640 1 -- 172.21.15.44:0/2724646892 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8493+0+0 (secure 0 0 0) 0x7fb4f8004450 con 0x5567eb0a3d40 2024-08-20T03:10:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.681+0000 7fb4f0ff9640 1 -- 172.21.15.44:0/2724646892 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb4f8040030 con 0x5567eb0a3d40 2024-08-20T03:10:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.681+0000 7fb51243b740 10 client.26500 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:07.683 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.681+0000 7fb51243b740 1 -- 172.21.15.44:0/2724646892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5567eade4620 con 0x5567eb0a3d40 2024-08-20T03:10:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.684+0000 7fb4f0ff9640 1 -- 172.21.15.44:0/2724646892 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5567eafb7e00 con 0x5567eb0a3d40 2024-08-20T03:10:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.684+0000 7fb4f0ff9640 1 -- 172.21.15.44:0/2724646892 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7fb4f8004bc0 con 0x5567eb0a3d40 2024-08-20T03:10:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.685+0000 7fb51243b740 10 client.26500 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.685+0000 7fb51243b740 10 client.26500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.685+0000 7fb51243b740 20 client.26500 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.685+0000 7fb51243b740 1 --2- 172.21.15.44:0/2724646892 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5567eb0d16b0 0x5567eb0d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.685+0000 7fb51243b740 4 client.26500 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.685+0000 7fb51243b740 1 -- 172.21.15.44:0/2724646892 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567eb01b2b0 con 0x5567eb0d16b0 2024-08-20T03:10:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.686+0000 7fb50cbe3640 1 --2- 172.21.15.44:0/2724646892 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5567eb0d16b0 0x5567eb0d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.689+0000 7fb50cbe3640 1 --2- 172.21.15.44:0/2724646892 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5567eb0d16b0 0x5567eb0d3b30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df9a185f7d075ff4 server_cookie=1d05207a031a30af in_seq=0 out_seq=0 2024-08-20T03:10:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.689+0000 7fb4f0ff9640 10 client.26500 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.690+0000 7fb4f0ff9640 1 -- 172.21.15.44:0/2724646892 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567eb01b2b0 con 0x5567eb0d16b0 2024-08-20T03:10:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.690+0000 7fb4f0ff9640 10 client.26500 handle_command_reply: tid=0 2024-08-20T03:10:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.691+0000 7fb51243b740 1 client.26500 shutdown 2024-08-20T03:10:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.698+0000 7fb51243b740 1 -- 172.21.15.44:0/2724646892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a3d40 msgr2=0x5567eb0ad590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.698+0000 7fb51243b740 1 --2- 172.21.15.44:0/2724646892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a3d40 0x5567eb0ad590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x5567eb0a2850 tx=0x7fb4f8030a80 comp rx=0 tx=0).stop 2024-08-20T03:10:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.699+0000 7fb51243b740 1 -- 172.21.15.44:0/2724646892 shutdown_connections 2024-08-20T03:10:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.699+0000 7fb51243b740 1 --2- 172.21.15.44:0/2724646892 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5567eb0d16b0 0x5567eb0d3b30 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.699+0000 7fb51243b740 1 --2- 172.21.15.44:0/2724646892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.699+0000 7fb51243b740 1 --2- 172.21.15.44:0/2724646892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0b05f0 0x5567eb0adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.699+0000 7fb51243b740 1 --2- 172.21.15.44:0/2724646892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a3d40 0x5567eb0ad590 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.699+0000 7fb51243b740 1 -- 172.21.15.44:0/2724646892 >> 172.21.15.44:0/2724646892 conn(0x5567eb0a1b70 msgr2=0x5567eb0a7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.700+0000 7fb4f0ff9640 0 client.26500 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.702+0000 7fb51243b740 1 -- 172.21.15.44:0/2724646892 shutdown_connections 2024-08-20T03:10:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.704+0000 7fb51243b740 1 -- 172.21.15.44:0/2724646892 wait complete. 2024-08-20T03:10:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.704+0000 7fb51243b740 20 client.26500 trim_cache size 0 max 16384 2024-08-20T03:10:07.734 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.732+0000 7fb51243b740 1 Processor -- start 2024-08-20T03:10:07.738 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.735+0000 7fb51243b740 1 -- start start 2024-08-20T03:10:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.736+0000 7fb51243b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0a3d40 0x5567eb0afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.736+0000 7fb51243b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0a1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.736+0000 7fb51243b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0b05f0 0x5567eb0cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.736+0000 7fb51243b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb50014feb0 con 0x5567eb0a3d40 2024-08-20T03:10:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.736+0000 7fb51243b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb500151b70 con 0x5567eb0ae010 2024-08-20T03:10:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.737+0000 7fb51243b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb500150610 con 0x5567eb0b05f0 2024-08-20T03:10:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.737+0000 7fb50dbe5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0b05f0 0x5567eb0cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.737+0000 7fb50dbe5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0b05f0 0x5567eb0cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:42560/0 (socket says 172.21.15.44:42560) 2024-08-20T03:10:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.737+0000 7fb50dbe5640 1 -- 172.21.15.44:0/2450928853 learned_addr learned my addr 172.21.15.44:0/2450928853 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.738+0000 7fb50d3e4640 1 --2- 172.21.15.44:0/2450928853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0a3d40 0x5567eb0afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.740 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.739+0000 7fb50cbe3640 1 --2- 172.21.15.44:0/2450928853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.739+0000 7fb50dbe5640 1 -- 172.21.15.44:0/2450928853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 msgr2=0x5567eb0a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.739+0000 7fb50dbe5640 1 --2- 172.21.15.44:0/2450928853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.739+0000 7fb50dbe5640 1 -- 172.21.15.44:0/2450928853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0a3d40 msgr2=0x5567eb0afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.739+0000 7fb50dbe5640 1 --2- 172.21.15.44:0/2450928853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0a3d40 0x5567eb0afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.739+0000 7fb50dbe5640 1 -- 172.21.15.44:0/2450928853 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567eb09bfe0 con 0x5567eb0b05f0 2024-08-20T03:10:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.740+0000 7fb50dbe5640 1 --2- 172.21.15.44:0/2450928853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0b05f0 0x5567eb0cd480 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb508146420 tx=0x7fb50810b830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=752bdf96d21aa715 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.741+0000 7fb4f2ffd640 1 -- 172.21.15.44:0/2450928853 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508118360 con 0x5567eb0b05f0 2024-08-20T03:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.741+0000 7fb4f2ffd640 1 -- 172.21.15.44:0/2450928853 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb50810f480 con 0x5567eb0b05f0 2024-08-20T03:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.741+0000 7fb4f2ffd640 1 -- 172.21.15.44:0/2450928853 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50812a260 con 0x5567eb0b05f0 2024-08-20T03:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.742+0000 7fb51243b740 1 -- 172.21.15.44:0/2450928853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0b05f0 msgr2=0x5567eb0cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.742+0000 7fb51243b740 1 --2- 172.21.15.44:0/2450928853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0b05f0 0x5567eb0cd480 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb508146420 tx=0x7fb50810b830 comp rx=0 tx=0).stop 2024-08-20T03:10:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.744+0000 7fb51243b740 1 -- 172.21.15.44:0/2450928853 shutdown_connections 2024-08-20T03:10:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.744+0000 7fb51243b740 1 --2- 172.21.15.44:0/2450928853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0b05f0 0x5567eb0cd480 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.744+0000 7fb51243b740 1 --2- 172.21.15.44:0/2450928853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.744+0000 7fb51243b740 1 --2- 172.21.15.44:0/2450928853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0a3d40 0x5567eb0afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.744+0000 7fb51243b740 1 -- 172.21.15.44:0/2450928853 >> 172.21.15.44:0/2450928853 conn(0x5567eb0a1b70 msgr2=0x5567eb0a76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.748+0000 7fb51243b740 1 -- 172.21.15.44:0/2450928853 shutdown_connections 2024-08-20T03:10:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.748+0000 7fb51243b740 1 -- 172.21.15.44:0/2450928853 wait complete. 2024-08-20T03:10:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.749+0000 7fb51243b740 1 -- start start 2024-08-20T03:10:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.749+0000 7fb51243b740 1 Processor -- start 2024-08-20T03:10:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.751+0000 7fb51243b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a3d40 0x5567eb0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.751+0000 7fb51243b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.751+0000 7fb51243b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0b05f0 0x5567eb0b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.753 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.751+0000 7fb51243b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb5001514a0 con 0x5567eb0b05f0 2024-08-20T03:10:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.751+0000 7fb51243b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb500150d70 con 0x5567eb0ae010 2024-08-20T03:10:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.751+0000 7fb51243b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb50014deb0 con 0x5567eb0a3d40 2024-08-20T03:10:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.751+0000 7fb51243b740 10 client.0 _ll_register_callbacks cb 0x5567eb0b0540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.752+0000 7fb50cbe3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.752+0000 7fb50cbe3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:39856/0 (socket says 172.21.15.44:39856) 2024-08-20T03:10:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.752+0000 7fb50cbe3640 1 -- 172.21.15.44:0/674499702 learned_addr learned my addr 172.21.15.44:0/674499702 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.752+0000 7fb50d3e4640 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a3d40 0x5567eb0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.752+0000 7fb50dbe5640 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0b05f0 0x5567eb0b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.754+0000 7fb50d3e4640 1 -- 172.21.15.44:0/674499702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 msgr2=0x5567eb0b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.754+0000 7fb50d3e4640 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.754+0000 7fb50d3e4640 1 -- 172.21.15.44:0/674499702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0b05f0 msgr2=0x5567eb0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.754+0000 7fb50d3e4640 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0b05f0 0x5567eb0b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.754+0000 7fb50d3e4640 1 -- 172.21.15.44:0/674499702 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567eb0c8720 con 0x5567eb0a3d40 2024-08-20T03:10:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.755+0000 7fb50d3e4640 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a3d40 0x5567eb0b0a70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8030af0 tx=0x7fb4f80059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=444f4b2fd26656e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.756+0000 7fb4f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:10:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.757+0000 7fb4f0ff9640 1 -- 172.21.15.44:0/674499702 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f804a040 con 0x5567eb0a3d40 2024-08-20T03:10:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.757+0000 7fb51243b740 1 -- 172.21.15.44:0/674499702 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb50810f480 con 0x5567eb0a3d40 2024-08-20T03:10:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.758+0000 7fb4f0ff9640 1 -- 172.21.15.44:0/674499702 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8493+0+0 (secure 0 0 0) 0x7fb4f80053f0 con 0x5567eb0a3d40 2024-08-20T03:10:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.759+0000 7fb4f0ff9640 1 -- 172.21.15.44:0/674499702 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb4f8037070 con 0x5567eb0a3d40 2024-08-20T03:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.762+0000 7fb51243b740 10 client.26506 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.762+0000 7fb51243b740 1 -- 172.21.15.44:0/674499702 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5567eb0c9a30 con 0x5567eb0a3d40 2024-08-20T03:10:07.765 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.763+0000 7fb4f0ff9640 1 -- 172.21.15.44:0/674499702 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb50810f480 con 0x5567eb0a3d40 2024-08-20T03:10:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.764+0000 7fb4f0ff9640 1 -- 172.21.15.44:0/674499702 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7fb4f805d030 con 0x5567eb0a3d40 2024-08-20T03:10:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.764+0000 7fb51243b740 10 client.26506 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.764+0000 7fb51243b740 10 client.26506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.764+0000 7fb51243b740 20 client.26506 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.764+0000 7fb51243b740 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5567eb13c3d0 0x5567eb13e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.764+0000 7fb51243b740 4 client.26506 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:10:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.764+0000 7fb51243b740 1 -- 172.21.15.44:0/674499702 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567eb016d50 con 0x5567eb13c3d0 2024-08-20T03:10:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.767+0000 7fb50cbe3640 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5567eb13c3d0 0x5567eb13e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.771+0000 7fb50cbe3640 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5567eb13c3d0 0x5567eb13e7c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dace21b73ada1679 server_cookie=48a9d2b467ea96cf in_seq=0 out_seq=0 2024-08-20T03:10:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:07.772+0000 7fb4f0ff9640 10 client.26506 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:08.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:07 smithi044 ceph-mon[18790]: pgmap v552: 97 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 84 active+clean; 873 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 484 op/s 2024-08-20T03:10:08.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:07 smithi165 ceph-mon[22357]: pgmap v552: 97 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 84 active+clean; 873 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 484 op/s 2024-08-20T03:10:08.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:07 smithi099 ceph-mon[21984]: pgmap v552: 97 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 84 active+clean; 873 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 484 op/s 2024-08-20T03:10:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb4f0ff9640 1 -- 172.21.15.44:0/674499702 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5567eb016d50 con 0x5567eb13c3d0 2024-08-20T03:10:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb4f0ff9640 10 client.26506 handle_command_reply: tid=0 2024-08-20T03:10:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb51243b740 1 client.26506 shutdown 2024-08-20T03:10:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb51243b740 1 -- 172.21.15.44:0/674499702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a3d40 msgr2=0x5567eb0b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb51243b740 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a3d40 0x5567eb0b0a70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8030af0 tx=0x7fb4f80059b0 comp rx=0 tx=0).stop 2024-08-20T03:10:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb51243b740 1 -- 172.21.15.44:0/674499702 shutdown_connections 2024-08-20T03:10:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb51243b740 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5567eb13c3d0 0x5567eb13e7c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb51243b740 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5567eb0b05f0 0x5567eb0b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb51243b740 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5567eb0ae010 0x5567eb0b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb51243b740 1 --2- 172.21.15.44:0/674499702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5567eb0a3d40 0x5567eb0b0a70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb51243b740 1 -- 172.21.15.44:0/674499702 >> 172.21.15.44:0/674499702 conn(0x5567eb0a1b70 msgr2=0x5567eb0aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb4f0ff9640 0 client.26506 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb51243b740 1 -- 172.21.15.44:0/674499702 shutdown_connections 2024-08-20T03:10:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.727+0000 7fb51243b740 1 -- 172.21.15.44:0/674499702 wait complete. 2024-08-20T03:10:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.728+0000 7fb51243b740 20 client.26506 trim_cache size 0 max 16384 2024-08-20T03:10:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:10:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:10:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:10:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:10:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.728+0000 7fb50e3e6640 1 -- 172.21.15.44:0/3746993397 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb4f40b3f50 msgr2=0x7fb4f40b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.729+0000 7fb50e3e6640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb4f40b3f50 0x7fb4f40b6410 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb504006dc0 tx=0x7fb504008040 comp rx=0 tx=0).stop 2024-08-20T03:10:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.729+0000 7fb50e3e6640 1 -- 172.21.15.44:0/3746993397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb50009aa90 msgr2=0x7fb5000a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.729+0000 7fb50e3e6640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb50009aa90 0x7fb5000a2950 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb5080687d0 tx=0x7fb50806c650 comp rx=0 tx=0).stop 2024-08-20T03:10:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.730+0000 7fb50e3e6640 1 -- 172.21.15.44:0/3746993397 shutdown_connections 2024-08-20T03:10:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.730+0000 7fb50e3e6640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb4f40b3f50 0x7fb4f40b6410 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.730+0000 7fb50e3e6640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb50009aa90 0x7fb5000a2950 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.730+0000 7fb50e3e6640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5000963f0 0x7fb5000a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.730+0000 7fb50e3e6640 1 --2- 172.21.15.44:0/3746993397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb500004d60 0x7fb5000a8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.730+0000 7fb50e3e6640 1 -- 172.21.15.44:0/3746993397 >> 172.21.15.44:0/3746993397 conn(0x7fb50000a180 msgr2=0x7fb5000924e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.730+0000 7fb50e3e6640 1 -- 172.21.15.44:0/3746993397 shutdown_connections 2024-08-20T03:10:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:08.730+0000 7fb50e3e6640 1 -- 172.21.15.44:0/3746993397 wait complete. 2024-08-20T03:10:10.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:09 smithi044 ceph-mon[18790]: pgmap v553: 97 pgs: 16 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 57 active+clean; 873 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 453 op/s 2024-08-20T03:10:10.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:09 smithi165 ceph-mon[22357]: pgmap v553: 97 pgs: 16 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 57 active+clean; 873 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 453 op/s 2024-08-20T03:10:10.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:09 smithi099 ceph-mon[21984]: pgmap v553: 97 pgs: 16 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 57 active+clean; 873 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 453 op/s 2024-08-20T03:10:12.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:11 smithi044 ceph-mon[18790]: pgmap v554: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 864 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 568 op/s 2024-08-20T03:10:12.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:11 smithi165 ceph-mon[22357]: pgmap v554: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 864 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 568 op/s 2024-08-20T03:10:12.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:11 smithi099 ceph-mon[21984]: pgmap v554: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 864 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 568 op/s 2024-08-20T03:10:14.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:13 smithi044 ceph-mon[18790]: pgmap v555: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 847 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 626 op/s 2024-08-20T03:10:14.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:13 smithi165 ceph-mon[22357]: pgmap v555: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 847 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 626 op/s 2024-08-20T03:10:14.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:13 smithi099 ceph-mon[21984]: pgmap v555: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 847 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 626 op/s 2024-08-20T03:10:16.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:15 smithi044 ceph-mon[18790]: pgmap v556: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 829 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 884 op/s 2024-08-20T03:10:16.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:15 smithi165 ceph-mon[22357]: pgmap v556: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 829 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 884 op/s 2024-08-20T03:10:16.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:15 smithi099 ceph-mon[21984]: pgmap v556: 97 pgs: 20 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 37 active+clean; 829 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 884 op/s 2024-08-20T03:10:18.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:17 smithi044 ceph-mon[18790]: pgmap v557: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 828 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 894 op/s 2024-08-20T03:10:18.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:17 smithi165 ceph-mon[22357]: pgmap v557: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 828 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 894 op/s 2024-08-20T03:10:18.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:17 smithi099 ceph-mon[21984]: pgmap v557: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 828 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 894 op/s 2024-08-20T03:10:19.966 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:19 smithi099 ceph-mon[21984]: pgmap v558: 97 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 49 active+clean; 833 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 919 op/s 2024-08-20T03:10:20.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:19 smithi044 ceph-mon[18790]: pgmap v558: 97 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 49 active+clean; 833 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 919 op/s 2024-08-20T03:10:20.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:19 smithi165 ceph-mon[22357]: pgmap v558: 97 pgs: 16 active+clean+snaptrim, 32 active+clean+snaptrim_wait, 49 active+clean; 833 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.8 MiB/s wr, 919 op/s 2024-08-20T03:10:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.861+0000 7f7f720a8640 1 -- 172.21.15.44:0/3284945720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c101920 msgr2=0x7f7f6c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.861+0000 7f7f720a8640 1 --2- 172.21.15.44:0/3284945720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c101920 0x7f7f6c101d00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009ca0 tx=0x7f7f5c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.861+0000 7f7f720a8640 1 -- 172.21.15.44:0/3284945720 shutdown_connections 2024-08-20T03:10:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.861+0000 7f7f720a8640 1 --2- 172.21.15.44:0/3284945720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c107c50 0x7f7f6c10e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.861+0000 7f7f720a8640 1 --2- 172.21.15.44:0/3284945720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f6c1072d0 0x7f7f6c107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.861+0000 7f7f720a8640 1 --2- 172.21.15.44:0/3284945720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c101920 0x7f7f6c101d00 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.861+0000 7f7f720a8640 1 -- 172.21.15.44:0/3284945720 >> 172.21.15.44:0/3284945720 conn(0x7f7f6c078620 msgr2=0x7f7f6c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.862+0000 7f7f720a8640 1 -- 172.21.15.44:0/3284945720 shutdown_connections 2024-08-20T03:10:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.862+0000 7f7f720a8640 1 -- 172.21.15.44:0/3284945720 wait complete. 2024-08-20T03:10:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.862+0000 7f7f720a8640 1 Processor -- start 2024-08-20T03:10:21.865 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.863+0000 7f7f720a8640 1 -- start start 2024-08-20T03:10:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.863+0000 7f7f720a8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c101920 0x7f7f6c19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.863+0000 7f7f720a8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f6c1072d0 0x7f7f6c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.863+0000 7f7f720a8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c107c50 0x7f7f6c1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.863+0000 7f7f720a8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c110850 con 0x7f7f6c101920 2024-08-20T03:10:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.863+0000 7f7f720a8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1106d0 con 0x7f7f6c1072d0 2024-08-20T03:10:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.863+0000 7f7f720a8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1109d0 con 0x7f7f6c107c50 2024-08-20T03:10:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.863+0000 7f7f6b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c101920 0x7f7f6c19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.863+0000 7f7f6b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c101920 0x7f7f6c19b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:40990/0 (socket says 172.21.15.44:40990) 2024-08-20T03:10:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.863+0000 7f7f6b7fe640 1 -- 172.21.15.44:0/4052257493 learned_addr learned my addr 172.21.15.44:0/4052257493 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.863+0000 7f7f6affd640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f6c1072d0 0x7f7f6c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.867 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.864+0000 7f7f6bfff640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c107c50 0x7f7f6c1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.864+0000 7f7f6b7fe640 1 -- 172.21.15.44:0/4052257493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c107c50 msgr2=0x7f7f6c1a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.864+0000 7f7f6b7fe640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c107c50 0x7f7f6c1a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.864+0000 7f7f6b7fe640 1 -- 172.21.15.44:0/4052257493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f6c1072d0 msgr2=0x7f7f6c19b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.864+0000 7f7f6b7fe640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f6c1072d0 0x7f7f6c19b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.864+0000 7f7f6b7fe640 1 -- 172.21.15.44:0/4052257493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c1a1240 con 0x7f7f6c101920 2024-08-20T03:10:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.864+0000 7f7f6bfff640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c107c50 0x7f7f6c1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:10:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.864+0000 7f7f6b7fe640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c101920 0x7f7f6c19b1a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c02aeb0 tx=0x7f7f5c0317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.865+0000 7f7f68ff9640 1 -- 172.21.15.44:0/4052257493 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c003b70 con 0x7f7f6c101920 2024-08-20T03:10:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.865+0000 7f7f720a8640 1 -- 172.21.15.44:0/4052257493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c1a14d0 con 0x7f7f6c101920 2024-08-20T03:10:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.865+0000 7f7f68ff9640 1 -- 172.21.15.44:0/4052257493 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f5c003d10 con 0x7f7f6c101920 2024-08-20T03:10:21.868 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.865+0000 7f7f68ff9640 1 -- 172.21.15.44:0/4052257493 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c02f9e0 con 0x7f7f6c101920 2024-08-20T03:10:21.869 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.865+0000 7f7f720a8640 1 -- 172.21.15.44:0/4052257493 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c1a7bf0 con 0x7f7f6c101920 2024-08-20T03:10:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.868+0000 7f7f68ff9640 1 -- 172.21.15.44:0/4052257493 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f7f5c004580 con 0x7f7f6c101920 2024-08-20T03:10:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.869+0000 7f7f74e9a740 1 Processor -- start 2024-08-20T03:10:21.871 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.869+0000 7f7f74e9a740 1 -- start start 2024-08-20T03:10:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.869+0000 7f7f74e9a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 0x55a389d2bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f74e9a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2c510 0x55a389d37f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f74e9a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 0x55a389d3a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f74e9a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a389ca1360 con 0x55a389d2bbf0 2024-08-20T03:10:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f74e9a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a389c90860 con 0x55a389d2c510 2024-08-20T03:10:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f74e9a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a389c948a0 con 0x55a389d384a0 2024-08-20T03:10:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f6bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 0x55a389d2bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f6bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 0x55a389d2bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41000/0 (socket says 172.21.15.44:41000) 2024-08-20T03:10:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f6bfff640 1 -- 172.21.15.44:0/1603652775 learned_addr learned my addr 172.21.15.44:0/1603652775 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f6b7fe640 1 --2- 172.21.15.44:0/1603652775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2c510 0x55a389d37f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f68ff9640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7f580b3df0 0x7f7f580b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.873 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f6bfff640 1 -- 172.21.15.44:0/1603652775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 msgr2=0x55a389d3a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f6bfff640 1 --2- 172.21.15.44:0/1603652775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 0x55a389d3a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f6bfff640 1 -- 172.21.15.44:0/1603652775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2c510 msgr2=0x55a389d37f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f6bfff640 1 --2- 172.21.15.44:0/1603652775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2c510 0x55a389d37f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f6bfff640 1 -- 172.21.15.44:0/1603652775 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a389c37e40 con 0x55a389d2bbf0 2024-08-20T03:10:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.870+0000 7f7f68ff9640 1 -- 172.21.15.44:0/4052257493 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8493+0+0 (secure 0 0 0) 0x7f7f5c105410 con 0x7f7f6c101920 2024-08-20T03:10:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.871+0000 7f7f6bfff640 1 --2- 172.21.15.44:0/1603652775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 0x55a389d2bfd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400bb00 tx=0x7f7f64008230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3af64ae9160087 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.871+0000 7f7f6bfff640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7f580b3df0 0x7f7f580b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.871+0000 7f7f61ffb640 1 -- 172.21.15.44:0/1603652775 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64014820 con 0x55a389d2bbf0 2024-08-20T03:10:21.874 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.871+0000 7f7f61ffb640 1 -- 172.21.15.44:0/1603652775 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f6401fd50 con 0x55a389d2bbf0 2024-08-20T03:10:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.871+0000 7f7f61ffb640 1 -- 172.21.15.44:0/1603652775 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64020ba0 con 0x55a389d2bbf0 2024-08-20T03:10:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.871+0000 7f7f6bfff640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7f580b3df0 0x7f7f580b62b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7f64008fd0 tx=0x7f7f64020000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.872+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1603652775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 msgr2=0x55a389d2bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.872+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1603652775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 0x55a389d2bfd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400bb00 tx=0x7f7f64008230 comp rx=0 tx=0).stop 2024-08-20T03:10:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.872+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1603652775 shutdown_connections 2024-08-20T03:10:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.872+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1603652775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 0x55a389d3a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.872+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1603652775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2c510 0x55a389d37f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.872+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1603652775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 0x55a389d2bfd0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.872+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1603652775 >> 172.21.15.44:0/1603652775 conn(0x55a389d29a20 msgr2=0x55a389d2abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.872+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1603652775 shutdown_connections 2024-08-20T03:10:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.873+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1603652775 wait complete. 2024-08-20T03:10:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.873+0000 7f7f74e9a740 1 -- start start 2024-08-20T03:10:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.873+0000 7f7f74e9a740 1 Processor -- start 2024-08-20T03:10:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.873+0000 7f7f74e9a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 0x55a389d37e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.873+0000 7f7f74e9a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2c510 0x55a389d35240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.873+0000 7f7f74e9a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 0x55a389d358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.873+0000 7f7f74e9a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a389bc9ab0 con 0x55a389d2bbf0 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.873+0000 7f7f74e9a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a389c17780 con 0x55a389d2c510 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.873+0000 7f7f74e9a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a389d3b180 con 0x55a389d384a0 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.873+0000 7f7f74e9a740 10 client.0 _ll_register_callbacks cb 0x55a38999fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f6b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 0x55a389d37e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f6b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 0x55a389d37e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41004/0 (socket says 172.21.15.44:41004) 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f6b7fe640 1 -- 172.21.15.44:0/1022241215 learned_addr learned my addr 172.21.15.44:0/1022241215 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f6affd640 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2c510 0x55a389d35240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f6b7fe640 1 -- 172.21.15.44:0/1022241215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 msgr2=0x55a389d358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f6b7fe640 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 0x55a389d358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f6b7fe640 1 -- 172.21.15.44:0/1022241215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2c510 msgr2=0x55a389d35240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f6b7fe640 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2c510 0x55a389d35240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f6b7fe640 1 -- 172.21.15.44:0/1022241215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3899f3790 con 0x55a389d2bbf0 2024-08-20T03:10:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f6affd640 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2c510 0x55a389d35240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:10:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f6b7fe640 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 0x55a389d37e00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c005e30 tx=0x7f7f5c103020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed33d7f01892b93 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f4affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f4affd640 1 -- 172.21.15.44:0/1022241215 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0cbec0 con 0x55a389d2bbf0 2024-08-20T03:10:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1022241215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a389c3fe00 con 0x55a389d2bbf0 2024-08-20T03:10:21.879 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.874+0000 7f7f4affd640 1 -- 172.21.15.44:0/1022241215 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8493+0+0 (secure 0 0 0) 0x7f7f5c0d4d10 con 0x55a389d2bbf0 2024-08-20T03:10:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.875+0000 7f7f4affd640 1 -- 172.21.15.44:0/1022241215 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7f5c0cb860 con 0x55a389d2bbf0 2024-08-20T03:10:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.875+0000 7f7f74e9a740 10 client.17127 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:21.880 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.875+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1022241215 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a389a6c620 con 0x55a389d2bbf0 2024-08-20T03:10:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.875+0000 7f7f4affd640 1 -- 172.21.15.44:0/1022241215 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a389c3fe00 con 0x55a389d2bbf0 2024-08-20T03:10:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.875+0000 7f7f4affd640 1 -- 172.21.15.44:0/1022241215 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f7f5c0cb350 con 0x55a389d2bbf0 2024-08-20T03:10:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.875+0000 7f7f74e9a740 10 client.17127 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.875+0000 7f7f74e9a740 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.875+0000 7f7f74e9a740 20 client.17127 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.875+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a389d47f00 0x55a389d4a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.875+0000 7f7f74e9a740 4 client.17127 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.875+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1022241215 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a389ca32b0 con 0x55a389d47f00 2024-08-20T03:10:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.876+0000 7f7f6affd640 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a389d47f00 0x55a389d4a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.877+0000 7f7f4affd640 10 client.17127 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.877+0000 7f7f6affd640 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a389d47f00 0x55a389d4a380 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b39a6fd471e0d658 server_cookie=4182674b2a8e8679 in_seq=0 out_seq=0 2024-08-20T03:10:21.881 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.877+0000 7f7f4affd640 1 -- 172.21.15.44:0/1022241215 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a389ca32b0 con 0x55a389d47f00 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.877+0000 7f7f4affd640 10 client.17127 handle_command_reply: tid=0 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f74e9a740 1 client.17127 shutdown 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1022241215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 msgr2=0x55a389d37e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 0x55a389d37e00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c005e30 tx=0x7f7f5c103020 comp rx=0 tx=0).stop 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1022241215 shutdown_connections 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a389d47f00 0x55a389d4a380 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 0x55a389d358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2c510 0x55a389d35240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1022241215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2bbf0 0x55a389d37e00 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1022241215 >> 172.21.15.44:0/1022241215 conn(0x55a389d29a20 msgr2=0x55a389d2f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f4affd640 0 client.17127 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1022241215 shutdown_connections 2024-08-20T03:10:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1022241215 wait complete. 2024-08-20T03:10:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.878+0000 7f7f74e9a740 20 client.17127 trim_cache size 0 max 16384 2024-08-20T03:10:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.905+0000 7f7f74e9a740 1 Processor -- start 2024-08-20T03:10:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.905+0000 7f7f74e9a740 1 -- start start 2024-08-20T03:10:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.905+0000 7f7f74e9a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2bbf0 0x55a389d24780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.905+0000 7f7f74e9a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2c510 0x55a389d442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.905+0000 7f7f74e9a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 0x55a389d292d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.905+0000 7f7f74e9a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b0650 con 0x55a389d2c510 2024-08-20T03:10:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.905+0000 7f7f74e9a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b2310 con 0x55a389d2bbf0 2024-08-20T03:10:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.905+0000 7f7f74e9a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b0db0 con 0x55a389d384a0 2024-08-20T03:10:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.906+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2c510 0x55a389d442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.906+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2c510 0x55a389d442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41014/0 (socket says 172.21.15.44:41014) 2024-08-20T03:10:21.908 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.906+0000 7f7f6affd640 1 -- 172.21.15.44:0/236022616 learned_addr learned my addr 172.21.15.44:0/236022616 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.906+0000 7f7f6b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2bbf0 0x55a389d24780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.906+0000 7f7f6bfff640 1 --2- 172.21.15.44:0/236022616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 0x55a389d292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.906+0000 7f7f6affd640 1 -- 172.21.15.44:0/236022616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 msgr2=0x55a389d292d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.906+0000 7f7f6affd640 1 --2- 172.21.15.44:0/236022616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 0x55a389d292d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.906+0000 7f7f6affd640 1 -- 172.21.15.44:0/236022616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2bbf0 msgr2=0x55a389d24780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.906+0000 7f7f6affd640 1 --2- 172.21.15.44:0/236022616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2bbf0 0x55a389d24780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.906+0000 7f7f6affd640 1 -- 172.21.15.44:0/236022616 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a389d3b060 con 0x55a389d2c510 2024-08-20T03:10:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.906+0000 7f7f6affd640 1 --2- 172.21.15.44:0/236022616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2c510 0x55a389d442a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7f54000c00 tx=0x7f7f54010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884c24d05cdb6b79 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.906+0000 7f7f617fa640 1 -- 172.21.15.44:0/236022616 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54004250 con 0x55a389d2c510 2024-08-20T03:10:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.907+0000 7f7f617fa640 1 -- 172.21.15.44:0/236022616 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f54009e20 con 0x55a389d2c510 2024-08-20T03:10:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.907+0000 7f7f617fa640 1 -- 172.21.15.44:0/236022616 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5401c7a0 con 0x55a389d2c510 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.908+0000 7f7f74e9a740 1 -- 172.21.15.44:0/236022616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2c510 msgr2=0x55a389d442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.908+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/236022616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2c510 0x55a389d442a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7f54000c00 tx=0x7f7f54010800 comp rx=0 tx=0).stop 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.908+0000 7f7f74e9a740 1 -- 172.21.15.44:0/236022616 shutdown_connections 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.908+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/236022616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d384a0 0x55a389d292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.908+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/236022616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2c510 0x55a389d442a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.908+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/236022616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d2bbf0 0x55a389d24780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.908+0000 7f7f74e9a740 1 -- 172.21.15.44:0/236022616 >> 172.21.15.44:0/236022616 conn(0x55a389d29a20 msgr2=0x55a389d33aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.908+0000 7f7f74e9a740 1 -- 172.21.15.44:0/236022616 shutdown_connections 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.908+0000 7f7f74e9a740 1 -- 172.21.15.44:0/236022616 wait complete. 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.908+0000 7f7f74e9a740 1 -- start start 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f74e9a740 1 Processor -- start 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f74e9a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d2bbf0 0x55a389d38e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f74e9a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2c510 0x55a389d393a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f74e9a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d384a0 0x55a389d3f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f74e9a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b1c40 con 0x55a389d2c510 2024-08-20T03:10:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f74e9a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1b1510 con 0x55a389d384a0 2024-08-20T03:10:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f74e9a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c1ae650 con 0x55a389d2bbf0 2024-08-20T03:10:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f6b7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d2bbf0 0x55a389d38e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f6b7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d2bbf0 0x55a389d38e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:60124/0 (socket says 172.21.15.44:60124) 2024-08-20T03:10:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2c510 0x55a389d393a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f6b7fe640 1 -- 172.21.15.44:0/1565589478 learned_addr learned my addr 172.21.15.44:0/1565589478 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f74e9a740 10 client.0 _ll_register_callbacks cb 0x7f7f5c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.909+0000 7f7f6bfff640 1 --2- 172.21.15.44:0/1565589478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d384a0 0x55a389d3f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.910+0000 7f7f6b7fe640 1 -- 172.21.15.44:0/1565589478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d384a0 msgr2=0x55a389d3f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.910+0000 7f7f6b7fe640 1 --2- 172.21.15.44:0/1565589478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d384a0 0x55a389d3f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.910+0000 7f7f6b7fe640 1 -- 172.21.15.44:0/1565589478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2c510 msgr2=0x55a389d393a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.910+0000 7f7f6b7fe640 1 --2- 172.21.15.44:0/1565589478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2c510 0x55a389d393a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.910+0000 7f7f6b7fe640 1 -- 172.21.15.44:0/1565589478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a389d3f9d0 con 0x55a389d2bbf0 2024-08-20T03:10:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.910+0000 7f7f6b7fe640 1 --2- 172.21.15.44:0/1565589478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d2bbf0 0x55a389d38e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0c8170 tx=0x7f7f5c003ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9170dd1b4bc75d2e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.910+0000 7f7f4affd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:10:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.911+0000 7f7f4affd640 1 -- 172.21.15.44:0/1565589478 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0cba40 con 0x55a389d2bbf0 2024-08-20T03:10:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.911+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1565589478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f54009e20 con 0x55a389d2bbf0 2024-08-20T03:10:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.911+0000 7f7f4affd640 1 -- 172.21.15.44:0/1565589478 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8493+0+0 (secure 0 0 0) 0x7f7f5c004d50 con 0x55a389d2bbf0 2024-08-20T03:10:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.911+0000 7f7f4affd640 1 -- 172.21.15.44:0/1565589478 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7f5c0cbc00 con 0x55a389d2bbf0 2024-08-20T03:10:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.911+0000 7f7f74e9a740 10 client.26515 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.911+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1565589478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a389d3f9d0 con 0x55a389d2bbf0 2024-08-20T03:10:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.912+0000 7f7f4affd640 1 -- 172.21.15.44:0/1565589478 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f54009e20 con 0x55a389d2bbf0 2024-08-20T03:10:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.912+0000 7f7f4affd640 1 -- 172.21.15.44:0/1565589478 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f7f5c0912a0 con 0x55a389d2bbf0 2024-08-20T03:10:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.912+0000 7f7f74e9a740 10 client.26515 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.912+0000 7f7f74e9a740 10 client.26515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.912+0000 7f7f74e9a740 20 client.26515 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.912+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1565589478 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a389dbb470 0x55a389dbd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.912+0000 7f7f74e9a740 4 client.26515 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:10:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.912+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1565589478 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a389c9ed50 con 0x55a389dbb470 2024-08-20T03:10:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.912+0000 7f7f6affd640 1 --2- 172.21.15.44:0/1565589478 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a389dbb470 0x55a389dbd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.913+0000 7f7f4affd640 10 client.26515 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:21.913+0000 7f7f6affd640 1 --2- 172.21.15.44:0/1565589478 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a389dbb470 0x55a389dbd860 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa5d4cba1d76d47 server_cookie=ce7c1a9eb182319b in_seq=0 out_seq=0 2024-08-20T03:10:22.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:21 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:22.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:21 smithi044 ceph-mon[18790]: pgmap v559: 97 pgs: 15 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 54 active+clean; 831 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 894 op/s 2024-08-20T03:10:22.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:21 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:22.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:21 smithi165 ceph-mon[22357]: pgmap v559: 97 pgs: 15 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 54 active+clean; 831 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 894 op/s 2024-08-20T03:10:22.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:21 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:22.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:21 smithi099 ceph-mon[21984]: pgmap v559: 97 pgs: 15 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 54 active+clean; 831 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 894 op/s 2024-08-20T03:10:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.397+0000 7f7f4affd640 1 -- 172.21.15.44:0/1565589478 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a389c9ed50 con 0x55a389dbb470 2024-08-20T03:10:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.397+0000 7f7f4affd640 10 client.26515 handle_command_reply: tid=0 2024-08-20T03:10:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.397+0000 7f7f74e9a740 1 client.26515 shutdown 2024-08-20T03:10:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.397+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1565589478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d2bbf0 msgr2=0x55a389d38e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.397+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1565589478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d2bbf0 0x55a389d38e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0c8170 tx=0x7f7f5c003ec0 comp rx=0 tx=0).stop 2024-08-20T03:10:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.397+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1565589478 shutdown_connections 2024-08-20T03:10:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.397+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1565589478 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a389dbb470 0x55a389dbd860 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.397+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1565589478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a389d384a0 0x55a389d3f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.397+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1565589478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a389d2c510 0x55a389d393a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.397+0000 7f7f4affd640 0 client.26515 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.397+0000 7f7f74e9a740 1 --2- 172.21.15.44:0/1565589478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a389d2bbf0 0x55a389d38e60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.397+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1565589478 >> 172.21.15.44:0/1565589478 conn(0x55a389d29a20 msgr2=0x55a389d33e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.398+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1565589478 shutdown_connections 2024-08-20T03:10:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.398+0000 7f7f74e9a740 1 -- 172.21.15.44:0/1565589478 wait complete. 2024-08-20T03:10:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.398+0000 7f7f74e9a740 20 client.26515 trim_cache size 0 max 16384 2024-08-20T03:10:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:10:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:10:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:10:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:10:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.398+0000 7f7f720a8640 1 -- 172.21.15.44:0/4052257493 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7f580b3df0 msgr2=0x7f7f580b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.398+0000 7f7f720a8640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7f580b3df0 0x7f7f580b62b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7f64008fd0 tx=0x7f7f64020000 comp rx=0 tx=0).stop 2024-08-20T03:10:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.398+0000 7f7f720a8640 1 -- 172.21.15.44:0/4052257493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c101920 msgr2=0x7f7f6c19b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.398+0000 7f7f720a8640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c101920 0x7f7f6c19b1a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c02aeb0 tx=0x7f7f5c0317c0 comp rx=0 tx=0).stop 2024-08-20T03:10:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.398+0000 7f7f720a8640 1 -- 172.21.15.44:0/4052257493 shutdown_connections 2024-08-20T03:10:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.398+0000 7f7f720a8640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f7f580b3df0 0x7f7f580b62b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.398+0000 7f7f720a8640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f6c107c50 0x7f7f6c1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.398+0000 7f7f720a8640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f6c1072d0 0x7f7f6c19b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.398+0000 7f7f720a8640 1 --2- 172.21.15.44:0/4052257493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f6c101920 0x7f7f6c19b1a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.399+0000 7f7f720a8640 1 -- 172.21.15.44:0/4052257493 >> 172.21.15.44:0/4052257493 conn(0x7f7f6c078620 msgr2=0x7f7f6c105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.399+0000 7f7f720a8640 1 -- 172.21.15.44:0/4052257493 shutdown_connections 2024-08-20T03:10:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:23.399+0000 7f7f720a8640 1 -- 172.21.15.44:0/4052257493 wait complete. 2024-08-20T03:10:23.959 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:23 smithi165 ceph-mon[22357]: pgmap v560: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 830 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 830 op/s 2024-08-20T03:10:24.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:23 smithi044 ceph-mon[18790]: pgmap v560: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 830 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 830 op/s 2024-08-20T03:10:24.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:23 smithi099 ceph-mon[21984]: pgmap v560: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 830 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 830 op/s 2024-08-20T03:10:26.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:25 smithi044 ceph-mon[18790]: pgmap v561: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 831 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 973 op/s 2024-08-20T03:10:26.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:25 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:10:26.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:25 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:10:26.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:25 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:10:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:25 smithi165 ceph-mon[22357]: pgmap v561: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 831 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 973 op/s 2024-08-20T03:10:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:25 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:10:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:25 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:10:26.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:25 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:10:26.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:25 smithi099 ceph-mon[21984]: pgmap v561: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 831 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 973 op/s 2024-08-20T03:10:26.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:25 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:10:26.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:25 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:10:26.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:25 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:10:27.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:26 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:10:27.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:26 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:10:27.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:26 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:10:27.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:26 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:10:27.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:26 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:10:27.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:26 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:10:28.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:27 smithi044 ceph-mon[18790]: pgmap v562: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 833 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 722 op/s 2024-08-20T03:10:28.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:27 smithi165 ceph-mon[22357]: pgmap v562: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 833 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 722 op/s 2024-08-20T03:10:28.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:27 smithi099 ceph-mon[21984]: pgmap v562: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 833 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 722 op/s 2024-08-20T03:10:30.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:29 smithi044 ceph-mon[18790]: pgmap v563: 97 pgs: 97 active+clean; 823 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 728 op/s 2024-08-20T03:10:30.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:29 smithi165 ceph-mon[22357]: pgmap v563: 97 pgs: 97 active+clean; 823 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 728 op/s 2024-08-20T03:10:30.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:29 smithi099 ceph-mon[21984]: pgmap v563: 97 pgs: 97 active+clean; 823 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 728 op/s 2024-08-20T03:10:32.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:31 smithi044 ceph-mon[18790]: pgmap v564: 97 pgs: 97 active+clean; 824 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 625 op/s 2024-08-20T03:10:32.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:31 smithi165 ceph-mon[22357]: pgmap v564: 97 pgs: 97 active+clean; 824 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 625 op/s 2024-08-20T03:10:32.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:31 smithi099 ceph-mon[21984]: pgmap v564: 97 pgs: 97 active+clean; 824 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 625 op/s 2024-08-20T03:10:33.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:32 smithi044 ceph-mon[18790]: osdmap e81: 12 total, 12 up, 12 in 2024-08-20T03:10:33.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:32 smithi165 ceph-mon[22357]: osdmap e81: 12 total, 12 up, 12 in 2024-08-20T03:10:33.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:32 smithi099 ceph-mon[21984]: osdmap e81: 12 total, 12 up, 12 in 2024-08-20T03:10:34.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:33 smithi044 ceph-mon[18790]: pgmap v566: 97 pgs: 97 active+clean; 834 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 524 op/s 2024-08-20T03:10:34.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:33 smithi165 ceph-mon[22357]: pgmap v566: 97 pgs: 97 active+clean; 834 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 524 op/s 2024-08-20T03:10:34.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:33 smithi099 ceph-mon[21984]: pgmap v566: 97 pgs: 97 active+clean; 834 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 524 op/s 2024-08-20T03:10:36.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:35 smithi165 ceph-mon[22357]: pgmap v567: 97 pgs: 97 active+clean; 838 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 386 op/s 2024-08-20T03:10:36.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:35 smithi099 ceph-mon[21984]: pgmap v567: 97 pgs: 97 active+clean; 838 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 386 op/s 2024-08-20T03:10:36.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:35 smithi044 ceph-mon[18790]: pgmap v567: 97 pgs: 97 active+clean; 838 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 386 op/s 2024-08-20T03:10:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.501+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1656126585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bf0108da0 msgr2=0x7f4bf010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.501+0000 7f4bf5acc640 1 --2- 172.21.15.44:0/1656126585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bf0108da0 0x7f4bf010f5a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009ca0 tx=0x7f4be802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.502+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1656126585 shutdown_connections 2024-08-20T03:10:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.502+0000 7f4bf5acc640 1 --2- 172.21.15.44:0/1656126585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bf0108da0 0x7f4bf010f5a0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.502+0000 7f4bf5acc640 1 --2- 172.21.15.44:0/1656126585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bf0108420 0x7f4bf0108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.502+0000 7f4bf5acc640 1 --2- 172.21.15.44:0/1656126585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bf0102a20 0x7f4bf0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.502+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1656126585 >> 172.21.15.44:0/1656126585 conn(0x7f4bf00fe480 msgr2=0x7f4bf01008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.502+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1656126585 shutdown_connections 2024-08-20T03:10:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.502+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1656126585 wait complete. 2024-08-20T03:10:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.502+0000 7f4bf5acc640 1 Processor -- start 2024-08-20T03:10:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.502+0000 7f4bf5acc640 1 -- start start 2024-08-20T03:10:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4bf5acc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bf0102a20 0x7f4bf010e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4bf5acc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bf0108420 0x7f4bf010ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4bf5acc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bf0108da0 0x7f4bf019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4bf5acc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0112a20 con 0x7f4bf0108420 2024-08-20T03:10:36.506 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4bf5acc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01128a0 con 0x7f4bf0108da0 2024-08-20T03:10:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4bf5acc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0112ba0 con 0x7f4bf0102a20 2024-08-20T03:10:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bf0108420 0x7f4bf010ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bf0108420 0x7f4bf010ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48628/0 (socket says 172.21.15.44:48628) 2024-08-20T03:10:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4be7fff640 1 -- 172.21.15.44:0/1926289262 learned_addr learned my addr 172.21.15.44:0/1926289262 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bf0102a20 0x7f4bf010e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4be7fff640 1 -- 172.21.15.44:0/1926289262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bf0102a20 msgr2=0x7f4bf010e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4be7fff640 1 --2- 172.21.15.44:0/1926289262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bf0102a20 0x7f4bf010e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4be7fff640 1 -- 172.21.15.44:0/1926289262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bf0108da0 msgr2=0x7f4bf019c5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T03:10:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4be7fff640 1 --2- 172.21.15.44:0/1926289262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bf0108da0 0x7f4bf019c5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.503+0000 7f4be7fff640 1 -- 172.21.15.44:0/1926289262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf019cd90 con 0x7f4bf0108420 2024-08-20T03:10:36.507 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.504+0000 7f4be7fff640 1 --2- 172.21.15.44:0/1926289262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bf0108420 0x7f4bf010ec90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800d770 tx=0x7f4bd800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.504+0000 7f4bed7fa640 1 -- 172.21.15.44:0/1926289262 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8017070 con 0x7f4bf0108420 2024-08-20T03:10:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.504+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1926289262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf019d080 con 0x7f4bf0108420 2024-08-20T03:10:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.504+0000 7f4bed7fa640 1 -- 172.21.15.44:0/1926289262 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bd80054d0 con 0x7f4bf0108420 2024-08-20T03:10:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.504+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1926289262 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf01a3760 con 0x7f4bf0108420 2024-08-20T03:10:36.508 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.505+0000 7f4bed7fa640 1 -- 172.21.15.44:0/1926289262 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8004d00 con 0x7f4bf0108420 2024-08-20T03:10:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.508+0000 7f4bf88b9740 1 Processor -- start 2024-08-20T03:10:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.509+0000 7f4bed7fa640 1 -- 172.21.15.44:0/1926289262 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4bd8004ea0 con 0x7f4bf0108420 2024-08-20T03:10:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.510+0000 7f4bf88b9740 1 -- start start 2024-08-20T03:10:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.510+0000 7f4bf88b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56379335cbf0 0x56379335cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.510+0000 7f4bf88b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335d510 0x563793368f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.510+0000 7f4bf88b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637933694a0 0x56379336b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.510+0000 7f4bf88b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5637932d2360 con 0x56379335d510 2024-08-20T03:10:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.510+0000 7f4bf88b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5637932c1860 con 0x5637933694a0 2024-08-20T03:10:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.510+0000 7f4bf88b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5637932c58a0 con 0x56379335cbf0 2024-08-20T03:10:36.513 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.510+0000 7f4bed7fa640 1 --2- 172.21.15.44:0/1926289262 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4bdc0b3f40 0x7f4bdc0b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335d510 0x563793368f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335d510 0x563793368f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48634/0 (socket says 172.21.15.44:48634) 2024-08-20T03:10:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637933694a0 0x56379336b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4bef7fe640 1 -- 172.21.15.44:0/1251691395 learned_addr learned my addr 172.21.15.44:0/1251691395 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4bed7fa640 1 -- 172.21.15.44:0/1926289262 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f4bd8004aa0 con 0x7f4bf0108420 2024-08-20T03:10:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4beffff640 1 --2- 172.21.15.44:0/1251691395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56379335cbf0 0x56379335cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4bef7fe640 1 -- 172.21.15.44:0/1251691395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56379335cbf0 msgr2=0x56379335cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4bef7fe640 1 --2- 172.21.15.44:0/1251691395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56379335cbf0 0x56379335cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4bef7fe640 1 -- 172.21.15.44:0/1251691395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637933694a0 msgr2=0x56379336b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4bef7fe640 1 --2- 172.21.15.44:0/1251691395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637933694a0 0x56379336b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4bef7fe640 1 -- 172.21.15.44:0/1251691395 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563793268e40 con 0x56379335d510 2024-08-20T03:10:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4beffff640 1 --2- 172.21.15.44:0/1251691395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56379335cbf0 0x56379335cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:10:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4beffff640 1 --2- 172.21.15.44:0/1926289262 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4bdc0b3f40 0x7f4bdc0b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.511+0000 7f4bef7fe640 1 --2- 172.21.15.44:0/1251691395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335d510 0x563793368f60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009ce0 tx=0x7f4be0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff09e4bfc9167a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.512+0000 7f4beffff640 1 --2- 172.21.15.44:0/1926289262 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4bdc0b3f40 0x7f4bdc0b6400 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005e30 tx=0x7f4be8031590 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.512+0000 7f4be5ffb640 1 -- 172.21.15.44:0/1251691395 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0004220 con 0x56379335d510 2024-08-20T03:10:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.512+0000 7f4be5ffb640 1 -- 172.21.15.44:0/1251691395 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4be00043c0 con 0x56379335d510 2024-08-20T03:10:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.512+0000 7f4be5ffb640 1 -- 172.21.15.44:0/1251691395 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0019810 con 0x56379335d510 2024-08-20T03:10:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.513+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1251691395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335d510 msgr2=0x563793368f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.513+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/1251691395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335d510 0x563793368f60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009ce0 tx=0x7f4be0011810 comp rx=0 tx=0).stop 2024-08-20T03:10:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.513+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1251691395 shutdown_connections 2024-08-20T03:10:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.513+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/1251691395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5637933694a0 0x56379336b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.513+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/1251691395 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335d510 0x563793368f60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.513+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/1251691395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56379335cbf0 0x56379335cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.513+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1251691395 >> 172.21.15.44:0/1251691395 conn(0x56379335aa20 msgr2=0x56379335bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.513+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1251691395 shutdown_connections 2024-08-20T03:10:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.513+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1251691395 wait complete. 2024-08-20T03:10:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.514+0000 7f4bf88b9740 1 -- start start 2024-08-20T03:10:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.514+0000 7f4bf88b9740 1 Processor -- start 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.514+0000 7f4bf88b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335cbf0 0x563793368df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.514+0000 7f4bf88b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335d510 0x563793366230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.514+0000 7f4bf88b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637933694a0 0x563793366890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.514+0000 7f4bf88b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5637931faab0 con 0x56379335cbf0 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.514+0000 7f4bf88b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563793248780 con 0x56379335d510 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.514+0000 7f4bf88b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56379336c180 con 0x5637933694a0 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.514+0000 7f4bf88b9740 10 client.0 _ll_register_callbacks cb 0x563792fd0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335cbf0 0x563793368df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335d510 0x563793366230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4beffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637933694a0 0x563793366890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4be7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335d510 0x563793366230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52642/0 (socket says 172.21.15.44:52642) 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335cbf0 0x563793368df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48642/0 (socket says 172.21.15.44:48642) 2024-08-20T03:10:36.518 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4be7fff640 1 -- 172.21.15.44:0/2163112396 learned_addr learned my addr 172.21.15.44:0/2163112396 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4be7fff640 1 -- 172.21.15.44:0/2163112396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637933694a0 msgr2=0x563793366890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4be7fff640 1 --2- 172.21.15.44:0/2163112396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637933694a0 0x563793366890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4be7fff640 1 -- 172.21.15.44:0/2163112396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335cbf0 msgr2=0x563793368df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4be7fff640 1 --2- 172.21.15.44:0/2163112396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335cbf0 0x563793368df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4be7fff640 1 -- 172.21.15.44:0/2163112396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563793024790 con 0x56379335d510 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4be7fff640 1 --2- 172.21.15.44:0/2163112396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335d510 0x563793366230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80e1040 tx=0x7f4bd80203f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a851b9267969ec3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.515+0000 7f4bceffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.516+0000 7f4bceffd640 1 -- 172.21.15.44:0/2163112396 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8005680 con 0x56379335d510 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.516+0000 7f4bf88b9740 1 -- 172.21.15.44:0/2163112396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563793270e00 con 0x56379335d510 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.516+0000 7f4bceffd640 1 -- 172.21.15.44:0/2163112396 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f4bd80a6730 con 0x56379335d510 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.516+0000 7f4bceffd640 1 -- 172.21.15.44:0/2163112396 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4bd80aaba0 con 0x56379335d510 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.516+0000 7f4bf88b9740 10 client.26564 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.516+0000 7f4bf88b9740 1 -- 172.21.15.44:0/2163112396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56379309d620 con 0x56379335d510 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.516+0000 7f4bceffd640 1 -- 172.21.15.44:0/2163112396 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563793270e00 con 0x56379335d510 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.516+0000 7f4bceffd640 1 -- 172.21.15.44:0/2163112396 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f4bd8005190 con 0x56379335d510 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.516+0000 7f4bf88b9740 10 client.26564 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.516+0000 7f4bf88b9740 10 client.26564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.516+0000 7f4bf88b9740 20 client.26564 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.517+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/2163112396 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563793378d70 0x56379337b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.517+0000 7f4bf88b9740 4 client.26564 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.517+0000 7f4bf88b9740 1 -- 172.21.15.44:0/2163112396 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637932d42b0 con 0x563793378d70 2024-08-20T03:10:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.519+0000 7f4bef7fe640 1 --2- 172.21.15.44:0/2163112396 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563793378d70 0x56379337b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.520+0000 7f4bceffd640 10 client.26564 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.520+0000 7f4bef7fe640 1 --2- 172.21.15.44:0/2163112396 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563793378d70 0x56379337b160 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3731dd20789248cf server_cookie=5ffe4ed361da6cb3 in_seq=0 out_seq=0 2024-08-20T03:10:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.521+0000 7f4bceffd640 1 -- 172.21.15.44:0/2163112396 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637932d42b0 con 0x563793378d70 2024-08-20T03:10:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.521+0000 7f4bceffd640 10 client.26564 handle_command_reply: tid=0 2024-08-20T03:10:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.521+0000 7f4bf88b9740 1 client.26564 shutdown 2024-08-20T03:10:36.524 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.522+0000 7f4bf88b9740 1 -- 172.21.15.44:0/2163112396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335d510 msgr2=0x563793366230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.522+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/2163112396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335d510 0x563793366230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80e1040 tx=0x7f4bd80203f0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.522+0000 7f4bf88b9740 1 -- 172.21.15.44:0/2163112396 shutdown_connections 2024-08-20T03:10:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.522+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/2163112396 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563793378d70 0x56379337b160 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.522+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/2163112396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637933694a0 0x563793366890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.522+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/2163112396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335d510 0x563793366230 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.522+0000 7f4bceffd640 0 client.26564 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.522+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/2163112396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335cbf0 0x563793368df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.522+0000 7f4bf88b9740 1 -- 172.21.15.44:0/2163112396 >> 172.21.15.44:0/2163112396 conn(0x56379335aa20 msgr2=0x5637933605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.522+0000 7f4bf88b9740 1 -- 172.21.15.44:0/2163112396 shutdown_connections 2024-08-20T03:10:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.522+0000 7f4bf88b9740 1 -- 172.21.15.44:0/2163112396 wait complete. 2024-08-20T03:10:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.522+0000 7f4bf88b9740 20 client.26564 trim_cache size 0 max 16384 2024-08-20T03:10:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.544+0000 7f4bf88b9740 1 Processor -- start 2024-08-20T03:10:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.544+0000 7f4bf88b9740 1 -- start start 2024-08-20T03:10:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.544+0000 7f4bf88b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335cbf0 0x563793355780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.545+0000 7f4bf88b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335d510 0x5637933752a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.545+0000 7f4bf88b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637933694a0 0x563793380a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.545+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335cbf0 0x563793355780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.545+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335cbf0 0x563793355780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:52646/0 (socket says 172.21.15.44:52646) 2024-08-20T03:10:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.545+0000 7f4bef7fe640 1 -- 172.21.15.44:0/3016201562 learned_addr learned my addr 172.21.15.44:0/3016201562 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.545+0000 7f4bf88b9740 1 -- 172.21.15.44:0/3016201562 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01ac020 con 0x56379335d510 2024-08-20T03:10:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.545+0000 7f4bf88b9740 1 -- 172.21.15.44:0/3016201562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01adce0 con 0x56379335cbf0 2024-08-20T03:10:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.545+0000 7f4bf88b9740 1 -- 172.21.15.44:0/3016201562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01ac780 con 0x5637933694a0 2024-08-20T03:10:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.545+0000 7f4beffff640 1 --2- 172.21.15.44:0/3016201562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637933694a0 0x563793380a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.546+0000 7f4bef7fe640 1 -- 172.21.15.44:0/3016201562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637933694a0 msgr2=0x563793380a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.546+0000 7f4bef7fe640 1 --2- 172.21.15.44:0/3016201562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637933694a0 0x563793380a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.546+0000 7f4bef7fe640 1 -- 172.21.15.44:0/3016201562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335d510 msgr2=0x5637933752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.546+0000 7f4bef7fe640 1 --2- 172.21.15.44:0/3016201562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335d510 0x5637933752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.546+0000 7f4bef7fe640 1 -- 172.21.15.44:0/3016201562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563793354da0 con 0x56379335cbf0 2024-08-20T03:10:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.546+0000 7f4bef7fe640 1 --2- 172.21.15.44:0/3016201562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335cbf0 0x563793355780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4be0003f50 tx=0x7f4be0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79af58d0e724647a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.546+0000 7f4be57fa640 1 -- 172.21.15.44:0/3016201562 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0031070 con 0x56379335cbf0 2024-08-20T03:10:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.547+0000 7f4be57fa640 1 -- 172.21.15.44:0/3016201562 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4be0005430 con 0x56379335cbf0 2024-08-20T03:10:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.547+0000 7f4be57fa640 1 -- 172.21.15.44:0/3016201562 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be002b610 con 0x56379335cbf0 2024-08-20T03:10:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.548+0000 7f4bf88b9740 1 -- 172.21.15.44:0/3016201562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335cbf0 msgr2=0x563793355780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.548+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/3016201562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335cbf0 0x563793355780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4be0003f50 tx=0x7f4be0011810 comp rx=0 tx=0).stop 2024-08-20T03:10:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.548+0000 7f4bf88b9740 1 -- 172.21.15.44:0/3016201562 shutdown_connections 2024-08-20T03:10:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.548+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/3016201562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5637933694a0 0x563793380a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.548+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/3016201562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56379335d510 0x5637933752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.548+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/3016201562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335cbf0 0x563793355780 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.548+0000 7f4bf88b9740 1 -- 172.21.15.44:0/3016201562 >> 172.21.15.44:0/3016201562 conn(0x56379335aa20 msgr2=0x563793364a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.548+0000 7f4bf88b9740 1 -- 172.21.15.44:0/3016201562 shutdown_connections 2024-08-20T03:10:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.548+0000 7f4bf88b9740 1 -- 172.21.15.44:0/3016201562 wait complete. 2024-08-20T03:10:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.548+0000 7f4bf88b9740 1 -- start start 2024-08-20T03:10:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.549+0000 7f4bf88b9740 1 Processor -- start 2024-08-20T03:10:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.549+0000 7f4bf88b9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56379335cbf0 0x56379336b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.549+0000 7f4bf88b9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335d510 0x563793371460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.549+0000 7f4bf88b9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637933694a0 0x563793371bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.549+0000 7f4bf88b9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01ad610 con 0x5637933694a0 2024-08-20T03:10:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.550+0000 7f4bf88b9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01acee0 con 0x56379335d510 2024-08-20T03:10:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.550+0000 7f4bf88b9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf01aa020 con 0x56379335cbf0 2024-08-20T03:10:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.550+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56379335cbf0 0x56379336b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.550+0000 7f4bf88b9740 10 client.0 _ll_register_callbacks cb 0x7f4bd800d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.550+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56379335cbf0 0x56379336b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:36846/0 (socket says 172.21.15.44:36846) 2024-08-20T03:10:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.550+0000 7f4bef7fe640 1 -- 172.21.15.44:0/1703720171 learned_addr learned my addr 172.21.15.44:0/1703720171 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.550+0000 7f4be7fff640 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335d510 0x563793371460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.550+0000 7f4beffff640 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637933694a0 0x563793371bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.551+0000 7f4beffff640 1 -- 172.21.15.44:0/1703720171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56379335cbf0 msgr2=0x56379336b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.551+0000 7f4beffff640 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56379335cbf0 0x56379336b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.551+0000 7f4beffff640 1 -- 172.21.15.44:0/1703720171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335d510 msgr2=0x563793371460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.551+0000 7f4beffff640 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335d510 0x563793371460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.551+0000 7f4beffff640 1 -- 172.21.15.44:0/1703720171 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56379336c580 con 0x5637933694a0 2024-08-20T03:10:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.551+0000 7f4beffff640 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637933694a0 0x563793371bf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4be8004af0 tx=0x7f4be80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed534eea3b9125f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.551+0000 7f4bceffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.552+0000 7f4bceffd640 1 -- 172.21.15.44:0/1703720171 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be80307c0 con 0x5637933694a0 2024-08-20T03:10:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.552+0000 7f4bceffd640 1 -- 172.21.15.44:0/1703720171 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f4be8030960 con 0x5637933694a0 2024-08-20T03:10:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.552+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1703720171 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4be0005430 con 0x5637933694a0 2024-08-20T03:10:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.552+0000 7f4bceffd640 1 -- 172.21.15.44:0/1703720171 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4be8030b60 con 0x5637933694a0 2024-08-20T03:10:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.552+0000 7f4bf88b9740 10 client.17148 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.553+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1703720171 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5637933e3ed0 con 0x5637933694a0 2024-08-20T03:10:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.553+0000 7f4bceffd640 1 -- 172.21.15.44:0/1703720171 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4be8041040 con 0x5637933694a0 2024-08-20T03:10:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.553+0000 7f4bceffd640 1 -- 172.21.15.44:0/1703720171 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f4be8051070 con 0x5637933694a0 2024-08-20T03:10:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.553+0000 7f4bf88b9740 10 client.17148 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.553+0000 7f4bf88b9740 10 client.17148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.553+0000 7f4bf88b9740 20 client.17148 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.554+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5637933e4680 0x5637933e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.554+0000 7f4bf88b9740 4 client.17148 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:10:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.554+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1703720171 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4be0005430 con 0x5637933e4680 2024-08-20T03:10:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.554+0000 7f4bef7fe640 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5637933e4680 0x5637933e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.555+0000 7f4bef7fe640 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5637933e4680 0x5637933e6aa0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607304e6bcc3094e server_cookie=2eaf3f4e2c0de9bb in_seq=0 out_seq=0 2024-08-20T03:10:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:36.555+0000 7f4bceffd640 10 client.17148 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:36.794 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:10:37.080 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:10:37.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:36 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:10:37.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:36 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:10:37.243 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:36 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:10:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bceffd640 1 -- 172.21.15.44:0/1703720171 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4be0005430 con 0x5637933e4680 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bceffd640 10 client.17148 handle_command_reply: tid=0 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bf88b9740 1 client.17148 shutdown 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1703720171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637933694a0 msgr2=0x563793371bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637933694a0 0x563793371bf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4be8004af0 tx=0x7f4be80095c0 comp rx=0 tx=0).stop 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1703720171 shutdown_connections 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5637933e4680 0x5637933e6aa0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5637933694a0 0x563793371bf0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56379335d510 0x563793371460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bf88b9740 1 --2- 172.21.15.44:0/1703720171 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56379335cbf0 0x56379336b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1703720171 >> 172.21.15.44:0/1703720171 conn(0x56379335aa20 msgr2=0x563793368080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bceffd640 0 client.17148 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1703720171 shutdown_connections 2024-08-20T03:10:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bf88b9740 1 -- 172.21.15.44:0/1703720171 wait complete. 2024-08-20T03:10:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.321+0000 7f4bf88b9740 20 client.17148 trim_cache size 0 max 16384 2024-08-20T03:10:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:10:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:10:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:10:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:10:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.322+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1926289262 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4bdc0b3f40 msgr2=0x7f4bdc0b6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.322+0000 7f4bf5acc640 1 --2- 172.21.15.44:0/1926289262 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4bdc0b3f40 0x7f4bdc0b6400 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4be8005e30 tx=0x7f4be8031590 comp rx=0 tx=0).stop 2024-08-20T03:10:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.322+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1926289262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bf0108420 msgr2=0x7f4bf010ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.322+0000 7f4bf5acc640 1 --2- 172.21.15.44:0/1926289262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bf0108420 0x7f4bf010ec90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4bd800d770 tx=0x7f4bd800dc30 comp rx=0 tx=0).stop 2024-08-20T03:10:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.322+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1926289262 shutdown_connections 2024-08-20T03:10:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.322+0000 7f4bf5acc640 1 --2- 172.21.15.44:0/1926289262 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f4bdc0b3f40 0x7f4bdc0b6400 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.322+0000 7f4bf5acc640 1 --2- 172.21.15.44:0/1926289262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bf0108da0 0x7f4bf019c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.322+0000 7f4bf5acc640 1 --2- 172.21.15.44:0/1926289262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4bf0108420 0x7f4bf010ec90 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.322+0000 7f4bf5acc640 1 --2- 172.21.15.44:0/1926289262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bf0102a20 0x7f4bf010e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.322+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1926289262 >> 172.21.15.44:0/1926289262 conn(0x7f4bf00fe480 msgr2=0x7f4bf0106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.322+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1926289262 shutdown_connections 2024-08-20T03:10:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:37.322+0000 7f4bf5acc640 1 -- 172.21.15.44:0/1926289262 wait complete. 2024-08-20T03:10:38.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:37 smithi165 ceph-mon[22357]: pgmap v568: 97 pgs: 97 active+clean; 843 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 380 op/s 2024-08-20T03:10:38.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:10:38.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:10:38.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:10:38.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:37 smithi044 ceph-mon[18790]: pgmap v568: 97 pgs: 97 active+clean; 843 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 380 op/s 2024-08-20T03:10:38.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:10:38.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:10:38.145 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:10:38.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:37 smithi099 ceph-mon[21984]: pgmap v568: 97 pgs: 97 active+clean; 843 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 380 op/s 2024-08-20T03:10:38.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:10:38.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:10:38.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:10:38.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.229+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/3096068271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59c101990 msgr2=0x7fc59c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:38.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.229+0000 7fc5a0a4b640 1 --2- 172.21.15.44:0/3096068271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59c101990 0x7fc59c101d70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009900 tx=0x7fc58c02a410 comp rx=0 tx=0).stop 2024-08-20T03:10:38.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.229+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/3096068271 shutdown_connections 2024-08-20T03:10:38.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.229+0000 7fc5a0a4b640 1 --2- 172.21.15.44:0/3096068271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc59c107d20 0x7fc59c10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.229+0000 7fc5a0a4b640 1 --2- 172.21.15.44:0/3096068271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59c1073a0 0x7fc59c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.229+0000 7fc5a0a4b640 1 --2- 172.21.15.44:0/3096068271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59c101990 0x7fc59c101d70 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.229+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/3096068271 >> 172.21.15.44:0/3096068271 conn(0x7fc59c0fd400 msgr2=0x7fc59c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.229+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/3096068271 shutdown_connections 2024-08-20T03:10:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.229+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/3096068271 wait complete. 2024-08-20T03:10:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.230+0000 7fc5a0a4b640 1 Processor -- start 2024-08-20T03:10:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.230+0000 7fc5a0a4b640 1 -- start start 2024-08-20T03:10:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.230+0000 7fc5a0a4b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59c101990 0x7fc59c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.230+0000 7fc5a0a4b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59c1073a0 0x7fc59c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.230+0000 7fc5a0a4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc59c107d20 0x7fc59c1a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.230+0000 7fc5a0a4b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1119e0 con 0x7fc59c1073a0 2024-08-20T03:10:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.230+0000 7fc5a0a4b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c111860 con 0x7fc59c107d20 2024-08-20T03:10:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.230+0000 7fc5a0a4b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c111b60 con 0x7fc59c101990 2024-08-20T03:10:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc599d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59c1073a0 0x7fc59c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc599d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59c1073a0 0x7fc59c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46068/0 (socket says 172.21.15.44:46068) 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc59a59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59c101990 0x7fc59c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc599d9b640 1 -- 172.21.15.44:0/2389493506 learned_addr learned my addr 172.21.15.44:0/2389493506 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc59ad9d640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc59c107d20 0x7fc59c1a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc599d9b640 1 -- 172.21.15.44:0/2389493506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59c101990 msgr2=0x7fc59c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc599d9b640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59c101990 0x7fc59c199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc599d9b640 1 -- 172.21.15.44:0/2389493506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc59c107d20 msgr2=0x7fc59c1a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc599d9b640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc59c107d20 0x7fc59c1a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc599d9b640 1 -- 172.21.15.44:0/2389493506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c1a5370 con 0x7fc59c1073a0 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc599d9b640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59c1073a0 0x7fc59c19a2f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc59000e9f0 tx=0x7fc59000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc58b7fe640 1 -- 172.21.15.44:0/2389493506 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59000cd80 con 0x7fc59c1073a0 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/2389493506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59c1a5540 con 0x7fc59c1073a0 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc58b7fe640 1 -- 172.21.15.44:0/2389493506 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5900054d0 con 0x7fc59c1073a0 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc58b7fe640 1 -- 172.21.15.44:0/2389493506 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590010690 con 0x7fc59c1073a0 2024-08-20T03:10:38.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.231+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/2389493506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59c1a5a20 con 0x7fc59c1073a0 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.234+0000 7fc5a2e44740 1 Processor -- start 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.234+0000 7fc5a2e44740 1 -- start start 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.234+0000 7fc5a2e44740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e990020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.234+0000 7fc5a2e44740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e99bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.234+0000 7fc5a2e44740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e99e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.234+0000 7fc5a2e44740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a24e9028e0 con 0x55a24e990560 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc5a2e44740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a24e8f1de0 con 0x55a24e99c4f0 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc5a2e44740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a24e8f5e20 con 0x55a24e98fc40 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc59ad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e990020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc59ad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e990020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44674/0 (socket says 172.21.15.44:44674) 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc59ad9d640 1 -- 172.21.15.44:0/1387646472 learned_addr learned my addr 172.21.15.44:0/1387646472 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc59a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e99bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc58b7fe640 1 -- 172.21.15.44:0/2389493506 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc590004e90 con 0x7fc59c1073a0 2024-08-20T03:10:38.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc59ad9d640 1 -- 172.21.15.44:0/1387646472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 msgr2=0x55a24e99e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc59ad9d640 1 --2- 172.21.15.44:0/1387646472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e99e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc59ad9d640 1 -- 172.21.15.44:0/1387646472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 msgr2=0x55a24e99bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc59ad9d640 1 --2- 172.21.15.44:0/1387646472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e99bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc59ad9d640 1 -- 172.21.15.44:0/1387646472 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24e651650 con 0x55a24e98fc40 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.235+0000 7fc59a59c640 1 --2- 172.21.15.44:0/1387646472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e99bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.236+0000 7fc59ad9d640 1 --2- 172.21.15.44:0/1387646472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e990020 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc594009fd0 tx=0x7fc5940118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc8d210bb8b78011 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.236+0000 7fc588ff9640 1 -- 172.21.15.44:0/1387646472 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594004d20 con 0x55a24e98fc40 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.236+0000 7fc588ff9640 1 -- 172.21.15.44:0/1387646472 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc594004050 con 0x55a24e98fc40 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.236+0000 7fc588ff9640 1 -- 172.21.15.44:0/1387646472 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594005220 con 0x55a24e98fc40 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.236+0000 7fc58b7fe640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc5840b3d40 0x7fc5840b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.236+0000 7fc58b7fe640 1 -- 172.21.15.44:0/2389493506 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fc5900e4350 con 0x7fc59c1073a0 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.236+0000 7fc59a59c640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc5840b3d40 0x7fc5840b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.237+0000 7fc5a2e44740 1 -- 172.21.15.44:0/1387646472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 msgr2=0x55a24e990020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.237+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/1387646472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e990020 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc594009fd0 tx=0x7fc5940118d0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.237+0000 7fc5a2e44740 1 -- 172.21.15.44:0/1387646472 shutdown_connections 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.237+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/1387646472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e99e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.237+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/1387646472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e99bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.237+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/1387646472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e990020 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.237+0000 7fc5a2e44740 1 -- 172.21.15.44:0/1387646472 >> 172.21.15.44:0/1387646472 conn(0x55a24e98da70 msgr2=0x55a24e98ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.237+0000 7fc59a59c640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc5840b3d40 0x7fc5840b6200 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc58c031440 tx=0x7fc58c005b90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.237+0000 7fc5a2e44740 1 -- 172.21.15.44:0/1387646472 shutdown_connections 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.237+0000 7fc5a2e44740 1 -- 172.21.15.44:0/1387646472 wait complete. 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.237+0000 7fc5a2e44740 1 -- start start 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.237+0000 7fc5a2e44740 1 Processor -- start 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc5a2e44740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e999500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc5a2e44740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e999a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc5a2e44740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e999f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc5a2e44740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a24e754700 con 0x55a24e990560 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc5a2e44740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a24e8787c0 con 0x55a24e99c4f0 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc5a2e44740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a24e99f170 con 0x55a24e98fc40 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc5a2e44740 10 client.0 _ll_register_callbacks cb 0x55a24e8eb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc599d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e999a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc599d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e999a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46086/0 (socket says 172.21.15.44:46086) 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc599d9b640 1 -- 172.21.15.44:0/26431091 learned_addr learned my addr 172.21.15.44:0/26431091 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc59a59c640 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e999500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc59ad9d640 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e999f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc599d9b640 1 -- 172.21.15.44:0/26431091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 msgr2=0x55a24e999500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc599d9b640 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e999500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc599d9b640 1 -- 172.21.15.44:0/26431091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 msgr2=0x55a24e999f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:38.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc599d9b640 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e999f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc599d9b640 1 -- 172.21.15.44:0/26431091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24e9a85a0 con 0x55a24e990560 2024-08-20T03:10:38.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc59ad9d640 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e999f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:10:38.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc599d9b640 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e999a40 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc5900e20e0 tx=0x7fc5900aad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7748869497d6a706 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:38.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc581ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:38.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc581ffb640 1 -- 172.21.15.44:0/26431091 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590005180 con 0x55a24e990560 2024-08-20T03:10:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc581ffb640 1 -- 172.21.15.44:0/26431091 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fc5900b3dd0 con 0x55a24e990560 2024-08-20T03:10:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.238+0000 7fc5a2e44740 1 -- 172.21.15.44:0/26431091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a24e8a1100 con 0x55a24e990560 2024-08-20T03:10:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.239+0000 7fc581ffb640 1 -- 172.21.15.44:0/26431091 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc5900aaab0 con 0x55a24e990560 2024-08-20T03:10:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.239+0000 7fc5a2e44740 10 client.17160 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.239+0000 7fc5a2e44740 1 -- 172.21.15.44:0/26431091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a24e9aafb0 con 0x55a24e990560 2024-08-20T03:10:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.239+0000 7fc581ffb640 1 -- 172.21.15.44:0/26431091 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a24e8a1100 con 0x55a24e990560 2024-08-20T03:10:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.239+0000 7fc581ffb640 1 -- 172.21.15.44:0/26431091 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7fc5900a6910 con 0x55a24e990560 2024-08-20T03:10:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.239+0000 7fc5a2e44740 10 client.17160 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.239+0000 7fc5a2e44740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.239+0000 7fc5a2e44740 20 client.17160 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.239+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a24e9ac460 0x55a24e9ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.239+0000 7fc5a2e44740 4 client.17160 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.239+0000 7fc5a2e44740 1 -- 172.21.15.44:0/26431091 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a24e904830 con 0x55a24e9ac460 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.239+0000 7fc59a59c640 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a24e9ac460 0x55a24e9ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.240+0000 7fc59a59c640 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a24e9ac460 0x55a24e9ae850 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1dd6eadb9b20de server_cookie=15d52b08e4a463a6 in_seq=0 out_seq=0 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.240+0000 7fc581ffb640 10 client.17160 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc581ffb640 1 -- 172.21.15.44:0/26431091 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a24e904830 con 0x55a24e9ac460 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc581ffb640 10 client.17160 handle_command_reply: tid=0 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc5a2e44740 1 client.17160 shutdown 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc5a2e44740 1 -- 172.21.15.44:0/26431091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 msgr2=0x55a24e999a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e999a40 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc5900e20e0 tx=0x7fc5900aad70 comp rx=0 tx=0).stop 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc5a2e44740 1 -- 172.21.15.44:0/26431091 shutdown_connections 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a24e9ac460 0x55a24e9ae850 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e999f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e999a40 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc581ffb640 0 client.17160 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:38.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/26431091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e999500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc5a2e44740 1 -- 172.21.15.44:0/26431091 >> 172.21.15.44:0/26431091 conn(0x55a24e98da70 msgr2=0x55a24e993610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:38.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc5a2e44740 1 -- 172.21.15.44:0/26431091 shutdown_connections 2024-08-20T03:10:38.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc5a2e44740 1 -- 172.21.15.44:0/26431091 wait complete. 2024-08-20T03:10:38.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.241+0000 7fc5a2e44740 20 client.17160 trim_cache size 0 max 16384 2024-08-20T03:10:38.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.259+0000 7fc5a2e44740 1 Processor -- start 2024-08-20T03:10:38.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.259+0000 7fc5a2e44740 1 -- start start 2024-08-20T03:10:38.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.259+0000 7fc5a2e44740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e988570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.259+0000 7fc5a2e44740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e9b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.259+0000 7fc5a2e44740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e9b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.259+0000 7fc5a2e44740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1ac060 con 0x55a24e990560 2024-08-20T03:10:38.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.259+0000 7fc5a2e44740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1add20 con 0x55a24e99c4f0 2024-08-20T03:10:38.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.259+0000 7fc5a2e44740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1ac7c0 con 0x55a24e98fc40 2024-08-20T03:10:38.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.259+0000 7fc599d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e9b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.259+0000 7fc59a59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e988570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc59a59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e988570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44688/0 (socket says 172.21.15.44:44688) 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc59a59c640 1 -- 172.21.15.44:0/2642636146 learned_addr learned my addr 172.21.15.44:0/2642636146 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc599d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e9b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46098/0 (socket says 172.21.15.44:46098) 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc59ad9d640 1 --2- 172.21.15.44:0/2642636146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e9b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc59a59c640 1 -- 172.21.15.44:0/2642636146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 msgr2=0x55a24e9b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc59a59c640 1 --2- 172.21.15.44:0/2642636146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e9b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc59a59c640 1 -- 172.21.15.44:0/2642636146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 msgr2=0x55a24e9b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc59a59c640 1 --2- 172.21.15.44:0/2642636146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e9b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc59a59c640 1 -- 172.21.15.44:0/2642636146 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24e990390 con 0x55a24e98fc40 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc599d9b640 1 --2- 172.21.15.44:0/2642636146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e9b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc59ad9d640 1 --2- 172.21.15.44:0/2642636146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e9b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc59a59c640 1 --2- 172.21.15.44:0/2642636146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e988570 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0096c0 tx=0x7fc58c03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ece91d35c56255c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:38.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc583fff640 1 -- 172.21.15.44:0/2642636146 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c035040 con 0x55a24e98fc40 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc583fff640 1 -- 172.21.15.44:0/2642636146 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc58c0035f0 con 0x55a24e98fc40 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.260+0000 7fc583fff640 1 -- 172.21.15.44:0/2642636146 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0535f0 con 0x55a24e98fc40 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.261+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2642636146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 msgr2=0x55a24e988570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.261+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/2642636146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e988570 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0096c0 tx=0x7fc58c03f000 comp rx=0 tx=0).stop 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.261+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2642636146 shutdown_connections 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.261+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/2642636146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e99c4f0 0x55a24e9b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.261+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/2642636146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e990560 0x55a24e9b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.261+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/2642636146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e988570 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.261+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2642636146 >> 172.21.15.44:0/2642636146 conn(0x55a24e98da70 msgr2=0x55a24e99b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.261+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2642636146 shutdown_connections 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.261+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2642636146 wait complete. 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc5a2e44740 1 -- start start 2024-08-20T03:10:38.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc5a2e44740 1 Processor -- start 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc5a2e44740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e9a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc5a2e44740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e990560 0x55a24e9a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc5a2e44740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e99c4f0 0x55a24e9a2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc5a2e44740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1ad650 con 0x55a24e99c4f0 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc5a2e44740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1acf20 con 0x55a24e990560 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc5a2e44740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1aa080 con 0x55a24e98fc40 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc5a2e44740 10 client.0 _ll_register_callbacks cb 0x7fc59000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc59ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e99c4f0 0x55a24e9a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc59ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e99c4f0 0x55a24e9a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46102/0 (socket says 172.21.15.44:46102) 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc599d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e990560 0x55a24e9a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc59ad9d640 1 -- 172.21.15.44:0/2595214746 learned_addr learned my addr 172.21.15.44:0/2595214746 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc59a59c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e9a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc59ad9d640 1 -- 172.21.15.44:0/2595214746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 msgr2=0x55a24e9a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc59ad9d640 1 --2- 172.21.15.44:0/2595214746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e9a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc59ad9d640 1 -- 172.21.15.44:0/2595214746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e990560 msgr2=0x55a24e9a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc59ad9d640 1 --2- 172.21.15.44:0/2595214746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e990560 0x55a24e9a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc59ad9d640 1 -- 172.21.15.44:0/2595214746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a24e9a27e0 con 0x55a24e99c4f0 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.262+0000 7fc599d9b640 1 --2- 172.21.15.44:0/2595214746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e990560 0x55a24e9a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.263+0000 7fc59ad9d640 1 --2- 172.21.15.44:0/2595214746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e99c4f0 0x55a24e9a2220 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc594014020 tx=0x7fc594003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd809345128af8c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.263+0000 7fc581ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.263+0000 7fc581ffb640 1 -- 172.21.15.44:0/2595214746 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594005550 con 0x55a24e99c4f0 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.263+0000 7fc581ffb640 1 -- 172.21.15.44:0/2595214746 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fc5940056f0 con 0x55a24e99c4f0 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.263+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2595214746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc58c0035f0 con 0x55a24e99c4f0 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.263+0000 7fc581ffb640 1 -- 172.21.15.44:0/2595214746 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc5940058f0 con 0x55a24e99c4f0 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.263+0000 7fc5a2e44740 10 client.17166 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:38.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.263+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2595214746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a24ea16ea0 con 0x55a24e99c4f0 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.263+0000 7fc581ffb640 1 -- 172.21.15.44:0/2595214746 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc58c0035f0 con 0x55a24e99c4f0 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.263+0000 7fc581ffb640 1 -- 172.21.15.44:0/2595214746 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7fc594019550 con 0x55a24e99c4f0 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.264+0000 7fc5a2e44740 10 client.17166 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.264+0000 7fc5a2e44740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.264+0000 7fc5a2e44740 20 client.17166 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.264+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/2595214746 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a24ea17420 0x55a24ea19840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.264+0000 7fc5a2e44740 4 client.17166 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.264+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2595214746 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a24e9002d0 con 0x55a24ea17420 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.264+0000 7fc59a59c640 1 --2- 172.21.15.44:0/2595214746 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a24ea17420 0x55a24ea19840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.264+0000 7fc59a59c640 1 --2- 172.21.15.44:0/2595214746 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a24ea17420 0x55a24ea19840 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=304c6ff21361e6cb server_cookie=dee5918ce7daaee2 in_seq=0 out_seq=0 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.264+0000 7fc581ffb640 10 client.17166 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc581ffb640 1 -- 172.21.15.44:0/2595214746 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a24e9002d0 con 0x55a24ea17420 2024-08-20T03:10:38.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc581ffb640 10 client.17166 handle_command_reply: tid=0 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc5a2e44740 1 client.17166 shutdown 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2595214746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e99c4f0 msgr2=0x55a24e9a2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/2595214746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e99c4f0 0x55a24e9a2220 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc594014020 tx=0x7fc594003eb0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2595214746 shutdown_connections 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/2595214746 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a24ea17420 0x55a24ea19840 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/2595214746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a24e99c4f0 0x55a24e9a2220 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/2595214746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a24e990560 0x55a24e9a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc5a2e44740 1 --2- 172.21.15.44:0/2595214746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a24e98fc40 0x55a24e9a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2595214746 >> 172.21.15.44:0/2595214746 conn(0x55a24e98da70 msgr2=0x55a24e992ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc581ffb640 0 client.17166 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2595214746 shutdown_connections 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc5a2e44740 1 -- 172.21.15.44:0/2595214746 wait complete. 2024-08-20T03:10:38.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.265+0000 7fc5a2e44740 20 client.17166 trim_cache size 0 max 16384 2024-08-20T03:10:38.276 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:10:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-08-20T03:10:38.276 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-08-20T03:10:38.276 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:10:38.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.266+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/2389493506 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc5840b3d40 msgr2=0x7fc5840b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:38.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.266+0000 7fc5a0a4b640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc5840b3d40 0x7fc5840b6200 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc58c031440 tx=0x7fc58c005b90 comp rx=0 tx=0).stop 2024-08-20T03:10:38.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.266+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/2389493506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59c1073a0 msgr2=0x7fc59c19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:38.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.266+0000 7fc5a0a4b640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59c1073a0 0x7fc59c19a2f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc59000e9f0 tx=0x7fc59000edf0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.266+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/2389493506 shutdown_connections 2024-08-20T03:10:38.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.266+0000 7fc5a0a4b640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc5840b3d40 0x7fc5840b6200 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.266+0000 7fc5a0a4b640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc59c107d20 0x7fc59c1a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.266+0000 7fc5a0a4b640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59c1073a0 0x7fc59c19a2f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.266+0000 7fc5a0a4b640 1 --2- 172.21.15.44:0/2389493506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc59c101990 0x7fc59c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:38.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.266+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/2389493506 >> 172.21.15.44:0/2389493506 conn(0x7fc59c0fd400 msgr2=0x7fc59c0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:38.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.266+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/2389493506 shutdown_connections 2024-08-20T03:10:38.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:38.266+0000 7fc5a0a4b640 1 -- 172.21.15.44:0/2389493506 wait complete. 2024-08-20T03:10:39.046 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T03:10:39.047 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:10:39.317 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:10:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.855+0000 7f4058720640 1 -- 172.21.15.44:0/752359358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4050108500 msgr2=0x7f4050108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.855+0000 7f4058720640 1 --2- 172.21.15.44:0/752359358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4050108500 0x7f4050108940 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4040009900 tx=0x7f404002a410 comp rx=0 tx=0).stop 2024-08-20T03:10:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.855+0000 7f4058720640 1 -- 172.21.15.44:0/752359358 shutdown_connections 2024-08-20T03:10:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.855+0000 7f4058720640 1 --2- 172.21.15.44:0/752359358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4050108e80 0x7f405010f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.855+0000 7f4058720640 1 --2- 172.21.15.44:0/752359358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4050108500 0x7f4050108940 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.855+0000 7f4058720640 1 --2- 172.21.15.44:0/752359358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4050102af0 0x7f4050102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.855+0000 7f4058720640 1 -- 172.21.15.44:0/752359358 >> 172.21.15.44:0/752359358 conn(0x7f40500fe540 msgr2=0x7f4050100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:39.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.855+0000 7f4058720640 1 -- 172.21.15.44:0/752359358 shutdown_connections 2024-08-20T03:10:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.856+0000 7f4058720640 1 -- 172.21.15.44:0/752359358 wait complete. 2024-08-20T03:10:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.856+0000 7f4058720640 1 Processor -- start 2024-08-20T03:10:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.856+0000 7f4058720640 1 -- start start 2024-08-20T03:10:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4058720640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4050102af0 0x7f40500751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4058720640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4050108500 0x7f4050075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4058720640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4050108e80 0x7f4050075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4058720640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f4050112b00 con 0x7f4050108e80 2024-08-20T03:10:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4058720640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4050112980 con 0x7f4050108500 2024-08-20T03:10:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4058720640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f4050112c80 con 0x7f4050102af0 2024-08-20T03:10:39.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4056cbd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4050108e80 0x7f4050075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4056cbd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4050108e80 0x7f4050075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46112/0 (socket says 172.21.15.44:46112) 2024-08-20T03:10:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4056cbd640 1 -- 172.21.15.44:0/3332862693 learned_addr learned my addr 172.21.15.44:0/3332862693 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f40564bc640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4050102af0 0x7f40500751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4056cbd640 1 -- 172.21.15.44:0/3332862693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4050102af0 msgr2=0x7f40500751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4056cbd640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4050102af0 0x7f40500751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4056cbd640 1 -- 172.21.15.44:0/3332862693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4050108500 msgr2=0x7f4050075710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T03:10:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4056cbd640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4050108500 0x7f4050075710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4056cbd640 1 -- 172.21.15.44:0/3332862693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40501a7410 con 0x7f4050108e80 2024-08-20T03:10:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.857+0000 7f4056cbd640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4050108e80 0x7f4050075c50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f404800cad0 tx=0x7f404800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.858+0000 7f40477fe640 1 -- 172.21.15.44:0/3332862693 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404800ae20 con 0x7f4050108e80 2024-08-20T03:10:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.858+0000 7f4058720640 1 -- 172.21.15.44:0/3332862693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40501a7700 con 0x7f4050108e80 2024-08-20T03:10:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.858+0000 7f4058720640 1 -- 172.21.15.44:0/3332862693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40501a7c90 con 0x7f4050108e80 2024-08-20T03:10:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.859+0000 7f40477fe640 1 -- 172.21.15.44:0/3332862693 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4048005020 con 0x7f4050108e80 2024-08-20T03:10:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.859+0000 7f40477fe640 1 -- 172.21.15.44:0/3332862693 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404800f690 con 0x7f4050108e80 2024-08-20T03:10:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.860+0000 7f4058720640 1 -- 172.21.15.44:0/3332862693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4034005290 con 0x7f4050108e80 2024-08-20T03:10:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.863+0000 7f40477fe640 1 -- 172.21.15.44:0/3332862693 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f4048004a40 con 0x7f4050108e80 2024-08-20T03:10:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.864+0000 7f40477fe640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f403c0b3dc0 0x7f403c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.864+0000 7f40477fe640 1 -- 172.21.15.44:0/3332862693 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f40480e4130 con 0x7f4050108e80 2024-08-20T03:10:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.864+0000 7f40564bc640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f403c0b3dc0 0x7f403c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.865+0000 7f40564bc640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f403c0b3dc0 0x7f403c0b6280 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f404c009bd0 tx=0x7f404c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:39.868+0000 7f40477fe640 1 -- 172.21.15.44:0/3332862693 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4048014030 con 0x7f4050108e80 2024-08-20T03:10:40.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:39 smithi044 ceph-mon[18790]: pgmap v569: 97 pgs: 97 active+clean; 835 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 418 op/s 2024-08-20T03:10:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.098+0000 7f4058720640 1 -- 172.21.15.44:0/3332862693 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4034005580 con 0x7f4050108e80 2024-08-20T03:10:40.101 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.099+0000 7f40477fe640 1 -- 172.21.15.44:0/3332862693 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11979 (secure 0 0 0) 0x7f40480ab520 con 0x7f4050108e80 2024-08-20T03:10:40.102 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:10:40.102 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":41,"btime":"2024-08-20T03:09:50:618324+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:09:49.873550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24604,24589,24596,24583,24569]},"id":1}]} 2024-08-20T03:10:40.102 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 41 2024-08-20T03:10:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.102+0000 7f4058720640 1 -- 172.21.15.44:0/3332862693 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f403c0b3dc0 msgr2=0x7f403c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.102+0000 7f4058720640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f403c0b3dc0 0x7f403c0b6280 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f404c009bd0 tx=0x7f404c005f90 comp rx=0 tx=0).stop 2024-08-20T03:10:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.102+0000 7f4058720640 1 -- 172.21.15.44:0/3332862693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4050108e80 msgr2=0x7f4050075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.102+0000 7f4058720640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4050108e80 0x7f4050075c50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f404800cad0 tx=0x7f404800cf90 comp rx=0 tx=0).stop 2024-08-20T03:10:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.102+0000 7f4058720640 1 -- 172.21.15.44:0/3332862693 shutdown_connections 2024-08-20T03:10:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.102+0000 7f4058720640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f403c0b3dc0 0x7f403c0b6280 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.102+0000 7f4058720640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4050108e80 0x7f4050075c50 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.102+0000 7f4058720640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4050108500 0x7f4050075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:40.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.102+0000 7f4058720640 1 --2- 172.21.15.44:0/3332862693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4050102af0 0x7f40500751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.102+0000 7f4058720640 1 -- 172.21.15.44:0/3332862693 >> 172.21.15.44:0/3332862693 conn(0x7f40500fe540 msgr2=0x7f4050100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.102+0000 7f4058720640 1 -- 172.21.15.44:0/3332862693 shutdown_connections 2024-08-20T03:10:40.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:40.102+0000 7f4058720640 1 -- 172.21.15.44:0/3332862693 wait complete. 2024-08-20T03:10:40.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:39 smithi165 ceph-mon[22357]: pgmap v569: 97 pgs: 97 active+clean; 835 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 418 op/s 2024-08-20T03:10:40.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:39 smithi099 ceph-mon[21984]: pgmap v569: 97 pgs: 97 active+clean; 835 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 418 op/s 2024-08-20T03:10:40.893 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-20T03:10:41.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:40 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3332862693' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:10:41.149 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:10:41.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:40 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3332862693' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:10:41.271 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:40 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3332862693' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:10:41.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.705+0000 7f2c4c130640 1 -- 172.21.15.44:0/2624940454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c44104270 msgr2=0x7f2c4410c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.705+0000 7f2c4c130640 1 --2- 172.21.15.44:0/2624940454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c44104270 0x7f2c4410c850 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009900 tx=0x7f2c3402a410 comp rx=0 tx=0).stop 2024-08-20T03:10:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.705+0000 7f2c4c130640 1 -- 172.21.15.44:0/2624940454 shutdown_connections 2024-08-20T03:10:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.705+0000 7f2c4c130640 1 --2- 172.21.15.44:0/2624940454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c4410ce20 0x7f2c4410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.705+0000 7f2c4c130640 1 --2- 172.21.15.44:0/2624940454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c44104270 0x7f2c4410c850 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.705+0000 7f2c4c130640 1 --2- 172.21.15.44:0/2624940454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c44103870 0x7f2c44103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.705+0000 7f2c4c130640 1 -- 172.21.15.44:0/2624940454 >> 172.21.15.44:0/2624940454 conn(0x7f2c440fd440 msgr2=0x7f2c440ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.706+0000 7f2c4c130640 1 -- 172.21.15.44:0/2624940454 shutdown_connections 2024-08-20T03:10:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.706+0000 7f2c4c130640 1 -- 172.21.15.44:0/2624940454 wait complete. 2024-08-20T03:10:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.706+0000 7f2c4c130640 1 Processor -- start 2024-08-20T03:10:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.706+0000 7f2c4c130640 1 -- start start 2024-08-20T03:10:41.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.706+0000 7f2c4c130640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c44103870 0x7f2c44199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.706+0000 7f2c4c130640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c44104270 0x7f2c44199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.706+0000 7f2c4c130640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c4410ce20 0x7f2c4419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.706+0000 7f2c4c130640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44112aa0 con 0x7f2c4410ce20 2024-08-20T03:10:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.706+0000 7f2c4c130640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44112920 con 0x7f2c44103870 2024-08-20T03:10:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.706+0000 7f2c4c130640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44112c20 con 0x7f2c44104270 2024-08-20T03:10:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c49ecc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c44103870 0x7f2c44199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c4a6cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c4410ce20 0x7f2c4419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c49ecc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c44103870 0x7f2c44199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45200/0 (socket says 172.21.15.44:45200) 2024-08-20T03:10:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c49ecc640 1 -- 172.21.15.44:0/2573135400 learned_addr learned my addr 172.21.15.44:0/2573135400 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:41.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c496cb640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c44104270 0x7f2c44199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c49ecc640 1 -- 172.21.15.44:0/2573135400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c44104270 msgr2=0x7f2c44199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c49ecc640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c44104270 0x7f2c44199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c49ecc640 1 -- 172.21.15.44:0/2573135400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c4410ce20 msgr2=0x7f2c4419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c49ecc640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c4410ce20 0x7f2c4419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c49ecc640 1 -- 172.21.15.44:0/2573135400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4419f0c0 con 0x7f2c44103870 2024-08-20T03:10:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c4a6cd640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c4410ce20 0x7f2c4419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c496cb640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c44104270 0x7f2c44199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c49ecc640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c44103870 0x7f2c44199010 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009c40 tx=0x7f2c4000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c3affd640 1 -- 172.21.15.44:0/2573135400 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40005020 con 0x7f2c44103870 2024-08-20T03:10:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.707+0000 7f2c3affd640 1 -- 172.21.15.44:0/2573135400 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c40005670 con 0x7f2c44103870 2024-08-20T03:10:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.708+0000 7f2c4c130640 1 -- 172.21.15.44:0/2573135400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4419f3b0 con 0x7f2c44103870 2024-08-20T03:10:41.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.708+0000 7f2c3affd640 1 -- 172.21.15.44:0/2573135400 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c40019670 con 0x7f2c44103870 2024-08-20T03:10:41.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.708+0000 7f2c4c130640 1 -- 172.21.15.44:0/2573135400 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c441015d0 con 0x7f2c44103870 2024-08-20T03:10:41.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c4e529740 1 Processor -- start 2024-08-20T03:10:41.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c4e529740 1 -- start start 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c3affd640 1 -- 172.21.15.44:0/2573135400 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2c40019810 con 0x7f2c44103870 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c4e529740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a81eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c4e529740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a82aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c4e529740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a82d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c4e529740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55802a795930 con 0x55802a81f050 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c4e529740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55802a784e30 con 0x55802a82afe0 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c4e529740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55802a788e70 con 0x55802a81e730 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c49ecc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a82aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c496cb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a82d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c496cb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a82d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45208/0 (socket says 172.21.15.44:45208) 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c496cb640 1 -- 172.21.15.44:0/901637850 learned_addr learned my addr 172.21.15.44:0/901637850 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.711+0000 7f2c4a6cd640 1 --2- 172.21.15.44:0/901637850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a81eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c496cb640 1 -- 172.21.15.44:0/901637850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 msgr2=0x55802a81eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c496cb640 1 --2- 172.21.15.44:0/901637850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a81eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c496cb640 1 -- 172.21.15.44:0/901637850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 msgr2=0x55802a82aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c496cb640 1 --2- 172.21.15.44:0/901637850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a82aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c496cb640 1 -- 172.21.15.44:0/901637850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55802a4e4650 con 0x55802a82afe0 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c49ecc640 1 --2- 172.21.15.44:0/901637850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a82aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c496cb640 1 --2- 172.21.15.44:0/901637850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a82d3d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009900 tx=0x7f2c34005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efe43e5ea3e82b05 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c3affd640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2c300b3e90 0x7f2c300b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c4a6cd640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2c300b3e90 0x7f2c300b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c2ffff640 1 -- 172.21.15.44:0/901637850 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34040070 con 0x55802a82afe0 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c2ffff640 1 -- 172.21.15.44:0/901637850 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c34031860 con 0x55802a82afe0 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c2ffff640 1 -- 172.21.15.44:0/901637850 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34035040 con 0x55802a82afe0 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.712+0000 7f2c3affd640 1 -- 172.21.15.44:0/2573135400 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f2c400e78e0 con 0x7f2c44103870 2024-08-20T03:10:41.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.713+0000 7f2c4e529740 1 -- 172.21.15.44:0/901637850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 msgr2=0x55802a82d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.713+0000 7f2c4e529740 1 --2- 172.21.15.44:0/901637850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a82d3d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009900 tx=0x7f2c34005d90 comp rx=0 tx=0).stop 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.713+0000 7f2c4a6cd640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2c300b3e90 0x7f2c300b6350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009fd0 tx=0x7f2c3c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.713+0000 7f2c4e529740 1 -- 172.21.15.44:0/901637850 shutdown_connections 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.713+0000 7f2c4e529740 1 --2- 172.21.15.44:0/901637850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a82d3d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.713+0000 7f2c4e529740 1 --2- 172.21.15.44:0/901637850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a82aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.713+0000 7f2c4e529740 1 --2- 172.21.15.44:0/901637850 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a81eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.713+0000 7f2c4e529740 1 -- 172.21.15.44:0/901637850 >> 172.21.15.44:0/901637850 conn(0x55802a81c560 msgr2=0x55802a81d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.713+0000 7f2c4e529740 1 -- 172.21.15.44:0/901637850 shutdown_connections 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.713+0000 7f2c4e529740 1 -- 172.21.15.44:0/901637850 wait complete. 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.713+0000 7f2c4e529740 1 -- start start 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c4e529740 1 Processor -- start 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c4e529740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a827f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c4e529740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a8284a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c4e529740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a8289e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c4e529740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55802a5e7700 con 0x55802a81f050 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c4e529740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55802a70b810 con 0x55802a82afe0 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c4e529740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55802a82dc30 con 0x55802a81e730 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c4e529740 10 client.0 _ll_register_callbacks cb 0x55802a77e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c496cb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a8284a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c49ecc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a827f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c49ecc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a827f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44750/0 (socket says 172.21.15.44:44750) 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c49ecc640 1 -- 172.21.15.44:0/2999780655 learned_addr learned my addr 172.21.15.44:0/2999780655 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c496cb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a8284a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46144/0 (socket says 172.21.15.44:46144) 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.714+0000 7f2c4a6cd640 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a8289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c49ecc640 1 -- 172.21.15.44:0/2999780655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 msgr2=0x55802a8289e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c49ecc640 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a8289e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c49ecc640 1 -- 172.21.15.44:0/2999780655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 msgr2=0x55802a8284a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:41.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c49ecc640 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a8284a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c49ecc640 1 -- 172.21.15.44:0/2999780655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55802a836fe0 con 0x55802a81e730 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c496cb640 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a8284a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c49ecc640 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a827f60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009c10 tx=0x7f2c4000bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcbd4388e84875fc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c2d7fa640 1 -- 172.21.15.44:0/2999780655 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c400aa4a0 con 0x55802a81e730 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c2d7fa640 1 -- 172.21.15.44:0/2999780655 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f2c400051c0 con 0x55802a81e730 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c4e529740 1 -- 172.21.15.44:0/2999780655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55802a734150 con 0x55802a81e730 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c2d7fa640 1 -- 172.21.15.44:0/2999780655 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2c400add50 con 0x55802a81e730 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c4e529740 10 client.26563 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c4e529740 1 -- 172.21.15.44:0/2999780655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55802a839970 con 0x55802a81e730 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.715+0000 7f2c2d7fa640 1 -- 172.21.15.44:0/2999780655 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55802a734150 con 0x55802a81e730 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.716+0000 7f2c2d7fa640 1 -- 172.21.15.44:0/2999780655 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f2c400c3bd0 con 0x55802a81e730 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.716+0000 7f2c4e529740 10 client.26563 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:41.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.716+0000 7f2c4e529740 10 client.26563 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:10:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.716+0000 7f2c4e529740 20 client.26563 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.716+0000 7f2c4e529740 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55802a83ae80 0x55802a83d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.716+0000 7f2c4e529740 4 client.26563 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.716+0000 7f2c4e529740 1 -- 172.21.15.44:0/2999780655 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55802a797880 con 0x55802a83ae80 2024-08-20T03:10:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.716+0000 7f2c496cb640 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55802a83ae80 0x55802a83d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.717+0000 7f2c2d7fa640 10 client.26563 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:10:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.717+0000 7f2c496cb640 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55802a83ae80 0x55802a83d320 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=94f60631744b8326 server_cookie=fe080d8c8d46e878 in_seq=0 out_seq=0 2024-08-20T03:10:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.717+0000 7f2c2d7fa640 1 -- 172.21.15.44:0/2999780655 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55802a797880 con 0x55802a83ae80 2024-08-20T03:10:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.717+0000 7f2c2d7fa640 10 client.26563 handle_command_reply: tid=0 2024-08-20T03:10:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.717+0000 7f2c4e529740 1 client.26563 shutdown 2024-08-20T03:10:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.718+0000 7f2c4e529740 1 -- 172.21.15.44:0/2999780655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 msgr2=0x55802a827f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:41.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.718+0000 7f2c4e529740 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a827f60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009c10 tx=0x7f2c4000bf30 comp rx=0 tx=0).stop 2024-08-20T03:10:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.718+0000 7f2c4e529740 1 -- 172.21.15.44:0/2999780655 shutdown_connections 2024-08-20T03:10:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.718+0000 7f2c4e529740 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55802a83ae80 0x55802a83d320 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.718+0000 7f2c4e529740 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a8289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.718+0000 7f2c4e529740 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a8284a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.718+0000 7f2c4e529740 1 --2- 172.21.15.44:0/2999780655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a827f60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.718+0000 7f2c4e529740 1 -- 172.21.15.44:0/2999780655 >> 172.21.15.44:0/2999780655 conn(0x55802a81c560 msgr2=0x55802a8220d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.718+0000 7f2c2d7fa640 0 client.26563 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:10:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.718+0000 7f2c4e529740 1 -- 172.21.15.44:0/2999780655 shutdown_connections 2024-08-20T03:10:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.718+0000 7f2c4e529740 1 -- 172.21.15.44:0/2999780655 wait complete. 2024-08-20T03:10:41.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.718+0000 7f2c4e529740 20 client.26563 trim_cache size 0 max 16384 2024-08-20T03:10:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.731+0000 7f2c4e529740 1 Processor -- start 2024-08-20T03:10:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.731+0000 7f2c4e529740 1 -- start start 2024-08-20T03:10:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.731+0000 7f2c4e529740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a842c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.731+0000 7f2c4e529740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a843170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.731+0000 7f2c4e529740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a81d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.731+0000 7f2c4e529740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441ac0f0 con 0x55802a81f050 2024-08-20T03:10:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.731+0000 7f2c4e529740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441addb0 con 0x55802a82afe0 2024-08-20T03:10:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.731+0000 7f2c4e529740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441ac850 con 0x55802a81e730 2024-08-20T03:10:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.731+0000 7f2c496cb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a843170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.731+0000 7f2c496cb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a843170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46154/0 (socket says 172.21.15.44:46154) 2024-08-20T03:10:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.731+0000 7f2c496cb640 1 -- 172.21.15.44:0/169962843 learned_addr learned my addr 172.21.15.44:0/169962843 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.732+0000 7f2c49ecc640 1 --2- 172.21.15.44:0/169962843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a842c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.732+0000 7f2c4a6cd640 1 --2- 172.21.15.44:0/169962843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a81d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.732+0000 7f2c496cb640 1 -- 172.21.15.44:0/169962843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 msgr2=0x55802a842c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.732+0000 7f2c496cb640 1 --2- 172.21.15.44:0/169962843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a842c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.732+0000 7f2c496cb640 1 -- 172.21.15.44:0/169962843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 msgr2=0x55802a81d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.732+0000 7f2c496cb640 1 --2- 172.21.15.44:0/169962843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a81d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.732+0000 7f2c496cb640 1 -- 172.21.15.44:0/169962843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55802a81cde0 con 0x55802a81f050 2024-08-20T03:10:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.732+0000 7f2c496cb640 1 --2- 172.21.15.44:0/169962843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a843170 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2c34033000 tx=0x7f2c34005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7359c6e448bc2a1e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.732+0000 7f2c2f7fe640 1 -- 172.21.15.44:0/169962843 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34035040 con 0x55802a81f050 2024-08-20T03:10:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.732+0000 7f2c2f7fe640 1 -- 172.21.15.44:0/169962843 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c34031860 con 0x55802a81f050 2024-08-20T03:10:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.732+0000 7f2c2f7fe640 1 -- 172.21.15.44:0/169962843 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34040040 con 0x55802a81f050 2024-08-20T03:10:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.733+0000 7f2c4e529740 1 -- 172.21.15.44:0/169962843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 msgr2=0x55802a843170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.733+0000 7f2c4e529740 1 --2- 172.21.15.44:0/169962843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a843170 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2c34033000 tx=0x7f2c34005e80 comp rx=0 tx=0).stop 2024-08-20T03:10:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.733+0000 7f2c4e529740 1 -- 172.21.15.44:0/169962843 shutdown_connections 2024-08-20T03:10:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.733+0000 7f2c4e529740 1 --2- 172.21.15.44:0/169962843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a82afe0 0x55802a81d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.733+0000 7f2c4e529740 1 --2- 172.21.15.44:0/169962843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81f050 0x55802a843170 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.733+0000 7f2c4e529740 1 --2- 172.21.15.44:0/169962843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a81e730 0x55802a842c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.733+0000 7f2c4e529740 1 -- 172.21.15.44:0/169962843 >> 172.21.15.44:0/169962843 conn(0x55802a81c560 msgr2=0x55802a8267e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.733+0000 7f2c4e529740 1 -- 172.21.15.44:0/169962843 shutdown_connections 2024-08-20T03:10:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.733+0000 7f2c4e529740 1 -- 172.21.15.44:0/169962843 wait complete. 2024-08-20T03:10:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.733+0000 7f2c4e529740 1 -- start start 2024-08-20T03:10:41.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.733+0000 7f2c4e529740 1 Processor -- start 2024-08-20T03:10:41.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c4e529740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81e730 0x55802a833160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c4e529740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a81f050 0x55802a8336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c4e529740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a82afe0 0x55802a82dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c4e529740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441ad6e0 con 0x55802a81e730 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c4e529740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441acfb0 con 0x55802a81f050 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c4e529740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441aa0f0 con 0x55802a82afe0 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c4e529740 10 client.0 _ll_register_callbacks cb 0x7f2c400ed240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c49ecc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81e730 0x55802a833160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c4a6cd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a82afe0 0x55802a82dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c496cb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a81f050 0x55802a8336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c49ecc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81e730 0x55802a833160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46156/0 (socket says 172.21.15.44:46156) 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c49ecc640 1 -- 172.21.15.44:0/504633325 learned_addr learned my addr 172.21.15.44:0/504633325 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c49ecc640 1 -- 172.21.15.44:0/504633325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a82afe0 msgr2=0x55802a82dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:41.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c49ecc640 1 --2- 172.21.15.44:0/504633325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a82afe0 0x55802a82dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c49ecc640 1 -- 172.21.15.44:0/504633325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a81f050 msgr2=0x55802a8336c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c49ecc640 1 --2- 172.21.15.44:0/504633325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a81f050 0x55802a8336c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c49ecc640 1 -- 172.21.15.44:0/504633325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55802a822c80 con 0x55802a81e730 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c496cb640 1 --2- 172.21.15.44:0/504633325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a81f050 0x55802a8336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c49ecc640 1 --2- 172.21.15.44:0/504633325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81e730 0x55802a833160 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f2c400b2c20 tx=0x7f2c400053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe2adcc1678001a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.734+0000 7f2c2d7fa640 1 -- 172.21.15.44:0/504633325 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c400adf30 con 0x55802a81e730 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c2d7fa640 1 -- 172.21.15.44:0/504633325 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f2c400ada50 con 0x55802a81e730 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c4e529740 1 -- 172.21.15.44:0/504633325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c34031860 con 0x55802a81e730 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c2d7fa640 1 -- 172.21.15.44:0/504633325 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f2c400ae3b0 con 0x55802a81e730 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c4e529740 10 client.17190 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c4e529740 1 -- 172.21.15.44:0/504633325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55802a81cfe0 con 0x55802a81e730 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c2d7fa640 1 -- 172.21.15.44:0/504633325 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c34031860 con 0x55802a81e730 2024-08-20T03:10:41.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c2d7fa640 1 -- 172.21.15.44:0/504633325 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f2c400a9f50 con 0x55802a81e730 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c4e529740 10 client.17190 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c4e529740 10 client.17190 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c4e529740 20 client.17190 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c4e529740 1 --2- 172.21.15.44:0/504633325 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55802a8a4cb0 0x55802a8a70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c4e529740 4 client.17190 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c4e529740 1 -- 172.21.15.44:0/504633325 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55802a793320 con 0x55802a8a4cb0 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.735+0000 7f2c496cb640 1 --2- 172.21.15.44:0/504633325 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55802a8a4cb0 0x55802a8a70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c2d7fa640 10 client.17190 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c496cb640 1 --2- 172.21.15.44:0/504633325 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55802a8a4cb0 0x55802a8a70d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d82feb528348ab97 server_cookie=a19dffe591a11bcf in_seq=0 out_seq=0 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c2d7fa640 1 -- 172.21.15.44:0/504633325 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55802a793320 con 0x55802a8a4cb0 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c2d7fa640 10 client.17190 handle_command_reply: tid=0 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c4e529740 1 client.17190 shutdown 2024-08-20T03:10:41.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c4e529740 1 -- 172.21.15.44:0/504633325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81e730 msgr2=0x55802a833160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c4e529740 1 --2- 172.21.15.44:0/504633325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81e730 0x55802a833160 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f2c400b2c20 tx=0x7f2c400053f0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c4e529740 1 -- 172.21.15.44:0/504633325 shutdown_connections 2024-08-20T03:10:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c4e529740 1 --2- 172.21.15.44:0/504633325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55802a82afe0 0x55802a82dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c4e529740 1 --2- 172.21.15.44:0/504633325 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55802a8a4cb0 0x55802a8a70d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c4e529740 1 --2- 172.21.15.44:0/504633325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55802a81f050 0x55802a8336c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c4e529740 1 --2- 172.21.15.44:0/504633325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55802a81e730 0x55802a833160 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c2d7fa640 0 client.17190 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:10:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c4e529740 1 -- 172.21.15.44:0/504633325 >> 172.21.15.44:0/504633325 conn(0x55802a81c560 msgr2=0x55802a81d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c4e529740 1 -- 172.21.15.44:0/504633325 shutdown_connections 2024-08-20T03:10:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c4e529740 1 -- 172.21.15.44:0/504633325 wait complete. 2024-08-20T03:10:41.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.736+0000 7f2c4e529740 20 client.17190 trim_cache size 0 max 16384 2024-08-20T03:10:41.745 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:10:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.737+0000 7f2c4c130640 1 -- 172.21.15.44:0/2573135400 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2c300b3e90 msgr2=0x7f2c300b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:41.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.737+0000 7f2c4c130640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2c300b3e90 0x7f2c300b6350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009fd0 tx=0x7f2c3c009500 comp rx=0 tx=0).stop 2024-08-20T03:10:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.737+0000 7f2c4c130640 1 -- 172.21.15.44:0/2573135400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c44103870 msgr2=0x7f2c44199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.737+0000 7f2c4c130640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c44103870 0x7f2c44199010 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009c40 tx=0x7f2c4000b690 comp rx=0 tx=0).stop 2024-08-20T03:10:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.737+0000 7f2c4c130640 1 -- 172.21.15.44:0/2573135400 shutdown_connections 2024-08-20T03:10:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.737+0000 7f2c4c130640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2c300b3e90 0x7f2c300b6350 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.737+0000 7f2c4c130640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c4410ce20 0x7f2c4419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.737+0000 7f2c4c130640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c44104270 0x7f2c44199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.737+0000 7f2c4c130640 1 --2- 172.21.15.44:0/2573135400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c44103870 0x7f2c44199010 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.737+0000 7f2c4c130640 1 -- 172.21.15.44:0/2573135400 >> 172.21.15.44:0/2573135400 conn(0x7f2c440fd440 msgr2=0x7f2c440fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.737+0000 7f2c4c130640 1 -- 172.21.15.44:0/2573135400 shutdown_connections 2024-08-20T03:10:41.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:41.737+0000 7f2c4c130640 1 -- 172.21.15.44:0/2573135400 wait complete. 2024-08-20T03:10:42.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:41 smithi044 ceph-mon[18790]: pgmap v570: 97 pgs: 97 active+clean; 835 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 453 op/s 2024-08-20T03:10:42.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:41 smithi165 ceph-mon[22357]: pgmap v570: 97 pgs: 97 active+clean; 835 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 453 op/s 2024-08-20T03:10:42.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:41 smithi099 ceph-mon[21984]: pgmap v570: 97 pgs: 97 active+clean; 835 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 453 op/s 2024-08-20T03:10:42.913 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-20T03:10:43.218 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:10:44.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.984+0000 7ff28648a640 1 -- 172.21.15.44:0/4242993797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28010a6c0 msgr2=0x7ff28010ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.984+0000 7ff28648a640 1 --2- 172.21.15.44:0/4242993797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28010a6c0 0x7ff28010ab00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff274009900 tx=0x7ff27402a410 comp rx=0 tx=0).stop 2024-08-20T03:10:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.984+0000 7ff28648a640 1 -- 172.21.15.44:0/4242993797 shutdown_connections 2024-08-20T03:10:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.984+0000 7ff28648a640 1 --2- 172.21.15.44:0/4242993797 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff28010b040 0x7ff280111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.984+0000 7ff28648a640 1 --2- 172.21.15.44:0/4242993797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28010a6c0 0x7ff28010ab00 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.984+0000 7ff28648a640 1 --2- 172.21.15.44:0/4242993797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff280104cb0 0x7ff280105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.984+0000 7ff28648a640 1 -- 172.21.15.44:0/4242993797 >> 172.21.15.44:0/4242993797 conn(0x7ff280100720 msgr2=0x7ff280102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.984+0000 7ff28648a640 1 -- 172.21.15.44:0/4242993797 shutdown_connections 2024-08-20T03:10:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.985+0000 7ff28648a640 1 -- 172.21.15.44:0/4242993797 wait complete. 2024-08-20T03:10:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.985+0000 7ff28648a640 1 Processor -- start 2024-08-20T03:10:44.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.985+0000 7ff28648a640 1 -- start start 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.985+0000 7ff28648a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff280104cb0 0x7ff28019b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff28648a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28010a6c0 0x7ff28019b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff28648a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28010b040 0x7ff2801a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff28648a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff280114cc0 con 0x7ff28010b040 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff28648a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff280114b40 con 0x7ff28010a6c0 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff28648a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff280114e40 con 0x7ff280104cb0 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff27ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff280104cb0 0x7ff28019b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff284a27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28010b040 0x7ff2801a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff27ffff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff280104cb0 0x7ff28019b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44790/0 (socket says 172.21.15.44:44790) 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff27ffff640 1 -- 172.21.15.44:0/3789181549 learned_addr learned my addr 172.21.15.44:0/3789181549 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff27f7fe640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28010a6c0 0x7ff28019b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff284a27640 1 -- 172.21.15.44:0/3789181549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff280104cb0 msgr2=0x7ff28019b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:44.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff284a27640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff280104cb0 0x7ff28019b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff284a27640 1 -- 172.21.15.44:0/3789181549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28010a6c0 msgr2=0x7ff28019b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff284a27640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28010a6c0 0x7ff28019b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff284a27640 1 -- 172.21.15.44:0/3789181549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2801a1170 con 0x7ff28010b040 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff27ffff640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff280104cb0 0x7ff28019b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff27f7fe640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28010a6c0 0x7ff28019b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff284a27640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28010b040 0x7ff2801a09a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff278009fd0 tx=0x7ff27800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff27d7fa640 1 -- 172.21.15.44:0/3789181549 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27800ce20 con 0x7ff28010b040 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff27d7fa640 1 -- 172.21.15.44:0/3789181549 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff278005020 con 0x7ff28010b040 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff28648a640 1 -- 172.21.15.44:0/3789181549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2801a1460 con 0x7ff28010b040 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.986+0000 7ff27d7fa640 1 -- 172.21.15.44:0/3789181549 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278010690 con 0x7ff28010b040 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.987+0000 7ff28648a640 1 -- 172.21.15.44:0/3789181549 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2801a7cf0 con 0x7ff28010b040 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff288883740 1 Processor -- start 2024-08-20T03:10:44.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff27d7fa640 1 -- 172.21.15.44:0/3789181549 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7ff2780054d0 con 0x7ff28010b040 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff288883740 1 -- start start 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff288883740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e68880 0x5609a8e68c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff288883740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e691a0 0x5609a8e74bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff288883740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e75130 0x5609a8e77520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff288883740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5609a8ddf930 con 0x5609a8e68880 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff288883740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5609a8dcee30 con 0x5609a8e75130 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff288883740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5609a8dd2e70 con 0x5609a8e691a0 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff27f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e68880 0x5609a8e68c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff27f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e68880 0x5609a8e68c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46184/0 (socket says 172.21.15.44:46184) 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff27f7fe640 1 -- 172.21.15.44:0/2106494859 learned_addr learned my addr 172.21.15.44:0/2106494859 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.989+0000 7ff27ffff640 1 --2- 172.21.15.44:0/2106494859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e691a0 0x5609a8e74bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff27f7fe640 1 --2- 172.21.15.44:0/2106494859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e75130 0x5609a8e77520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff27ffff640 1 -- 172.21.15.44:0/2106494859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e75130 msgr2=0x5609a8e77520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff27ffff640 1 --2- 172.21.15.44:0/2106494859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e75130 0x5609a8e77520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff27ffff640 1 -- 172.21.15.44:0/2106494859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e68880 msgr2=0x5609a8e68c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff27ffff640 1 --2- 172.21.15.44:0/2106494859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e68880 0x5609a8e68c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff27ffff640 1 -- 172.21.15.44:0/2106494859 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a8b2e650 con 0x5609a8e691a0 2024-08-20T03:10:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff27f7fe640 1 --2- 172.21.15.44:0/2106494859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e68880 0x5609a8e68c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff27d7fa640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2680b3dc0 0x7ff2680b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff27f7fe640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2680b3dc0 0x7ff2680b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff27ffff640 1 --2- 172.21.15.44:0/2106494859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e691a0 0x5609a8e74bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff27000f470 tx=0x7ff27000f930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a67d10eb9fd21e8a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff27d7fa640 1 -- 172.21.15.44:0/3789181549 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7ff2780e46f0 con 0x7ff28010b040 2024-08-20T03:10:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff2667fc640 1 -- 172.21.15.44:0/2106494859 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270005220 con 0x5609a8e691a0 2024-08-20T03:10:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff2667fc640 1 -- 172.21.15.44:0/2106494859 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2700047a0 con 0x5609a8e691a0 2024-08-20T03:10:44.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.990+0000 7ff2667fc640 1 -- 172.21.15.44:0/2106494859 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270010ba0 con 0x5609a8e691a0 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.991+0000 7ff288883740 1 -- 172.21.15.44:0/2106494859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e691a0 msgr2=0x5609a8e74bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.991+0000 7ff288883740 1 --2- 172.21.15.44:0/2106494859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e691a0 0x5609a8e74bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff27000f470 tx=0x7ff27000f930 comp rx=0 tx=0).stop 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.991+0000 7ff27f7fe640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2680b3dc0 0x7ff2680b6280 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x5609a8e68830 tx=0x7ff27403f040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.991+0000 7ff288883740 1 -- 172.21.15.44:0/2106494859 shutdown_connections 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.991+0000 7ff288883740 1 --2- 172.21.15.44:0/2106494859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e75130 0x5609a8e77520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.991+0000 7ff288883740 1 --2- 172.21.15.44:0/2106494859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e691a0 0x5609a8e74bf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.991+0000 7ff288883740 1 --2- 172.21.15.44:0/2106494859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e68880 0x5609a8e68c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.991+0000 7ff288883740 1 -- 172.21.15.44:0/2106494859 >> 172.21.15.44:0/2106494859 conn(0x5609a8e666b0 msgr2=0x5609a8e67850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.991+0000 7ff288883740 1 -- 172.21.15.44:0/2106494859 shutdown_connections 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.991+0000 7ff288883740 1 -- 172.21.15.44:0/2106494859 wait complete. 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.991+0000 7ff288883740 1 -- start start 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff288883740 1 Processor -- start 2024-08-20T03:10:44.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff288883740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e68880 0x5609a8e720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff288883740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e691a0 0x5609a8e72600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff288883740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 0x5609a8e72b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff288883740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5609a8c31700 con 0x5609a8e75130 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff288883740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5609a8d55810 con 0x5609a8e68880 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff288883740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5609a8e77de0 con 0x5609a8e691a0 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff288883740 10 client.0 _ll_register_callbacks cb 0x5609a8dc8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff284a27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 0x5609a8e72b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff27ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e68880 0x5609a8e720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff284a27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 0x5609a8e72b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46192/0 (socket says 172.21.15.44:46192) 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff284a27640 1 -- 172.21.15.44:0/221715137 learned_addr learned my addr 172.21.15.44:0/221715137 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff27ffff640 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e68880 0x5609a8e720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45280/0 (socket says 172.21.15.44:45280) 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff27f7fe640 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e691a0 0x5609a8e72600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff284a27640 1 -- 172.21.15.44:0/221715137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e691a0 msgr2=0x5609a8e72600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff284a27640 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e691a0 0x5609a8e72600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff284a27640 1 -- 172.21.15.44:0/221715137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e68880 msgr2=0x5609a8e720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff284a27640 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e68880 0x5609a8e720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff284a27640 1 -- 172.21.15.44:0/221715137 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a8e81190 con 0x5609a8e75130 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.992+0000 7ff27ffff640 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e68880 0x5609a8e720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff284a27640 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 0x5609a8e72b40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff2780ab270 tx=0x7ff2780abaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9290001c0de75d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff25f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff25f7fe640 1 -- 172.21.15.44:0/221715137 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780b4390 con 0x5609a8e75130 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff25f7fe640 1 -- 172.21.15.44:0/221715137 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7ff2780a78d0 con 0x5609a8e75130 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff288883740 1 -- 172.21.15.44:0/221715137 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5609a8d7e150 con 0x5609a8e75130 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff25f7fe640 1 -- 172.21.15.44:0/221715137 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff2780cc050 con 0x5609a8e75130 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff288883740 10 client.17202 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:44.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff288883740 1 -- 172.21.15.44:0/221715137 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5609a8e83ba0 con 0x5609a8e75130 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff25f7fe640 1 -- 172.21.15.44:0/221715137 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5609a8d7e150 con 0x5609a8e75130 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff25f7fe640 1 -- 172.21.15.44:0/221715137 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7ff2780a9fb0 con 0x5609a8e75130 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff288883740 10 client.17202 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff288883740 10 client.17202 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff288883740 20 client.17202 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff288883740 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5609a8e85000 0x5609a8e87410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff288883740 4 client.17202 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.993+0000 7ff288883740 1 -- 172.21.15.44:0/221715137 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609a8de1880 con 0x5609a8e85000 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.994+0000 7ff27ffff640 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5609a8e85000 0x5609a8e87410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.994+0000 7ff27ffff640 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5609a8e85000 0x5609a8e87410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce327437ec11a42f server_cookie=ae4339b145f7001a in_seq=0 out_seq=0 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.994+0000 7ff25f7fe640 10 client.17202 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff25f7fe640 1 -- 172.21.15.44:0/221715137 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609a8de1880 con 0x5609a8e85000 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff25f7fe640 10 client.17202 handle_command_reply: tid=0 2024-08-20T03:10:44.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff288883740 1 client.17202 shutdown 2024-08-20T03:10:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff288883740 1 -- 172.21.15.44:0/221715137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 msgr2=0x5609a8e72b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff288883740 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 0x5609a8e72b40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff2780ab270 tx=0x7ff2780abaa0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff288883740 1 -- 172.21.15.44:0/221715137 shutdown_connections 2024-08-20T03:10:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff288883740 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5609a8e85000 0x5609a8e87410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff288883740 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 0x5609a8e72b40 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff25f7fe640 0 client.17202 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:10:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff288883740 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e691a0 0x5609a8e72600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff288883740 1 --2- 172.21.15.44:0/221715137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e68880 0x5609a8e720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff288883740 1 -- 172.21.15.44:0/221715137 >> 172.21.15.44:0/221715137 conn(0x5609a8e666b0 msgr2=0x5609a8e6c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff288883740 1 -- 172.21.15.44:0/221715137 shutdown_connections 2024-08-20T03:10:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff288883740 1 -- 172.21.15.44:0/221715137 wait complete. 2024-08-20T03:10:44.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:43.995+0000 7ff288883740 20 client.17202 trim_cache size 0 max 16384 2024-08-20T03:10:44.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.009+0000 7ff288883740 1 Processor -- start 2024-08-20T03:10:44.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.009+0000 7ff288883740 1 -- start start 2024-08-20T03:10:44.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.009+0000 7ff288883740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e68880 0x5609a8e71820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.009+0000 7ff288883740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e691a0 0x5609a8e8cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.009+0000 7ff288883740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e75130 0x5609a8e8d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.009+0000 7ff288883740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b0450 con 0x5609a8e691a0 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.009+0000 7ff288883740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b2110 con 0x5609a8e75130 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.009+0000 7ff288883740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b0bb0 con 0x5609a8e68880 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.009+0000 7ff27f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e691a0 0x5609a8e8cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.009+0000 7ff27f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e691a0 0x5609a8e8cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46196/0 (socket says 172.21.15.44:46196) 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.009+0000 7ff27f7fe640 1 -- 172.21.15.44:0/2184399516 learned_addr learned my addr 172.21.15.44:0/2184399516 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.009+0000 7ff27ffff640 1 --2- 172.21.15.44:0/2184399516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e68880 0x5609a8e71820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.010+0000 7ff284a27640 1 --2- 172.21.15.44:0/2184399516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e75130 0x5609a8e8d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.010+0000 7ff27f7fe640 1 -- 172.21.15.44:0/2184399516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e68880 msgr2=0x5609a8e71820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.010+0000 7ff27f7fe640 1 --2- 172.21.15.44:0/2184399516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e68880 0x5609a8e71820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.010+0000 7ff27f7fe640 1 -- 172.21.15.44:0/2184399516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e75130 msgr2=0x5609a8e8d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:44.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.010+0000 7ff27f7fe640 1 --2- 172.21.15.44:0/2184399516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e75130 0x5609a8e8d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.010+0000 7ff27f7fe640 1 -- 172.21.15.44:0/2184399516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a8e67130 con 0x5609a8e691a0 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.010+0000 7ff27f7fe640 1 --2- 172.21.15.44:0/2184399516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e691a0 0x5609a8e8cce0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff274009900 tx=0x7ff274039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea341aa85cbedec server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.010+0000 7ff265ffb640 1 -- 172.21.15.44:0/2184399516 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274004440 con 0x5609a8e691a0 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.010+0000 7ff265ffb640 1 -- 172.21.15.44:0/2184399516 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff274003aa0 con 0x5609a8e691a0 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.010+0000 7ff265ffb640 1 -- 172.21.15.44:0/2184399516 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27403d9f0 con 0x5609a8e691a0 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.011+0000 7ff288883740 1 -- 172.21.15.44:0/2184399516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e691a0 msgr2=0x5609a8e8cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.011+0000 7ff288883740 1 --2- 172.21.15.44:0/2184399516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e691a0 0x5609a8e8cce0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff274009900 tx=0x7ff274039000 comp rx=0 tx=0).stop 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.011+0000 7ff288883740 1 -- 172.21.15.44:0/2184399516 shutdown_connections 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.011+0000 7ff288883740 1 --2- 172.21.15.44:0/2184399516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e75130 0x5609a8e8d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.011+0000 7ff288883740 1 --2- 172.21.15.44:0/2184399516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e691a0 0x5609a8e8cce0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.011+0000 7ff288883740 1 --2- 172.21.15.44:0/2184399516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e68880 0x5609a8e71820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.011+0000 7ff288883740 1 -- 172.21.15.44:0/2184399516 >> 172.21.15.44:0/2184399516 conn(0x5609a8e666b0 msgr2=0x5609a8e73f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:44.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.011+0000 7ff288883740 1 -- 172.21.15.44:0/2184399516 shutdown_connections 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.011+0000 7ff288883740 1 -- 172.21.15.44:0/2184399516 wait complete. 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.011+0000 7ff288883740 1 -- start start 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff288883740 1 Processor -- start 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff288883740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e68880 0x5609a8e6fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff288883740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e691a0 0x5609a8e7d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff288883740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 0x5609a8e7d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff288883740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b1a40 con 0x5609a8e75130 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff288883740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b1310 con 0x5609a8e691a0 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff288883740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801ae450 con 0x5609a8e68880 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff288883740 10 client.0 _ll_register_callbacks cb 0x7ff2780a71a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff284a27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 0x5609a8e7d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff284a27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 0x5609a8e7d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46208/0 (socket says 172.21.15.44:46208) 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff284a27640 1 -- 172.21.15.44:0/3341962824 learned_addr learned my addr 172.21.15.44:0/3341962824 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:44.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff27ffff640 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e68880 0x5609a8e6fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff27f7fe640 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e691a0 0x5609a8e7d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff284a27640 1 -- 172.21.15.44:0/3341962824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e68880 msgr2=0x5609a8e6fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff284a27640 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e68880 0x5609a8e6fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff284a27640 1 -- 172.21.15.44:0/3341962824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e691a0 msgr2=0x5609a8e7d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff284a27640 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e691a0 0x5609a8e7d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff284a27640 1 -- 172.21.15.44:0/3341962824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a8e7dcf0 con 0x5609a8e75130 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.012+0000 7ff27ffff640 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e68880 0x5609a8e6fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff284a27640 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 0x5609a8e7d5b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff2780a7360 tx=0x7ff2780cc5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e56adac801547a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff25f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff25f7fe640 1 -- 172.21.15.44:0/3341962824 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278089070 con 0x5609a8e75130 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff288883740 1 -- 172.21.15.44:0/3341962824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff274003aa0 con 0x5609a8e75130 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff25f7fe640 1 -- 172.21.15.44:0/3341962824 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7ff2780b4110 con 0x5609a8e75130 2024-08-20T03:10:44.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff25f7fe640 1 -- 172.21.15.44:0/3341962824 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff2780ab0b0 con 0x5609a8e75130 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff288883740 10 client.17208 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff288883740 1 -- 172.21.15.44:0/3341962824 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5609a8eeeda0 con 0x5609a8e75130 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff25f7fe640 1 -- 172.21.15.44:0/3341962824 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff274003aa0 con 0x5609a8e75130 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff25f7fe640 1 -- 172.21.15.44:0/3341962824 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7ff27808c020 con 0x5609a8e75130 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff288883740 10 client.17208 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff288883740 10 client.17208 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff288883740 20 client.17208 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff288883740 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5609a8eef4f0 0x5609a8ef1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff288883740 4 client.17208 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.013+0000 7ff288883740 1 -- 172.21.15.44:0/3341962824 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609a8ddd320 con 0x5609a8eef4f0 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff27ffff640 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5609a8eef4f0 0x5609a8ef1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff27ffff640 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5609a8eef4f0 0x5609a8ef1910 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a8444f759890015 server_cookie=c5d1011da1ffc66d in_seq=0 out_seq=0 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff25f7fe640 10 client.17208 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:10:44.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff25f7fe640 1 -- 172.21.15.44:0/3341962824 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609a8ddd320 con 0x5609a8eef4f0 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff25f7fe640 10 client.17208 handle_command_reply: tid=0 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff288883740 1 client.17208 shutdown 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff288883740 1 -- 172.21.15.44:0/3341962824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 msgr2=0x5609a8e7d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff288883740 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 0x5609a8e7d5b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff2780a7360 tx=0x7ff2780cc5e0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff288883740 1 -- 172.21.15.44:0/3341962824 shutdown_connections 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff288883740 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5609a8eef4f0 0x5609a8ef1910 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff288883740 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5609a8e75130 0x5609a8e7d5b0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff288883740 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609a8e691a0 0x5609a8e7d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff288883740 1 --2- 172.21.15.44:0/3341962824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5609a8e68880 0x5609a8e6fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff288883740 1 -- 172.21.15.44:0/3341962824 >> 172.21.15.44:0/3341962824 conn(0x5609a8e666b0 msgr2=0x5609a8e72030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.014+0000 7ff25f7fe640 0 client.17208 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff288883740 1 -- 172.21.15.44:0/3341962824 shutdown_connections 2024-08-20T03:10:44.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff288883740 1 -- 172.21.15.44:0/3341962824 wait complete. 2024-08-20T03:10:44.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff288883740 20 client.17208 trim_cache size 0 max 16384 2024-08-20T03:10:44.027 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:10:44.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff28648a640 1 -- 172.21.15.44:0/3789181549 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2680b3dc0 msgr2=0x7ff2680b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:44.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff28648a640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2680b3dc0 0x7ff2680b6280 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x5609a8e68830 tx=0x7ff27403f040 comp rx=0 tx=0).stop 2024-08-20T03:10:44.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff28648a640 1 -- 172.21.15.44:0/3789181549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28010b040 msgr2=0x7ff2801a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:44.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff28648a640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28010b040 0x7ff2801a09a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff278009fd0 tx=0x7ff27800ef10 comp rx=0 tx=0).stop 2024-08-20T03:10:44.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff28648a640 1 -- 172.21.15.44:0/3789181549 shutdown_connections 2024-08-20T03:10:44.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff28648a640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7ff2680b3dc0 0x7ff2680b6280 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff28648a640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28010b040 0x7ff2801a09a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff28648a640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff28010a6c0 0x7ff28019b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff28648a640 1 --2- 172.21.15.44:0/3789181549 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff280104cb0 0x7ff28019b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:44.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff28648a640 1 -- 172.21.15.44:0/3789181549 >> 172.21.15.44:0/3789181549 conn(0x7ff280100720 msgr2=0x7ff280102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:44.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff28648a640 1 -- 172.21.15.44:0/3789181549 shutdown_connections 2024-08-20T03:10:44.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:44.015+0000 7ff28648a640 1 -- 172.21.15.44:0/3789181549 wait complete. 2024-08-20T03:10:44.135 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:43 smithi165 ceph-mon[22357]: pgmap v571: 97 pgs: 97 active+clean; 827 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 456 op/s 2024-08-20T03:10:44.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:43 smithi099 ceph-mon[21984]: pgmap v571: 97 pgs: 97 active+clean; 827 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 456 op/s 2024-08-20T03:10:44.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:43 smithi044 ceph-mon[18790]: pgmap v571: 97 pgs: 97 active+clean; 827 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 456 op/s 2024-08-20T03:10:44.972 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T03:10:45.226 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:10:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.762+0000 7f4761f06640 1 -- 172.21.15.44:0/639136391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f475c10a7a0 msgr2=0x7f475c115200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.762+0000 7f4761f06640 1 --2- 172.21.15.44:0/639136391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f475c10a7a0 0x7f475c115200 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4750009900 tx=0x7f475002a410 comp rx=0 tx=0).stop 2024-08-20T03:10:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.762+0000 7f4761f06640 1 -- 172.21.15.44:0/639136391 shutdown_connections 2024-08-20T03:10:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.762+0000 7f4761f06640 1 --2- 172.21.15.44:0/639136391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f475c115740 0x7f475c117b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.762+0000 7f4761f06640 1 --2- 172.21.15.44:0/639136391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f475c10a7a0 0x7f475c115200 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.762+0000 7f4761f06640 1 --2- 172.21.15.44:0/639136391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f475c109e80 0x7f475c10a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.762+0000 7f4761f06640 1 -- 172.21.15.44:0/639136391 >> 172.21.15.44:0/639136391 conn(0x7f475c06dfe0 msgr2=0x7f475c06e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.763+0000 7f4761f06640 1 -- 172.21.15.44:0/639136391 shutdown_connections 2024-08-20T03:10:45.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.763+0000 7f4761f06640 1 -- 172.21.15.44:0/639136391 wait complete. 2024-08-20T03:10:45.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.764+0000 7f4761f06640 1 Processor -- start 2024-08-20T03:10:45.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.764+0000 7f4761f06640 1 -- start start 2024-08-20T03:10:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.764+0000 7f4761f06640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f475c109e80 0x7f475c1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.764+0000 7f4761f06640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f475c10a7a0 0x7f475c1a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.764+0000 7f4761f06640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f475c115740 0x7f475c1a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.764+0000 7f4761f06640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f475c11b290 con 0x7f475c109e80 2024-08-20T03:10:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.764+0000 7f4761f06640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f475c11b110 con 0x7f475c10a7a0 2024-08-20T03:10:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.764+0000 7f4761f06640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f475c11b410 con 0x7f475c115740 2024-08-20T03:10:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.765+0000 7f475b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f475c109e80 0x7f475c1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.765+0000 7f475b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f475c109e80 0x7f475c1a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46228/0 (socket says 172.21.15.44:46228) 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.765+0000 7f475b7fe640 1 -- 172.21.15.44:0/2966340153 learned_addr learned my addr 172.21.15.44:0/2966340153 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.765+0000 7f475bfff640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f475c115740 0x7f475c1a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.765+0000 7f475affd640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f475c10a7a0 0x7f475c1a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.765+0000 7f475b7fe640 1 -- 172.21.15.44:0/2966340153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f475c115740 msgr2=0x7f475c1a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.765+0000 7f475b7fe640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f475c115740 0x7f475c1a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.765+0000 7f475b7fe640 1 -- 172.21.15.44:0/2966340153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f475c10a7a0 msgr2=0x7f475c1a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.765+0000 7f475b7fe640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f475c10a7a0 0x7f475c1a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.765+0000 7f475b7fe640 1 -- 172.21.15.44:0/2966340153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475c1a78b0 con 0x7f475c109e80 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.765+0000 7f475bfff640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f475c115740 0x7f475c1a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.765+0000 7f475b7fe640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f475c109e80 0x7f475c1a1800 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f474c00ea60 tx=0x7f474c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.766+0000 7f4758ff9640 1 -- 172.21.15.44:0/2966340153 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c00cde0 con 0x7f475c109e80 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.766+0000 7f4761f06640 1 -- 172.21.15.44:0/2966340153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475c1a7ba0 con 0x7f475c109e80 2024-08-20T03:10:45.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.766+0000 7f4758ff9640 1 -- 172.21.15.44:0/2966340153 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f474c0035f0 con 0x7f475c109e80 2024-08-20T03:10:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.766+0000 7f4758ff9640 1 -- 172.21.15.44:0/2966340153 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c010690 con 0x7f475c109e80 2024-08-20T03:10:45.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.766+0000 7f4761f06640 1 -- 172.21.15.44:0/2966340153 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f475c10b160 con 0x7f475c109e80 2024-08-20T03:10:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.771+0000 7f4758ff9640 1 -- 172.21.15.44:0/2966340153 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f474c010830 con 0x7f475c109e80 2024-08-20T03:10:45.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.771+0000 7f47642ff740 1 Processor -- start 2024-08-20T03:10:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.772+0000 7f47642ff740 1 -- start start 2024-08-20T03:10:45.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.772+0000 7f47642ff740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9bacdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.772+0000 7f47642ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bace1a0 0x558d9bad9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.772+0000 7f47642ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bada130 0x558d9badc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.772+0000 7f47642ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558d9ba44930 con 0x558d9bada130 2024-08-20T03:10:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.772+0000 7f47642ff740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558d9ba33e30 con 0x558d9bacd880 2024-08-20T03:10:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.772+0000 7f47642ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558d9ba37e70 con 0x558d9bace1a0 2024-08-20T03:10:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.772+0000 7f475affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bada130 0x558d9badc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.772+0000 7f475affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bada130 0x558d9badc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46242/0 (socket says 172.21.15.44:46242) 2024-08-20T03:10:45.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.772+0000 7f475affd640 1 -- 172.21.15.44:0/943301170 learned_addr learned my addr 172.21.15.44:0/943301170 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.772+0000 7f475bfff640 1 --2- 172.21.15.44:0/943301170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9bacdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.772+0000 7f475b7fe640 1 --2- 172.21.15.44:0/943301170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bace1a0 0x558d9bad9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.773+0000 7f475bfff640 1 -- 172.21.15.44:0/943301170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bace1a0 msgr2=0x558d9bad9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:45.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.773+0000 7f475bfff640 1 --2- 172.21.15.44:0/943301170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bace1a0 0x558d9bad9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.773+0000 7f475bfff640 1 -- 172.21.15.44:0/943301170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bada130 msgr2=0x558d9badc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.773+0000 7f475bfff640 1 --2- 172.21.15.44:0/943301170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bada130 0x558d9badc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.773+0000 7f475bfff640 1 -- 172.21.15.44:0/943301170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d9b793650 con 0x558d9bacd880 2024-08-20T03:10:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.773+0000 7f475bfff640 1 --2- 172.21.15.44:0/943301170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9bacdc60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f475c06be30 tx=0x7f4754011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffccd27640eb5247 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.773+0000 7f4758ff9640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f47440b3d40 0x7f47440b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.773+0000 7f4741ffb640 1 -- 172.21.15.44:0/943301170 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754004d20 con 0x558d9bacd880 2024-08-20T03:10:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.773+0000 7f4758ff9640 1 -- 172.21.15.44:0/2966340153 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f474c0037a0 con 0x7f475c109e80 2024-08-20T03:10:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.773+0000 7f475b7fe640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f47440b3d40 0x7f47440b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.774+0000 7f4741ffb640 1 -- 172.21.15.44:0/943301170 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475401e070 con 0x558d9bacd880 2024-08-20T03:10:45.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.774+0000 7f4741ffb640 1 -- 172.21.15.44:0/943301170 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475401aab0 con 0x558d9bacd880 2024-08-20T03:10:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.775+0000 7f47642ff740 1 -- 172.21.15.44:0/943301170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 msgr2=0x558d9bacdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.775+0000 7f47642ff740 1 --2- 172.21.15.44:0/943301170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9bacdc60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f475c06be30 tx=0x7f4754011bf0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.775+0000 7f475b7fe640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f47440b3d40 0x7f47440b6200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f475c10b330 tx=0x7f474c0098b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.775+0000 7f47642ff740 1 -- 172.21.15.44:0/943301170 shutdown_connections 2024-08-20T03:10:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.775+0000 7f47642ff740 1 --2- 172.21.15.44:0/943301170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bada130 0x558d9badc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.775+0000 7f47642ff740 1 --2- 172.21.15.44:0/943301170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bace1a0 0x558d9bad9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.775+0000 7f47642ff740 1 --2- 172.21.15.44:0/943301170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9bacdc60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.775+0000 7f47642ff740 1 -- 172.21.15.44:0/943301170 >> 172.21.15.44:0/943301170 conn(0x558d9bacb6b0 msgr2=0x558d9bacc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.775+0000 7f47642ff740 1 -- 172.21.15.44:0/943301170 shutdown_connections 2024-08-20T03:10:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.775+0000 7f47642ff740 1 -- 172.21.15.44:0/943301170 wait complete. 2024-08-20T03:10:45.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.775+0000 7f47642ff740 1 -- start start 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.776+0000 7f47642ff740 1 Processor -- start 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.776+0000 7f47642ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bacd880 0x558d9bad9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.776+0000 7f47642ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bace1a0 0x558d9bad6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.776+0000 7f47642ff740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bada130 0x558d9bad7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.776+0000 7f47642ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x558d9b896700 con 0x558d9bacd880 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.776+0000 7f47642ff740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558d9b9ba810 con 0x558d9bada130 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.776+0000 7f47642ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x558d9badcde0 con 0x558d9bace1a0 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.776+0000 7f47642ff740 10 client.0 _ll_register_callbacks cb 0x558d9ba2d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.776+0000 7f475bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bacd880 0x558d9bad9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.776+0000 7f475bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bacd880 0x558d9bad9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46252/0 (socket says 172.21.15.44:46252) 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.776+0000 7f475bfff640 1 -- 172.21.15.44:0/4092557787 learned_addr learned my addr 172.21.15.44:0/4092557787 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.776+0000 7f475bfff640 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bada130 0x558d9bad7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.777+0000 7f475affd640 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bace1a0 0x558d9bad6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.777+0000 7f475bfff640 1 -- 172.21.15.44:0/4092557787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bace1a0 msgr2=0x558d9bad6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.777+0000 7f475bfff640 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bace1a0 0x558d9bad6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.777+0000 7f475bfff640 1 -- 172.21.15.44:0/4092557787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bacd880 msgr2=0x558d9bad9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.777+0000 7f475bfff640 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bacd880 0x558d9bad9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.777+0000 7f475bfff640 1 -- 172.21.15.44:0/4092557787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d9bae6170 con 0x558d9bada130 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.777+0000 7f475bfff640 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bada130 0x558d9bad7700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f475400fba0 tx=0x7f475400fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeb77525e1fd0f64 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.778+0000 7f473affd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.778+0000 7f473affd640 1 -- 172.21.15.44:0/4092557787 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754004050 con 0x558d9bada130 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.778+0000 7f473affd640 1 -- 172.21.15.44:0/4092557787 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f47540041f0 con 0x558d9bada130 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.778+0000 7f47642ff740 1 -- 172.21.15.44:0/4092557787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558d9b9e3150 con 0x558d9bada130 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.778+0000 7f473affd640 1 -- 172.21.15.44:0/4092557787 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f47540043f0 con 0x558d9bada130 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.778+0000 7f47642ff740 10 client.26630 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:45.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.778+0000 7f47642ff740 1 -- 172.21.15.44:0/4092557787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558d9bae8a40 con 0x558d9bada130 2024-08-20T03:10:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.779+0000 7f473affd640 1 -- 172.21.15.44:0/4092557787 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558d9b9e3150 con 0x558d9bada130 2024-08-20T03:10:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.779+0000 7f473affd640 1 -- 172.21.15.44:0/4092557787 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f475401ace0 con 0x558d9bada130 2024-08-20T03:10:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.779+0000 7f47642ff740 10 client.26630 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.779+0000 7f47642ff740 10 client.26630 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:10:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.779+0000 7f47642ff740 20 client.26630 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.779+0000 7f47642ff740 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558d9baea340 0x558d9baec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.779+0000 7f47642ff740 4 client.26630 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:45.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.779+0000 7f47642ff740 1 -- 172.21.15.44:0/4092557787 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d9ba46880 con 0x558d9baea340 2024-08-20T03:10:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.782+0000 7f475affd640 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558d9baea340 0x558d9baec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.782+0000 7f475affd640 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558d9baea340 0x558d9baec760 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=601e6d3cdab333d3 server_cookie=55836816f64139f6 in_seq=0 out_seq=0 2024-08-20T03:10:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.782+0000 7f473affd640 10 client.26630 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:10:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.784+0000 7f473affd640 1 -- 172.21.15.44:0/4092557787 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d9ba46880 con 0x558d9baea340 2024-08-20T03:10:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.784+0000 7f473affd640 10 client.26630 handle_command_reply: tid=0 2024-08-20T03:10:45.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.784+0000 7f47642ff740 1 client.26630 shutdown 2024-08-20T03:10:45.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.784+0000 7f47642ff740 1 -- 172.21.15.44:0/4092557787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bada130 msgr2=0x558d9bad7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:45.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.784+0000 7f47642ff740 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bada130 0x558d9bad7700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f475400fba0 tx=0x7f475400fc00 comp rx=0 tx=0).stop 2024-08-20T03:10:45.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.785+0000 7f47642ff740 1 -- 172.21.15.44:0/4092557787 shutdown_connections 2024-08-20T03:10:45.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.785+0000 7f47642ff740 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bada130 0x558d9bad7700 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.785+0000 7f47642ff740 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558d9baea340 0x558d9baec760 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.785+0000 7f47642ff740 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bace1a0 0x558d9bad6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.785+0000 7f47642ff740 1 --2- 172.21.15.44:0/4092557787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bacd880 0x558d9bad9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.785+0000 7f47642ff740 1 -- 172.21.15.44:0/4092557787 >> 172.21.15.44:0/4092557787 conn(0x558d9bacb6b0 msgr2=0x558d9bad1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:45.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.785+0000 7f473affd640 0 client.26630 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:10:45.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.785+0000 7f47642ff740 1 -- 172.21.15.44:0/4092557787 shutdown_connections 2024-08-20T03:10:45.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.785+0000 7f47642ff740 1 -- 172.21.15.44:0/4092557787 wait complete. 2024-08-20T03:10:45.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.785+0000 7f47642ff740 20 client.26630 trim_cache size 0 max 16384 2024-08-20T03:10:45.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.806+0000 7f47642ff740 1 Processor -- start 2024-08-20T03:10:45.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f47642ff740 1 -- start start 2024-08-20T03:10:45.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f47642ff740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9bad18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f47642ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bace1a0 0x558d9bac61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f47642ff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bada130 0x558d9bacc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f47642ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f474800e780 con 0x558d9bace1a0 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f47642ff740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4748010440 con 0x558d9bacd880 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f475affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bace1a0 0x558d9bac61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f47642ff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f474800eee0 con 0x558d9bada130 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f475bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9bad18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f475bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9bad18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45360/0 (socket says 172.21.15.44:45360) 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f475bfff640 1 -- 172.21.15.44:0/1581501504 learned_addr learned my addr 172.21.15.44:0/1581501504 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f475bfff640 1 --2- 172.21.15.44:0/1581501504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bada130 0x558d9bacc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f475bfff640 1 -- 172.21.15.44:0/1581501504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bada130 msgr2=0x558d9bacc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.807+0000 7f475bfff640 1 --2- 172.21.15.44:0/1581501504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bada130 0x558d9bacc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.808+0000 7f475bfff640 1 -- 172.21.15.44:0/1581501504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bace1a0 msgr2=0x558d9bac61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.808+0000 7f475bfff640 1 --2- 172.21.15.44:0/1581501504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bace1a0 0x558d9bac61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.808+0000 7f475bfff640 1 -- 172.21.15.44:0/1581501504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d9bacb030 con 0x558d9bacd880 2024-08-20T03:10:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.808+0000 7f475affd640 1 --2- 172.21.15.44:0/1581501504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bace1a0 0x558d9bac61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.808+0000 7f475bfff640 1 --2- 172.21.15.44:0/1581501504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9bad18f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f47540115a0 tx=0x7f4754009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e33955760551d152 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.808+0000 7f47417fa640 1 -- 172.21.15.44:0/1581501504 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475400fd10 con 0x558d9bacd880 2024-08-20T03:10:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.808+0000 7f47417fa640 1 -- 172.21.15.44:0/1581501504 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475401e070 con 0x558d9bacd880 2024-08-20T03:10:45.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.808+0000 7f47417fa640 1 -- 172.21.15.44:0/1581501504 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754004c60 con 0x558d9bacd880 2024-08-20T03:10:45.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.809+0000 7f47642ff740 1 -- 172.21.15.44:0/1581501504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 msgr2=0x558d9bad18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:45.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.809+0000 7f47642ff740 1 --2- 172.21.15.44:0/1581501504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9bad18f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f47540115a0 tx=0x7f4754009500 comp rx=0 tx=0).stop 2024-08-20T03:10:45.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.809+0000 7f47642ff740 1 -- 172.21.15.44:0/1581501504 shutdown_connections 2024-08-20T03:10:45.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.809+0000 7f47642ff740 1 --2- 172.21.15.44:0/1581501504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bada130 0x558d9bacc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.809+0000 7f47642ff740 1 --2- 172.21.15.44:0/1581501504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bace1a0 0x558d9bac61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.809+0000 7f47642ff740 1 --2- 172.21.15.44:0/1581501504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9bad18f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.809+0000 7f47642ff740 1 -- 172.21.15.44:0/1581501504 >> 172.21.15.44:0/1581501504 conn(0x558d9bacb6b0 msgr2=0x558d9bad57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:45.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.809+0000 7f47642ff740 1 -- 172.21.15.44:0/1581501504 shutdown_connections 2024-08-20T03:10:45.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.809+0000 7f47642ff740 1 -- 172.21.15.44:0/1581501504 wait complete. 2024-08-20T03:10:45.816 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.810+0000 7f47642ff740 1 -- start start 2024-08-20T03:10:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.810+0000 7f47642ff740 1 Processor -- start 2024-08-20T03:10:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.810+0000 7f47642ff740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9badbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.810+0000 7f47642ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bace1a0 0x558d9bada510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f475bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9badbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f475bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9badbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:45374/0 (socket says 172.21.15.44:45374) 2024-08-20T03:10:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f475bfff640 1 -- 172.21.15.44:0/413631935 learned_addr learned my addr 172.21.15.44:0/413631935 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f47642ff740 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bada130 0x558d9badaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f47642ff740 1 -- 172.21.15.44:0/413631935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f474800fd70 con 0x558d9bace1a0 2024-08-20T03:10:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f47642ff740 1 -- 172.21.15.44:0/413631935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f474800f640 con 0x558d9bacd880 2024-08-20T03:10:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f47642ff740 1 -- 172.21.15.44:0/413631935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f474800c780 con 0x558d9bada130 2024-08-20T03:10:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f475affd640 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bace1a0 0x558d9bada510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.817 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f47642ff740 10 client.0 _ll_register_callbacks cb 0x7f47540093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f475bfff640 1 -- 172.21.15.44:0/413631935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bada130 msgr2=0x558d9badaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f475bfff640 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bada130 0x558d9badaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f475bfff640 1 -- 172.21.15.44:0/413631935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bace1a0 msgr2=0x558d9bada510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f475bfff640 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bace1a0 0x558d9bada510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.811+0000 7f475bfff640 1 -- 172.21.15.44:0/413631935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d9bacc130 con 0x558d9bacd880 2024-08-20T03:10:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.812+0000 7f475bfff640 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9badbf50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4754000f80 tx=0x7f475400faa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ba93f90dab90e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:45.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.812+0000 7f473affd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:10:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.815+0000 7f473affd640 1 -- 172.21.15.44:0/413631935 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754004a00 con 0x558d9bacd880 2024-08-20T03:10:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.815+0000 7f473affd640 1 -- 172.21.15.44:0/413631935 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f4754004ba0 con 0x558d9bacd880 2024-08-20T03:10:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.815+0000 7f47642ff740 1 -- 172.21.15.44:0/413631935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475401e070 con 0x558d9bacd880 2024-08-20T03:10:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.815+0000 7f473affd640 1 -- 172.21.15.44:0/413631935 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4754004da0 con 0x558d9bacd880 2024-08-20T03:10:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.815+0000 7f47642ff740 10 client.26636 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.815+0000 7f47642ff740 1 -- 172.21.15.44:0/413631935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558d9bae21a0 con 0x558d9bacd880 2024-08-20T03:10:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.815+0000 7f473affd640 1 -- 172.21.15.44:0/413631935 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475401e070 con 0x558d9bacd880 2024-08-20T03:10:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.816+0000 7f473affd640 1 -- 172.21.15.44:0/413631935 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f4754043070 con 0x558d9bacd880 2024-08-20T03:10:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.816+0000 7f47642ff740 10 client.26636 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.816+0000 7f47642ff740 10 client.26636 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:10:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.816+0000 7f47642ff740 20 client.26636 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:45.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.816+0000 7f47642ff740 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558d9bae2840 0x558d9bb57010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.816+0000 7f47642ff740 4 client.26636 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:10:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.816+0000 7f47642ff740 1 -- 172.21.15.44:0/413631935 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d9bac6880 con 0x558d9bae2840 2024-08-20T03:10:45.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.816+0000 7f475affd640 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558d9bae2840 0x558d9bb57010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.817+0000 7f475affd640 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558d9bae2840 0x558d9bb57010 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c1d372193a9baea server_cookie=d0d04c95f36a4c0a in_seq=0 out_seq=0 2024-08-20T03:10:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.817+0000 7f473affd640 10 client.26636 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:10:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f473affd640 1 -- 172.21.15.44:0/413631935 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d9bac6880 con 0x558d9bae2840 2024-08-20T03:10:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f473affd640 10 client.26636 handle_command_reply: tid=0 2024-08-20T03:10:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f47642ff740 1 client.26636 shutdown 2024-08-20T03:10:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f47642ff740 1 -- 172.21.15.44:0/413631935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 msgr2=0x558d9badbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f47642ff740 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9badbf50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4754000f80 tx=0x7f475400faa0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f47642ff740 1 -- 172.21.15.44:0/413631935 shutdown_connections 2024-08-20T03:10:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f47642ff740 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x558d9bae2840 0x558d9bb57010 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f47642ff740 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558d9bada130 0x558d9badaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f473affd640 0 client.26636 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:10:45.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f47642ff740 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558d9bace1a0 0x558d9bada510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f47642ff740 1 --2- 172.21.15.44:0/413631935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d9bacd880 0x558d9badbf50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f47642ff740 1 -- 172.21.15.44:0/413631935 >> 172.21.15.44:0/413631935 conn(0x558d9bacb6b0 msgr2=0x558d9bacc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:45.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f47642ff740 1 -- 172.21.15.44:0/413631935 shutdown_connections 2024-08-20T03:10:45.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f47642ff740 1 -- 172.21.15.44:0/413631935 wait complete. 2024-08-20T03:10:45.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.818+0000 7f47642ff740 20 client.26636 trim_cache size 0 max 16384 2024-08-20T03:10:45.823 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:10:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.819+0000 7f4761f06640 1 -- 172.21.15.44:0/2966340153 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f47440b3d40 msgr2=0x7f47440b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.819+0000 7f4761f06640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f47440b3d40 0x7f47440b6200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f475c10b330 tx=0x7f474c0098b0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.819+0000 7f4761f06640 1 -- 172.21.15.44:0/2966340153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f475c109e80 msgr2=0x7f475c1a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.819+0000 7f4761f06640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f475c109e80 0x7f475c1a1800 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f474c00ea60 tx=0x7f474c00ef20 comp rx=0 tx=0).stop 2024-08-20T03:10:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.819+0000 7f4761f06640 1 -- 172.21.15.44:0/2966340153 shutdown_connections 2024-08-20T03:10:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.819+0000 7f4761f06640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f47440b3d40 0x7f47440b6200 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.819+0000 7f4761f06640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f475c115740 0x7f475c1a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.819+0000 7f4761f06640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f475c10a7a0 0x7f475c1a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.819+0000 7f4761f06640 1 --2- 172.21.15.44:0/2966340153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f475c109e80 0x7f475c1a1800 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:45.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.819+0000 7f4761f06640 1 -- 172.21.15.44:0/2966340153 >> 172.21.15.44:0/2966340153 conn(0x7f475c06dfe0 msgr2=0x7f475c0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:45.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.819+0000 7f4761f06640 1 -- 172.21.15.44:0/2966340153 shutdown_connections 2024-08-20T03:10:45.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:45.819+0000 7f4761f06640 1 -- 172.21.15.44:0/2966340153 wait complete. 2024-08-20T03:10:46.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:45 smithi044 ceph-mon[18790]: pgmap v572: 97 pgs: 97 active+clean; 832 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 519 op/s 2024-08-20T03:10:46.125 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:45 smithi165 ceph-mon[22357]: pgmap v572: 97 pgs: 97 active+clean; 832 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 519 op/s 2024-08-20T03:10:46.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:45 smithi099 ceph-mon[21984]: pgmap v572: 97 pgs: 97 active+clean; 832 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 519 op/s 2024-08-20T03:10:46.526 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T03:10:46.845 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:10:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.416+0000 7fb1a86d2640 1 -- 172.21.15.44:0/2546160713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a010ce00 msgr2=0x7fb1a010f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.416+0000 7fb1a86d2640 1 --2- 172.21.15.44:0/2546160713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a010ce00 0x7fb1a010f260 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb198009ca0 tx=0x7fb19802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.416+0000 7fb1a86d2640 1 -- 172.21.15.44:0/2546160713 shutdown_connections 2024-08-20T03:10:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.416+0000 7fb1a86d2640 1 --2- 172.21.15.44:0/2546160713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a010ce00 0x7fb1a010f260 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.416+0000 7fb1a86d2640 1 --2- 172.21.15.44:0/2546160713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a01042c0 0x7fb1a010c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.416+0000 7fb1a86d2640 1 --2- 172.21.15.44:0/2546160713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a01039a0 0x7fb1a0103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.416+0000 7fb1a86d2640 1 -- 172.21.15.44:0/2546160713 >> 172.21.15.44:0/2546160713 conn(0x7fb1a00fd400 msgr2=0x7fb1a00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.416+0000 7fb1a86d2640 1 -- 172.21.15.44:0/2546160713 shutdown_connections 2024-08-20T03:10:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.416+0000 7fb1a86d2640 1 -- 172.21.15.44:0/2546160713 wait complete. 2024-08-20T03:10:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.417+0000 7fb1a86d2640 1 Processor -- start 2024-08-20T03:10:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.417+0000 7fb1a86d2640 1 -- start start 2024-08-20T03:10:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.417+0000 7fb1a86d2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a01039a0 0x7fb1a0198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.417+0000 7fb1a86d2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a01042c0 0x7fb1a01994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.417+0000 7fb1a86d2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a010ce00 0x7fb1a019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.417+0000 7fb1a86d2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a0112a70 con 0x7fb1a010ce00 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.417+0000 7fb1a86d2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01128f0 con 0x7fb1a01042c0 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.417+0000 7fb1a86d2640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a0112bf0 con 0x7fb1a01039a0 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.417+0000 7fb1a6c6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a010ce00 0x7fb1a019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a646e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a01039a0 0x7fb1a0198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a646e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a01039a0 0x7fb1a0198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:44906/0 (socket says 172.21.15.44:44906) 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a6c6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a010ce00 0x7fb1a019e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46306/0 (socket says 172.21.15.44:46306) 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a646e640 1 -- 172.21.15.44:0/4046458857 learned_addr learned my addr 172.21.15.44:0/4046458857 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a5c6d640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a01042c0 0x7fb1a01994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a646e640 1 -- 172.21.15.44:0/4046458857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a01042c0 msgr2=0x7fb1a01994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a646e640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a01042c0 0x7fb1a01994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a646e640 1 -- 172.21.15.44:0/4046458857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a010ce00 msgr2=0x7fb1a019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a646e640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a010ce00 0x7fb1a019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a646e640 1 -- 172.21.15.44:0/4046458857 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a019f050 con 0x7fb1a01039a0 2024-08-20T03:10:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a5c6d640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a01042c0 0x7fb1a01994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:10:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a646e640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a01039a0 0x7fb1a0198fa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00e9f0 tx=0x7fb19c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1977fe640 1 -- 172.21.15.44:0/4046458857 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00cd80 con 0x7fb1a01039a0 2024-08-20T03:10:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a86d2640 1 -- 172.21.15.44:0/4046458857 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a019f340 con 0x7fb1a01039a0 2024-08-20T03:10:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1977fe640 1 -- 172.21.15.44:0/4046458857 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb19c0108e0 con 0x7fb1a01039a0 2024-08-20T03:10:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1977fe640 1 -- 172.21.15.44:0/4046458857 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c018980 con 0x7fb1a01039a0 2024-08-20T03:10:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.418+0000 7fb1a86d2640 1 -- 172.21.15.44:0/4046458857 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a01a5a00 con 0x7fb1a01039a0 2024-08-20T03:10:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1aaacb740 1 Processor -- start 2024-08-20T03:10:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1aaacb740 1 -- start start 2024-08-20T03:10:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1aaacb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 0x555f541f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1aaacb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f541ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1aaacb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541fffe0 0x555f542023d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1aaacb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555f5416a930 con 0x555f541f3730 2024-08-20T03:10:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1aaacb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555f54159e30 con 0x555f541f4050 2024-08-20T03:10:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1aaacb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555f5415de70 con 0x555f541fffe0 2024-08-20T03:10:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1a6c6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 0x555f541f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1a6c6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 0x555f541f3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46318/0 (socket says 172.21.15.44:46318) 2024-08-20T03:10:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/2159276424 learned_addr learned my addr 172.21.15.44:0/2159276424 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1a5c6d640 1 --2- 172.21.15.44:0/2159276424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541fffe0 0x555f542023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/2159276424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541fffe0 msgr2=0x555f542023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/2159276424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541fffe0 0x555f542023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/2159276424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 msgr2=0x555f541ffaa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:10:47.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/2159276424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f541ffaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.421+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/2159276424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f53eb9650 con 0x555f541f3730 2024-08-20T03:10:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.422+0000 7fb1977fe640 1 -- 172.21.15.44:0/4046458857 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb19c010430 con 0x7fb1a01039a0 2024-08-20T03:10:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.422+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/2159276424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 0x555f541f3b10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb198005ef0 tx=0x7fb198005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9354855cd8239f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.422+0000 7fb194ff9640 1 -- 172.21.15.44:0/2159276424 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198031cf0 con 0x555f541f3730 2024-08-20T03:10:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.422+0000 7fb194ff9640 1 -- 172.21.15.44:0/2159276424 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb198036070 con 0x555f541f3730 2024-08-20T03:10:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.422+0000 7fb194ff9640 1 -- 172.21.15.44:0/2159276424 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1980042a0 con 0x555f541f3730 2024-08-20T03:10:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.422+0000 7fb1977fe640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb18c0b3d40 0x7fb18c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.422+0000 7fb1a646e640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb18c0b3d40 0x7fb18c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1aaacb740 1 -- 172.21.15.44:0/2159276424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 msgr2=0x555f541f3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/2159276424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 0x555f541f3b10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb198005ef0 tx=0x7fb198005f20 comp rx=0 tx=0).stop 2024-08-20T03:10:47.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1977fe640 1 -- 172.21.15.44:0/4046458857 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fb19c0e43e0 con 0x7fb1a01039a0 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1aaacb740 1 -- 172.21.15.44:0/2159276424 shutdown_connections 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/2159276424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541fffe0 0x555f542023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/2159276424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f541ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/2159276424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 0x555f541f3b10 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1aaacb740 1 -- 172.21.15.44:0/2159276424 >> 172.21.15.44:0/2159276424 conn(0x555f541f1560 msgr2=0x555f541f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1aaacb740 1 -- 172.21.15.44:0/2159276424 shutdown_connections 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1aaacb740 1 -- 172.21.15.44:0/2159276424 wait complete. 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1a646e640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb18c0b3d40 0x7fb18c0b6200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x555f541f68b0 tx=0x7fb19c0744b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1aaacb740 1 -- start start 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.423+0000 7fb1aaacb740 1 Processor -- start 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1aaacb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541f3730 0x555f541fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1aaacb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f541fd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1aaacb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 0x555f541fd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1aaacb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555f53fbc700 con 0x555f541fffe0 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1aaacb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555f540e0810 con 0x555f541f4050 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1aaacb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x555f54202ab0 con 0x555f541f3730 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1aaacb740 10 client.0 _ll_register_callbacks cb 0x555f54153700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1a6c6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 0x555f541fd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1a5c6d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f541fd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1a6c6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 0x555f541fd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46324/0 (socket says 172.21.15.44:46324) 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/3336005325 learned_addr learned my addr 172.21.15.44:0/3336005325 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541f3730 0x555f541fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1a5c6d640 1 -- 172.21.15.44:0/3336005325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541f3730 msgr2=0x555f541fcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1a5c6d640 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541f3730 0x555f541fcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1a5c6d640 1 -- 172.21.15.44:0/3336005325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 msgr2=0x555f541fd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:47.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1a5c6d640 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 0x555f541fd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1a5c6d640 1 -- 172.21.15.44:0/3336005325 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f5420be90 con 0x555f541f4050 2024-08-20T03:10:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 0x555f541fd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb1a5c6d640 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f541fd4a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1a010f370 tx=0x7fb190012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb23bb5d41388336 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.424+0000 7fb189ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:10:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb189ffb640 1 -- 172.21.15.44:0/3336005325 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190004320 con 0x555f541f4050 2024-08-20T03:10:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb189ffb640 1 -- 172.21.15.44:0/3336005325 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fb190011970 con 0x555f541f4050 2024-08-20T03:10:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3336005325 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555f54109150 con 0x555f541f4050 2024-08-20T03:10:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb189ffb640 1 -- 172.21.15.44:0/3336005325 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb1900044c0 con 0x555f541f4050 2024-08-20T03:10:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb1aaacb740 10 client.26645 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3336005325 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555f5420e880 con 0x555f541f4050 2024-08-20T03:10:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb189ffb640 1 -- 172.21.15.44:0/3336005325 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555f54109150 con 0x555f541f4050 2024-08-20T03:10:47.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb189ffb640 1 -- 172.21.15.44:0/3336005325 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7fb19000f440 con 0x555f541f4050 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb1aaacb740 10 client.26645 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb1aaacb740 10 client.26645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb1aaacb740 20 client.26645 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x555f5420fd80 0x555f54212170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb1aaacb740 4 client.26645 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.425+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3336005325 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f5416c880 con 0x555f5420fd80 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.426+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x555f5420fd80 0x555f54212170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.426+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x555f5420fd80 0x555f54212170 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3e7783e1373d55c server_cookie=1df8f1408d6a6030 in_seq=0 out_seq=0 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.426+0000 7fb189ffb640 10 client.26645 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb189ffb640 1 -- 172.21.15.44:0/3336005325 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f5416c880 con 0x555f5420fd80 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb189ffb640 10 client.26645 handle_command_reply: tid=0 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb1aaacb740 1 client.26645 shutdown 2024-08-20T03:10:47.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3336005325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 msgr2=0x555f541fd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f541fd4a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1a010f370 tx=0x7fb190012b60 comp rx=0 tx=0).stop 2024-08-20T03:10:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3336005325 shutdown_connections 2024-08-20T03:10:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x555f5420fd80 0x555f54212170 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 0x555f541fd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f541fd4a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/3336005325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541f3730 0x555f541fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb189ffb640 0 client.26645 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:10:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3336005325 >> 172.21.15.44:0/3336005325 conn(0x555f541f1560 msgr2=0x555f541f7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3336005325 shutdown_connections 2024-08-20T03:10:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3336005325 wait complete. 2024-08-20T03:10:47.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.427+0000 7fb1aaacb740 20 client.26645 trim_cache size 0 max 16384 2024-08-20T03:10:47.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1aaacb740 1 Processor -- start 2024-08-20T03:10:47.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1aaacb740 1 -- start start 2024-08-20T03:10:47.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1aaacb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541f3730 0x555f54217a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1aaacb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f54217f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1aaacb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 0x555f54207d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1aaacb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01ae2e0 con 0x555f541fffe0 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1aaacb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01affa0 con 0x555f541f4050 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1aaacb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01aea40 con 0x555f541f3730 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1a6c6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 0x555f54207d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1a6c6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 0x555f54207d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60010/0 (socket says 172.21.15.44:60010) 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/3046313466 learned_addr learned my addr 172.21.15.44:0/3046313466 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1a5c6d640 1 --2- 172.21.15.44:0/3046313466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f54217f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.441+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/3046313466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541f3730 0x555f54217a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.442+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/3046313466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541f3730 msgr2=0x555f54217a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.442+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/3046313466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541f3730 0x555f54217a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.442+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/3046313466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 msgr2=0x555f54217f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.442+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/3046313466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f54217f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.442+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/3046313466 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f541f3e80 con 0x555f541fffe0 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.442+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/3046313466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 0x555f54207d60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb198005ef0 tx=0x7fb1980093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3557d1595baeddb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:47.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.442+0000 7fb18bfff640 1 -- 172.21.15.44:0/3046313466 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198031bd0 con 0x555f541fffe0 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.442+0000 7fb18bfff640 1 -- 172.21.15.44:0/3046313466 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb198036040 con 0x555f541fffe0 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.442+0000 7fb18bfff640 1 -- 172.21.15.44:0/3046313466 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19802f660 con 0x555f541fffe0 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3046313466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 msgr2=0x555f54207d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/3046313466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 0x555f54207d60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb198005ef0 tx=0x7fb1980093a0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3046313466 shutdown_connections 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/3046313466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541fffe0 0x555f54207d60 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/3046313466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f54217f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/3046313466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541f3730 0x555f54217a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3046313466 >> 172.21.15.44:0/3046313466 conn(0x555f541f1560 msgr2=0x555f541f04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3046313466 shutdown_connections 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 -- 172.21.15.44:0/3046313466 wait complete. 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 -- start start 2024-08-20T03:10:47.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 Processor -- start 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 0x555f54205440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f542059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541fffe0 0x555f54278a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01af8d0 con 0x555f541f3730 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01af1a0 con 0x555f541f4050 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01ac2e0 con 0x555f541fffe0 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1aaacb740 10 client.0 _ll_register_callbacks cb 0x7fb190009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.443+0000 7fb1a6c6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 0x555f54205440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb1a6c6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 0x555f54205440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60022/0 (socket says 172.21.15.44:60022) 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/1443838856 learned_addr learned my addr 172.21.15.44:0/1443838856 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb1a5c6d640 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f542059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541fffe0 0x555f54278a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/1443838856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541fffe0 msgr2=0x555f54278a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541fffe0 0x555f54278a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/1443838856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 msgr2=0x555f542059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f542059a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb1a6c6f640 1 -- 172.21.15.44:0/1443838856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f54279100 con 0x555f541f3730 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb1a6c6f640 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 0x555f54205440 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb198003ba0 tx=0x7fb19802b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd4e6a01d9147b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb189ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb189ffb640 1 -- 172.21.15.44:0/1443838856 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198059020 con 0x555f541f3730 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb1aaacb740 1 -- 172.21.15.44:0/1443838856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb198036040 con 0x555f541f3730 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.444+0000 7fb189ffb640 1 -- 172.21.15.44:0/1443838856 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fb198031650 con 0x555f541f3730 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.445+0000 7fb189ffb640 1 -- 172.21.15.44:0/1443838856 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb19804f040 con 0x555f541f3730 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.445+0000 7fb1aaacb740 10 client.17244 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.445+0000 7fb1aaacb740 1 -- 172.21.15.44:0/1443838856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555f5427a400 con 0x555f541f3730 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.445+0000 7fb189ffb640 1 -- 172.21.15.44:0/1443838856 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb198036040 con 0x555f541f3730 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.445+0000 7fb189ffb640 1 -- 172.21.15.44:0/1443838856 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7fb198052030 con 0x555f541f3730 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.445+0000 7fb1aaacb740 10 client.17244 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.445+0000 7fb1aaacb740 10 client.17244 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:10:47.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.445+0000 7fb1aaacb740 20 client.17244 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.445+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x555f5427a990 0x555f5427cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.445+0000 7fb1aaacb740 4 client.17244 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.445+0000 7fb1aaacb740 1 -- 172.21.15.44:0/1443838856 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f54168320 con 0x555f5427a990 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.445+0000 7fb1a5c6d640 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x555f5427a990 0x555f5427cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb189ffb640 10 client.17244 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1a5c6d640 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x555f5427a990 0x555f5427cdb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79e47811a57db07b server_cookie=c694b9d28217ca88 in_seq=0 out_seq=0 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb189ffb640 1 -- 172.21.15.44:0/1443838856 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f54168320 con 0x555f5427a990 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb189ffb640 10 client.17244 handle_command_reply: tid=0 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1aaacb740 1 client.17244 shutdown 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1aaacb740 1 -- 172.21.15.44:0/1443838856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 msgr2=0x555f54205440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 0x555f54205440 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb198003ba0 tx=0x7fb19802b040 comp rx=0 tx=0).stop 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1aaacb740 1 -- 172.21.15.44:0/1443838856 shutdown_connections 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x555f5427a990 0x555f5427cdb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555f541fffe0 0x555f54278a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555f541f4050 0x555f542059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb189ffb640 0 client.17244 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:10:47.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1aaacb740 1 --2- 172.21.15.44:0/1443838856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555f541f3730 0x555f54205440 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1aaacb740 1 -- 172.21.15.44:0/1443838856 >> 172.21.15.44:0/1443838856 conn(0x555f541f1560 msgr2=0x555f541fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:47.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1aaacb740 1 -- 172.21.15.44:0/1443838856 shutdown_connections 2024-08-20T03:10:47.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1aaacb740 1 -- 172.21.15.44:0/1443838856 wait complete. 2024-08-20T03:10:47.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.446+0000 7fb1aaacb740 20 client.17244 trim_cache size 0 max 16384 2024-08-20T03:10:47.458 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:10:47.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.447+0000 7fb1a86d2640 1 -- 172.21.15.44:0/4046458857 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb18c0b3d40 msgr2=0x7fb18c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:47.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.447+0000 7fb1a86d2640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb18c0b3d40 0x7fb18c0b6200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x555f541f68b0 tx=0x7fb19c0744b0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.447+0000 7fb1a86d2640 1 -- 172.21.15.44:0/4046458857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a01039a0 msgr2=0x7fb1a0198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:47.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.447+0000 7fb1a86d2640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a01039a0 0x7fb1a0198fa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00e9f0 tx=0x7fb19c00eeb0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.447+0000 7fb1a86d2640 1 -- 172.21.15.44:0/4046458857 shutdown_connections 2024-08-20T03:10:47.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.447+0000 7fb1a86d2640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb18c0b3d40 0x7fb18c0b6200 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.447+0000 7fb1a86d2640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb1a010ce00 0x7fb1a019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.447+0000 7fb1a86d2640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1a01042c0 0x7fb1a01994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.447+0000 7fb1a86d2640 1 --2- 172.21.15.44:0/4046458857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1a01039a0 0x7fb1a0198fa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:47.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.447+0000 7fb1a86d2640 1 -- 172.21.15.44:0/4046458857 >> 172.21.15.44:0/4046458857 conn(0x7fb1a00fd400 msgr2=0x7fb1a00fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:47.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.447+0000 7fb1a86d2640 1 -- 172.21.15.44:0/4046458857 shutdown_connections 2024-08-20T03:10:47.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:47.447+0000 7fb1a86d2640 1 -- 172.21.15.44:0/4046458857 wait complete. 2024-08-20T03:10:48.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:47 smithi165 ceph-mon[22357]: pgmap v573: 97 pgs: 97 active+clean; 841 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 449 op/s 2024-08-20T03:10:48.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:47 smithi099 ceph-mon[21984]: pgmap v573: 97 pgs: 97 active+clean; 841 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 449 op/s 2024-08-20T03:10:48.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:47 smithi044 ceph-mon[18790]: pgmap v573: 97 pgs: 97 active+clean; 841 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 449 op/s 2024-08-20T03:10:48.345 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T03:10:48.672 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:10:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.178+0000 7f806c5af640 1 -- 172.21.15.44:0/3869330709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80641042c0 msgr2=0x7f806410c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.178+0000 7f806c5af640 1 --2- 172.21.15.44:0/3869330709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80641042c0 0x7f806410c7b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8058009900 tx=0x7f805802a410 comp rx=0 tx=0).stop 2024-08-20T03:10:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.178+0000 7f806c5af640 1 -- 172.21.15.44:0/3869330709 shutdown_connections 2024-08-20T03:10:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.178+0000 7f806c5af640 1 --2- 172.21.15.44:0/3869330709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806410ce00 0x7f806410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.178+0000 7f806c5af640 1 --2- 172.21.15.44:0/3869330709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80641042c0 0x7f806410c7b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.178+0000 7f806c5af640 1 --2- 172.21.15.44:0/3869330709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80641039a0 0x7f8064103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.178+0000 7f806c5af640 1 -- 172.21.15.44:0/3869330709 >> 172.21.15.44:0/3869330709 conn(0x7f80640fd400 msgr2=0x7f80640ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.178+0000 7f806c5af640 1 -- 172.21.15.44:0/3869330709 shutdown_connections 2024-08-20T03:10:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.178+0000 7f806c5af640 1 -- 172.21.15.44:0/3869330709 wait complete. 2024-08-20T03:10:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.178+0000 7f806c5af640 1 Processor -- start 2024-08-20T03:10:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.178+0000 7f806c5af640 1 -- start start 2024-08-20T03:10:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f806c5af640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80641039a0 0x7f8064199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f806c5af640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80641042c0 0x7f8064199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f806c5af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806410ce00 0x7f806419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f806c5af640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8064112a70 con 0x7f80641042c0 2024-08-20T03:10:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f806c5af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80641128f0 con 0x7f806410ce00 2024-08-20T03:10:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f806c5af640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8064112bf0 con 0x7f80641039a0 2024-08-20T03:10:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f8069b4a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80641042c0 0x7f8064199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f8069b4a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80641042c0 0x7f8064199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60032/0 (socket says 172.21.15.44:60032) 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f8069b4a640 1 -- 172.21.15.44:0/1688492075 learned_addr learned my addr 172.21.15.44:0/1688492075 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f806ab4c640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806410ce00 0x7f806419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f806a34b640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80641039a0 0x7f8064199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f8069b4a640 1 -- 172.21.15.44:0/1688492075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80641039a0 msgr2=0x7f8064199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f8069b4a640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80641039a0 0x7f8064199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f8069b4a640 1 -- 172.21.15.44:0/1688492075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806410ce00 msgr2=0x7f806419e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f8069b4a640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806410ce00 0x7f806419e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f8069b4a640 1 -- 172.21.15.44:0/1688492075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806419f0d0 con 0x7f80641042c0 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f806ab4c640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806410ce00 0x7f806419e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f806a34b640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80641039a0 0x7f8064199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.179+0000 7f8069b4a640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80641042c0 0x7f8064199560 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8058032fd0 tx=0x7f8058031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.180+0000 7f80637fe640 1 -- 172.21.15.44:0/1688492075 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805803f030 con 0x7f80641042c0 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.180+0000 7f806c5af640 1 -- 172.21.15.44:0/1688492075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806419f360 con 0x7f80641042c0 2024-08-20T03:10:49.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.180+0000 7f80637fe640 1 -- 172.21.15.44:0/1688492075 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8058004410 con 0x7f80641042c0 2024-08-20T03:10:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.180+0000 7f80637fe640 1 -- 172.21.15.44:0/1688492075 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058035070 con 0x7f80641042c0 2024-08-20T03:10:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.180+0000 7f806c5af640 1 -- 172.21.15.44:0/1688492075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80641a5a80 con 0x7f80641042c0 2024-08-20T03:10:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.182+0000 7f80637fe640 1 -- 172.21.15.44:0/1688492075 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8058042030 con 0x7f80641042c0 2024-08-20T03:10:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.182+0000 7f806e9a8740 1 Processor -- start 2024-08-20T03:10:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.182+0000 7f806e9a8740 1 -- start start 2024-08-20T03:10:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806e9a8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186a730 0x55e98186ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806e9a8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98186b050 0x55e981876aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806e9a8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e981876fe0 0x55e9818793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806e9a8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e9817e1930 con 0x55e98186b050 2024-08-20T03:10:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806e9a8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e9817d0e30 con 0x55e981876fe0 2024-08-20T03:10:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806e9a8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e9817d4e70 con 0x55e98186a730 2024-08-20T03:10:49.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806ab4c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186a730 0x55e98186ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806ab4c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186a730 0x55e98186ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:38768/0 (socket says 172.21.15.44:38768) 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806ab4c640 1 -- 172.21.15.44:0/3527187912 learned_addr learned my addr 172.21.15.44:0/3527187912 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806a34b640 1 --2- 172.21.15.44:0/3527187912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98186b050 0x55e981876aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f8069b4a640 1 --2- 172.21.15.44:0/3527187912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e981876fe0 0x55e9818793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f80637fe640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f804c0b3d40 0x7f804c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806ab4c640 1 -- 172.21.15.44:0/3527187912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e981876fe0 msgr2=0x55e9818793d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806ab4c640 1 --2- 172.21.15.44:0/3527187912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e981876fe0 0x55e9818793d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806ab4c640 1 -- 172.21.15.44:0/3527187912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98186b050 msgr2=0x55e981876aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806ab4c640 1 --2- 172.21.15.44:0/3527187912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98186b050 0x55e981876aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806ab4c640 1 -- 172.21.15.44:0/3527187912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e981530650 con 0x55e98186a730 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806ab4c640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f804c0b3d40 0x7f804c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f80637fe640 1 -- 172.21.15.44:0/1688492075 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f8058104250 con 0x7f80641042c0 2024-08-20T03:10:49.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.183+0000 7f806ab4c640 1 --2- 172.21.15.44:0/3527187912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186a730 0x55e98186ab10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f805c0116f0 tx=0x7f805c011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b58687ced064f63a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:49.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.184+0000 7f8060ff9640 1 -- 172.21.15.44:0/3527187912 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c017050 con 0x55e98186a730 2024-08-20T03:10:49.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.184+0000 7f806ab4c640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f804c0b3d40 0x7f804c0b6200 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f805c023000 tx=0x7f805c024000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:49.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.184+0000 7f8060ff9640 1 -- 172.21.15.44:0/3527187912 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805c01bd80 con 0x55e98186a730 2024-08-20T03:10:49.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.184+0000 7f8060ff9640 1 -- 172.21.15.44:0/3527187912 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c026c20 con 0x55e98186a730 2024-08-20T03:10:49.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.184+0000 7f806e9a8740 1 -- 172.21.15.44:0/3527187912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186a730 msgr2=0x55e98186ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:49.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.184+0000 7f806e9a8740 1 --2- 172.21.15.44:0/3527187912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186a730 0x55e98186ab10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f805c0116f0 tx=0x7f805c011bb0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.184+0000 7f806e9a8740 1 -- 172.21.15.44:0/3527187912 shutdown_connections 2024-08-20T03:10:49.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.184+0000 7f806e9a8740 1 --2- 172.21.15.44:0/3527187912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e981876fe0 0x55e9818793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.184+0000 7f806e9a8740 1 --2- 172.21.15.44:0/3527187912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98186b050 0x55e981876aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.184+0000 7f806e9a8740 1 --2- 172.21.15.44:0/3527187912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186a730 0x55e98186ab10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.184+0000 7f806e9a8740 1 -- 172.21.15.44:0/3527187912 >> 172.21.15.44:0/3527187912 conn(0x55e981868560 msgr2=0x55e981869700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:49.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806e9a8740 1 -- 172.21.15.44:0/3527187912 shutdown_connections 2024-08-20T03:10:49.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806e9a8740 1 -- 172.21.15.44:0/3527187912 wait complete. 2024-08-20T03:10:49.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806e9a8740 1 -- start start 2024-08-20T03:10:49.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806e9a8740 1 Processor -- start 2024-08-20T03:10:49.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806e9a8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98186a730 0x55e981873fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806e9a8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186b050 0x55e9818744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806e9a8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e981876fe0 0x55e981874a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806e9a8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e981633700 con 0x55e98186a730 2024-08-20T03:10:49.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806e9a8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e981757810 con 0x55e98186b050 2024-08-20T03:10:49.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806e9a8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e981879c90 con 0x55e981876fe0 2024-08-20T03:10:49.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806e9a8740 10 client.0 _ll_register_callbacks cb 0x55e9817ca700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:49.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806a34b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98186a730 0x55e981873fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806ab4c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e981876fe0 0x55e981874a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.185+0000 7f806a34b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98186a730 0x55e981873fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60046/0 (socket says 172.21.15.44:60046) 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f806a34b640 1 -- 172.21.15.44:0/2433385206 learned_addr learned my addr 172.21.15.44:0/2433385206 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f8069b4a640 1 --2- 172.21.15.44:0/2433385206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186b050 0x55e9818744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f806ab4c640 1 -- 172.21.15.44:0/2433385206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186b050 msgr2=0x55e9818744e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f806ab4c640 1 --2- 172.21.15.44:0/2433385206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186b050 0x55e9818744e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f806ab4c640 1 -- 172.21.15.44:0/2433385206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98186a730 msgr2=0x55e981873fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f806ab4c640 1 --2- 172.21.15.44:0/2433385206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98186a730 0x55e981873fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f806ab4c640 1 -- 172.21.15.44:0/2433385206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9818830a0 con 0x55e981876fe0 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f806ab4c640 1 --2- 172.21.15.44:0/2433385206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e981876fe0 0x55e981874a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f805c0116f0 tx=0x7f805c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b74f1212731e3f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f8049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f8049ffb640 1 -- 172.21.15.44:0/2433385206 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c034060 con 0x55e981876fe0 2024-08-20T03:10:49.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f806e9a8740 1 -- 172.21.15.44:0/2433385206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e981780150 con 0x55e981876fe0 2024-08-20T03:10:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f8049ffb640 1 -- 172.21.15.44:0/2433385206 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f805c01bae0 con 0x55e981876fe0 2024-08-20T03:10:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f8049ffb640 1 -- 172.21.15.44:0/2433385206 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f805c017020 con 0x55e981876fe0 2024-08-20T03:10:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f806e9a8740 10 client.26632 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.186+0000 7f806e9a8740 1 -- 172.21.15.44:0/2433385206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e981885a50 con 0x55e981876fe0 2024-08-20T03:10:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.187+0000 7f8049ffb640 1 -- 172.21.15.44:0/2433385206 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e981780150 con 0x55e981876fe0 2024-08-20T03:10:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.187+0000 7f8049ffb640 1 -- 172.21.15.44:0/2433385206 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f805c02e070 con 0x55e981876fe0 2024-08-20T03:10:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.187+0000 7f806e9a8740 10 client.26632 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.187+0000 7f806e9a8740 10 client.26632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.187+0000 7f806e9a8740 20 client.26632 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.187+0000 7f806e9a8740 1 --2- 172.21.15.44:0/2433385206 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e981886e50 0x55e981889240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.187+0000 7f806e9a8740 4 client.26632 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:49.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.187+0000 7f806e9a8740 1 -- 172.21.15.44:0/2433385206 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9817e3880 con 0x55e981886e50 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.187+0000 7f806a34b640 1 --2- 172.21.15.44:0/2433385206 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e981886e50 0x55e981889240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.187+0000 7f806a34b640 1 --2- 172.21.15.44:0/2433385206 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e981886e50 0x55e981889240 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc169c9ef0b17115 server_cookie=263083245d21fee0 in_seq=0 out_seq=0 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.187+0000 7f8049ffb640 10 client.26632 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f8049ffb640 1 -- 172.21.15.44:0/2433385206 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e9817e3880 con 0x55e981886e50 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f8049ffb640 10 client.26632 handle_command_reply: tid=0 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f806e9a8740 1 client.26632 shutdown 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f806e9a8740 1 -- 172.21.15.44:0/2433385206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e981876fe0 msgr2=0x55e981874a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f806e9a8740 1 --2- 172.21.15.44:0/2433385206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e981876fe0 0x55e981874a20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f805c0116f0 tx=0x7f805c009310 comp rx=0 tx=0).stop 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f806e9a8740 1 -- 172.21.15.44:0/2433385206 shutdown_connections 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f806e9a8740 1 --2- 172.21.15.44:0/2433385206 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e981886e50 0x55e981889240 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f806e9a8740 1 --2- 172.21.15.44:0/2433385206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e981876fe0 0x55e981874a20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f806e9a8740 1 --2- 172.21.15.44:0/2433385206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186b050 0x55e9818744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f8049ffb640 0 client.26632 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:49.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f806e9a8740 1 --2- 172.21.15.44:0/2433385206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e98186a730 0x55e981873fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f806e9a8740 1 -- 172.21.15.44:0/2433385206 >> 172.21.15.44:0/2433385206 conn(0x55e981868560 msgr2=0x55e98186e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:49.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f806e9a8740 1 -- 172.21.15.44:0/2433385206 shutdown_connections 2024-08-20T03:10:49.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f806e9a8740 1 -- 172.21.15.44:0/2433385206 wait complete. 2024-08-20T03:10:49.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.189+0000 7f806e9a8740 20 client.26632 trim_cache size 0 max 16384 2024-08-20T03:10:49.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.202+0000 7f806e9a8740 1 Processor -- start 2024-08-20T03:10:49.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.202+0000 7f806e9a8740 1 -- start start 2024-08-20T03:10:49.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806e9a8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186a730 0x55e9818677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806e9a8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186b050 0x55e981873830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806e9a8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e981876fe0 0x55e98186e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806e9a8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f80641ae2f0 con 0x55e981876fe0 2024-08-20T03:10:49.677 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:10:49.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806e9a8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80641affb0 con 0x55e98186b050 2024-08-20T03:10:49.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806e9a8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f80641aea50 con 0x55e98186a730 2024-08-20T03:10:49.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806ab4c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e981876fe0 0x55e98186e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806ab4c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e981876fe0 0x55e98186e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60056/0 (socket says 172.21.15.44:60056) 2024-08-20T03:10:49.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806ab4c640 1 -- 172.21.15.44:0/2251001135 learned_addr learned my addr 172.21.15.44:0/2251001135 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:49.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806a34b640 1 --2- 172.21.15.44:0/2251001135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186a730 0x55e9818677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f8069b4a640 1 --2- 172.21.15.44:0/2251001135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186b050 0x55e981873830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806ab4c640 1 -- 172.21.15.44:0/2251001135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186a730 msgr2=0x55e9818677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:49.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806ab4c640 1 --2- 172.21.15.44:0/2251001135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186a730 0x55e9818677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806ab4c640 1 -- 172.21.15.44:0/2251001135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186b050 msgr2=0x55e981873830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:49.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806ab4c640 1 --2- 172.21.15.44:0/2251001135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186b050 0x55e981873830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806ab4c640 1 -- 172.21.15.44:0/2251001135 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e981868fe0 con 0x55e981876fe0 2024-08-20T03:10:49.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f806ab4c640 1 --2- 172.21.15.44:0/2251001135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e981876fe0 0x55e98186e7a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f805c029040 tx=0x7f805c011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9c7373544fe613 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:49.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f804bfff640 1 -- 172.21.15.44:0/2251001135 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c015030 con 0x55e981876fe0 2024-08-20T03:10:49.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f804bfff640 1 -- 172.21.15.44:0/2251001135 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805c009d90 con 0x55e981876fe0 2024-08-20T03:10:49.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.203+0000 7f804bfff640 1 -- 172.21.15.44:0/2251001135 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c034040 con 0x55e981876fe0 2024-08-20T03:10:49.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.204+0000 7f806e9a8740 1 -- 172.21.15.44:0/2251001135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e981876fe0 msgr2=0x55e98186e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:49.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.204+0000 7f806e9a8740 1 --2- 172.21.15.44:0/2251001135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e981876fe0 0x55e98186e7a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f805c029040 tx=0x7f805c011bb0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.204+0000 7f806e9a8740 1 -- 172.21.15.44:0/2251001135 shutdown_connections 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.204+0000 7f806e9a8740 1 --2- 172.21.15.44:0/2251001135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e981876fe0 0x55e98186e7a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.204+0000 7f806e9a8740 1 --2- 172.21.15.44:0/2251001135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186b050 0x55e981873830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.204+0000 7f806e9a8740 1 --2- 172.21.15.44:0/2251001135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186a730 0x55e9818677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.204+0000 7f806e9a8740 1 -- 172.21.15.44:0/2251001135 >> 172.21.15.44:0/2251001135 conn(0x55e981868560 msgr2=0x55e981875df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.204+0000 7f806e9a8740 1 -- 172.21.15.44:0/2251001135 shutdown_connections 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.204+0000 7f806e9a8740 1 -- 172.21.15.44:0/2251001135 wait complete. 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.204+0000 7f806e9a8740 1 -- start start 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f806e9a8740 1 Processor -- start 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f806e9a8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186a730 0x55e981879ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f806e9a8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186b050 0x55e98187a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f806e9a8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e981876fe0 0x55e98187a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f806e9a8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f80641af8e0 con 0x55e981876fe0 2024-08-20T03:10:49.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f806e9a8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80641af1b0 con 0x55e98186a730 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f806e9a8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f80641ac2f0 con 0x55e98186b050 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f806e9a8740 10 client.0 _ll_register_callbacks cb 0x7f8054009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f8069b4a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186b050 0x55e98187a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f806ab4c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e981876fe0 0x55e98187a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f8069b4a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186b050 0x55e98187a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:38796/0 (socket says 172.21.15.44:38796) 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f8069b4a640 1 -- 172.21.15.44:0/1157452153 learned_addr learned my addr 172.21.15.44:0/1157452153 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f806a34b640 1 --2- 172.21.15.44:0/1157452153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186a730 0x55e981879ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f8069b4a640 1 -- 172.21.15.44:0/1157452153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186a730 msgr2=0x55e981879ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f8069b4a640 1 --2- 172.21.15.44:0/1157452153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186a730 0x55e981879ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f8069b4a640 1 -- 172.21.15.44:0/1157452153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e981876fe0 msgr2=0x55e98187a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f8069b4a640 1 --2- 172.21.15.44:0/1157452153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e981876fe0 0x55e98187a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.205+0000 7f8069b4a640 1 -- 172.21.15.44:0/1157452153 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9818efb90 con 0x55e98186b050 2024-08-20T03:10:49.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f8069b4a640 1 --2- 172.21.15.44:0/1157452153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186b050 0x55e98187a430 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f80580c6f30 tx=0x7f805802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=244ba477330717ed server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:49.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f8049ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:10:49.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f8049ffb640 1 -- 172.21.15.44:0/1157452153 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80580d6030 con 0x55e98186b050 2024-08-20T03:10:49.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f8049ffb640 1 -- 172.21.15.44:0/1157452153 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f80580c74a0 con 0x55e98186b050 2024-08-20T03:10:49.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f806e9a8740 1 -- 172.21.15.44:0/1157452153 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f805c009d90 con 0x55e98186b050 2024-08-20T03:10:49.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f8049ffb640 1 -- 172.21.15.44:0/1157452153 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8058049040 con 0x55e98186b050 2024-08-20T03:10:49.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f806e9a8740 10 client.26638 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:49.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f806e9a8740 1 -- 172.21.15.44:0/1157452153 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e9818f0e70 con 0x55e98186b050 2024-08-20T03:10:49.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f8049ffb640 1 -- 172.21.15.44:0/1157452153 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805c009d90 con 0x55e98186b050 2024-08-20T03:10:49.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f8049ffb640 1 -- 172.21.15.44:0/1157452153 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f80580d3a20 con 0x55e98186b050 2024-08-20T03:10:49.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f806e9a8740 10 client.26638 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:49.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f806e9a8740 10 client.26638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:49.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f806e9a8740 20 client.26638 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:49.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f806e9a8740 1 --2- 172.21.15.44:0/1157452153 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e9818f15a0 0x55e9818f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:49.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f806e9a8740 4 client.26638 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:10:49.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.206+0000 7f806e9a8740 1 -- 172.21.15.44:0/1157452153 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9817df320 con 0x55e9818f15a0 2024-08-20T03:10:49.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806a34b640 1 --2- 172.21.15.44:0/1157452153 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e9818f15a0 0x55e9818f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:49.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f8049ffb640 10 client.26638 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:49.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806a34b640 1 --2- 172.21.15.44:0/1157452153 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e9818f15a0 0x55e9818f39c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=112a754730498ec8 server_cookie=f38a359f1f67a8e8 in_seq=0 out_seq=0 2024-08-20T03:10:49.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f8049ffb640 1 -- 172.21.15.44:0/1157452153 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9817df320 con 0x55e9818f15a0 2024-08-20T03:10:49.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f8049ffb640 10 client.26638 handle_command_reply: tid=0 2024-08-20T03:10:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806e9a8740 1 client.26638 shutdown 2024-08-20T03:10:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806e9a8740 1 -- 172.21.15.44:0/1157452153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186b050 msgr2=0x55e98187a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806e9a8740 1 --2- 172.21.15.44:0/1157452153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186b050 0x55e98187a430 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f80580c6f30 tx=0x7f805802a410 comp rx=0 tx=0).stop 2024-08-20T03:10:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806e9a8740 1 -- 172.21.15.44:0/1157452153 shutdown_connections 2024-08-20T03:10:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806e9a8740 1 --2- 172.21.15.44:0/1157452153 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e9818f15a0 0x55e9818f39c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806e9a8740 1 --2- 172.21.15.44:0/1157452153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e981876fe0 0x55e98187a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f8049ffb640 0 client.26638 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806e9a8740 1 --2- 172.21.15.44:0/1157452153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e98186b050 0x55e98187a430 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806e9a8740 1 --2- 172.21.15.44:0/1157452153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e98186a730 0x55e981879ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806e9a8740 1 -- 172.21.15.44:0/1157452153 >> 172.21.15.44:0/1157452153 conn(0x55e981868560 msgr2=0x55e981881dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:49.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806e9a8740 1 -- 172.21.15.44:0/1157452153 shutdown_connections 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.207+0000 7f806e9a8740 1 -- 172.21.15.44:0/1157452153 wait complete. 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806e9a8740 20 client.26638 trim_cache size 0 max 16384 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806c5af640 1 -- 172.21.15.44:0/1688492075 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f804c0b3d40 msgr2=0x7f804c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806c5af640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f804c0b3d40 0x7f804c0b6200 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f805c023000 tx=0x7f805c024000 comp rx=0 tx=0).stop 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806c5af640 1 -- 172.21.15.44:0/1688492075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80641042c0 msgr2=0x7f8064199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806c5af640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80641042c0 0x7f8064199560 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8058032fd0 tx=0x7f8058031970 comp rx=0 tx=0).stop 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806c5af640 1 -- 172.21.15.44:0/1688492075 shutdown_connections 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806c5af640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f804c0b3d40 0x7f804c0b6200 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806c5af640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f806410ce00 0x7f806419e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806c5af640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80641042c0 0x7f8064199560 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806c5af640 1 --2- 172.21.15.44:0/1688492075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80641039a0 0x7f8064199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806c5af640 1 -- 172.21.15.44:0/1688492075 >> 172.21.15.44:0/1688492075 conn(0x7f80640fd400 msgr2=0x7f80640fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:49.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806c5af640 1 -- 172.21.15.44:0/1688492075 shutdown_connections 2024-08-20T03:10:49.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:49.208+0000 7f806c5af640 1 -- 172.21.15.44:0/1688492075 wait complete. 2024-08-20T03:10:50.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:49 smithi044 ceph-mon[18790]: pgmap v574: 97 pgs: 97 active+clean; 844 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 474 op/s 2024-08-20T03:10:50.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:49 smithi165 ceph-mon[22357]: pgmap v574: 97 pgs: 97 active+clean; 844 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 474 op/s 2024-08-20T03:10:50.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:49 smithi099 ceph-mon[21984]: pgmap v574: 97 pgs: 97 active+clean; 844 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 474 op/s 2024-08-20T03:10:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.413+0000 7f3748731640 1 -- 172.21.15.44:0/3328225660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3740103910 msgr2=0x7f3740103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.413+0000 7f3748731640 1 --2- 172.21.15.44:0/3328225660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3740103910 0x7f3740103cf0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3730009ca0 tx=0x7f373002a8c0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.413+0000 7f3748731640 1 -- 172.21.15.44:0/3328225660 shutdown_connections 2024-08-20T03:10:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.413+0000 7f3748731640 1 --2- 172.21.15.44:0/3328225660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f374010cd30 0x7f374010f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.413+0000 7f3748731640 1 --2- 172.21.15.44:0/3328225660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3740104230 0x7f374010c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.413+0000 7f3748731640 1 --2- 172.21.15.44:0/3328225660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3740103910 0x7f3740103cf0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.413+0000 7f3748731640 1 -- 172.21.15.44:0/3328225660 >> 172.21.15.44:0/3328225660 conn(0x7f3740078620 msgr2=0x7f37400ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.413+0000 7f3748731640 1 -- 172.21.15.44:0/3328225660 shutdown_connections 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.413+0000 7f3748731640 1 -- 172.21.15.44:0/3328225660 wait complete. 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.413+0000 7f3748731640 1 Processor -- start 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.413+0000 7f3748731640 1 -- start start 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3748731640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740103910 0x7f3740198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3748731640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3740104230 0x7f3740199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3748731640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374010cd30 0x7f374019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3748731640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37401129c0 con 0x7f374010cd30 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3748731640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3740112840 con 0x7f3740103910 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3748731640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3740112b40 con 0x7f3740104230 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3746cce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374010cd30 0x7f374019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f37464cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740103910 0x7f3740198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3746cce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374010cd30 0x7f374019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60082/0 (socket says 172.21.15.44:60082) 2024-08-20T03:10:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3746cce640 1 -- 172.21.15.44:0/1874409320 learned_addr learned my addr 172.21.15.44:0/1874409320 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3745ccc640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3740104230 0x7f3740199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3746cce640 1 -- 172.21.15.44:0/1874409320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3740104230 msgr2=0x7f3740199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3746cce640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3740104230 0x7f3740199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3746cce640 1 -- 172.21.15.44:0/1874409320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740103910 msgr2=0x7f3740198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3746cce640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740103910 0x7f3740198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3746cce640 1 -- 172.21.15.44:0/1874409320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374019efe0 con 0x7f374010cd30 2024-08-20T03:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3746cce640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374010cd30 0x7f374019e810 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f373800e970 tx=0x7f373800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f373f7fe640 1 -- 172.21.15.44:0/1874409320 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373800ccc0 con 0x7f374010cd30 2024-08-20T03:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f373f7fe640 1 -- 172.21.15.44:0/1874409320 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373800ce60 con 0x7f374010cd30 2024-08-20T03:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.414+0000 7f3748731640 1 -- 172.21.15.44:0/1874409320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374019f2d0 con 0x7f374010cd30 2024-08-20T03:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.415+0000 7f373f7fe640 1 -- 172.21.15.44:0/1874409320 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738010430 con 0x7f374010cd30 2024-08-20T03:10:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.415+0000 7f3748731640 1 -- 172.21.15.44:0/1874409320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37401a5990 con 0x7f374010cd30 2024-08-20T03:10:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.417+0000 7f373f7fe640 1 -- 172.21.15.44:0/1874409320 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3738004ff0 con 0x7f374010cd30 2024-08-20T03:10:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.417+0000 7f374b600740 1 Processor -- start 2024-08-20T03:10:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f374b600740 1 -- start start 2024-08-20T03:10:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f374b600740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 0x55e447d1afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f374b600740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1b510 0x55e447d2b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f374b600740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d2b8b0 0x55e447d2dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f374b600740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e447c90360 con 0x55e447d1abf0 2024-08-20T03:10:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f374b600740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e447c7f860 con 0x55e447d2b8b0 2024-08-20T03:10:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f374b600740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e447c838a0 con 0x55e447d1b510 2024-08-20T03:10:50.426 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f3745ccc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 0x55e447d1afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f3745ccc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 0x55e447d1afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60094/0 (socket says 172.21.15.44:60094) 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f3745ccc640 1 -- 172.21.15.44:0/1873535264 learned_addr learned my addr 172.21.15.44:0/1873535264 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f37464cd640 1 --2- 172.21.15.44:0/1873535264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1b510 0x55e447d2b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f3745ccc640 1 --2- 172.21.15.44:0/1873535264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d2b8b0 0x55e447d2dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f3745ccc640 1 -- 172.21.15.44:0/1873535264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1b510 msgr2=0x55e447d2b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f3745ccc640 1 --2- 172.21.15.44:0/1873535264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1b510 0x55e447d2b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f3745ccc640 1 -- 172.21.15.44:0/1873535264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d2b8b0 msgr2=0x55e447d2dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f3745ccc640 1 --2- 172.21.15.44:0/1873535264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d2b8b0 0x55e447d2dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f3745ccc640 1 -- 172.21.15.44:0/1873535264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e447c26e40 con 0x55e447d1abf0 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f373f7fe640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f37280b3d40 0x7f37280b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f37464cd640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f37280b3d40 0x7f37280b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f3745ccc640 1 --2- 172.21.15.44:0/1873535264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 0x55e447d1afd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f37340137e0 tx=0x7f3734005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ed162610df9e4b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:50.427 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.418+0000 7f373f7fe640 1 -- 172.21.15.44:0/1874409320 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f37380e43d0 con 0x7f374010cd30 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.419+0000 7f373cff9640 1 -- 172.21.15.44:0/1873535264 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734013e20 con 0x55e447d1abf0 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.419+0000 7f373cff9640 1 -- 172.21.15.44:0/1873535264 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3734013170 con 0x55e447d1abf0 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.419+0000 7f373cff9640 1 -- 172.21.15.44:0/1873535264 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734004d00 con 0x55e447d1abf0 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.419+0000 7f374b600740 1 -- 172.21.15.44:0/1873535264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 msgr2=0x55e447d1afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.419+0000 7f374b600740 1 --2- 172.21.15.44:0/1873535264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 0x55e447d1afd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f37340137e0 tx=0x7f3734005850 comp rx=0 tx=0).stop 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.419+0000 7f37464cd640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f37280b3d40 0x7f37280b6200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3730005e30 tx=0x7f3730009360 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.419+0000 7f374b600740 1 -- 172.21.15.44:0/1873535264 shutdown_connections 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.419+0000 7f374b600740 1 --2- 172.21.15.44:0/1873535264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d2b8b0 0x55e447d2dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.419+0000 7f374b600740 1 --2- 172.21.15.44:0/1873535264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1b510 0x55e447d2b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.419+0000 7f374b600740 1 --2- 172.21.15.44:0/1873535264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 0x55e447d1afd0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.419+0000 7f374b600740 1 -- 172.21.15.44:0/1873535264 >> 172.21.15.44:0/1873535264 conn(0x55e447d18a20 msgr2=0x55e447d19bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f374b600740 1 -- 172.21.15.44:0/1873535264 shutdown_connections 2024-08-20T03:10:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f374b600740 1 -- 172.21.15.44:0/1873535264 wait complete. 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f374b600740 1 -- start start 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f374b600740 1 Processor -- start 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f374b600740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1abf0 0x55e447d2b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f374b600740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1b510 0x55e447d285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f374b600740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d2b8b0 0x55e447d28ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f374b600740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55e447bb8ab0 con 0x55e447d1b510 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f374b600740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e447c06780 con 0x55e447d2b8b0 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f374b600740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55e447d2e530 con 0x55e447d1abf0 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f374b600740 10 client.0 _ll_register_callbacks cb 0x55e44798ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f3745ccc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1b510 0x55e447d285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f37464cd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1abf0 0x55e447d2b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f3745ccc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1b510 0x55e447d285c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60106/0 (socket says 172.21.15.44:60106) 2024-08-20T03:10:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f3745ccc640 1 -- 172.21.15.44:0/564341322 learned_addr learned my addr 172.21.15.44:0/564341322 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.420+0000 7f3746cce640 1 --2- 172.21.15.44:0/564341322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d2b8b0 0x55e447d28ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f3745ccc640 1 -- 172.21.15.44:0/564341322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1abf0 msgr2=0x55e447d2b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f3745ccc640 1 --2- 172.21.15.44:0/564341322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1abf0 0x55e447d2b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f3745ccc640 1 -- 172.21.15.44:0/564341322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d2b8b0 msgr2=0x55e447d28ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f3745ccc640 1 --2- 172.21.15.44:0/564341322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d2b8b0 0x55e447d28ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f3745ccc640 1 -- 172.21.15.44:0/564341322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4479e2790 con 0x55e447d1b510 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f3745ccc640 1 --2- 172.21.15.44:0/564341322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1b510 0x55e447d285c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f37340049b0 tx=0x7f3734005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bd30fac4c35f1d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f3725ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f3725ffb640 1 -- 172.21.15.44:0/564341322 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734004460 con 0x55e447d1b510 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f3725ffb640 1 -- 172.21.15.44:0/564341322 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f3734005020 con 0x55e447d1b510 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f374b600740 1 -- 172.21.15.44:0/564341322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e447c2ee00 con 0x55e447d1b510 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f3725ffb640 1 -- 172.21.15.44:0/564341322 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3734030050 con 0x55e447d1b510 2024-08-20T03:10:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f374b600740 10 client.17274 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f374b600740 1 -- 172.21.15.44:0/564341322 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e447a5b620 con 0x55e447d1b510 2024-08-20T03:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.421+0000 7f3725ffb640 1 -- 172.21.15.44:0/564341322 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e447c2ee00 con 0x55e447d1b510 2024-08-20T03:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.422+0000 7f3725ffb640 1 -- 172.21.15.44:0/564341322 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f3734005220 con 0x55e447d1b510 2024-08-20T03:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.422+0000 7f374b600740 10 client.17274 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.422+0000 7f374b600740 10 client.17274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.422+0000 7f374b600740 20 client.17274 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.422+0000 7f374b600740 1 --2- 172.21.15.44:0/564341322 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e447d3aff0 0x55e447d3d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.422+0000 7f374b600740 4 client.17274 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.422+0000 7f374b600740 1 -- 172.21.15.44:0/564341322 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e447c922b0 con 0x55e447d3aff0 2024-08-20T03:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.422+0000 7f37464cd640 1 --2- 172.21.15.44:0/564341322 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e447d3aff0 0x55e447d3d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.422+0000 7f3725ffb640 10 client.17274 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.422+0000 7f37464cd640 1 --2- 172.21.15.44:0/564341322 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e447d3aff0 0x55e447d3d3e0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4428305c445f61c server_cookie=4cce9f21ef29bb36 in_seq=0 out_seq=0 2024-08-20T03:10:50.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.423+0000 7f3725ffb640 1 -- 172.21.15.44:0/564341322 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e447c922b0 con 0x55e447d3aff0 2024-08-20T03:10:50.432 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.423+0000 7f3725ffb640 10 client.17274 handle_command_reply: tid=0 2024-08-20T03:10:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f374b600740 1 client.17274 shutdown 2024-08-20T03:10:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f374b600740 1 -- 172.21.15.44:0/564341322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1b510 msgr2=0x55e447d285c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f374b600740 1 --2- 172.21.15.44:0/564341322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1b510 0x55e447d285c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f37340049b0 tx=0x7f3734005f90 comp rx=0 tx=0).stop 2024-08-20T03:10:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f374b600740 1 -- 172.21.15.44:0/564341322 shutdown_connections 2024-08-20T03:10:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f374b600740 1 --2- 172.21.15.44:0/564341322 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e447d3aff0 0x55e447d3d3e0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f374b600740 1 --2- 172.21.15.44:0/564341322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d2b8b0 0x55e447d28ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f374b600740 1 --2- 172.21.15.44:0/564341322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1b510 0x55e447d285c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f374b600740 1 --2- 172.21.15.44:0/564341322 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1abf0 0x55e447d2b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f3725ffb640 0 client.17274 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f374b600740 1 -- 172.21.15.44:0/564341322 >> 172.21.15.44:0/564341322 conn(0x55e447d18a20 msgr2=0x55e447d1e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f374b600740 1 -- 172.21.15.44:0/564341322 shutdown_connections 2024-08-20T03:10:50.433 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f374b600740 1 -- 172.21.15.44:0/564341322 wait complete. 2024-08-20T03:10:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.424+0000 7f374b600740 20 client.17274 trim_cache size 0 max 16384 2024-08-20T03:10:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.438+0000 7f374b600740 1 Processor -- start 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.438+0000 7f374b600740 1 -- start start 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f374b600740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d1abf0 0x55e447d182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f374b600740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1b510 0x55e447d37680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f374b600740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d2b8b0 0x55e447d27a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f374b600740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37401ac030 con 0x55e447d2b8b0 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f374b600740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f37401adcf0 con 0x55e447d1abf0 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f374b600740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f37401ac790 con 0x55e447d1b510 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3746cce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d2b8b0 0x55e447d27a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f37464cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d1abf0 0x55e447d182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3746cce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d2b8b0 0x55e447d27a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60112/0 (socket says 172.21.15.44:60112) 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3746cce640 1 -- 172.21.15.44:0/3992830278 learned_addr learned my addr 172.21.15.44:0/3992830278 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3745ccc640 1 --2- 172.21.15.44:0/3992830278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1b510 0x55e447d37680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3746cce640 1 -- 172.21.15.44:0/3992830278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1b510 msgr2=0x55e447d37680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3746cce640 1 --2- 172.21.15.44:0/3992830278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1b510 0x55e447d37680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3746cce640 1 -- 172.21.15.44:0/3992830278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d1abf0 msgr2=0x55e447d182d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3746cce640 1 --2- 172.21.15.44:0/3992830278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d1abf0 0x55e447d182d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3746cce640 1 -- 172.21.15.44:0/3992830278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e447d3d970 con 0x55e447d2b8b0 2024-08-20T03:10:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f37464cd640 1 --2- 172.21.15.44:0/3992830278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d1abf0 0x55e447d182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3746cce640 1 --2- 172.21.15.44:0/3992830278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d2b8b0 0x55e447d27a90 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f37380aaf60 tx=0x7f3738005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedc26b5737575a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3727fff640 1 -- 172.21.15.44:0/3992830278 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37380c4070 con 0x55e447d2b8b0 2024-08-20T03:10:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3727fff640 1 -- 172.21.15.44:0/3992830278 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37380ab020 con 0x55e447d2b8b0 2024-08-20T03:10:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.439+0000 7f3727fff640 1 -- 172.21.15.44:0/3992830278 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37380b3c40 con 0x55e447d2b8b0 2024-08-20T03:10:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.440+0000 7f374b600740 1 -- 172.21.15.44:0/3992830278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d2b8b0 msgr2=0x55e447d27a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.440+0000 7f374b600740 1 --2- 172.21.15.44:0/3992830278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d2b8b0 0x55e447d27a90 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f37380aaf60 tx=0x7f3738005730 comp rx=0 tx=0).stop 2024-08-20T03:10:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.440+0000 7f374b600740 1 -- 172.21.15.44:0/3992830278 shutdown_connections 2024-08-20T03:10:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.440+0000 7f374b600740 1 --2- 172.21.15.44:0/3992830278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d2b8b0 0x55e447d27a90 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.440+0000 7f374b600740 1 --2- 172.21.15.44:0/3992830278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d1b510 0x55e447d37680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.440+0000 7f374b600740 1 --2- 172.21.15.44:0/3992830278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d1abf0 0x55e447d182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.440+0000 7f374b600740 1 -- 172.21.15.44:0/3992830278 >> 172.21.15.44:0/3992830278 conn(0x55e447d18a20 msgr2=0x55e447d2d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.440+0000 7f374b600740 1 -- 172.21.15.44:0/3992830278 shutdown_connections 2024-08-20T03:10:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.440+0000 7f374b600740 1 -- 172.21.15.44:0/3992830278 wait complete. 2024-08-20T03:10:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.440+0000 7f374b600740 1 -- start start 2024-08-20T03:10:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.440+0000 7f374b600740 1 Processor -- start 2024-08-20T03:10:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f374b600740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 0x55e447d339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f374b600740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d1b510 0x55e447d33f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f374b600740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d2b8b0 0x55e447d42240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f374b600740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37401ad620 con 0x55e447d1abf0 2024-08-20T03:10:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f374b600740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f37401acef0 con 0x55e447d1b510 2024-08-20T03:10:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f374b600740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f37401aa030 con 0x55e447d2b8b0 2024-08-20T03:10:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f37464cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 0x55e447d339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f37464cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 0x55e447d339e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60118/0 (socket says 172.21.15.44:60118) 2024-08-20T03:10:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f37464cd640 1 -- 172.21.15.44:0/1312134475 learned_addr learned my addr 172.21.15.44:0/1312134475 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f3745ccc640 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d1b510 0x55e447d33f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f374b600740 10 client.0 _ll_register_callbacks cb 0x7f3734009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f3746cce640 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d2b8b0 0x55e447d42240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f37464cd640 1 -- 172.21.15.44:0/1312134475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d2b8b0 msgr2=0x55e447d42240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f37464cd640 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d2b8b0 0x55e447d42240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f37464cd640 1 -- 172.21.15.44:0/1312134475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d1b510 msgr2=0x55e447d33f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f37464cd640 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d1b510 0x55e447d33f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f37464cd640 1 -- 172.21.15.44:0/1312134475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e447d42a70 con 0x55e447d1abf0 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f37464cd640 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 0x55e447d339e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f373002ade0 tx=0x7f37300095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6f92374e146518 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f3725ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f3725ffb640 1 -- 172.21.15.44:0/1312134475 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730031780 con 0x55e447d1abf0 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f3725ffb640 1 -- 172.21.15.44:0/1312134475 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f3730003970 con 0x55e447d1abf0 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.441+0000 7f374b600740 1 -- 172.21.15.44:0/1312134475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37380ab020 con 0x55e447d1abf0 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f3725ffb640 1 -- 172.21.15.44:0/1312134475 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3730031920 con 0x55e447d1abf0 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f374b600740 10 client.17280 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f374b600740 1 -- 172.21.15.44:0/1312134475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e447da5760 con 0x55e447d1abf0 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f3725ffb640 1 -- 172.21.15.44:0/1312134475 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37380ab020 con 0x55e447d1abf0 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f3725ffb640 1 -- 172.21.15.44:0/1312134475 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f373002f620 con 0x55e447d1abf0 2024-08-20T03:10:50.450 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f374b600740 10 client.17280 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f374b600740 10 client.17280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f374b600740 20 client.17280 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f374b600740 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e447da5e30 0x55e447da8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f374b600740 4 client.17280 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T03:10:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f374b600740 1 -- 172.21.15.44:0/1312134475 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e447c8dd50 con 0x55e447da5e30 2024-08-20T03:10:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f3745ccc640 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e447da5e30 0x55e447da8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f3745ccc640 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e447da5e30 0x55e447da8220 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5744a6f1db477811 server_cookie=ead643f786bb0f10 in_seq=0 out_seq=0 2024-08-20T03:10:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:50.442+0000 7f3725ffb640 10 client.17280 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f3725ffb640 1 -- 172.21.15.44:0/1312134475 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e447c8dd50 con 0x55e447da5e30 2024-08-20T03:10:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f3725ffb640 10 client.17280 handle_command_reply: tid=0 2024-08-20T03:10:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f374b600740 1 client.17280 shutdown 2024-08-20T03:10:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f374b600740 1 -- 172.21.15.44:0/1312134475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 msgr2=0x55e447d339e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:51.206 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f374b600740 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 0x55e447d339e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f373002ade0 tx=0x7f37300095c0 comp rx=0 tx=0).stop 2024-08-20T03:10:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f374b600740 1 -- 172.21.15.44:0/1312134475 shutdown_connections 2024-08-20T03:10:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f374b600740 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55e447da5e30 0x55e447da8220 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f3725ffb640 0 client.17280 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f374b600740 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e447d2b8b0 0x55e447d42240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f374b600740 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e447d1b510 0x55e447d33f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f374b600740 1 --2- 172.21.15.44:0/1312134475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e447d1abf0 0x55e447d339e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f374b600740 1 -- 172.21.15.44:0/1312134475 >> 172.21.15.44:0/1312134475 conn(0x55e447d18a20 msgr2=0x55e447d29bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f374b600740 1 -- 172.21.15.44:0/1312134475 shutdown_connections 2024-08-20T03:10:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.202+0000 7f374b600740 1 -- 172.21.15.44:0/1312134475 wait complete. 2024-08-20T03:10:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f374b600740 20 client.17280 trim_cache size 0 max 16384 2024-08-20T03:10:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:{ 2024-08-20T03:10:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "message": "", 2024-08-20T03:10:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout: "return_code": 0 2024-08-20T03:10:51.208 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stdout:} 2024-08-20T03:10:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f3748731640 1 -- 172.21.15.44:0/1874409320 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f37280b3d40 msgr2=0x7f37280b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f3748731640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f37280b3d40 0x7f37280b6200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3730005e30 tx=0x7f3730009360 comp rx=0 tx=0).stop 2024-08-20T03:10:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f3748731640 1 -- 172.21.15.44:0/1874409320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374010cd30 msgr2=0x7f374019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f3748731640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374010cd30 0x7f374019e810 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f373800e970 tx=0x7f373800ed70 comp rx=0 tx=0).stop 2024-08-20T03:10:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f3748731640 1 -- 172.21.15.44:0/1874409320 shutdown_connections 2024-08-20T03:10:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f3748731640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f37280b3d40 0x7f37280b6200 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f3748731640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f374010cd30 0x7f374019e810 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f3748731640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3740104230 0x7f3740199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f3748731640 1 --2- 172.21.15.44:0/1874409320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3740103910 0x7f3740198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f3748731640 1 -- 172.21.15.44:0/1874409320 >> 172.21.15.44:0/1874409320 conn(0x7f3740078620 msgr2=0x7f37400ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:51.209 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f3748731640 1 -- 172.21.15.44:0/1874409320 shutdown_connections 2024-08-20T03:10:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi044.front.sepia.ceph.com.smithi044.stderr:2024-08-20T03:10:51.203+0000 7f3748731640 1 -- 172.21.15.44:0/1874409320 wait complete. 2024-08-20T03:10:51.559 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T03:10:51.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T03:10:51.902 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:10:52.022 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:51 smithi044 ceph-mon[18790]: pgmap v575: 97 pgs: 97 active+clean; 844 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 406 op/s 2024-08-20T03:10:52.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:51 smithi165 ceph-mon[22357]: pgmap v575: 97 pgs: 97 active+clean; 844 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 406 op/s 2024-08-20T03:10:52.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:51 smithi099 ceph-mon[21984]: pgmap v575: 97 pgs: 97 active+clean; 844 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 406 op/s 2024-08-20T03:10:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.516+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2527653947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b0106b50 msgr2=0x7fc0b0106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.516+0000 7fc0b64a5640 1 --2- 172.21.15.44:0/2527653947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b0106b50 0x7fc0b0106f90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009900 tx=0x7fc0a402a410 comp rx=0 tx=0).stop 2024-08-20T03:10:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.516+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2527653947 shutdown_connections 2024-08-20T03:10:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.516+0000 7fc0b64a5640 1 --2- 172.21.15.44:0/2527653947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b01074d0 0x7fc0b010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.516+0000 7fc0b64a5640 1 --2- 172.21.15.44:0/2527653947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b0106b50 0x7fc0b0106f90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.516+0000 7fc0b64a5640 1 --2- 172.21.15.44:0/2527653947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b0101520 0x7fc0b0101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.516+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2527653947 >> 172.21.15.44:0/2527653947 conn(0x7fc0b0076c80 msgr2=0x7fc0b0077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.516+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2527653947 shutdown_connections 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.516+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2527653947 wait complete. 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.517+0000 7fc0b64a5640 1 Processor -- start 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.517+0000 7fc0b64a5640 1 -- start start 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.517+0000 7fc0b64a5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b0101520 0x7fc0b019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0b64a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b01074d0 0x7fc0b019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0b64a5640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b019ab20 0x7fc0b01a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0b64a5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0111100 con 0x7fc0b01074d0 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0b64a5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0110f80 con 0x7fc0b0101520 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0b64a5640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0111280 con 0x7fc0b019ab20 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b01074d0 0x7fc0b019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.536 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b01074d0 0x7fc0b019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60142/0 (socket says 172.21.15.44:60142) 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0af7fe640 1 -- 172.21.15.44:0/2047021314 learned_addr learned my addr 172.21.15.44:0/2047021314 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0affff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b0101520 0x7fc0b019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0b4a42640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b019ab20 0x7fc0b01a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0af7fe640 1 -- 172.21.15.44:0/2047021314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b019ab20 msgr2=0x7fc0b01a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0af7fe640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b019ab20 0x7fc0b01a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0af7fe640 1 -- 172.21.15.44:0/2047021314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b0101520 msgr2=0x7fc0b019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0af7fe640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b0101520 0x7fc0b019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0af7fe640 1 -- 172.21.15.44:0/2047021314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b01a5870 con 0x7fc0b01074d0 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0af7fe640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b01074d0 0x7fc0b019a5e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4006010 tx=0x7fc0a40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0ad7fa640 1 -- 172.21.15.44:0/2047021314 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a403f030 con 0x7fc0b01074d0 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2047021314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b01a5b00 con 0x7fc0b01074d0 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0ad7fa640 1 -- 172.21.15.44:0/2047021314 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0a40044c0 con 0x7fc0b01074d0 2024-08-20T03:10:52.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0ad7fa640 1 -- 172.21.15.44:0/2047021314 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4035070 con 0x7fc0b01074d0 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.518+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2047021314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b01a5fe0 con 0x7fc0b01074d0 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.521+0000 7fc0ad7fa640 1 -- 172.21.15.44:0/2047021314 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fc0a4042030 con 0x7fc0b01074d0 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.521+0000 7fc0b889e740 1 Processor -- start 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.521+0000 7fc0b889e740 1 -- start start 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.521+0000 7fc0b889e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297aa3b0 0x561a297aa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.521+0000 7fc0b889e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aacd0 0x561a297b6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.521+0000 7fc0b889e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297b6c60 0x561a297b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.521+0000 7fc0b889e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561a29720940 con 0x561a297b6c60 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.521+0000 7fc0b889e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561a2970fe40 con 0x561a297aacd0 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.521+0000 7fc0b889e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561a29713e80 con 0x561a297aa3b0 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297b6c60 0x561a297b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0b4a42640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297aa3b0 0x561a297aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0b4a42640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297aa3b0 0x561a297aa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:38886/0 (socket says 172.21.15.44:38886) 2024-08-20T03:10:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0b4a42640 1 -- 172.21.15.44:0/2431193828 learned_addr learned my addr 172.21.15.44:0/2431193828 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297b6c60 0x561a297b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60156/0 (socket says 172.21.15.44:60156) 2024-08-20T03:10:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0affff640 1 --2- 172.21.15.44:0/2431193828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aacd0 0x561a297b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0b4a42640 1 -- 172.21.15.44:0/2431193828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aacd0 msgr2=0x561a297b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0b4a42640 1 --2- 172.21.15.44:0/2431193828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aacd0 0x561a297b6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0b4a42640 1 -- 172.21.15.44:0/2431193828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297b6c60 msgr2=0x561a297b9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0b4a42640 1 --2- 172.21.15.44:0/2431193828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297b6c60 0x561a297b9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0b4a42640 1 -- 172.21.15.44:0/2431193828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a296ac520 con 0x561a297aa3b0 2024-08-20T03:10:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0affff640 1 --2- 172.21.15.44:0/2431193828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aacd0 0x561a297b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:10:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0af7fe640 1 --2- 172.21.15.44:0/2431193828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297b6c60 0x561a297b9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0ad7fa640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc0980b3d70 0x7fc0980b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.539 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0b4a42640 1 --2- 172.21.15.44:0/2431193828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297aa3b0 0x561a297aa790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009ed0 tx=0x7fc0a8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dddde92fb07c6bf2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0affff640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc0980b3d70 0x7fc0980b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0967fc640 1 -- 172.21.15.44:0/2431193828 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8004d20 con 0x561a297aa3b0 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0ad7fa640 1 -- 172.21.15.44:0/2047021314 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fc0a4103ad0 con 0x7fc0b01074d0 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0967fc640 1 -- 172.21.15.44:0/2431193828 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0a8004030 con 0x561a297aa3b0 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.522+0000 7fc0967fc640 1 -- 172.21.15.44:0/2431193828 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8005240 con 0x561a297aa3b0 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.523+0000 7fc0b889e740 1 -- 172.21.15.44:0/2431193828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297aa3b0 msgr2=0x561a297aa790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.523+0000 7fc0affff640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc0980b3d70 0x7fc0980b6230 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x561a297ad530 tx=0x7fc0a000b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.523+0000 7fc0b889e740 1 --2- 172.21.15.44:0/2431193828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297aa3b0 0x561a297aa790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009ed0 tx=0x7fc0a8011810 comp rx=0 tx=0).stop 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.523+0000 7fc0b889e740 1 -- 172.21.15.44:0/2431193828 shutdown_connections 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.523+0000 7fc0b889e740 1 --2- 172.21.15.44:0/2431193828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297b6c60 0x561a297b9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.523+0000 7fc0b889e740 1 --2- 172.21.15.44:0/2431193828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aacd0 0x561a297b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.523+0000 7fc0b889e740 1 --2- 172.21.15.44:0/2431193828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297aa3b0 0x561a297aa790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.523+0000 7fc0b889e740 1 -- 172.21.15.44:0/2431193828 >> 172.21.15.44:0/2431193828 conn(0x561a297a81e0 msgr2=0x561a297a9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:52.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.523+0000 7fc0b889e740 1 -- 172.21.15.44:0/2431193828 shutdown_connections 2024-08-20T03:10:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.523+0000 7fc0b889e740 1 -- 172.21.15.44:0/2431193828 wait complete. 2024-08-20T03:10:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0b889e740 1 -- start start 2024-08-20T03:10:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0b889e740 1 Processor -- start 2024-08-20T03:10:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0b889e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aa3b0 0x561a297b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0b889e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297aacd0 0x561a297b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0b889e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297b6c60 0x561a297b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0b889e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561a29572890 con 0x561a297aa3b0 2024-08-20T03:10:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0b889e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561a29696700 con 0x561a297b6c60 2024-08-20T03:10:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0b889e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561a297b99b0 con 0x561a297aacd0 2024-08-20T03:10:52.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0b889e740 10 client.0 _ll_register_callbacks cb 0x561a2970b6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0affff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aa3b0 0x561a297b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0affff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aa3b0 0x561a297b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60168/0 (socket says 172.21.15.44:60168) 2024-08-20T03:10:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0affff640 1 -- 172.21.15.44:0/934754414 learned_addr learned my addr 172.21.15.44:0/934754414 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0af7fe640 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297aacd0 0x561a297b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0b4a42640 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297b6c60 0x561a297b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0affff640 1 -- 172.21.15.44:0/934754414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297aacd0 msgr2=0x561a297b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0affff640 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297aacd0 0x561a297b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0affff640 1 -- 172.21.15.44:0/934754414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297b6c60 msgr2=0x561a297b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.524+0000 7fc0affff640 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297b6c60 0x561a297b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc0affff640 1 -- 172.21.15.44:0/934754414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2946f930 con 0x561a297aa3b0 2024-08-20T03:10:52.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc0affff640 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aa3b0 0x561a297b3ba0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00107c0 tx=0x7fc0a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d00d4a18f171f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc08f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc08f7fe640 1 -- 172.21.15.44:0/934754414 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0023070 con 0x561a297aa3b0 2024-08-20T03:10:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc08f7fe640 1 -- 172.21.15.44:0/934754414 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fc0a000fac0 con 0x561a297aa3b0 2024-08-20T03:10:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc0b889e740 1 -- 172.21.15.44:0/934754414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0a8004030 con 0x561a297aa3b0 2024-08-20T03:10:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc08f7fe640 1 -- 172.21.15.44:0/934754414 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc0a002f050 con 0x561a297aa3b0 2024-08-20T03:10:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc0b889e740 10 client.17292 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc0b889e740 1 -- 172.21.15.44:0/934754414 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561a297c5580 con 0x561a297aa3b0 2024-08-20T03:10:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc08f7fe640 1 -- 172.21.15.44:0/934754414 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0a8004030 con 0x561a297aa3b0 2024-08-20T03:10:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc08f7fe640 1 -- 172.21.15.44:0/934754414 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7fc0a00273b0 con 0x561a297aa3b0 2024-08-20T03:10:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc0b889e740 10 client.17292 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:52.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.525+0000 7fc0b889e740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.526+0000 7fc0b889e740 20 client.17292 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.526+0000 7fc0b889e740 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561a297c6a50 0x561a297c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.526+0000 7fc0b889e740 4 client.17292 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.526+0000 7fc0b889e740 1 -- 172.21.15.44:0/934754414 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a29722890 con 0x561a297c6a50 2024-08-20T03:10:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.526+0000 7fc0af7fe640 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561a297c6a50 0x561a297c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.526+0000 7fc0af7fe640 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561a297c6a50 0x561a297c8e90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eef20334e9b115d server_cookie=b09a4be2017157ab in_seq=0 out_seq=0 2024-08-20T03:10:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.526+0000 7fc08f7fe640 10 client.17292 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc08f7fe640 1 -- 172.21.15.44:0/934754414 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a29722890 con 0x561a297c6a50 2024-08-20T03:10:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc08f7fe640 10 client.17292 handle_command_reply: tid=0 2024-08-20T03:10:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc0b889e740 1 client.17292 shutdown 2024-08-20T03:10:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc0b889e740 1 -- 172.21.15.44:0/934754414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aa3b0 msgr2=0x561a297b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:52.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc0b889e740 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aa3b0 0x561a297b3ba0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00107c0 tx=0x7fc0a00046c0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc0b889e740 1 -- 172.21.15.44:0/934754414 shutdown_connections 2024-08-20T03:10:52.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc0b889e740 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561a297c6a50 0x561a297c8e90 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc0b889e740 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297b6c60 0x561a297b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc0b889e740 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297aacd0 0x561a297b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc08f7fe640 0 client.17292 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:52.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc0b889e740 1 --2- 172.21.15.44:0/934754414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aa3b0 0x561a297b3ba0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc0b889e740 1 -- 172.21.15.44:0/934754414 >> 172.21.15.44:0/934754414 conn(0x561a297a81e0 msgr2=0x561a297add80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:52.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc0b889e740 1 -- 172.21.15.44:0/934754414 shutdown_connections 2024-08-20T03:10:52.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc0b889e740 1 -- 172.21.15.44:0/934754414 wait complete. 2024-08-20T03:10:52.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.528+0000 7fc0b889e740 20 client.17292 trim_cache size 0 max 16384 2024-08-20T03:10:52.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.546+0000 7fc0b889e740 1 Processor -- start 2024-08-20T03:10:52.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.546+0000 7fc0b889e740 1 -- start start 2024-08-20T03:10:52.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.546+0000 7fc0b889e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aa3b0 0x561a297b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.546+0000 7fc0b889e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 0x561a297b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.546+0000 7fc0b889e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297b6c60 0x561a297ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.546+0000 7fc0b889e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01ae9c0 con 0x561a297aacd0 2024-08-20T03:10:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.546+0000 7fc0b889e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01b0680 con 0x561a297aa3b0 2024-08-20T03:10:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.546+0000 7fc0b889e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01af120 con 0x561a297b6c60 2024-08-20T03:10:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 0x561a297b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc0affff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aa3b0 0x561a297b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 0x561a297b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60184/0 (socket says 172.21.15.44:60184) 2024-08-20T03:10:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc0af7fe640 1 -- 172.21.15.44:0/4177224008 learned_addr learned my addr 172.21.15.44:0/4177224008 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:52.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc0b4a42640 1 --2- 172.21.15.44:0/4177224008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297b6c60 0x561a297ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc0affff640 1 -- 172.21.15.44:0/4177224008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297b6c60 msgr2=0x561a297ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc0affff640 1 --2- 172.21.15.44:0/4177224008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297b6c60 0x561a297ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc0affff640 1 -- 172.21.15.44:0/4177224008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 msgr2=0x561a297b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc0affff640 1 --2- 172.21.15.44:0/4177224008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 0x561a297b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc0affff640 1 -- 172.21.15.44:0/4177224008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a297ae900 con 0x561a297aa3b0 2024-08-20T03:10:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc0af7fe640 1 --2- 172.21.15.44:0/4177224008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 0x561a297b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc0affff640 1 --2- 172.21.15.44:0/4177224008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aa3b0 0x561a297b30b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00277a0 tx=0x7fc0a0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95d6c0a1b4a8fa12 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:52.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc095ffb640 1 -- 172.21.15.44:0/4177224008 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0037030 con 0x561a297aa3b0 2024-08-20T03:10:52.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc095ffb640 1 -- 172.21.15.44:0/4177224008 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0a00273b0 con 0x561a297aa3b0 2024-08-20T03:10:52.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.547+0000 7fc095ffb640 1 -- 172.21.15.44:0/4177224008 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0023040 con 0x561a297aa3b0 2024-08-20T03:10:52.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.548+0000 7fc0b889e740 1 -- 172.21.15.44:0/4177224008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aa3b0 msgr2=0x561a297b30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:52.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.548+0000 7fc0b889e740 1 --2- 172.21.15.44:0/4177224008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aa3b0 0x561a297b30b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00277a0 tx=0x7fc0a0004730 comp rx=0 tx=0).stop 2024-08-20T03:10:52.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.548+0000 7fc0b889e740 1 -- 172.21.15.44:0/4177224008 shutdown_connections 2024-08-20T03:10:52.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.548+0000 7fc0b889e740 1 --2- 172.21.15.44:0/4177224008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297b6c60 0x561a297ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.548+0000 7fc0b889e740 1 --2- 172.21.15.44:0/4177224008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 0x561a297b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.548+0000 7fc0b889e740 1 --2- 172.21.15.44:0/4177224008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aa3b0 0x561a297b30b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.548+0000 7fc0b889e740 1 -- 172.21.15.44:0/4177224008 >> 172.21.15.44:0/4177224008 conn(0x561a297a81e0 msgr2=0x561a297b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:52.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.548+0000 7fc0b889e740 1 -- 172.21.15.44:0/4177224008 shutdown_connections 2024-08-20T03:10:52.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.548+0000 7fc0b889e740 1 -- 172.21.15.44:0/4177224008 wait complete. 2024-08-20T03:10:52.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.548+0000 7fc0b889e740 1 -- start start 2024-08-20T03:10:52.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0b889e740 1 Processor -- start 2024-08-20T03:10:52.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0b889e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aa3b0 0x561a297bf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0b889e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 0x561a297ba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0b889e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297b6c60 0x561a297baca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0b889e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01affb0 con 0x561a297aacd0 2024-08-20T03:10:52.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0b889e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01af880 con 0x561a297aa3b0 2024-08-20T03:10:52.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0b889e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01ac9c0 con 0x561a297b6c60 2024-08-20T03:10:52.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0b889e740 10 client.0 _ll_register_callbacks cb 0x7fc0a0005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:52.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 0x561a297ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0affff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aa3b0 0x561a297bf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 0x561a297ba760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60188/0 (socket says 172.21.15.44:60188) 2024-08-20T03:10:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0af7fe640 1 -- 172.21.15.44:0/287783242 learned_addr learned my addr 172.21.15.44:0/287783242 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0b4a42640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297b6c60 0x561a297baca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0af7fe640 1 -- 172.21.15.44:0/287783242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297b6c60 msgr2=0x561a297baca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0af7fe640 1 --2- 172.21.15.44:0/287783242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297b6c60 0x561a297baca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0af7fe640 1 -- 172.21.15.44:0/287783242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aa3b0 msgr2=0x561a297bf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.549+0000 7fc0af7fe640 1 --2- 172.21.15.44:0/287783242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aa3b0 0x561a297bf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc0af7fe640 1 -- 172.21.15.44:0/287783242 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a297bb4d0 con 0x561a297aacd0 2024-08-20T03:10:52.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc0affff640 1 --2- 172.21.15.44:0/287783242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aa3b0 0x561a297bf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc0af7fe640 1 --2- 172.21.15.44:0/287783242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 0x561a297ba760 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4005dc0 tx=0x7fc0a403b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9c30594fe6981b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc08f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc08f7fe640 1 -- 172.21.15.44:0/287783242 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4049040 con 0x561a297aacd0 2024-08-20T03:10:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc08f7fe640 1 -- 172.21.15.44:0/287783242 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7fc0a40c6a60 con 0x561a297aacd0 2024-08-20T03:10:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc0b889e740 1 -- 172.21.15.44:0/287783242 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0a00273b0 con 0x561a297aacd0 2024-08-20T03:10:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc08f7fe640 1 -- 172.21.15.44:0/287783242 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc0a40ca120 con 0x561a297aacd0 2024-08-20T03:10:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc0b889e740 10 client.17298 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc0b889e740 1 -- 172.21.15.44:0/287783242 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561a29831750 con 0x561a297aacd0 2024-08-20T03:10:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc08f7fe640 1 -- 172.21.15.44:0/287783242 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0a00273b0 con 0x561a297aacd0 2024-08-20T03:10:52.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc08f7fe640 1 -- 172.21.15.44:0/287783242 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7fc0a40ed0b0 con 0x561a297aacd0 2024-08-20T03:10:52.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc0b889e740 10 client.17298 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:52.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc0b889e740 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:52.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc0b889e740 20 client.17298 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:52.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc0b889e740 1 --2- 172.21.15.44:0/287783242 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561a297c6a50 0x561a29833ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:52.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc0b889e740 4 client.17298 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T03:10:52.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.550+0000 7fc0b889e740 1 -- 172.21.15.44:0/287783242 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561a2971e330 con 0x561a297c6a50 2024-08-20T03:10:52.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.551+0000 7fc0affff640 1 --2- 172.21.15.44:0/287783242 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561a297c6a50 0x561a29833ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:52.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.551+0000 7fc0affff640 1 --2- 172.21.15.44:0/287783242 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561a297c6a50 0x561a29833ec0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e83709c349f141a server_cookie=a4a401f18469da61 in_seq=0 out_seq=0 2024-08-20T03:10:52.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.551+0000 7fc08f7fe640 10 client.17298 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:52.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.686+0000 7fc08f7fe640 1 -- 172.21.15.44:0/287783242 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561a2971e330 con 0x561a297c6a50 2024-08-20T03:10:52.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.686+0000 7fc08f7fe640 10 client.17298 handle_command_reply: tid=0 2024-08-20T03:10:52.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.686+0000 7fc0b889e740 1 client.17298 shutdown 2024-08-20T03:10:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.687+0000 7fc0b889e740 1 -- 172.21.15.44:0/287783242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 msgr2=0x561a297ba760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.687+0000 7fc0b889e740 1 --2- 172.21.15.44:0/287783242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 0x561a297ba760 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4005dc0 tx=0x7fc0a403b560 comp rx=0 tx=0).stop 2024-08-20T03:10:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.687+0000 7fc0b889e740 1 -- 172.21.15.44:0/287783242 shutdown_connections 2024-08-20T03:10:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.687+0000 7fc0b889e740 1 --2- 172.21.15.44:0/287783242 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x561a297c6a50 0x561a29833ec0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.687+0000 7fc0b889e740 1 --2- 172.21.15.44:0/287783242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561a297b6c60 0x561a297baca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.687+0000 7fc08f7fe640 0 client.17298 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.687+0000 7fc0b889e740 1 --2- 172.21.15.44:0/287783242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561a297aacd0 0x561a297ba760 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.687+0000 7fc0b889e740 1 --2- 172.21.15.44:0/287783242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561a297aa3b0 0x561a297bf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.687+0000 7fc0b889e740 1 -- 172.21.15.44:0/287783242 >> 172.21.15.44:0/287783242 conn(0x561a297a81e0 msgr2=0x561a297add80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.687+0000 7fc0b889e740 1 -- 172.21.15.44:0/287783242 shutdown_connections 2024-08-20T03:10:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.687+0000 7fc0b889e740 1 -- 172.21.15.44:0/287783242 wait complete. 2024-08-20T03:10:52.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.687+0000 7fc0b889e740 20 client.17298 trim_cache size 0 max 16384 2024-08-20T03:10:52.692 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:10:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-08-20T03:10:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "6372c1ac-b401-482e-8856-984029d6b3c3", 2024-08-20T03:10:52.692 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-08-20T03:10:52.692 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:10:52.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.688+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2047021314 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc0980b3d70 msgr2=0x7fc0980b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:52.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.688+0000 7fc0b64a5640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc0980b3d70 0x7fc0980b6230 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x561a297ad530 tx=0x7fc0a000b410 comp rx=0 tx=0).stop 2024-08-20T03:10:52.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.688+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2047021314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b01074d0 msgr2=0x7fc0b019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:52.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.688+0000 7fc0b64a5640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b01074d0 0x7fc0b019a5e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4006010 tx=0x7fc0a40040c0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.688+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2047021314 shutdown_connections 2024-08-20T03:10:52.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.688+0000 7fc0b64a5640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fc0980b3d70 0x7fc0980b6230 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.688+0000 7fc0b64a5640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0b019ab20 0x7fc0b01a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.688+0000 7fc0b64a5640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0b01074d0 0x7fc0b019a5e0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.688+0000 7fc0b64a5640 1 --2- 172.21.15.44:0/2047021314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0b0101520 0x7fc0b019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:52.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.688+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2047021314 >> 172.21.15.44:0/2047021314 conn(0x7fc0b0076c80 msgr2=0x7fc0b0103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:52.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.688+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2047021314 shutdown_connections 2024-08-20T03:10:52.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:52.688+0000 7fc0b64a5640 1 -- 172.21.15.44:0/2047021314 wait complete. 2024-08-20T03:10:53.470 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:10:53.736 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:10:54.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:53 smithi165 ceph-mon[22357]: pgmap v576: 97 pgs: 97 active+clean; 838 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 389 op/s 2024-08-20T03:10:54.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:53 smithi165 ceph-mon[22357]: scrub queued for path: / 2024-08-20T03:10:54.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:53 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:54.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:53 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:10:54.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:53 smithi099 ceph-mon[21984]: pgmap v576: 97 pgs: 97 active+clean; 838 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 389 op/s 2024-08-20T03:10:54.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:53 smithi099 ceph-mon[21984]: scrub queued for path: / 2024-08-20T03:10:54.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:53 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:54.165 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:53 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:10:54.188 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:53 smithi044 ceph-mon[18790]: pgmap v576: 97 pgs: 97 active+clean; 838 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 389 op/s 2024-08-20T03:10:54.188 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:53 smithi044 ceph-mon[18790]: scrub queued for path: / 2024-08-20T03:10:54.188 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:53 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:54.188 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:53 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:10:54.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.267+0000 7f3e01161640 1 -- 172.21.15.44:0/1390393005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc108e40 msgr2=0x7f3dfc10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:54.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.267+0000 7f3e01161640 1 --2- 172.21.15.44:0/1390393005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc108e40 0x7f3dfc10f640 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009ca0 tx=0x7f3df402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.267+0000 7f3e01161640 1 -- 172.21.15.44:0/1390393005 shutdown_connections 2024-08-20T03:10:54.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.267+0000 7f3e01161640 1 --2- 172.21.15.44:0/1390393005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc108e40 0x7f3dfc10f640 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.267+0000 7f3e01161640 1 --2- 172.21.15.44:0/1390393005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dfc1084c0 0x7f3dfc108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.267+0000 7f3e01161640 1 --2- 172.21.15.44:0/1390393005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3dfc102ab0 0x7f3dfc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.267+0000 7f3e01161640 1 -- 172.21.15.44:0/1390393005 >> 172.21.15.44:0/1390393005 conn(0x7f3dfc0fe500 msgr2=0x7f3dfc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.267+0000 7f3e01161640 1 -- 172.21.15.44:0/1390393005 shutdown_connections 2024-08-20T03:10:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.267+0000 7f3e01161640 1 -- 172.21.15.44:0/1390393005 wait complete. 2024-08-20T03:10:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.268+0000 7f3e01161640 1 Processor -- start 2024-08-20T03:10:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.268+0000 7f3e01161640 1 -- start start 2024-08-20T03:10:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.268+0000 7f3e01161640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3dfc102ab0 0x7f3dfc0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.268+0000 7f3e01161640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc1084c0 0x7f3dfc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.268+0000 7f3e01161640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dfc108e40 0x7f3dfc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.268+0000 7f3e01161640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc112b50 con 0x7f3dfc1084c0 2024-08-20T03:10:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.268+0000 7f3e01161640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1129d0 con 0x7f3dfc108e40 2024-08-20T03:10:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.268+0000 7f3e01161640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc112cd0 con 0x7f3dfc102ab0 2024-08-20T03:10:54.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3dfa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc1084c0 0x7f3dfc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3dfa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc1084c0 0x7f3dfc075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60216/0 (socket says 172.21.15.44:60216) 2024-08-20T03:10:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3dfa59c640 1 -- 172.21.15.44:0/2119810135 learned_addr learned my addr 172.21.15.44:0/2119810135 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3dfc102ab0 0x7f3dfc0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3dfb59e640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dfc108e40 0x7f3dfc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3dfa59c640 1 -- 172.21.15.44:0/2119810135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3dfc102ab0 msgr2=0x7f3dfc0751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3dfa59c640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3dfc102ab0 0x7f3dfc0751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3dfa59c640 1 -- 172.21.15.44:0/2119810135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dfc108e40 msgr2=0x7f3dfc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3dfa59c640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dfc108e40 0x7f3dfc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3dfa59c640 1 -- 172.21.15.44:0/2119810135 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dfc1a7340 con 0x7f3dfc1084c0 2024-08-20T03:10:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3dfb59e640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dfc108e40 0x7f3dfc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:10:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3dfa59c640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc1084c0 0x7f3dfc075710 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3df000d950 tx=0x7f3df000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:54.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3de3fff640 1 -- 172.21.15.44:0/2119810135 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0017070 con 0x7f3dfc1084c0 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3e01161640 1 -- 172.21.15.44:0/2119810135 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dfc1a7630 con 0x7f3dfc1084c0 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3de3fff640 1 -- 172.21.15.44:0/2119810135 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df0009c30 con 0x7f3dfc1084c0 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3de3fff640 1 -- 172.21.15.44:0/2119810135 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df000f620 con 0x7f3dfc1084c0 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.269+0000 7f3e01161640 1 -- 172.21.15.44:0/2119810135 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dfc1a7bc0 con 0x7f3dfc1084c0 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.272+0000 7f3e0355a740 1 Processor -- start 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.272+0000 7f3e0355a740 1 -- start start 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3de3fff640 1 -- 172.21.15.44:0/2119810135 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f3df0010460 con 0x7f3dfc1084c0 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3e0355a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 0x56484ab85c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3e0355a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab86150 0x56484ab91ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3e0355a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab920e0 0x56484ab944d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3e0355a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56484aafc8e0 con 0x56484ab920e0 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3e0355a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56484aaebde0 con 0x56484ab86150 2024-08-20T03:10:54.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3e0355a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56484aaefe20 con 0x56484ab85830 2024-08-20T03:10:54.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab920e0 0x56484ab944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfb59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 0x56484ab85c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfb59e640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 0x56484ab85c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:38928/0 (socket says 172.21.15.44:38928) 2024-08-20T03:10:54.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab86150 0x56484ab91ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfb59e640 1 -- 172.21.15.44:0/1319067508 learned_addr learned my addr 172.21.15.44:0/1319067508 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:54.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfb59e640 1 -- 172.21.15.44:0/1319067508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab86150 msgr2=0x56484ab91ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:54.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfb59e640 1 --2- 172.21.15.44:0/1319067508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab86150 0x56484ab91ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfb59e640 1 -- 172.21.15.44:0/1319067508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab920e0 msgr2=0x56484ab944d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:54.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfb59e640 1 --2- 172.21.15.44:0/1319067508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab920e0 0x56484ab944d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfb59e640 1 -- 172.21.15.44:0/1319067508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56484a84b650 con 0x56484ab85830 2024-08-20T03:10:54.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfa59c640 1 --2- 172.21.15.44:0/1319067508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab920e0 0x56484ab944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:54.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/1319067508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab86150 0x56484ab91ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:54.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3de3fff640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3de40b3d40 0x7f3de40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.274+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3de40b3d40 0x7f3de40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.273+0000 7f3dfb59e640 1 --2- 172.21.15.44:0/1319067508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 0x56484ab85c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3df4035000 tx=0x7f3df4009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56dc7a4e6e52e5e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:54.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.274+0000 7f3de3fff640 1 -- 172.21.15.44:0/2119810135 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f3df00e3c00 con 0x7f3dfc1084c0 2024-08-20T03:10:54.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.274+0000 7f3de17fa640 1 -- 172.21.15.44:0/1319067508 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4031cf0 con 0x56484ab85830 2024-08-20T03:10:54.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.274+0000 7f3de17fa640 1 -- 172.21.15.44:0/1319067508 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df4040070 con 0x56484ab85830 2024-08-20T03:10:54.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.274+0000 7f3de17fa640 1 -- 172.21.15.44:0/1319067508 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df40042a0 con 0x56484ab85830 2024-08-20T03:10:54.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.274+0000 7f3e0355a740 1 -- 172.21.15.44:0/1319067508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 msgr2=0x56484ab85c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:54.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.274+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3de40b3d40 0x7f3de40b6200 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x56484ab889b0 tx=0x7f3dec005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:54.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.274+0000 7f3e0355a740 1 --2- 172.21.15.44:0/1319067508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 0x56484ab85c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3df4035000 tx=0x7f3df4009800 comp rx=0 tx=0).stop 2024-08-20T03:10:54.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 -- 172.21.15.44:0/1319067508 shutdown_connections 2024-08-20T03:10:54.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 --2- 172.21.15.44:0/1319067508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab920e0 0x56484ab944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 --2- 172.21.15.44:0/1319067508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab86150 0x56484ab91ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 --2- 172.21.15.44:0/1319067508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 0x56484ab85c10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 -- 172.21.15.44:0/1319067508 >> 172.21.15.44:0/1319067508 conn(0x56484ab83660 msgr2=0x56484ab84800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:54.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 -- 172.21.15.44:0/1319067508 shutdown_connections 2024-08-20T03:10:54.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 -- 172.21.15.44:0/1319067508 wait complete. 2024-08-20T03:10:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 -- start start 2024-08-20T03:10:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 Processor -- start 2024-08-20T03:10:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 0x56484ab8f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab86150 0x56484ab8f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab920e0 0x56484ab8fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56484a94e700 con 0x56484ab85830 2024-08-20T03:10:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56484aa727c0 con 0x56484ab920e0 2024-08-20T03:10:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56484ab94d30 con 0x56484ab86150 2024-08-20T03:10:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3e0355a740 10 client.0 _ll_register_callbacks cb 0x56484aae56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3dfad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 0x56484ab8f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3dfad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 0x56484ab8f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60246/0 (socket says 172.21.15.44:60246) 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3dfad9d640 1 -- 172.21.15.44:0/3438793402 learned_addr learned my addr 172.21.15.44:0/3438793402 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3dfa59c640 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab86150 0x56484ab8f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.275+0000 7f3dfb59e640 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab920e0 0x56484ab8fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3dfad9d640 1 -- 172.21.15.44:0/3438793402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab86150 msgr2=0x56484ab8f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab86150 0x56484ab8f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3dfad9d640 1 -- 172.21.15.44:0/3438793402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab920e0 msgr2=0x56484ab8fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab920e0 0x56484ab8fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3dfad9d640 1 -- 172.21.15.44:0/3438793402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56484ab9e160 con 0x56484ab85830 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 0x56484ab8f0e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3dec01e9d0 tx=0x7f3dec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95ee4732458f5d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3ddeff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3ddeff5640 1 -- 172.21.15.44:0/3438793402 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec009be0 con 0x56484ab85830 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3ddeff5640 1 -- 172.21.15.44:0/3438793402 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f3dec018030 con 0x56484ab85830 2024-08-20T03:10:54.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3e0355a740 1 -- 172.21.15.44:0/3438793402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56484aa9b100 con 0x56484ab85830 2024-08-20T03:10:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3ddeff5640 1 -- 172.21.15.44:0/3438793402 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3dec004020 con 0x56484ab85830 2024-08-20T03:10:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3e0355a740 10 client.17310 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.276+0000 7f3e0355a740 1 -- 172.21.15.44:0/3438793402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56484aba0b50 con 0x56484ab85830 2024-08-20T03:10:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.277+0000 7f3ddeff5640 1 -- 172.21.15.44:0/3438793402 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56484aa9b100 con 0x56484ab85830 2024-08-20T03:10:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.277+0000 7f3ddeff5640 1 -- 172.21.15.44:0/3438793402 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f3dec029420 con 0x56484ab85830 2024-08-20T03:10:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.277+0000 7f3e0355a740 10 client.17310 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.277+0000 7f3e0355a740 10 client.17310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.277+0000 7f3e0355a740 20 client.17310 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.277+0000 7f3e0355a740 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56484aba20b0 0x56484aba44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.277+0000 7f3e0355a740 4 client.17310 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:10:54.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.277+0000 7f3e0355a740 1 -- 172.21.15.44:0/3438793402 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56484aafe830 con 0x56484aba20b0 2024-08-20T03:10:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.277+0000 7f3dfa59c640 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56484aba20b0 0x56484aba44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.277+0000 7f3dfa59c640 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56484aba20b0 0x56484aba44c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c38d83c3f1587a16 server_cookie=66411080da44c85 in_seq=0 out_seq=0 2024-08-20T03:10:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.277+0000 7f3ddeff5640 10 client.17310 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3ddeff5640 1 -- 172.21.15.44:0/3438793402 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56484aafe830 con 0x56484aba20b0 2024-08-20T03:10:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3ddeff5640 10 client.17310 handle_command_reply: tid=0 2024-08-20T03:10:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3e0355a740 1 client.17310 shutdown 2024-08-20T03:10:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3e0355a740 1 -- 172.21.15.44:0/3438793402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 msgr2=0x56484ab8f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3e0355a740 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 0x56484ab8f0e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3dec01e9d0 tx=0x7f3dec004730 comp rx=0 tx=0).stop 2024-08-20T03:10:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3e0355a740 1 -- 172.21.15.44:0/3438793402 shutdown_connections 2024-08-20T03:10:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3e0355a740 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56484aba20b0 0x56484aba44c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3e0355a740 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab920e0 0x56484ab8fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.318 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3e0355a740 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab86150 0x56484ab8f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3e0355a740 1 --2- 172.21.15.44:0/3438793402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 0x56484ab8f0e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3ddeff5640 0 client.17310 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3e0355a740 1 -- 172.21.15.44:0/3438793402 >> 172.21.15.44:0/3438793402 conn(0x56484ab83660 msgr2=0x56484ab89200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3e0355a740 1 -- 172.21.15.44:0/3438793402 shutdown_connections 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3e0355a740 1 -- 172.21.15.44:0/3438793402 wait complete. 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.280+0000 7f3e0355a740 20 client.17310 trim_cache size 0 max 16384 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.298+0000 7f3e0355a740 1 Processor -- start 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.298+0000 7f3e0355a740 1 -- start start 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.298+0000 7f3e0355a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 0x56484ab7e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.298+0000 7f3e0355a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab86150 0x56484ab8e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.298+0000 7f3e0355a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab920e0 0x56484ab8f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.298+0000 7f3e0355a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1b04e0 con 0x56484ab85830 2024-08-20T03:10:54.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.298+0000 7f3e0355a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1b21a0 con 0x56484ab86150 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.298+0000 7f3e0355a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1b0c40 con 0x56484ab920e0 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3dfad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 0x56484ab7e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3dfad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 0x56484ab7e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60262/0 (socket says 172.21.15.44:60262) 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3dfad9d640 1 -- 172.21.15.44:0/2509776760 learned_addr learned my addr 172.21.15.44:0/2509776760 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3dfa59c640 1 --2- 172.21.15.44:0/2509776760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab86150 0x56484ab8e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3dfb59e640 1 --2- 172.21.15.44:0/2509776760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab920e0 0x56484ab8f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3dfad9d640 1 -- 172.21.15.44:0/2509776760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab920e0 msgr2=0x56484ab8f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/2509776760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab920e0 0x56484ab8f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3dfad9d640 1 -- 172.21.15.44:0/2509776760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab86150 msgr2=0x56484ab8e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/2509776760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab86150 0x56484ab8e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3dfad9d640 1 -- 172.21.15.44:0/2509776760 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56484ab840e0 con 0x56484ab85830 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3dfa59c640 1 --2- 172.21.15.44:0/2509776760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab86150 0x56484ab8e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/2509776760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 0x56484ab7e380 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3dec019040 tx=0x7f3dec005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae21a52b8d41c0e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:54.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3de0ff9640 1 -- 172.21.15.44:0/2509776760 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec029e60 con 0x56484ab85830 2024-08-20T03:10:54.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3de0ff9640 1 -- 172.21.15.44:0/2509776760 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dec029ab0 con 0x56484ab85830 2024-08-20T03:10:54.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.299+0000 7f3de0ff9640 1 -- 172.21.15.44:0/2509776760 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec003810 con 0x56484ab85830 2024-08-20T03:10:54.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.300+0000 7f3e0355a740 1 -- 172.21.15.44:0/2509776760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 msgr2=0x56484ab7e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:54.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.300+0000 7f3e0355a740 1 --2- 172.21.15.44:0/2509776760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 0x56484ab7e380 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3dec019040 tx=0x7f3dec005b60 comp rx=0 tx=0).stop 2024-08-20T03:10:54.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.300+0000 7f3e0355a740 1 -- 172.21.15.44:0/2509776760 shutdown_connections 2024-08-20T03:10:54.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.300+0000 7f3e0355a740 1 --2- 172.21.15.44:0/2509776760 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab920e0 0x56484ab8f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.300+0000 7f3e0355a740 1 --2- 172.21.15.44:0/2509776760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab86150 0x56484ab8e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.300+0000 7f3e0355a740 1 --2- 172.21.15.44:0/2509776760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab85830 0x56484ab7e380 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.300+0000 7f3e0355a740 1 -- 172.21.15.44:0/2509776760 >> 172.21.15.44:0/2509776760 conn(0x56484ab83660 msgr2=0x56484ab8f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:54.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.300+0000 7f3e0355a740 1 -- 172.21.15.44:0/2509776760 shutdown_connections 2024-08-20T03:10:54.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.300+0000 7f3e0355a740 1 -- 172.21.15.44:0/2509776760 wait complete. 2024-08-20T03:10:54.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.300+0000 7f3e0355a740 1 -- start start 2024-08-20T03:10:54.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.300+0000 7f3e0355a740 1 Processor -- start 2024-08-20T03:10:54.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3e0355a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 0x56484ab9a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3e0355a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab86150 0x56484ab94f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3e0355a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab920e0 0x56484ab95600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3e0355a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1b1ad0 con 0x56484ab86150 2024-08-20T03:10:54.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3e0355a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1b13a0 con 0x56484ab920e0 2024-08-20T03:10:54.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3e0355a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1ae4e0 con 0x56484ab85830 2024-08-20T03:10:54.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3e0355a740 10 client.0 _ll_register_callbacks cb 0x7f3dec005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:10:54.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3dfa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab86150 0x56484ab94f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3dfa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab86150 0x56484ab94f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60274/0 (socket says 172.21.15.44:60274) 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3dfa59c640 1 -- 172.21.15.44:0/1610315161 learned_addr learned my addr 172.21.15.44:0/1610315161 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3dfad9d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 0x56484ab9a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3dfb59e640 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab920e0 0x56484ab95600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3dfa59c640 1 -- 172.21.15.44:0/1610315161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 msgr2=0x56484ab9a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3dfa59c640 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 0x56484ab9a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3dfa59c640 1 -- 172.21.15.44:0/1610315161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab920e0 msgr2=0x56484ab95600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3dfa59c640 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab920e0 0x56484ab95600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3dfa59c640 1 -- 172.21.15.44:0/1610315161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56484ac0ad50 con 0x56484ab86150 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 0x56484ab9a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3dfa59c640 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab86150 0x56484ab94f80 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3df00a7ac0 tx=0x7f3df00e7750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fefced052515d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3ddeff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.301+0000 7f3ddeff5640 1 -- 172.21.15.44:0/1610315161 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0020040 con 0x56484ab86150 2024-08-20T03:10:54.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3ddeff5640 1 -- 172.21.15.44:0/1610315161 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8449+0+0 (secure 0 0 0) 0x7f3df00a6e40 con 0x56484ab86150 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3e0355a740 1 -- 172.21.15.44:0/1610315161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dec029ab0 con 0x56484ab86150 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3ddeff5640 1 -- 172.21.15.44:0/1610315161 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3df00b3240 con 0x56484ab86150 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3e0355a740 10 client.17316 fetch_fsmap learned FSMap version 41 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3e0355a740 1 -- 172.21.15.44:0/1610315161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56484ac0bf30 con 0x56484ab86150 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3ddeff5640 1 -- 172.21.15.44:0/1610315161 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dec029ab0 con 0x56484ab86150 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3ddeff5640 1 -- 172.21.15.44:0/1610315161 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 41) ==== 8398+0+0 (secure 0 0 0) 0x7f3df00eb040 con 0x56484ab86150 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3e0355a740 10 client.17316 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3e0355a740 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3e0355a740 20 client.17316 populate_metadata read hostname 'smithi044' 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3e0355a740 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56484ac0c690 0x56484ac0eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3e0355a740 4 client.17316 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3e0355a740 1 -- 172.21.15.44:0/1610315161 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56484aafa2d0 con 0x56484ac0c690 2024-08-20T03:10:54.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.302+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56484ac0c690 0x56484ac0eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3dfad9d640 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56484ac0c690 0x56484ac0eab0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68312f70c97ffb23 server_cookie=2e588e01b64483c5 in_seq=0 out_seq=0 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3ddeff5640 10 client.17316 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3ddeff5640 1 -- 172.21.15.44:0/1610315161 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56484aafa2d0 con 0x56484ac0c690 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3ddeff5640 10 client.17316 handle_command_reply: tid=0 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3e0355a740 1 client.17316 shutdown 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3e0355a740 1 -- 172.21.15.44:0/1610315161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab86150 msgr2=0x56484ab94f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3e0355a740 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab86150 0x56484ab94f80 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3df00a7ac0 tx=0x7f3df00e7750 comp rx=0 tx=0).stop 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3e0355a740 1 -- 172.21.15.44:0/1610315161 shutdown_connections 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3e0355a740 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x56484ac0c690 0x56484ac0eab0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3e0355a740 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56484ab920e0 0x56484ab95600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3e0355a740 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56484ab86150 0x56484ab94f80 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3e0355a740 1 --2- 172.21.15.44:0/1610315161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56484ab85830 0x56484ab9a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3e0355a740 1 -- 172.21.15.44:0/1610315161 >> 172.21.15.44:0/1610315161 conn(0x56484ab83660 msgr2=0x56484ab89200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:54.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.303+0000 7f3ddeff5640 0 client.17316 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:10:54.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e0355a740 1 -- 172.21.15.44:0/1610315161 shutdown_connections 2024-08-20T03:10:54.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e0355a740 1 -- 172.21.15.44:0/1610315161 wait complete. 2024-08-20T03:10:54.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e0355a740 20 client.17316 trim_cache size 0 max 16384 2024-08-20T03:10:54.326 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:10:54.326 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-20T03:10:54.326 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-08-20T03:10:54.326 INFO:teuthology.orchestra.run.smithi044.stdout: "6372c1ac-b401-482e-8856-984029d6b3c3": { 2024-08-20T03:10:54.326 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-08-20T03:10:54.327 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "6372c1ac-b401-482e-8856-984029d6b3c3", 2024-08-20T03:10:54.327 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-08-20T03:10:54.327 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:10:54.327 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:10:54.327 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:10:54.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e01161640 1 -- 172.21.15.44:0/2119810135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3de40b3d40 msgr2=0x7f3de40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:54.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e01161640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3de40b3d40 0x7f3de40b6200 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x56484ab889b0 tx=0x7f3dec005f90 comp rx=0 tx=0).stop 2024-08-20T03:10:54.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e01161640 1 -- 172.21.15.44:0/2119810135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc1084c0 msgr2=0x7f3dfc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:10:54.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e01161640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc1084c0 0x7f3dfc075710 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3df000d950 tx=0x7f3df000de10 comp rx=0 tx=0).stop 2024-08-20T03:10:54.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e01161640 1 -- 172.21.15.44:0/2119810135 shutdown_connections 2024-08-20T03:10:54.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e01161640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f3de40b3d40 0x7f3de40b6200 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e01161640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dfc108e40 0x7f3dfc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e01161640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dfc1084c0 0x7f3dfc075710 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e01161640 1 --2- 172.21.15.44:0/2119810135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3dfc102ab0 0x7f3dfc0751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:10:54.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e01161640 1 -- 172.21.15.44:0/2119810135 >> 172.21.15.44:0/2119810135 conn(0x7f3dfc0fe500 msgr2=0x7f3dfc1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:10:54.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e01161640 1 -- 172.21.15.44:0/2119810135 shutdown_connections 2024-08-20T03:10:54.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:10:54.304+0000 7f3e01161640 1 -- 172.21.15.44:0/2119810135 wait complete. 2024-08-20T03:10:55.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:54 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:55.115 INFO:tasks.cephfs.filesystem:scrub status for tag:6372c1ac-b401-482e-8856-984029d6b3c3 - {'path': '/', 'tag': '6372c1ac-b401-482e-8856-984029d6b3c3', 'options': 'recursive,force'} 2024-08-20T03:10:55.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:54 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:55.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:54 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:10:56.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:55 smithi165 ceph-mon[22357]: pgmap v577: 97 pgs: 97 active+clean; 827 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 537 op/s 2024-08-20T03:10:56.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:55 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:10:56.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:55 smithi099 ceph-mon[21984]: pgmap v577: 97 pgs: 97 active+clean; 827 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 537 op/s 2024-08-20T03:10:56.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:55 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:10:56.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:55 smithi044 ceph-mon[18790]: pgmap v577: 97 pgs: 97 active+clean; 827 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 537 op/s 2024-08-20T03:10:56.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:55 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:10:57.413 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-08-20T03:10:57.413 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-08-20T03:10:57.735 DEBUG:teuthology.parallel:result is None 2024-08-20T03:10:57.735 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-08-20T03:10:57.783 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-08-20T03:10:57.783 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-08-20T03:10:57.793 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi044.front.sepia.ceph.com 2024-08-20T03:10:57.803 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-08-20T03:10:57.814 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-08-20T03:10:57.815 INFO:teuthology.task.exec:Executing custom commands... 2024-08-20T03:10:57.815 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi044.front.sepia.ceph.com 2024-08-20T03:10:57.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-08-20T03:10:57.870 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-08-20T03:10:58.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:57 smithi044 ceph-mon[18790]: pgmap v578: 97 pgs: 97 active+clean; 831 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 489 op/s 2024-08-20T03:10:58.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:57 smithi165 ceph-mon[22357]: pgmap v578: 97 pgs: 97 active+clean; 831 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 489 op/s 2024-08-20T03:10:58.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:57 smithi099 ceph-mon[21984]: pgmap v578: 97 pgs: 97 active+clean; 831 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 489 op/s 2024-08-20T03:11:00.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:10:59 smithi165 ceph-mon[22357]: pgmap v579: 97 pgs: 97 active+clean; 836 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 554 op/s 2024-08-20T03:11:00.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:10:59 smithi099 ceph-mon[21984]: pgmap v579: 97 pgs: 97 active+clean; 836 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 554 op/s 2024-08-20T03:11:00.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:10:59 smithi044 ceph-mon[18790]: pgmap v579: 97 pgs: 97 active+clean; 836 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 554 op/s 2024-08-20T03:11:02.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:01 smithi165 ceph-mon[22357]: pgmap v580: 97 pgs: 97 active+clean; 836 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 537 op/s 2024-08-20T03:11:02.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:01 smithi099 ceph-mon[21984]: pgmap v580: 97 pgs: 97 active+clean; 836 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 537 op/s 2024-08-20T03:11:02.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:01 smithi044 ceph-mon[18790]: pgmap v580: 97 pgs: 97 active+clean; 836 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 537 op/s 2024-08-20T03:11:04.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:03 smithi165 ceph-mon[22357]: pgmap v581: 97 pgs: 97 active+clean; 839 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 512 op/s 2024-08-20T03:11:04.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:03 smithi099 ceph-mon[21984]: pgmap v581: 97 pgs: 97 active+clean; 839 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 512 op/s 2024-08-20T03:11:04.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:03 smithi044 ceph-mon[18790]: pgmap v581: 97 pgs: 97 active+clean; 839 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 512 op/s 2024-08-20T03:11:06.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:05 smithi165 ceph-mon[22357]: pgmap v582: 97 pgs: 97 active+clean; 839 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 470 op/s 2024-08-20T03:11:06.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:05 smithi099 ceph-mon[21984]: pgmap v582: 97 pgs: 97 active+clean; 839 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 470 op/s 2024-08-20T03:11:06.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:05 smithi044 ceph-mon[18790]: pgmap v582: 97 pgs: 97 active+clean; 839 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 470 op/s 2024-08-20T03:11:07.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:06 smithi165 ceph-mon[22357]: osdmap e82: 12 total, 12 up, 12 in 2024-08-20T03:11:07.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:06 smithi099 ceph-mon[21984]: osdmap e82: 12 total, 12 up, 12 in 2024-08-20T03:11:07.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:06 smithi044 ceph-mon[18790]: osdmap e82: 12 total, 12 up, 12 in 2024-08-20T03:11:08.136 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:07 smithi165 ceph-mon[22357]: pgmap v584: 97 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 84 active+clean; 839 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 891 KiB/s wr, 238 op/s 2024-08-20T03:11:08.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:07 smithi099 ceph-mon[21984]: pgmap v584: 97 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 84 active+clean; 839 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 891 KiB/s wr, 238 op/s 2024-08-20T03:11:08.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:07 smithi044 ceph-mon[18790]: pgmap v584: 97 pgs: 6 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 84 active+clean; 839 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 891 KiB/s wr, 238 op/s 2024-08-20T03:11:10.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:09 smithi099 ceph-mon[21984]: pgmap v585: 97 pgs: 16 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 57 active+clean; 838 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 464 KiB/s wr, 117 op/s 2024-08-20T03:11:10.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:09 smithi044 ceph-mon[18790]: pgmap v585: 97 pgs: 16 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 57 active+clean; 838 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 464 KiB/s wr, 117 op/s 2024-08-20T03:11:10.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:09 smithi165 ceph-mon[22357]: pgmap v585: 97 pgs: 16 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 57 active+clean; 838 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 464 KiB/s wr, 117 op/s 2024-08-20T03:11:12.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:11 smithi099 ceph-mon[21984]: pgmap v586: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 836 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 464 KiB/s wr, 66 op/s 2024-08-20T03:11:12.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:11 smithi044 ceph-mon[18790]: pgmap v586: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 836 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 464 KiB/s wr, 66 op/s 2024-08-20T03:11:12.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:11 smithi165 ceph-mon[22357]: pgmap v586: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 836 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 464 KiB/s wr, 66 op/s 2024-08-20T03:11:14.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:13 smithi099 ceph-mon[21984]: pgmap v587: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 835 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 46 op/s 2024-08-20T03:11:14.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:13 smithi044 ceph-mon[18790]: pgmap v587: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 835 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 46 op/s 2024-08-20T03:11:14.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:13 smithi165 ceph-mon[22357]: pgmap v587: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 835 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 38 KiB/s wr, 46 op/s 2024-08-20T03:11:16.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:15 smithi099 ceph-mon[21984]: pgmap v588: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 830 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-08-20T03:11:16.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:15 smithi044 ceph-mon[18790]: pgmap v588: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 830 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-08-20T03:11:16.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:15 smithi165 ceph-mon[22357]: pgmap v588: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 830 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-08-20T03:11:18.164 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:17 smithi099 ceph-mon[21984]: pgmap v589: 97 pgs: 20 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 39 active+clean; 827 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-08-20T03:11:18.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:17 smithi044 ceph-mon[18790]: pgmap v589: 97 pgs: 20 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 39 active+clean; 827 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-08-20T03:11:18.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:17 smithi165 ceph-mon[22357]: pgmap v589: 97 pgs: 20 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 39 active+clean; 827 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-08-20T03:11:20.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:19 smithi044 ceph-mon[18790]: pgmap v590: 97 pgs: 18 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 43 active+clean; 825 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-08-20T03:11:20.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:19 smithi165 ceph-mon[22357]: pgmap v590: 97 pgs: 18 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 43 active+clean; 825 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-08-20T03:11:20.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:19 smithi099 ceph-mon[21984]: pgmap v590: 97 pgs: 18 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 43 active+clean; 825 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-08-20T03:11:22.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:21 smithi044 ceph-mon[18790]: pgmap v591: 97 pgs: 18 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 43 active+clean; 822 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-08-20T03:11:22.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:21 smithi165 ceph-mon[22357]: pgmap v591: 97 pgs: 18 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 43 active+clean; 822 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-08-20T03:11:22.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:21 smithi099 ceph-mon[21984]: pgmap v591: 97 pgs: 18 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 43 active+clean; 822 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-08-20T03:11:24.188 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:23 smithi165 ceph-mon[22357]: pgmap v592: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 820 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-08-20T03:11:24.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:23 smithi044 ceph-mon[18790]: pgmap v592: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 820 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-08-20T03:11:24.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:23 smithi099 ceph-mon[21984]: pgmap v592: 97 pgs: 14 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 52 active+clean; 820 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-08-20T03:11:25.115 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:11:25.384 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:11:26.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:25 smithi044 ceph-mon[18790]: pgmap v593: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 816 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-08-20T03:11:26.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:25 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:11:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:25 smithi165 ceph-mon[22357]: pgmap v593: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 816 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-08-20T03:11:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:25 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:11:26.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:25 smithi099 ceph-mon[21984]: pgmap v593: 97 pgs: 4 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 87 active+clean; 816 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-08-20T03:11:26.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:25 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:11:26.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.892+0000 7fd62c7fc640 1 -- 172.21.15.44:0/127500005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6241073a0 msgr2=0x7fd6241077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.892+0000 7fd62c7fc640 1 --2- 172.21.15.44:0/127500005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6241073a0 0x7fd6241077e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd614009900 tx=0x7fd61402a410 comp rx=0 tx=0).stop 2024-08-20T03:11:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.893+0000 7fd62c7fc640 1 -- 172.21.15.44:0/127500005 shutdown_connections 2024-08-20T03:11:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.893+0000 7fd62c7fc640 1 --2- 172.21.15.44:0/127500005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd624107d20 0x7fd62410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.893+0000 7fd62c7fc640 1 --2- 172.21.15.44:0/127500005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd6241073a0 0x7fd6241077e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.893+0000 7fd62c7fc640 1 --2- 172.21.15.44:0/127500005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd624101990 0x7fd624101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.893+0000 7fd62c7fc640 1 -- 172.21.15.44:0/127500005 >> 172.21.15.44:0/127500005 conn(0x7fd6240fd400 msgr2=0x7fd6240ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.893+0000 7fd62c7fc640 1 -- 172.21.15.44:0/127500005 shutdown_connections 2024-08-20T03:11:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.893+0000 7fd62c7fc640 1 -- 172.21.15.44:0/127500005 wait complete. 2024-08-20T03:11:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.893+0000 7fd62c7fc640 1 Processor -- start 2024-08-20T03:11:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.893+0000 7fd62c7fc640 1 -- start start 2024-08-20T03:11:26.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62c7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd624101990 0x7fd62410f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62c7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6241073a0 0x7fd62410f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62c7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd624107d20 0x7fd62410fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62c7fc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd624112b00 con 0x7fd624107d20 2024-08-20T03:11:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62c7fc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd624112980 con 0x7fd6241073a0 2024-08-20T03:11:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62c7fc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd624112c80 con 0x7fd624101990 2024-08-20T03:11:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62ad99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd624107d20 0x7fd62410fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62ad99640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd624107d20 0x7fd62410fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55704/0 (socket says 172.21.15.44:55704) 2024-08-20T03:11:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62ad99640 1 -- 172.21.15.44:0/1673789208 learned_addr learned my addr 172.21.15.44:0/1673789208 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd629d97640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6241073a0 0x7fd62410f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62a598640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd624101990 0x7fd62410f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62ad99640 1 -- 172.21.15.44:0/1673789208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd624101990 msgr2=0x7fd62410f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:26.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62ad99640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd624101990 0x7fd62410f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62ad99640 1 -- 172.21.15.44:0/1673789208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6241073a0 msgr2=0x7fd62410f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62ad99640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6241073a0 0x7fd62410f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62ad99640 1 -- 172.21.15.44:0/1673789208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6241a73b0 con 0x7fd624107d20 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.894+0000 7fd62ad99640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd624107d20 0x7fd62410fab0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00eaa0 tx=0x7fd61c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.895+0000 7fd61b7fe640 1 -- 172.21.15.44:0/1673789208 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c00cdb0 con 0x7fd624107d20 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.895+0000 7fd61b7fe640 1 -- 172.21.15.44:0/1673789208 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd61c005020 con 0x7fd624107d20 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.895+0000 7fd62c7fc640 1 -- 172.21.15.44:0/1673789208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6241a76a0 con 0x7fd624107d20 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.895+0000 7fd61b7fe640 1 -- 172.21.15.44:0/1673789208 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0116d0 con 0x7fd624107d20 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.895+0000 7fd62c7fc640 1 -- 172.21.15.44:0/1673789208 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6241a7bb0 con 0x7fd624107d20 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.897+0000 7fd61b7fe640 1 -- 172.21.15.44:0/1673789208 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd61c0051d0 con 0x7fd624107d20 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.897+0000 7fd62ebf5740 1 Processor -- start 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62ebf5740 1 -- start start 2024-08-20T03:11:26.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62ebf5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb63830 0x5622fdb63c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62ebf5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb64150 0x5622fdb6fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62ebf5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62ebf5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5622fdada8e0 con 0x5622fdb64150 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62ebf5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5622fdac9de0 con 0x5622fdb700e0 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62ebf5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5622fdacde20 con 0x5622fdb63830 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62a598640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb64150 0x5622fdb6fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62a598640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb64150 0x5622fdb6fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55720/0 (socket says 172.21.15.44:55720) 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62a598640 1 -- 172.21.15.44:0/2435770845 learned_addr learned my addr 172.21.15.44:0/2435770845 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd629d97640 1 --2- 172.21.15.44:0/2435770845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd61b7fe640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd6100b3d40 0x7fd6100b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd629d97640 1 --2- 172.21.15.44:0/2435770845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb63830 0x5622fdb63c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62a598640 1 -- 172.21.15.44:0/2435770845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb63830 msgr2=0x5622fdb63c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:26.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62a598640 1 --2- 172.21.15.44:0/2435770845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb63830 0x5622fdb63c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62a598640 1 -- 172.21.15.44:0/2435770845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 msgr2=0x5622fdb724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62ad99640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd6100b3d40 0x7fd6100b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62a598640 1 --2- 172.21.15.44:0/2435770845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd62a598640 1 -- 172.21.15.44:0/2435770845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622fd829650 con 0x5622fdb64150 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd629d97640 1 --2- 172.21.15.44:0/2435770845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb63830 0x5622fdb63c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.898+0000 7fd61b7fe640 1 -- 172.21.15.44:0/1673789208 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7fd61c0e4150 con 0x7fd624107d20 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.899+0000 7fd62a598640 1 --2- 172.21.15.44:0/2435770845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb64150 0x5622fdb6fba0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd62000bd40 tx=0x7fd6200107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19811751ee400f84 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.899+0000 7fd618ff9640 1 -- 172.21.15.44:0/2435770845 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd620004230 con 0x5622fdb64150 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.899+0000 7fd618ff9640 1 -- 172.21.15.44:0/2435770845 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd62000be30 con 0x5622fdb64150 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.899+0000 7fd618ff9640 1 -- 172.21.15.44:0/2435770845 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62001a7a0 con 0x5622fdb64150 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.899+0000 7fd62ebf5740 1 -- 172.21.15.44:0/2435770845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb64150 msgr2=0x5622fdb6fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.899+0000 7fd62ad99640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd6100b3d40 0x7fd6100b6200 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0a8040 tx=0x7fd61c092040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.899+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/2435770845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb64150 0x5622fdb6fba0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd62000bd40 tx=0x7fd6200107a0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 -- 172.21.15.44:0/2435770845 shutdown_connections 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/2435770845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/2435770845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb64150 0x5622fdb6fba0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/2435770845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb63830 0x5622fdb63c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 -- 172.21.15.44:0/2435770845 >> 172.21.15.44:0/2435770845 conn(0x5622fdb61660 msgr2=0x5622fdb62800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 -- 172.21.15.44:0/2435770845 shutdown_connections 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 -- 172.21.15.44:0/2435770845 wait complete. 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 -- start start 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 Processor -- start 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb6d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb6d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb6db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5622fd92c700 con 0x5622fdb63830 2024-08-20T03:11:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5622fda507c0 con 0x5622fdb700e0 2024-08-20T03:11:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5622fdb72d90 con 0x5622fdb64150 2024-08-20T03:11:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd62ebf5740 10 client.0 _ll_register_callbacks cb 0x5622fdac36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd629d97640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb6d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd629d97640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb6d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55732/0 (socket says 172.21.15.44:55732) 2024-08-20T03:11:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.900+0000 7fd629d97640 1 -- 172.21.15.44:0/3951919852 learned_addr learned my addr 172.21.15.44:0/3951919852 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd629d97640 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb6db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd62a598640 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb6d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd629d97640 1 -- 172.21.15.44:0/3951919852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 msgr2=0x5622fdb6d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd629d97640 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb6d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd629d97640 1 -- 172.21.15.44:0/3951919852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 msgr2=0x5622fdb6db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd629d97640 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb6db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd629d97640 1 -- 172.21.15.44:0/3951919852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622fdb7c170 con 0x5622fdb63830 2024-08-20T03:11:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd62a598640 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb6d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd629d97640 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb6d090 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd61402a930 tx=0x7fd614041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54d4a55f5644605 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd60dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd60dffb640 1 -- 172.21.15.44:0/3951919852 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614035030 con 0x5622fdb63830 2024-08-20T03:11:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd60dffb640 1 -- 172.21.15.44:0/3951919852 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7fd614004040 con 0x5622fdb63830 2024-08-20T03:11:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3951919852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5622fda79100 con 0x5622fdb63830 2024-08-20T03:11:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd60dffb640 1 -- 172.21.15.44:0/3951919852 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd614004240 con 0x5622fdb63830 2024-08-20T03:11:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd62ebf5740 10 client.17328 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3951919852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5622fdb7eb00 con 0x5622fdb63830 2024-08-20T03:11:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.901+0000 7fd60dffb640 1 -- 172.21.15.44:0/3951919852 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5622fda79100 con 0x5622fdb63830 2024-08-20T03:11:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.902+0000 7fd60dffb640 1 -- 172.21.15.44:0/3951919852 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7fd614053020 con 0x5622fdb63830 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.902+0000 7fd62ebf5740 10 client.17328 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.902+0000 7fd62ebf5740 10 client.17328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.902+0000 7fd62ebf5740 20 client.17328 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.902+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5622fdb7ff60 0x5622fdb823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.902+0000 7fd62ebf5740 4 client.17328 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.902+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3951919852 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622fdadc830 con 0x5622fdb7ff60 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.902+0000 7fd62a598640 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5622fdb7ff60 0x5622fdb823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.902+0000 7fd62a598640 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5622fdb7ff60 0x5622fdb823c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9571d66473d107f server_cookie=5f4879ce21494987 in_seq=0 out_seq=0 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.902+0000 7fd60dffb640 10 client.17328 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.903+0000 7fd60dffb640 1 -- 172.21.15.44:0/3951919852 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622fdadc830 con 0x5622fdb7ff60 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.903+0000 7fd60dffb640 10 client.17328 handle_command_reply: tid=0 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.903+0000 7fd62ebf5740 1 client.17328 shutdown 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.903+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3951919852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 msgr2=0x5622fdb6d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:26.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.903+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb6d090 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd61402a930 tx=0x7fd614041040 comp rx=0 tx=0).stop 2024-08-20T03:11:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.903+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3951919852 shutdown_connections 2024-08-20T03:11:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.903+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5622fdb7ff60 0x5622fdb823c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.903+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb6db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.903+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb6d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.903+0000 7fd60dffb640 0 client.17328 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.903+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/3951919852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb6d090 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.903+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3951919852 >> 172.21.15.44:0/3951919852 conn(0x5622fdb61660 msgr2=0x5622fdb67200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.904+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3951919852 shutdown_connections 2024-08-20T03:11:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.904+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3951919852 wait complete. 2024-08-20T03:11:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.904+0000 7fd62ebf5740 20 client.17328 trim_cache size 0 max 16384 2024-08-20T03:11:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd62ebf5740 1 Processor -- start 2024-08-20T03:11:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd62ebf5740 1 -- start start 2024-08-20T03:11:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd62ebf5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb679f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd62ebf5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb6c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd62ebf5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb87c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd62ebf5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241b0550 con 0x5622fdb63830 2024-08-20T03:11:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd62ebf5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241b21e0 con 0x5622fdb700e0 2024-08-20T03:11:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd62ebf5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241b0cb0 con 0x5622fdb64150 2024-08-20T03:11:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd629d97640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd629d97640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb679f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55746/0 (socket says 172.21.15.44:55746) 2024-08-20T03:11:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd629d97640 1 -- 172.21.15.44:0/1653929151 learned_addr learned my addr 172.21.15.44:0/1653929151 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd62a598640 1 --2- 172.21.15.44:0/1653929151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb6c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd629d97640 1 --2- 172.21.15.44:0/1653929151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb87c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd629d97640 1 -- 172.21.15.44:0/1653929151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 msgr2=0x5622fdb6c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd629d97640 1 --2- 172.21.15.44:0/1653929151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb6c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd629d97640 1 -- 172.21.15.44:0/1653929151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 msgr2=0x5622fdb87c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd629d97640 1 --2- 172.21.15.44:0/1653929151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb87c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd629d97640 1 -- 172.21.15.44:0/1653929151 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622fdb61490 con 0x5622fdb63830 2024-08-20T03:11:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.922+0000 7fd62a598640 1 --2- 172.21.15.44:0/1653929151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb6c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:26.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.923+0000 7fd629d97640 1 --2- 172.21.15.44:0/1653929151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb679f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd614031990 tx=0x7fd614001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738158131376ed37 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:26.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.923+0000 7fd60ffff640 1 -- 172.21.15.44:0/1653929151 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614035070 con 0x5622fdb63830 2024-08-20T03:11:26.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.923+0000 7fd60ffff640 1 -- 172.21.15.44:0/1653929151 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6140314a0 con 0x5622fdb63830 2024-08-20T03:11:26.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.923+0000 7fd60ffff640 1 -- 172.21.15.44:0/1653929151 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614031a70 con 0x5622fdb63830 2024-08-20T03:11:26.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.924+0000 7fd62ebf5740 1 -- 172.21.15.44:0/1653929151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 msgr2=0x5622fdb679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:26.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.924+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/1653929151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb679f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd614031990 tx=0x7fd614001e70 comp rx=0 tx=0).stop 2024-08-20T03:11:26.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.924+0000 7fd62ebf5740 1 -- 172.21.15.44:0/1653929151 shutdown_connections 2024-08-20T03:11:26.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.924+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/1653929151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb87c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.924+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/1653929151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb6c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.924+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/1653929151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb679f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.924+0000 7fd62ebf5740 1 -- 172.21.15.44:0/1653929151 >> 172.21.15.44:0/1653929151 conn(0x5622fdb61660 msgr2=0x5622fdb6eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:26.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.924+0000 7fd62ebf5740 1 -- 172.21.15.44:0/1653929151 shutdown_connections 2024-08-20T03:11:26.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.924+0000 7fd62ebf5740 1 -- 172.21.15.44:0/1653929151 wait complete. 2024-08-20T03:11:26.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.925+0000 7fd62ebf5740 1 -- start start 2024-08-20T03:11:26.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.925+0000 7fd62ebf5740 1 Processor -- start 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.925+0000 7fd62ebf5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.925+0000 7fd62ebf5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb73610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.925+0000 7fd62ebf5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb73b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.925+0000 7fd62ebf5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241b1b10 con 0x5622fdb63830 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.925+0000 7fd62ebf5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241b13e0 con 0x5622fdb700e0 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.925+0000 7fd62ebf5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241ae550 con 0x5622fdb64150 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.925+0000 7fd62ebf5740 10 client.0 _ll_register_callbacks cb 0x7fd614005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.925+0000 7fd629d97640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.925+0000 7fd629d97640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55750/0 (socket says 172.21.15.44:55750) 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.925+0000 7fd629d97640 1 -- 172.21.15.44:0/3640055504 learned_addr learned my addr 172.21.15.44:0/3640055504 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.926+0000 7fd62a598640 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb73610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.926+0000 7fd629d97640 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb73b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.926+0000 7fd629d97640 1 -- 172.21.15.44:0/3640055504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 msgr2=0x5622fdb73610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.926+0000 7fd629d97640 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb73610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.926+0000 7fd629d97640 1 -- 172.21.15.44:0/3640055504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 msgr2=0x5622fdb73b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.926+0000 7fd629d97640 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb73b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.926+0000 7fd629d97640 1 -- 172.21.15.44:0/3640055504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622fdb62d40 con 0x5622fdb63830 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.926+0000 7fd629d97640 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb730d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd61404e0f0 tx=0x7fd614009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5da9a86ce473c49 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.926+0000 7fd60dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.926+0000 7fd60dffb640 1 -- 172.21.15.44:0/3640055504 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614041030 con 0x5622fdb63830 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.926+0000 7fd60dffb640 1 -- 172.21.15.44:0/3640055504 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7fd61404eae0 con 0x5622fdb63830 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.926+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3640055504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6140314a0 con 0x5622fdb63830 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.927+0000 7fd60dffb640 1 -- 172.21.15.44:0/3640055504 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd614035040 con 0x5622fdb63830 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.927+0000 7fd62ebf5740 10 client.17334 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.927+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3640055504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5622fdbea060 con 0x5622fdb63830 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.927+0000 7fd60dffb640 1 -- 172.21.15.44:0/3640055504 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6140314a0 con 0x5622fdb63830 2024-08-20T03:11:26.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.927+0000 7fd60dffb640 1 -- 172.21.15.44:0/3640055504 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7fd61404ece0 con 0x5622fdb63830 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.927+0000 7fd62ebf5740 10 client.17334 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.927+0000 7fd62ebf5740 10 client.17334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.927+0000 7fd62ebf5740 20 client.17334 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.927+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5622fdb7ff60 0x5622fdbec4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.927+0000 7fd62ebf5740 4 client.17334 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.927+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3640055504 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5622fdad82d0 con 0x5622fdb7ff60 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.927+0000 7fd62a598640 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5622fdb7ff60 0x5622fdbec4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd60dffb640 10 client.17334 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd62a598640 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5622fdb7ff60 0x5622fdbec4f0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fddc07e2f9bc900 server_cookie=251ed621828085c in_seq=0 out_seq=0 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd60dffb640 1 -- 172.21.15.44:0/3640055504 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5622fdad82d0 con 0x5622fdb7ff60 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd60dffb640 10 client.17334 handle_command_reply: tid=0 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd62ebf5740 1 client.17334 shutdown 2024-08-20T03:11:26.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3640055504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 msgr2=0x5622fdb730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb730d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd61404e0f0 tx=0x7fd614009830 comp rx=0 tx=0).stop 2024-08-20T03:11:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3640055504 shutdown_connections 2024-08-20T03:11:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x5622fdb7ff60 0x5622fdbec4f0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd60dffb640 0 client.17334 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622fdb700e0 0x5622fdb73b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622fdb64150 0x5622fdb73610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd62ebf5740 1 --2- 172.21.15.44:0/3640055504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622fdb63830 0x5622fdb730d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.928+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3640055504 >> 172.21.15.44:0/3640055504 conn(0x5622fdb61660 msgr2=0x5622fdb6d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.929+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3640055504 shutdown_connections 2024-08-20T03:11:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.929+0000 7fd62ebf5740 1 -- 172.21.15.44:0/3640055504 wait complete. 2024-08-20T03:11:26.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.929+0000 7fd62ebf5740 20 client.17334 trim_cache size 0 max 16384 2024-08-20T03:11:26.936 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:11:26.936 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-08-20T03:11:26.936 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-08-20T03:11:26.936 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:11:26.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.929+0000 7fd62c7fc640 1 -- 172.21.15.44:0/1673789208 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd6100b3d40 msgr2=0x7fd6100b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.929+0000 7fd62c7fc640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd6100b3d40 0x7fd6100b6200 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0a8040 tx=0x7fd61c092040 comp rx=0 tx=0).stop 2024-08-20T03:11:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.929+0000 7fd62c7fc640 1 -- 172.21.15.44:0/1673789208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd624107d20 msgr2=0x7fd62410fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.929+0000 7fd62c7fc640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd624107d20 0x7fd62410fab0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00eaa0 tx=0x7fd61c00ef60 comp rx=0 tx=0).stop 2024-08-20T03:11:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.929+0000 7fd62c7fc640 1 -- 172.21.15.44:0/1673789208 shutdown_connections 2024-08-20T03:11:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.929+0000 7fd62c7fc640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd6100b3d40 0x7fd6100b6200 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.929+0000 7fd62c7fc640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd624107d20 0x7fd62410fab0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.930+0000 7fd62c7fc640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6241073a0 0x7fd62410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.930+0000 7fd62c7fc640 1 --2- 172.21.15.44:0/1673789208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd624101990 0x7fd62410f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.930+0000 7fd62c7fc640 1 -- 172.21.15.44:0/1673789208 >> 172.21.15.44:0/1673789208 conn(0x7fd6240fd400 msgr2=0x7fd6240ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.930+0000 7fd62c7fc640 1 -- 172.21.15.44:0/1673789208 shutdown_connections 2024-08-20T03:11:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:26.930+0000 7fd62c7fc640 1 -- 172.21.15.44:0/1673789208 wait complete. 2024-08-20T03:11:27.641 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T03:11:27.641 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:11:27.897 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:11:28.282 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:27 smithi044 ceph-mon[18790]: pgmap v594: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 815 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-08-20T03:11:28.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:27 smithi165 ceph-mon[22357]: pgmap v594: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 815 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-08-20T03:11:28.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.361+0000 7f8cc3702640 1 -- 172.21.15.44:0/1045756211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cbc103850 msgr2=0x7f8cbc103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:28.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.361+0000 7f8cc3702640 1 --2- 172.21.15.44:0/1045756211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cbc103850 0x7f8cbc103c30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009900 tx=0x7f8cb802a410 comp rx=0 tx=0).stop 2024-08-20T03:11:28.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.361+0000 7f8cc3702640 1 -- 172.21.15.44:0/1045756211 shutdown_connections 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.361+0000 7f8cc3702640 1 --2- 172.21.15.44:0/1045756211 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc10ce00 0x7f8cbc10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.361+0000 7f8cc3702640 1 --2- 172.21.15.44:0/1045756211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cbc104250 0x7f8cbc10c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.361+0000 7f8cc3702640 1 --2- 172.21.15.44:0/1045756211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cbc103850 0x7f8cbc103c30 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.361+0000 7f8cc3702640 1 -- 172.21.15.44:0/1045756211 >> 172.21.15.44:0/1045756211 conn(0x7f8cbc0fd400 msgr2=0x7f8cbc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.361+0000 7f8cc3702640 1 -- 172.21.15.44:0/1045756211 shutdown_connections 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.361+0000 7f8cc3702640 1 -- 172.21.15.44:0/1045756211 wait complete. 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.361+0000 7f8cc3702640 1 Processor -- start 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.361+0000 7f8cc3702640 1 -- start start 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc3702640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc103850 0x7f8cbc1016d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc3702640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cbc104250 0x7f8cbc101c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc3702640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cbc10ce00 0x7f8cbc102150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc3702640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc112aa0 con 0x7f8cbc10ce00 2024-08-20T03:11:28.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc3702640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc112920 con 0x7f8cbc103850 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc3702640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc112c20 con 0x7f8cbc104250 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc1c9f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cbc10ce00 0x7f8cbc102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc149e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc103850 0x7f8cbc1016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc1c9f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cbc10ce00 0x7f8cbc102150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38850/0 (socket says 172.21.15.44:38850) 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc1c9f640 1 -- 172.21.15.44:0/2612371384 learned_addr learned my addr 172.21.15.44:0/2612371384 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc0c9d640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cbc104250 0x7f8cbc101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc1c9f640 1 -- 172.21.15.44:0/2612371384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cbc104250 msgr2=0x7f8cbc101c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc1c9f640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cbc104250 0x7f8cbc101c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc1c9f640 1 -- 172.21.15.44:0/2612371384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc103850 msgr2=0x7f8cbc1016d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc1c9f640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc103850 0x7f8cbc1016d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc1c9f640 1 -- 172.21.15.44:0/2612371384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc1a74b0 con 0x7f8cbc10ce00 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc149e640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc103850 0x7f8cbc1016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:11:28.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.362+0000 7f8cc1c9f640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cbc10ce00 0x7f8cbc102150 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400eaa0 tx=0x7f8cb400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.363+0000 7f8cb27fc640 1 -- 172.21.15.44:0/2612371384 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4004d20 con 0x7f8cbc10ce00 2024-08-20T03:11:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.363+0000 7f8cc3702640 1 -- 172.21.15.44:0/2612371384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cbc1a77a0 con 0x7f8cbc10ce00 2024-08-20T03:11:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.363+0000 7f8cb27fc640 1 -- 172.21.15.44:0/2612371384 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb4005020 con 0x7f8cbc10ce00 2024-08-20T03:11:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.363+0000 7f8cb27fc640 1 -- 172.21.15.44:0/2612371384 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb40116b0 con 0x7f8cbc10ce00 2024-08-20T03:11:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.363+0000 7f8cc3702640 1 -- 172.21.15.44:0/2612371384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cbc1a7ce0 con 0x7f8cbc10ce00 2024-08-20T03:11:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.364+0000 7f8cc3702640 1 -- 172.21.15.44:0/2612371384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cbc103c30 con 0x7f8cbc10ce00 2024-08-20T03:11:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.366+0000 7f8cb27fc640 1 -- 172.21.15.44:0/2612371384 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8cb40051d0 con 0x7f8cbc10ce00 2024-08-20T03:11:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.366+0000 7f8cb27fc640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8ca80b3d40 0x7f8ca80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.367+0000 7f8cc149e640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8ca80b3d40 0x7f8ca80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.367+0000 7f8cb27fc640 1 -- 172.21.15.44:0/2612371384 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f8cb40e45f0 con 0x7f8cbc10ce00 2024-08-20T03:11:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.367+0000 7f8cc149e640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8ca80b3d40 0x7f8ca80b6200 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8031440 tx=0x7f8cb8005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:28.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.370+0000 7f8cb27fc640 1 -- 172.21.15.44:0/2612371384 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8cb4016030 con 0x7f8cbc10ce00 2024-08-20T03:11:28.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:27 smithi099 ceph-mon[21984]: pgmap v594: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 815 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-08-20T03:11:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.594+0000 7f8cc3702640 1 -- 172.21.15.44:0/2612371384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cbc107ed0 con 0x7f8cbc10ce00 2024-08-20T03:11:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.594+0000 7f8cb27fc640 1 -- 172.21.15.44:0/2612371384 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11977 (secure 0 0 0) 0x7f8cb40ab8c0 con 0x7f8cbc10ce00 2024-08-20T03:11:28.597 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:11:28.597 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":42,"btime":"2024-08-20T03:11:24:922103+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:11:23.996320+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24569,24583,24596,24589,24604]},"id":1}]} 2024-08-20T03:11:28.597 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 42 2024-08-20T03:11:28.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.596+0000 7f8cc3702640 1 -- 172.21.15.44:0/2612371384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8ca80b3d40 msgr2=0x7f8ca80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:28.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.596+0000 7f8cc3702640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8ca80b3d40 0x7f8ca80b6200 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8031440 tx=0x7f8cb8005b50 comp rx=0 tx=0).stop 2024-08-20T03:11:28.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.596+0000 7f8cc3702640 1 -- 172.21.15.44:0/2612371384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cbc10ce00 msgr2=0x7f8cbc102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.596+0000 7f8cc3702640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cbc10ce00 0x7f8cbc102150 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400eaa0 tx=0x7f8cb400eea0 comp rx=0 tx=0).stop 2024-08-20T03:11:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.596+0000 7f8cc3702640 1 -- 172.21.15.44:0/2612371384 shutdown_connections 2024-08-20T03:11:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.596+0000 7f8cc3702640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8ca80b3d40 0x7f8ca80b6200 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.596+0000 7f8cc3702640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cbc10ce00 0x7f8cbc102150 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.596+0000 7f8cc3702640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cbc104250 0x7f8cbc101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.596+0000 7f8cc3702640 1 --2- 172.21.15.44:0/2612371384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cbc103850 0x7f8cbc1016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.596+0000 7f8cc3702640 1 -- 172.21.15.44:0/2612371384 >> 172.21.15.44:0/2612371384 conn(0x7f8cbc0fd400 msgr2=0x7f8cbc0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.596+0000 7f8cc3702640 1 -- 172.21.15.44:0/2612371384 shutdown_connections 2024-08-20T03:11:28.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:28.596+0000 7f8cc3702640 1 -- 172.21.15.44:0/2612371384 wait complete. 2024-08-20T03:11:29.255 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:28 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2612371384' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:11:29.281 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-20T03:11:29.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:28 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2612371384' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:11:29.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:28 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2612371384' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:11:29.534 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:11:30.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.058+0000 7f1b3d920640 1 -- 172.21.15.44:0/1383828145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b381073a0 msgr2=0x7f1b381077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:30.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.058+0000 7f1b3d920640 1 --2- 172.21.15.44:0/1383828145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b381073a0 0x7f1b381077e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009900 tx=0x7f1b2002a410 comp rx=0 tx=0).stop 2024-08-20T03:11:30.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.058+0000 7f1b3d920640 1 -- 172.21.15.44:0/1383828145 shutdown_connections 2024-08-20T03:11:30.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.058+0000 7f1b3d920640 1 --2- 172.21.15.44:0/1383828145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b38107d20 0x7f1b3810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.058+0000 7f1b3d920640 1 --2- 172.21.15.44:0/1383828145 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b381073a0 0x7f1b381077e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.058+0000 7f1b3d920640 1 --2- 172.21.15.44:0/1383828145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b38101990 0x7f1b38101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.058+0000 7f1b3d920640 1 -- 172.21.15.44:0/1383828145 >> 172.21.15.44:0/1383828145 conn(0x7f1b380fd400 msgr2=0x7f1b380ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.058+0000 7f1b3d920640 1 -- 172.21.15.44:0/1383828145 shutdown_connections 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.058+0000 7f1b3d920640 1 -- 172.21.15.44:0/1383828145 wait complete. 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.058+0000 7f1b3d920640 1 Processor -- start 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.058+0000 7f1b3d920640 1 -- start start 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b3d920640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b38101990 0x7f1b3810f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b3d920640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b381073a0 0x7f1b3810f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b3d920640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b38107d20 0x7f1b3810fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b3d920640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38112b00 con 0x7f1b38107d20 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b3d920640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38112980 con 0x7f1b381073a0 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b3d920640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38112c80 con 0x7f1b38101990 2024-08-20T03:11:30.077 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b377fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b38107d20 0x7f1b3810fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b367fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b381073a0 0x7f1b3810f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b36ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b38101990 0x7f1b3810f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b377fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b38107d20 0x7f1b3810fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38866/0 (socket says 172.21.15.44:38866) 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b377fe640 1 -- 172.21.15.44:0/1617283209 learned_addr learned my addr 172.21.15.44:0/1617283209 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b377fe640 1 -- 172.21.15.44:0/1617283209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b38101990 msgr2=0x7f1b3810f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b377fe640 1 --2- 172.21.15.44:0/1617283209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b38101990 0x7f1b3810f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b377fe640 1 -- 172.21.15.44:0/1617283209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b381073a0 msgr2=0x7f1b3810f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b377fe640 1 --2- 172.21.15.44:0/1617283209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b381073a0 0x7f1b3810f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b377fe640 1 -- 172.21.15.44:0/1617283209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b381a73b0 con 0x7f1b38107d20 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.059+0000 7f1b377fe640 1 --2- 172.21.15.44:0/1617283209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b38107d20 0x7f1b3810fab0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000eaa0 tx=0x7f1b3000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.060+0000 7f1b3c91e640 1 -- 172.21.15.44:0/1617283209 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000cdb0 con 0x7f1b38107d20 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.060+0000 7f1b3c91e640 1 -- 172.21.15.44:0/1617283209 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b30005020 con 0x7f1b38107d20 2024-08-20T03:11:30.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.060+0000 7f1b3c91e640 1 -- 172.21.15.44:0/1617283209 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b300116d0 con 0x7f1b38107d20 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.060+0000 7f1b3d920640 1 -- 172.21.15.44:0/1617283209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b381a76a0 con 0x7f1b38107d20 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.060+0000 7f1b3d920640 1 -- 172.21.15.44:0/1617283209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b381a7b60 con 0x7f1b38107d20 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.062+0000 7f1b3c91e640 1 -- 172.21.15.44:0/1617283209 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1b300051d0 con 0x7f1b38107d20 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.062+0000 7f1b3fd19740 1 Processor -- start 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.062+0000 7f1b3fd19740 1 -- start start 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b3fd19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009768880 0x55c009768c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b3fd19740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c009774bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b3fd19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c009777520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b3fd19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c0096df930 con 0x55c009775130 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b3fd19740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c0096cee30 con 0x55c0097691a0 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b3fd19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c0096d2e70 con 0x55c009768880 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b367fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c009777520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b367fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c009777520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38874/0 (socket says 172.21.15.44:38874) 2024-08-20T03:11:30.079 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b367fc640 1 -- 172.21.15.44:0/1937287955 learned_addr learned my addr 172.21.15.44:0/1937287955 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b36ffd640 1 --2- 172.21.15.44:0/1937287955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c009774bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b367fc640 1 --2- 172.21.15.44:0/1937287955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009768880 0x55c009768c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b36ffd640 1 -- 172.21.15.44:0/1937287955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009768880 msgr2=0x55c009768c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b36ffd640 1 --2- 172.21.15.44:0/1937287955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009768880 0x55c009768c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b36ffd640 1 -- 172.21.15.44:0/1937287955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 msgr2=0x55c009777520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b36ffd640 1 --2- 172.21.15.44:0/1937287955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c009777520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b36ffd640 1 -- 172.21.15.44:0/1937287955 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c00942e650 con 0x55c0097691a0 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b3c91e640 1 --2- 172.21.15.44:0/1617283209 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1b240b3d40 0x7f1b240b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b367fc640 1 --2- 172.21.15.44:0/1617283209 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1b240b3d40 0x7f1b240b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b36ffd640 1 --2- 172.21.15.44:0/1937287955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c009774bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800bbf0 tx=0x7f1b280107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede8be36dea0e712 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.063+0000 7f1b3c91e640 1 -- 172.21.15.44:0/1617283209 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f1b300e40f0 con 0x7f1b38107d20 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b2dffb640 1 -- 172.21.15.44:0/1937287955 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28004030 con 0x55c0097691a0 2024-08-20T03:11:30.080 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b2dffb640 1 -- 172.21.15.44:0/1937287955 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b280035f0 con 0x55c0097691a0 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b2dffb640 1 -- 172.21.15.44:0/1937287955 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b280116a0 con 0x55c0097691a0 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1937287955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 msgr2=0x55c009774bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b367fc640 1 --2- 172.21.15.44:0/1617283209 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1b240b3d40 0x7f1b240b6200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55c009768830 tx=0x7f1b20005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/1937287955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c009774bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800bbf0 tx=0x7f1b280107a0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1937287955 shutdown_connections 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/1937287955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c009777520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/1937287955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c009774bf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/1937287955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009768880 0x55c009768c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1937287955 >> 172.21.15.44:0/1937287955 conn(0x55c0097666b0 msgr2=0x55c009767850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1937287955 shutdown_connections 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.064+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1937287955 wait complete. 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b3fd19740 1 -- start start 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b3fd19740 1 Processor -- start 2024-08-20T03:11:30.081 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b3fd19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009768880 0x55c009772080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b3fd19740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c0097725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b3fd19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009775130 0x55c009772b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b3fd19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55c009531700 con 0x55c009768880 2024-08-20T03:11:30.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b3fd19740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c009655810 con 0x55c0097691a0 2024-08-20T03:11:30.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b3fd19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55c009777de0 con 0x55c009775130 2024-08-20T03:11:30.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b3fd19740 10 client.0 _ll_register_callbacks cb 0x55c0096c8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:30.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b36ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009768880 0x55c009772080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b36ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009768880 0x55c009772080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38884/0 (socket says 172.21.15.44:38884) 2024-08-20T03:11:30.082 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b36ffd640 1 -- 172.21.15.44:0/1812170240 learned_addr learned my addr 172.21.15.44:0/1812170240 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.065+0000 7f1b367fc640 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c0097725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b377fe640 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009775130 0x55c009772b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b36ffd640 1 -- 172.21.15.44:0/1812170240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009775130 msgr2=0x55c009772b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b36ffd640 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009775130 0x55c009772b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b36ffd640 1 -- 172.21.15.44:0/1812170240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 msgr2=0x55c0097725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b36ffd640 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c0097725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b36ffd640 1 -- 172.21.15.44:0/1812170240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c009781140 con 0x55c009768880 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b377fe640 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009775130 0x55c009772b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b36ffd640 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009768880 0x55c009772080 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1b28005890 tx=0x7f1b2800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d756cb67a9307170 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b16ffd640 1 -- 172.21.15.44:0/1812170240 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2802a060 con 0x55c009768880 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b16ffd640 1 -- 172.21.15.44:0/1812170240 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f1b28004360 con 0x55c009768880 2024-08-20T03:11:30.083 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1812170240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c00967e150 con 0x55c009768880 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b16ffd640 1 -- 172.21.15.44:0/1812170240 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1b28005040 con 0x55c009768880 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b3fd19740 10 client.17352 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1812170240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c009783af0 con 0x55c009768880 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.066+0000 7f1b16ffd640 1 -- 172.21.15.44:0/1812170240 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c00967e150 con 0x55c009768880 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.067+0000 7f1b16ffd640 1 -- 172.21.15.44:0/1812170240 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f1b280046c0 con 0x55c009768880 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.067+0000 7f1b3fd19740 10 client.17352 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.067+0000 7f1b3fd19740 10 client.17352 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.067+0000 7f1b3fd19740 20 client.17352 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.067+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c009784ef0 0x55c0097872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.067+0000 7f1b3fd19740 4 client.17352 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.067+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1812170240 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0096e1880 con 0x55c009784ef0 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.067+0000 7f1b367fc640 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c009784ef0 0x55c0097872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.067+0000 7f1b367fc640 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c009784ef0 0x55c0097872e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eec3d91cbcba1ec9 server_cookie=20e7b8fe96a65a1e in_seq=0 out_seq=0 2024-08-20T03:11:30.084 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.067+0000 7f1b16ffd640 10 client.17352 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b16ffd640 1 -- 172.21.15.44:0/1812170240 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0096e1880 con 0x55c009784ef0 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b16ffd640 10 client.17352 handle_command_reply: tid=0 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b3fd19740 1 client.17352 shutdown 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1812170240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009768880 msgr2=0x55c009772080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009768880 0x55c009772080 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f1b28005890 tx=0x7f1b2800bf10 comp rx=0 tx=0).stop 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1812170240 shutdown_connections 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009775130 0x55c009772b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c009784ef0 0x55c0097872e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c0097725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/1812170240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009768880 0x55c009772080 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1812170240 >> 172.21.15.44:0/1812170240 conn(0x55c0097666b0 msgr2=0x55c00976c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b16ffd640 0 client.17352 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1812170240 shutdown_connections 2024-08-20T03:11:30.085 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b3fd19740 1 -- 172.21.15.44:0/1812170240 wait complete. 2024-08-20T03:11:30.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.068+0000 7f1b3fd19740 20 client.17352 trim_cache size 0 max 16384 2024-08-20T03:11:30.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.081+0000 7f1b3fd19740 1 Processor -- start 2024-08-20T03:11:30.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.081+0000 7f1b3fd19740 1 -- start start 2024-08-20T03:11:30.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b3fd19740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c009768880 0x55c009780600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b3fd19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c0097691a0 0x55c009780b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b3fd19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c00978c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.090 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b3fd19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381b05d0 con 0x55c009775130 2024-08-20T03:11:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b3fd19740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381b2260 con 0x55c009768880 2024-08-20T03:11:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b3fd19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381b0d30 con 0x55c0097691a0 2024-08-20T03:11:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b377fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c00978c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b377fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c00978c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38888/0 (socket says 172.21.15.44:38888) 2024-08-20T03:11:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b377fe640 1 -- 172.21.15.44:0/2672718437 learned_addr learned my addr 172.21.15.44:0/2672718437 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b367fc640 1 --2- 172.21.15.44:0/2672718437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c0097691a0 0x55c009780b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b36ffd640 1 --2- 172.21.15.44:0/2672718437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c009768880 0x55c009780600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b377fe640 1 -- 172.21.15.44:0/2672718437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c0097691a0 msgr2=0x55c009780b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b377fe640 1 --2- 172.21.15.44:0/2672718437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c0097691a0 0x55c009780b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b377fe640 1 -- 172.21.15.44:0/2672718437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c009768880 msgr2=0x55c009780600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b377fe640 1 --2- 172.21.15.44:0/2672718437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c009768880 0x55c009780600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.091 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b377fe640 1 -- 172.21.15.44:0/2672718437 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c009767130 con 0x55c009775130 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b377fe640 1 --2- 172.21.15.44:0/2672718437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c00978c940 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1b300a8040 tx=0x7f1b300aacd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48ccb891cddad2c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b2d7fa640 1 -- 172.21.15.44:0/2672718437 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3001f040 con 0x55c009775130 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b2d7fa640 1 -- 172.21.15.44:0/2672718437 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b300b3dd0 con 0x55c009775130 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.082+0000 7f1b2d7fa640 1 -- 172.21.15.44:0/2672718437 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b300a69a0 con 0x55c009775130 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.083+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2672718437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 msgr2=0x55c00978c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.083+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/2672718437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c00978c940 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1b300a8040 tx=0x7f1b300aacd0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.083+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2672718437 shutdown_connections 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.083+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/2672718437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c00978c940 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.083+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/2672718437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c0097691a0 0x55c009780b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.083+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/2672718437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c009768880 0x55c009780600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.083+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2672718437 >> 172.21.15.44:0/2672718437 conn(0x55c0097666b0 msgr2=0x55c00977fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.083+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2672718437 shutdown_connections 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.083+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2672718437 wait complete. 2024-08-20T03:11:30.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.083+0000 7f1b3fd19740 1 -- start start 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b3fd19740 1 Processor -- start 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b3fd19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009768880 0x55c00977a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b3fd19740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c00977af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b3fd19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c0097e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b3fd19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381b1b90 con 0x55c009775130 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b3fd19740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381b1460 con 0x55c0097691a0 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b3fd19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381ae5d0 con 0x55c009768880 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b3fd19740 10 client.0 _ll_register_callbacks cb 0x7f1b200095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b377fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c0097e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b377fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c0097e68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38898/0 (socket says 172.21.15.44:38898) 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b377fe640 1 -- 172.21.15.44:0/2296846044 learned_addr learned my addr 172.21.15.44:0/2296846044 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b36ffd640 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009768880 0x55c00977a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b367fc640 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c00977af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b377fe640 1 -- 172.21.15.44:0/2296846044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009768880 msgr2=0x55c00977a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b377fe640 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009768880 0x55c00977a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b377fe640 1 -- 172.21.15.44:0/2296846044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 msgr2=0x55c00977af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b377fe640 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c00977af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b377fe640 1 -- 172.21.15.44:0/2296846044 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0097e7010 con 0x55c009775130 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.084+0000 7f1b367fc640 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c00977af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b377fe640 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c0097e68d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000cfd0 tx=0x7f1b300a6d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf1352a21f099b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b16ffd640 1 -- 172.21.15.44:0/2296846044 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b300aa240 con 0x55c009775130 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b16ffd640 1 -- 172.21.15.44:0/2296846044 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7f1b300a6740 con 0x55c009775130 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2296846044 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b300b3dd0 con 0x55c009775130 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b16ffd640 1 -- 172.21.15.44:0/2296846044 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1b300a6940 con 0x55c009775130 2024-08-20T03:11:30.094 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b3fd19740 10 client.17358 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2296846044 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c0097ef530 con 0x55c009775130 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b16ffd640 1 -- 172.21.15.44:0/2296846044 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b300b3dd0 con 0x55c009775130 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b16ffd640 1 -- 172.21.15.44:0/2296846044 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f1b3000f040 con 0x55c009775130 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b3fd19740 10 client.17358 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b3fd19740 10 client.17358 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b3fd19740 20 client.17358 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c0097efce0 0x55c0097f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b3fd19740 4 client.17358 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.085+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2296846044 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0096dd320 con 0x55c0097efce0 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b36ffd640 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c0097efce0 0x55c0097f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b36ffd640 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c0097efce0 0x55c0097f2100 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ddf54de4afc8275c server_cookie=bc38fd0711f6c8ee in_seq=0 out_seq=0 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b16ffd640 10 client.17358 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b16ffd640 1 -- 172.21.15.44:0/2296846044 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0096dd320 con 0x55c0097efce0 2024-08-20T03:11:30.095 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b16ffd640 10 client.17358 handle_command_reply: tid=0 2024-08-20T03:11:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b3fd19740 1 client.17358 shutdown 2024-08-20T03:11:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2296846044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 msgr2=0x55c0097e68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c0097e68d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000cfd0 tx=0x7f1b300a6d10 comp rx=0 tx=0).stop 2024-08-20T03:11:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2296846044 shutdown_connections 2024-08-20T03:11:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c009775130 0x55c0097e68d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55c0097efce0 0x55c0097f2100 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b16ffd640 0 client.17358 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:11:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c0097691a0 0x55c00977af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b3fd19740 1 --2- 172.21.15.44:0/2296846044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c009768880 0x55c00977a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.086+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2296846044 >> 172.21.15.44:0/2296846044 conn(0x55c0097666b0 msgr2=0x55c009780710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2296846044 shutdown_connections 2024-08-20T03:11:30.096 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3fd19740 1 -- 172.21.15.44:0/2296846044 wait complete. 2024-08-20T03:11:30.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3fd19740 20 client.17358 trim_cache size 0 max 16384 2024-08-20T03:11:30.097 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:11:30.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3d920640 1 -- 172.21.15.44:0/1617283209 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1b240b3d40 msgr2=0x7f1b240b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:30.097 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3d920640 1 --2- 172.21.15.44:0/1617283209 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1b240b3d40 0x7f1b240b6200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55c009768830 tx=0x7f1b20005b50 comp rx=0 tx=0).stop 2024-08-20T03:11:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3d920640 1 -- 172.21.15.44:0/1617283209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b38107d20 msgr2=0x7f1b3810fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3d920640 1 --2- 172.21.15.44:0/1617283209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b38107d20 0x7f1b3810fab0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000eaa0 tx=0x7f1b3000ef60 comp rx=0 tx=0).stop 2024-08-20T03:11:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3d920640 1 -- 172.21.15.44:0/1617283209 shutdown_connections 2024-08-20T03:11:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3d920640 1 --2- 172.21.15.44:0/1617283209 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1b240b3d40 0x7f1b240b6200 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3d920640 1 --2- 172.21.15.44:0/1617283209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b38107d20 0x7f1b3810fab0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3d920640 1 --2- 172.21.15.44:0/1617283209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b381073a0 0x7f1b3810f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3d920640 1 --2- 172.21.15.44:0/1617283209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b38101990 0x7f1b3810f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3d920640 1 -- 172.21.15.44:0/1617283209 >> 172.21.15.44:0/1617283209 conn(0x7f1b380fd400 msgr2=0x7f1b380ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3d920640 1 -- 172.21.15.44:0/1617283209 shutdown_connections 2024-08-20T03:11:30.098 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:30.087+0000 7f1b3d920640 1 -- 172.21.15.44:0/1617283209 wait complete. 2024-08-20T03:11:30.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:29 smithi044 ceph-mon[18790]: pgmap v595: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-08-20T03:11:30.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:29 smithi165 ceph-mon[22357]: pgmap v595: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-08-20T03:11:30.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:29 smithi099 ceph-mon[21984]: pgmap v595: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-08-20T03:11:30.718 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-20T03:11:30.971 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:11:31.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.426+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/3965946474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf34102ad0 msgr2=0x7fbf34102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:31.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.426+0000 7fbf3bc4c640 1 --2- 172.21.15.44:0/3965946474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf34102ad0 0x7fbf34102eb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009900 tx=0x7fbf2c02a410 comp rx=0 tx=0).stop 2024-08-20T03:11:31.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.426+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/3965946474 shutdown_connections 2024-08-20T03:11:31.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.426+0000 7fbf3bc4c640 1 --2- 172.21.15.44:0/3965946474 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf34108e60 0x7fbf3410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.426+0000 7fbf3bc4c640 1 --2- 172.21.15.44:0/3965946474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf341084e0 0x7fbf34108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.426+0000 7fbf3bc4c640 1 --2- 172.21.15.44:0/3965946474 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf34102ad0 0x7fbf34102eb0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.427+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/3965946474 >> 172.21.15.44:0/3965946474 conn(0x7fbf340fe540 msgr2=0x7fbf34100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.427+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/3965946474 shutdown_connections 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.427+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/3965946474 wait complete. 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.427+0000 7fbf3bc4c640 1 Processor -- start 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.427+0000 7fbf3bc4c640 1 -- start start 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.427+0000 7fbf3bc4c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf34102ad0 0x7fbf34199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.427+0000 7fbf3bc4c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf341084e0 0x7fbf34199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.427+0000 7fbf3bc4c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf34108e60 0x7fbf3419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.427+0000 7fbf3bc4c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf34112b70 con 0x7fbf34102ad0 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.427+0000 7fbf3bc4c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341129f0 con 0x7fbf341084e0 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.427+0000 7fbf3bc4c640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf34112cf0 con 0x7fbf34108e60 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf399e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf34102ad0 0x7fbf34199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf399e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf34102ad0 0x7fbf34199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38922/0 (socket says 172.21.15.44:38922) 2024-08-20T03:11:31.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf399e8640 1 -- 172.21.15.44:0/60580526 learned_addr learned my addr 172.21.15.44:0/60580526 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:31.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf34108e60 0x7fbf3419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf391e7640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf341084e0 0x7fbf34199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf399e8640 1 -- 172.21.15.44:0/60580526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf34108e60 msgr2=0x7fbf3419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:31.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf399e8640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf34108e60 0x7fbf3419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf399e8640 1 -- 172.21.15.44:0/60580526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf341084e0 msgr2=0x7fbf34199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:31.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf399e8640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf341084e0 0x7fbf34199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf399e8640 1 -- 172.21.15.44:0/60580526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3419f0b0 con 0x7fbf34102ad0 2024-08-20T03:11:31.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf391e7640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf341084e0 0x7fbf34199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:31.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf399e8640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf34102ad0 0x7fbf34199000 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c005ab0 tx=0x7fbf2c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:31.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf32ffd640 1 -- 172.21.15.44:0/60580526 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c03f070 con 0x7fbf34102ad0 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/60580526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3419f340 con 0x7fbf34102ad0 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf32ffd640 1 -- 172.21.15.44:0/60580526 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf2c031d60 con 0x7fbf34102ad0 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf32ffd640 1 -- 172.21.15.44:0/60580526 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c035030 con 0x7fbf34102ad0 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.428+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/60580526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf341a5a60 con 0x7fbf34102ad0 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf3e045740 1 Processor -- start 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf32ffd640 1 -- 172.21.15.44:0/60580526 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbf2c042070 con 0x7fbf34102ad0 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf3e045740 1 -- start start 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf3e045740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf3e045740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c71a0 0x5654388d7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf3e045740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 0x5654388d9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf3e045740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56543883d930 con 0x5654388c6880 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf3e045740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56543882ce30 con 0x5654388c71a0 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf3e045740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x565438830e70 con 0x5654388d7540 2024-08-20T03:11:31.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf3a1e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf3a1e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388c6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38938/0 (socket says 172.21.15.44:38938) 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf3a1e9640 1 -- 172.21.15.44:0/2933725367 learned_addr learned my addr 172.21.15.44:0/2933725367 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf391e7640 1 --2- 172.21.15.44:0/2933725367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 0x5654388d9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.431+0000 7fbf399e8640 1 --2- 172.21.15.44:0/2933725367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c71a0 0x5654388d7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf3a1e9640 1 -- 172.21.15.44:0/2933725367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 msgr2=0x5654388d9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/2933725367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 0x5654388d9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf3a1e9640 1 -- 172.21.15.44:0/2933725367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c71a0 msgr2=0x5654388d7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/2933725367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c71a0 0x5654388d7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf3a1e9640 1 -- 172.21.15.44:0/2933725367 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56543858c650 con 0x5654388c6880 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf399e8640 1 --2- 172.21.15.44:0/2933725367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c71a0 0x5654388d7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf32ffd640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbf1c0b3d40 0x7fbf1c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/2933725367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388c6c60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fbf240103a0 tx=0x7fbf24010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9220968a6a76c4f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf399e8640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbf1c0b3d40 0x7fbf1c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf1bfff640 1 -- 172.21.15.44:0/2933725367 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24016070 con 0x5654388c6880 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf32ffd640 1 -- 172.21.15.44:0/60580526 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7fbf2c1048c0 con 0x7fbf34102ad0 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf1bfff640 1 -- 172.21.15.44:0/2933725367 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf24004d20 con 0x5654388c6880 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.432+0000 7fbf1bfff640 1 -- 172.21.15.44:0/2933725367 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24005390 con 0x5654388c6880 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.433+0000 7fbf3e045740 1 -- 172.21.15.44:0/2933725367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 msgr2=0x5654388c6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.433+0000 7fbf399e8640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbf1c0b3d40 0x7fbf1c0b6200 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c1020e0 tx=0x7fbf2c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.433+0000 7fbf3e045740 1 --2- 172.21.15.44:0/2933725367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388c6c60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fbf240103a0 tx=0x7fbf24010860 comp rx=0 tx=0).stop 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.433+0000 7fbf3e045740 1 -- 172.21.15.44:0/2933725367 shutdown_connections 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.433+0000 7fbf3e045740 1 --2- 172.21.15.44:0/2933725367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 0x5654388d9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.433+0000 7fbf3e045740 1 --2- 172.21.15.44:0/2933725367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c71a0 0x5654388d7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.433+0000 7fbf3e045740 1 --2- 172.21.15.44:0/2933725367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388c6c60 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.433+0000 7fbf3e045740 1 -- 172.21.15.44:0/2933725367 >> 172.21.15.44:0/2933725367 conn(0x5654388c46b0 msgr2=0x5654388c5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.433+0000 7fbf3e045740 1 -- 172.21.15.44:0/2933725367 shutdown_connections 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.433+0000 7fbf3e045740 1 -- 172.21.15.44:0/2933725367 wait complete. 2024-08-20T03:11:31.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.433+0000 7fbf3e045740 1 -- start start 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.433+0000 7fbf3e045740 1 Processor -- start 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf3e045740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c6880 0x5654388d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf3e045740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c71a0 0x5654388d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf3e045740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 0x5654388d4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf3e045740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56543868f700 con 0x5654388c71a0 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf3e045740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5654387b3810 con 0x5654388c6880 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf3e045740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5654388da1f0 con 0x5654388d7540 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf3e045740 10 client.0 _ll_register_callbacks cb 0x565438826700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf391e7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c71a0 0x5654388d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf391e7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c71a0 0x5654388d4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38954/0 (socket says 172.21.15.44:38954) 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf391e7640 1 -- 172.21.15.44:0/3548859453 learned_addr learned my addr 172.21.15.44:0/3548859453 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 0x5654388d4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c6880 0x5654388d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf391e7640 1 -- 172.21.15.44:0/3548859453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 msgr2=0x5654388d4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf391e7640 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 0x5654388d4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf391e7640 1 -- 172.21.15.44:0/3548859453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c6880 msgr2=0x5654388d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf391e7640 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c6880 0x5654388d4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf391e7640 1 -- 172.21.15.44:0/3548859453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654388e35f0 con 0x5654388c71a0 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf391e7640 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c71a0 0x5654388d4a70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fbf28010810 tx=0x7fbf28010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7177d50d1ed235a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.434+0000 7fbf197fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf197fa640 1 -- 172.21.15.44:0/3548859453 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2800ecc0 con 0x5654388c71a0 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf197fa640 1 -- 172.21.15.44:0/3548859453 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7fbf28005020 con 0x5654388c71a0 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf3e045740 1 -- 172.21.15.44:0/3548859453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654387dc150 con 0x5654388c71a0 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf197fa640 1 -- 172.21.15.44:0/3548859453 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbf2800ee60 con 0x5654388c71a0 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf3e045740 10 client.17370 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf3e045740 1 -- 172.21.15.44:0/3548859453 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5654388e5f70 con 0x5654388c71a0 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf197fa640 1 -- 172.21.15.44:0/3548859453 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5654387dc150 con 0x5654388c71a0 2024-08-20T03:11:31.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf197fa640 1 -- 172.21.15.44:0/3548859453 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7fbf28011440 con 0x5654388c71a0 2024-08-20T03:11:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf3e045740 10 client.17370 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf3e045740 10 client.17370 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:11:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf3e045740 20 client.17370 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf3e045740 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5654388e7430 0x5654388e9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf3e045740 4 client.17370 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:11:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.435+0000 7fbf3e045740 1 -- 172.21.15.44:0/3548859453 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56543883f880 con 0x5654388e7430 2024-08-20T03:11:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.436+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5654388e7430 0x5654388e9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.436+0000 7fbf197fa640 10 client.17370 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:11:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.436+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5654388e7430 0x5654388e9840 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ba38fe1efe861e2b server_cookie=300f410832b098e8 in_seq=0 out_seq=0 2024-08-20T03:11:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf197fa640 1 -- 172.21.15.44:0/3548859453 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56543883f880 con 0x5654388e7430 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf197fa640 10 client.17370 handle_command_reply: tid=0 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf3e045740 1 client.17370 shutdown 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf3e045740 1 -- 172.21.15.44:0/3548859453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c71a0 msgr2=0x5654388d4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf3e045740 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c71a0 0x5654388d4a70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fbf28010810 tx=0x7fbf28010cd0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf3e045740 1 -- 172.21.15.44:0/3548859453 shutdown_connections 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf3e045740 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x5654388e7430 0x5654388e9840 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf3e045740 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 0x5654388d4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf3e045740 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c71a0 0x5654388d4a70 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf197fa640 0 client.17370 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf3e045740 1 --2- 172.21.15.44:0/3548859453 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c6880 0x5654388d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf3e045740 1 -- 172.21.15.44:0/3548859453 >> 172.21.15.44:0/3548859453 conn(0x5654388c46b0 msgr2=0x5654388ca1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf3e045740 1 -- 172.21.15.44:0/3548859453 shutdown_connections 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf3e045740 1 -- 172.21.15.44:0/3548859453 wait complete. 2024-08-20T03:11:31.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.437+0000 7fbf3e045740 20 client.17370 trim_cache size 0 max 16384 2024-08-20T03:11:31.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.450+0000 7fbf3e045740 1 Processor -- start 2024-08-20T03:11:31.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.450+0000 7fbf3e045740 1 -- start start 2024-08-20T03:11:31.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.450+0000 7fbf3e045740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.450+0000 7fbf3e045740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c71a0 0x5654388c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.450+0000 7fbf3e045740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 0x5654388c5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.450+0000 7fbf3e045740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341ac150 con 0x5654388c6880 2024-08-20T03:11:31.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.450+0000 7fbf3e045740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341adde0 con 0x5654388c71a0 2024-08-20T03:11:31.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.450+0000 7fbf3e045740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341ac8b0 con 0x5654388d7540 2024-08-20T03:11:31.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.450+0000 7fbf3a1e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.451+0000 7fbf3a1e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388d1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38966/0 (socket says 172.21.15.44:38966) 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.451+0000 7fbf3a1e9640 1 -- 172.21.15.44:0/3930925593 learned_addr learned my addr 172.21.15.44:0/3930925593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.451+0000 7fbf391e7640 1 --2- 172.21.15.44:0/3930925593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c71a0 0x5654388c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.451+0000 7fbf3a1e9640 1 -- 172.21.15.44:0/3930925593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 msgr2=0x5654388c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.451+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/3930925593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 0x5654388c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.451+0000 7fbf3a1e9640 1 -- 172.21.15.44:0/3930925593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c71a0 msgr2=0x5654388c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.451+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/3930925593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c71a0 0x5654388c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.451+0000 7fbf3a1e9640 1 -- 172.21.15.44:0/3930925593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654388c6fd0 con 0x5654388c6880 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.451+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/3930925593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388d1fd0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fbf240103a0 tx=0x7fbf24010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b688c9e242dc0b66 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.451+0000 7fbf1b7fe640 1 -- 172.21.15.44:0/3930925593 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24016040 con 0x5654388c6880 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.451+0000 7fbf1b7fe640 1 -- 172.21.15.44:0/3930925593 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf240059d0 con 0x5654388c6880 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.451+0000 7fbf1b7fe640 1 -- 172.21.15.44:0/3930925593 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24023040 con 0x5654388c6880 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.452+0000 7fbf3e045740 1 -- 172.21.15.44:0/3930925593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 msgr2=0x5654388d1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:31.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.452+0000 7fbf3e045740 1 --2- 172.21.15.44:0/3930925593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388d1fd0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fbf240103a0 tx=0x7fbf24010860 comp rx=0 tx=0).stop 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.452+0000 7fbf3e045740 1 -- 172.21.15.44:0/3930925593 shutdown_connections 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.452+0000 7fbf3e045740 1 --2- 172.21.15.44:0/3930925593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388d7540 0x5654388c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.452+0000 7fbf3e045740 1 --2- 172.21.15.44:0/3930925593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388c71a0 0x5654388c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.452+0000 7fbf3e045740 1 --2- 172.21.15.44:0/3930925593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388d1fd0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.452+0000 7fbf3e045740 1 -- 172.21.15.44:0/3930925593 >> 172.21.15.44:0/3930925593 conn(0x5654388c46b0 msgr2=0x5654388d8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.452+0000 7fbf3e045740 1 -- 172.21.15.44:0/3930925593 shutdown_connections 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.452+0000 7fbf3e045740 1 -- 172.21.15.44:0/3930925593 wait complete. 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.452+0000 7fbf3e045740 1 -- start start 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.452+0000 7fbf3e045740 1 Processor -- start 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf3e045740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388ef160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf3e045740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388c71a0 0x5654388ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf3e045740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388d7540 0x5654388df480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf3e045740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341ad710 con 0x5654388c6880 2024-08-20T03:11:31.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf3e045740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341acfe0 con 0x5654388d7540 2024-08-20T03:11:31.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf3e045740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbf341aa150 con 0x5654388c71a0 2024-08-20T03:11:31.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf3e045740 10 client.0 _ll_register_callbacks cb 0x7fbf2800c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:31.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf391e7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388c71a0 0x5654388ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf391e7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388c71a0 0x5654388ef6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:34972/0 (socket says 172.21.15.44:34972) 2024-08-20T03:11:31.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf391e7640 1 -- 172.21.15.44:0/2482927437 learned_addr learned my addr 172.21.15.44:0/2482927437 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:31.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf391e7640 1 -- 172.21.15.44:0/2482927437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388d7540 msgr2=0x5654388df480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:11:31.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf391e7640 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388d7540 0x5654388df480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf391e7640 1 -- 172.21.15.44:0/2482927437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 msgr2=0x5654388ef160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf391e7640 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388ef160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf391e7640 1 -- 172.21.15.44:0/2482927437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654388d0c00 con 0x5654388c71a0 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.453+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.454+0000 7fbf391e7640 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388c71a0 0x5654388ef6a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x5654388d6e20 tx=0x7fbf280116f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dda4f51afb1f1ea5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.454+0000 7fbf197fa640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.454+0000 7fbf197fa640 1 -- 172.21.15.44:0/2482927437 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf28032060 con 0x5654388c71a0 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.454+0000 7fbf3e045740 1 -- 172.21.15.44:0/2482927437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf240059d0 con 0x5654388c71a0 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.454+0000 7fbf197fa640 1 -- 172.21.15.44:0/2482927437 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8493+0+0 (secure 0 0 0) 0x7fbf28011cd0 con 0x5654388c71a0 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.454+0000 7fbf197fa640 1 -- 172.21.15.44:0/2482927437 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbf28028040 con 0x5654388c71a0 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.454+0000 7fbf3e045740 10 client.26743 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.454+0000 7fbf3e045740 1 -- 172.21.15.44:0/2482927437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5654388c8110 con 0x5654388c71a0 2024-08-20T03:11:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.454+0000 7fbf197fa640 1 -- 172.21.15.44:0/2482927437 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf240059d0 con 0x5654388c71a0 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf197fa640 1 -- 172.21.15.44:0/2482927437 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7fbf280046f0 con 0x5654388c71a0 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf3e045740 10 client.26743 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf3e045740 10 client.26743 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf3e045740 20 client.26743 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf3e045740 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x56543894c100 0x56543894e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf3e045740 4 client.26743 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf3e045740 1 -- 172.21.15.44:0/2482927437 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56543883b320 con 0x56543894c100 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x56543894c100 0x56543894e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf3a1e9640 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x56543894c100 0x56543894e540 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=af4c07d9140110ba server_cookie=fe5f9415706ba5c6 in_seq=0 out_seq=0 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf197fa640 10 client.26743 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf197fa640 1 -- 172.21.15.44:0/2482927437 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56543883b320 con 0x56543894c100 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf197fa640 10 client.26743 handle_command_reply: tid=0 2024-08-20T03:11:31.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.455+0000 7fbf3e045740 1 client.26743 shutdown 2024-08-20T03:11:31.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3e045740 1 -- 172.21.15.44:0/2482927437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388c71a0 msgr2=0x5654388ef6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:31.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3e045740 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388c71a0 0x5654388ef6a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x5654388d6e20 tx=0x7fbf280116f0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3e045740 1 -- 172.21.15.44:0/2482927437 shutdown_connections 2024-08-20T03:11:31.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3e045740 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x56543894c100 0x56543894e540 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3e045740 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5654388d7540 0x5654388df480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3e045740 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654388c71a0 0x5654388ef6a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf197fa640 0 client.26743 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:11:31.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3e045740 1 --2- 172.21.15.44:0/2482927437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5654388c6880 0x5654388ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3e045740 1 -- 172.21.15.44:0/2482927437 >> 172.21.15.44:0/2482927437 conn(0x5654388c46b0 msgr2=0x5654388c5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:31.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3e045740 1 -- 172.21.15.44:0/2482927437 shutdown_connections 2024-08-20T03:11:31.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3e045740 1 -- 172.21.15.44:0/2482927437 wait complete. 2024-08-20T03:11:31.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3e045740 20 client.26743 trim_cache size 0 max 16384 2024-08-20T03:11:31.475 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:11:31.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/60580526 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbf1c0b3d40 msgr2=0x7fbf1c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:31.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3bc4c640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbf1c0b3d40 0x7fbf1c0b6200 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c1020e0 tx=0x7fbf2c031400 comp rx=0 tx=0).stop 2024-08-20T03:11:31.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/60580526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf34102ad0 msgr2=0x7fbf34199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:31.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.456+0000 7fbf3bc4c640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf34102ad0 0x7fbf34199000 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c005ab0 tx=0x7fbf2c003cd0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.457+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/60580526 shutdown_connections 2024-08-20T03:11:31.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.457+0000 7fbf3bc4c640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fbf1c0b3d40 0x7fbf1c0b6200 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.457+0000 7fbf3bc4c640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf34108e60 0x7fbf3419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.457+0000 7fbf3bc4c640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf341084e0 0x7fbf34199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.457+0000 7fbf3bc4c640 1 --2- 172.21.15.44:0/60580526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf34102ad0 0x7fbf34199000 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:31.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.457+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/60580526 >> 172.21.15.44:0/60580526 conn(0x7fbf340fe540 msgr2=0x7fbf34100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:31.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.457+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/60580526 shutdown_connections 2024-08-20T03:11:31.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:31.457+0000 7fbf3bc4c640 1 -- 172.21.15.44:0/60580526 wait complete. 2024-08-20T03:11:32.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:31 smithi044 ceph-mon[18790]: pgmap v596: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-08-20T03:11:32.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:31 smithi044 ceph-mon[18790]: osdmap e83: 12 total, 12 up, 12 in 2024-08-20T03:11:32.155 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T03:11:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:31 smithi165 ceph-mon[22357]: pgmap v596: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-08-20T03:11:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:31 smithi165 ceph-mon[22357]: osdmap e83: 12 total, 12 up, 12 in 2024-08-20T03:11:32.405 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:11:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:31 smithi099 ceph-mon[21984]: pgmap v596: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-08-20T03:11:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:31 smithi099 ceph-mon[21984]: osdmap e83: 12 total, 12 up, 12 in 2024-08-20T03:11:32.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.863+0000 7f98a948f640 1 -- 172.21.15.44:0/635385232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a4102ab0 msgr2=0x7f98a4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:32.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.863+0000 7f98a948f640 1 --2- 172.21.15.44:0/635385232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a4102ab0 0x7f98a4102e90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9894009900 tx=0x7f989402a410 comp rx=0 tx=0).stop 2024-08-20T03:11:32.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.863+0000 7f98a948f640 1 -- 172.21.15.44:0/635385232 shutdown_connections 2024-08-20T03:11:32.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.863+0000 7f98a948f640 1 --2- 172.21.15.44:0/635385232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98a4108e40 0x7f98a410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.863+0000 7f98a948f640 1 --2- 172.21.15.44:0/635385232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98a41084c0 0x7f98a4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.863+0000 7f98a948f640 1 --2- 172.21.15.44:0/635385232 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a4102ab0 0x7f98a4102e90 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.863+0000 7f98a948f640 1 -- 172.21.15.44:0/635385232 >> 172.21.15.44:0/635385232 conn(0x7f98a40fe500 msgr2=0x7f98a4100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.863+0000 7f98a948f640 1 -- 172.21.15.44:0/635385232 shutdown_connections 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.863+0000 7f98a948f640 1 -- 172.21.15.44:0/635385232 wait complete. 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.863+0000 7f98a948f640 1 Processor -- start 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a948f640 1 -- start start 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a948f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98a4102ab0 0x7f98a419b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a948f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a41084c0 0x7f98a419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a948f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98a4108e40 0x7f98a41a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a948f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4112b00 con 0x7f98a41084c0 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a948f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4112980 con 0x7f98a4108e40 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a948f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4112c80 con 0x7f98a4102ab0 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a41084c0 0x7f98a419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a41084c0 0x7f98a419b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38982/0 (socket says 172.21.15.44:38982) 2024-08-20T03:11:32.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a27fc640 1 -- 172.21.15.44:0/2406162550 learned_addr learned my addr 172.21.15.44:0/2406162550 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:32.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a37fe640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98a4108e40 0x7f98a41a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a2ffd640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98a4102ab0 0x7f98a419b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a27fc640 1 -- 172.21.15.44:0/2406162550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98a4102ab0 msgr2=0x7f98a419b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:32.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a27fc640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98a4102ab0 0x7f98a419b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a27fc640 1 -- 172.21.15.44:0/2406162550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98a4108e40 msgr2=0x7f98a41a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:32.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a27fc640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98a4108e40 0x7f98a41a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a27fc640 1 -- 172.21.15.44:0/2406162550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a41a1280 con 0x7f98a41084c0 2024-08-20T03:11:32.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.864+0000 7f98a2ffd640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98a4102ab0 0x7f98a419b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:32.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.865+0000 7f98a27fc640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a41084c0 0x7f98a419b710 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f989800d950 tx=0x7f989800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:32.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.865+0000 7f988bfff640 1 -- 172.21.15.44:0/2406162550 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898017070 con 0x7f98a41084c0 2024-08-20T03:11:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.865+0000 7f988bfff640 1 -- 172.21.15.44:0/2406162550 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9898009c30 con 0x7f98a41084c0 2024-08-20T03:11:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.865+0000 7f98a948f640 1 -- 172.21.15.44:0/2406162550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a41a1570 con 0x7f98a41084c0 2024-08-20T03:11:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.865+0000 7f988bfff640 1 -- 172.21.15.44:0/2406162550 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989800f620 con 0x7f98a41084c0 2024-08-20T03:11:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.865+0000 7f98a948f640 1 -- 172.21.15.44:0/2406162550 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a41a7c30 con 0x7f98a41084c0 2024-08-20T03:11:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.867+0000 7f988bfff640 1 -- 172.21.15.44:0/2406162550 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9898010460 con 0x7f98a41084c0 2024-08-20T03:11:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.867+0000 7f98ab888740 1 Processor -- start 2024-08-20T03:11:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.867+0000 7f98ab888740 1 -- start start 2024-08-20T03:11:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98ab888740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31219880 0x562b31219c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98ab888740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3121a1a0 0x562b31225bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98ab888740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31226130 0x562b31228520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98ab888740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562b31190930 con 0x562b31226130 2024-08-20T03:11:32.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98ab888740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562b3117fe30 con 0x562b3121a1a0 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98ab888740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562b31183e70 con 0x562b31219880 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98a27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31226130 0x562b31228520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98a27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31226130 0x562b31228520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38998/0 (socket says 172.21.15.44:38998) 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98a37fe640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31219880 0x562b31219c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98a27fc640 1 -- 172.21.15.44:0/3488909102 learned_addr learned my addr 172.21.15.44:0/3488909102 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98a2ffd640 1 --2- 172.21.15.44:0/3488909102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3121a1a0 0x562b31225bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f988bfff640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f988c0b3d40 0x7f988c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98a37fe640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f988c0b3d40 0x7f988c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98a27fc640 1 -- 172.21.15.44:0/3488909102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31219880 msgr2=0x562b31219c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98a27fc640 1 --2- 172.21.15.44:0/3488909102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31219880 0x562b31219c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98a27fc640 1 -- 172.21.15.44:0/3488909102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3121a1a0 msgr2=0x562b31225bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98a27fc640 1 --2- 172.21.15.44:0/3488909102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3121a1a0 0x562b31225bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f988bfff640 1 -- 172.21.15.44:0/2406162550 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f98980e3c00 con 0x7f98a41084c0 2024-08-20T03:11:32.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98a27fc640 1 -- 172.21.15.44:0/3488909102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b30edf650 con 0x562b31226130 2024-08-20T03:11:32.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.868+0000 7f98a27fc640 1 --2- 172.21.15.44:0/3488909102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31226130 0x562b31228520 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f98980e1330 tx=0x7f9898005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1bb5b2b7ff9bda server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:32.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98a37fe640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f988c0b3d40 0x7f988c0b6200 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f989c00afd0 tx=0x7f989c004ab0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:32.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98897fa640 1 -- 172.21.15.44:0/3488909102 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898020070 con 0x562b31226130 2024-08-20T03:11:32.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98897fa640 1 -- 172.21.15.44:0/3488909102 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98980b3250 con 0x562b31226130 2024-08-20T03:11:32.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98897fa640 1 -- 172.21.15.44:0/3488909102 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98980aa850 con 0x562b31226130 2024-08-20T03:11:32.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98ab888740 1 -- 172.21.15.44:0/3488909102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31226130 msgr2=0x562b31228520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:32.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98ab888740 1 --2- 172.21.15.44:0/3488909102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31226130 0x562b31228520 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f98980e1330 tx=0x7f9898005880 comp rx=0 tx=0).stop 2024-08-20T03:11:32.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98ab888740 1 -- 172.21.15.44:0/3488909102 shutdown_connections 2024-08-20T03:11:32.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98ab888740 1 --2- 172.21.15.44:0/3488909102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31226130 0x562b31228520 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98ab888740 1 --2- 172.21.15.44:0/3488909102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3121a1a0 0x562b31225bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98ab888740 1 --2- 172.21.15.44:0/3488909102 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31219880 0x562b31219c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98ab888740 1 -- 172.21.15.44:0/3488909102 >> 172.21.15.44:0/3488909102 conn(0x562b312176b0 msgr2=0x562b31218850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98ab888740 1 -- 172.21.15.44:0/3488909102 shutdown_connections 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.869+0000 7f98ab888740 1 -- 172.21.15.44:0/3488909102 wait complete. 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98ab888740 1 -- start start 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98ab888740 1 Processor -- start 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98ab888740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 0x562b312230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98ab888740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b3121a1a0 0x562b312235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98ab888740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b31226130 0x562b31223b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98ab888740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x562b30fe2700 con 0x562b31219880 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98ab888740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562b31106810 con 0x562b31226130 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98ab888740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x562b31228d20 con 0x562b3121a1a0 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98ab888740 10 client.0 _ll_register_callbacks cb 0x562b31179700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98a2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 0x562b312230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98a2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 0x562b312230a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39014/0 (socket says 172.21.15.44:39014) 2024-08-20T03:11:32.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98a2ffd640 1 -- 172.21.15.44:0/1844856133 learned_addr learned my addr 172.21.15.44:0/1844856133 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:32.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98a27fc640 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b3121a1a0 0x562b312235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98a2ffd640 1 -- 172.21.15.44:0/1844856133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b3121a1a0 msgr2=0x562b312235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:32.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98a2ffd640 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b3121a1a0 0x562b312235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98a37fe640 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b31226130 0x562b31223b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98a2ffd640 1 -- 172.21.15.44:0/1844856133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b31226130 msgr2=0x562b31223b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:32.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98a2ffd640 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b31226130 0x562b31223b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.870+0000 7f98a2ffd640 1 -- 172.21.15.44:0/1844856133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b312320d0 con 0x562b31219880 2024-08-20T03:11:32.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.871+0000 7f98a27fc640 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b3121a1a0 0x562b312235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:32.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.871+0000 7f98a37fe640 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b31226130 0x562b31223b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:32.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.871+0000 7f98a2ffd640 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 0x562b312230a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9894009230 tx=0x7f98940316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bd2f0983c1cb25 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:32.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.871+0000 7f9886ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.871+0000 7f9886ff5640 1 -- 172.21.15.44:0/1844856133 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894040070 con 0x562b31219880 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.871+0000 7f9886ff5640 1 -- 172.21.15.44:0/1844856133 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f9894004250 con 0x562b31219880 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.871+0000 7f98ab888740 1 -- 172.21.15.44:0/1844856133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b3112f150 con 0x562b31219880 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.871+0000 7f9886ff5640 1 -- 172.21.15.44:0/1844856133 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9894035040 con 0x562b31219880 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.871+0000 7f98ab888740 10 client.17385 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.871+0000 7f98ab888740 1 -- 172.21.15.44:0/1844856133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562b31234a80 con 0x562b31219880 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.871+0000 7f9886ff5640 1 -- 172.21.15.44:0/1844856133 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562b3112f150 con 0x562b31219880 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.871+0000 7f9886ff5640 1 -- 172.21.15.44:0/1844856133 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f9894004960 con 0x562b31219880 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.872+0000 7f98ab888740 10 client.17385 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.872+0000 7f98ab888740 10 client.17385 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.872+0000 7f98ab888740 20 client.17385 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.872+0000 7f98ab888740 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x562b31235fe0 0x562b31238460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.872+0000 7f98ab888740 4 client.17385 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:11:32.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.872+0000 7f98ab888740 1 -- 172.21.15.44:0/1844856133 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b31192880 con 0x562b31235fe0 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.872+0000 7f98a27fc640 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x562b31235fe0 0x562b31238460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.872+0000 7f9886ff5640 10 client.17385 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.872+0000 7f98a27fc640 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x562b31235fe0 0x562b31238460 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2d655575fa3366d server_cookie=b49df09a87c9629e in_seq=0 out_seq=0 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f9886ff5640 1 -- 172.21.15.44:0/1844856133 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b31192880 con 0x562b31235fe0 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f9886ff5640 10 client.17385 handle_command_reply: tid=0 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f98ab888740 1 client.17385 shutdown 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f98ab888740 1 -- 172.21.15.44:0/1844856133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 msgr2=0x562b312230a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f98ab888740 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 0x562b312230a0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9894009230 tx=0x7f98940316d0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f98ab888740 1 -- 172.21.15.44:0/1844856133 shutdown_connections 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f98ab888740 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b31226130 0x562b31223b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f98ab888740 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x562b31235fe0 0x562b31238460 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f98ab888740 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b3121a1a0 0x562b312235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f98ab888740 1 --2- 172.21.15.44:0/1844856133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 0x562b312230a0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f9886ff5640 0 client.17385 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:11:32.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f98ab888740 1 -- 172.21.15.44:0/1844856133 >> 172.21.15.44:0/1844856133 conn(0x562b312176b0 msgr2=0x562b3121d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:32.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f98ab888740 1 -- 172.21.15.44:0/1844856133 shutdown_connections 2024-08-20T03:11:32.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f98ab888740 1 -- 172.21.15.44:0/1844856133 wait complete. 2024-08-20T03:11:32.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.873+0000 7f98ab888740 20 client.17385 trim_cache size 0 max 16384 2024-08-20T03:11:32.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.886+0000 7f98ab888740 1 Processor -- start 2024-08-20T03:11:32.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.886+0000 7f98ab888740 1 -- start start 2024-08-20T03:11:32.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.886+0000 7f98ab888740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 0x562b3121d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.886+0000 7f98ab888740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3121a1a0 0x562b31216660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.886+0000 7f98ab888740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31226130 0x562b3123dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.886+0000 7f98ab888740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b0660 con 0x562b31219880 2024-08-20T03:11:32.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.886+0000 7f98ab888740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b22f0 con 0x562b3121a1a0 2024-08-20T03:11:32.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.886+0000 7f98ab888740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b0dc0 con 0x562b31226130 2024-08-20T03:11:32.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f98a2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 0x562b3121d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f98a2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 0x562b3121d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39028/0 (socket says 172.21.15.44:39028) 2024-08-20T03:11:32.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f98a2ffd640 1 -- 172.21.15.44:0/3590456896 learned_addr learned my addr 172.21.15.44:0/3590456896 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:32.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f98a27fc640 1 --2- 172.21.15.44:0/3590456896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3121a1a0 0x562b31216660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f98a2ffd640 1 -- 172.21.15.44:0/3590456896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31226130 msgr2=0x562b3123dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:32.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f98a2ffd640 1 --2- 172.21.15.44:0/3590456896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31226130 0x562b3123dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f98a2ffd640 1 -- 172.21.15.44:0/3590456896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3121a1a0 msgr2=0x562b31216660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f98a2ffd640 1 --2- 172.21.15.44:0/3590456896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3121a1a0 0x562b31216660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f98a2ffd640 1 -- 172.21.15.44:0/3590456896 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b31219fd0 con 0x562b31219880 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f98a27fc640 1 --2- 172.21.15.44:0/3590456896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3121a1a0 0x562b31216660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f98a2ffd640 1 --2- 172.21.15.44:0/3590456896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 0x562b3121d8f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9894009850 tx=0x7f989402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befab2617dc38e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f9888ff9640 1 -- 172.21.15.44:0/3590456896 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894000e60 con 0x562b31219880 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f9888ff9640 1 -- 172.21.15.44:0/3590456896 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9894004250 con 0x562b31219880 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.887+0000 7f9888ff9640 1 -- 172.21.15.44:0/3590456896 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894040070 con 0x562b31219880 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 -- 172.21.15.44:0/3590456896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 msgr2=0x562b3121d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 --2- 172.21.15.44:0/3590456896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 0x562b3121d8f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9894009850 tx=0x7f989402a410 comp rx=0 tx=0).stop 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 -- 172.21.15.44:0/3590456896 shutdown_connections 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 --2- 172.21.15.44:0/3590456896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31226130 0x562b3123dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 --2- 172.21.15.44:0/3590456896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b3121a1a0 0x562b31216660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 --2- 172.21.15.44:0/3590456896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b31219880 0x562b3121d8f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 -- 172.21.15.44:0/3590456896 >> 172.21.15.44:0/3590456896 conn(0x562b312176b0 msgr2=0x562b31215f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 -- 172.21.15.44:0/3590456896 shutdown_connections 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 -- 172.21.15.44:0/3590456896 wait complete. 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 -- start start 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 Processor -- start 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31219880 0x562b3122b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b3121a1a0 0x562b3122b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b31226130 0x562b3129ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b1c20 con 0x562b3121a1a0 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b14f0 con 0x562b31226130 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41ae660 con 0x562b31219880 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.888+0000 7f98ab888740 10 client.0 _ll_register_callbacks cb 0x7f9894005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:32.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98a27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b3121a1a0 0x562b3122b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98a27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b3121a1a0 0x562b3122b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39038/0 (socket says 172.21.15.44:39038) 2024-08-20T03:11:32.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98a27fc640 1 -- 172.21.15.44:0/3521387329 learned_addr learned my addr 172.21.15.44:0/3521387329 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:32.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98a37fe640 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b31226130 0x562b3129ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98a2ffd640 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31219880 0x562b3122b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98a27fc640 1 -- 172.21.15.44:0/3521387329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31219880 msgr2=0x562b3122b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:32.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98a27fc640 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31219880 0x562b3122b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98a27fc640 1 -- 172.21.15.44:0/3521387329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b31226130 msgr2=0x562b3129ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:32.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98a27fc640 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b31226130 0x562b3129ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98a27fc640 1 -- 172.21.15.44:0/3521387329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b3129f240 con 0x562b3121a1a0 2024-08-20T03:11:32.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98a27fc640 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b3121a1a0 0x562b3122b790 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f98980a68d0 tx=0x7f98980a6830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257010dfab07e8ce server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:32.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f9886ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:32.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f9886ff5640 1 -- 172.21.15.44:0/3521387329 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898020040 con 0x562b3121a1a0 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f9886ff5640 1 -- 172.21.15.44:0/3521387329 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f98980b3470 con 0x562b3121a1a0 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98ab888740 1 -- 172.21.15.44:0/3521387329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9894004250 con 0x562b3121a1a0 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f9886ff5640 1 -- 172.21.15.44:0/3521387329 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f98980a9fc0 con 0x562b3121a1a0 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98ab888740 10 client.17391 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.889+0000 7f98ab888740 1 -- 172.21.15.44:0/3521387329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562b312a03c0 con 0x562b3121a1a0 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.890+0000 7f9886ff5640 1 -- 172.21.15.44:0/3521387329 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9894004250 con 0x562b3121a1a0 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.890+0000 7f9886ff5640 1 -- 172.21.15.44:0/3521387329 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f98980eb040 con 0x562b3121a1a0 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.890+0000 7f98ab888740 10 client.17391 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.890+0000 7f98ab888740 10 client.17391 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.890+0000 7f98ab888740 20 client.17391 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.890+0000 7f98ab888740 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x562b312a0750 0x562b312a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.890+0000 7f98ab888740 4 client.17391 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:11:32.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.890+0000 7f98ab888740 1 -- 172.21.15.44:0/3521387329 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b3118e320 con 0x562b312a0750 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.890+0000 7f98a2ffd640 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x562b312a0750 0x562b312a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.890+0000 7f9886ff5640 10 client.17391 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.890+0000 7f98a2ffd640 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x562b312a0750 0x562b312a2b70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6809021eaedc7cf server_cookie=a7c60d3458042650 in_seq=0 out_seq=0 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f9886ff5640 1 -- 172.21.15.44:0/3521387329 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b3118e320 con 0x562b312a0750 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f9886ff5640 10 client.17391 handle_command_reply: tid=0 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98ab888740 1 client.17391 shutdown 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98ab888740 1 -- 172.21.15.44:0/3521387329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b3121a1a0 msgr2=0x562b3122b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98ab888740 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b3121a1a0 0x562b3122b790 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f98980a68d0 tx=0x7f98980a6830 comp rx=0 tx=0).stop 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98ab888740 1 -- 172.21.15.44:0/3521387329 shutdown_connections 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98ab888740 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562b31226130 0x562b3129ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98ab888740 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b3121a1a0 0x562b3122b790 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98ab888740 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x562b312a0750 0x562b312a2b70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98ab888740 1 --2- 172.21.15.44:0/3521387329 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562b31219880 0x562b3122b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98ab888740 1 -- 172.21.15.44:0/3521387329 >> 172.21.15.44:0/3521387329 conn(0x562b312176b0 msgr2=0x562b31216770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:32.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f9886ff5640 0 client.17391 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:11:32.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98ab888740 1 -- 172.21.15.44:0/3521387329 shutdown_connections 2024-08-20T03:11:32.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98ab888740 1 -- 172.21.15.44:0/3521387329 wait complete. 2024-08-20T03:11:32.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98ab888740 20 client.17391 trim_cache size 0 max 16384 2024-08-20T03:11:32.904 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:11:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98a948f640 1 -- 172.21.15.44:0/2406162550 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f988c0b3d40 msgr2=0x7f988c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98a948f640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f988c0b3d40 0x7f988c0b6200 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f989c00afd0 tx=0x7f989c004ab0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98a948f640 1 -- 172.21.15.44:0/2406162550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a41084c0 msgr2=0x7f98a419b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.891+0000 7f98a948f640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a41084c0 0x7f98a419b710 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f989800d950 tx=0x7f989800de10 comp rx=0 tx=0).stop 2024-08-20T03:11:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.892+0000 7f98a948f640 1 -- 172.21.15.44:0/2406162550 shutdown_connections 2024-08-20T03:11:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.892+0000 7f98a948f640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f988c0b3d40 0x7f988c0b6200 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.892+0000 7f98a948f640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98a4108e40 0x7f98a41a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.892+0000 7f98a948f640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98a41084c0 0x7f98a419b710 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.892+0000 7f98a948f640 1 --2- 172.21.15.44:0/2406162550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98a4102ab0 0x7f98a419b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:32.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.892+0000 7f98a948f640 1 -- 172.21.15.44:0/2406162550 >> 172.21.15.44:0/2406162550 conn(0x7f98a40fe500 msgr2=0x7f98a40ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:32.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.892+0000 7f98a948f640 1 -- 172.21.15.44:0/2406162550 shutdown_connections 2024-08-20T03:11:32.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:32.892+0000 7f98a948f640 1 -- 172.21.15.44:0/2406162550 wait complete. 2024-08-20T03:11:33.531 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T03:11:33.784 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:11:34.175 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:33 smithi044 ceph-mon[18790]: pgmap v598: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:34.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.255+0000 7fbab64f1640 1 -- 172.21.15.44:0/3330595995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbab010a630 msgr2=0x7fbab010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.255+0000 7fbab64f1640 1 --2- 172.21.15.44:0/3330595995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbab010a630 0x7fbab010ca20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009900 tx=0x7fbaa402a410 comp rx=0 tx=0).stop 2024-08-20T03:11:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.255+0000 7fbab64f1640 1 -- 172.21.15.44:0/3330595995 shutdown_connections 2024-08-20T03:11:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.255+0000 7fbab64f1640 1 --2- 172.21.15.44:0/3330595995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab010cf60 0x7fbab010f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.255+0000 7fbab64f1640 1 --2- 172.21.15.44:0/3330595995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbab010a630 0x7fbab010ca20 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.255+0000 7fbab64f1640 1 --2- 172.21.15.44:0/3330595995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbab0101520 0x7fbab0101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.255+0000 7fbab64f1640 1 -- 172.21.15.44:0/3330595995 >> 172.21.15.44:0/3330595995 conn(0x7fbab0076c80 msgr2=0x7fbab0077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.255+0000 7fbab64f1640 1 -- 172.21.15.44:0/3330595995 shutdown_connections 2024-08-20T03:11:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.255+0000 7fbab64f1640 1 -- 172.21.15.44:0/3330595995 wait complete. 2024-08-20T03:11:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.255+0000 7fbab64f1640 1 Processor -- start 2024-08-20T03:11:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.255+0000 7fbab64f1640 1 -- start start 2024-08-20T03:11:34.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbab64f1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbab0101520 0x7fbab0198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbab64f1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbab010a630 0x7fbab01994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbab64f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab010cf60 0x7fbab019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbab64f1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0112aa0 con 0x7fbab010a630 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbab64f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0112920 con 0x7fbab010cf60 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbab64f1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0112c20 con 0x7fbab0101520 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbaaf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbab010a630 0x7fbab01994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbaaf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbab010a630 0x7fbab01994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39056/0 (socket says 172.21.15.44:39056) 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbaaf7fe640 1 -- 172.21.15.44:0/746055267 learned_addr learned my addr 172.21.15.44:0/746055267 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbab4a8e640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab010cf60 0x7fbab019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbaaffff640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbab0101520 0x7fbab0198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbaaf7fe640 1 -- 172.21.15.44:0/746055267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbab0101520 msgr2=0x7fbab0198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbab0101520 0x7fbab0198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbaaf7fe640 1 -- 172.21.15.44:0/746055267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab010cf60 msgr2=0x7fbab019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab010cf60 0x7fbab019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbaaf7fe640 1 -- 172.21.15.44:0/746055267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab019f060 con 0x7fbab010a630 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbab010a630 0x7fbab01994f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4005940 tx=0x7fbaa4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.256+0000 7fbaad7fa640 1 -- 172.21.15.44:0/746055267 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa403f030 con 0x7fbab010a630 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.257+0000 7fbaad7fa640 1 -- 172.21.15.44:0/746055267 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa4003aa0 con 0x7fbab010a630 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.257+0000 7fbab64f1640 1 -- 172.21.15.44:0/746055267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab019f2f0 con 0x7fbab010a630 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.257+0000 7fbaad7fa640 1 -- 172.21.15.44:0/746055267 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4035070 con 0x7fbab010a630 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.257+0000 7fbab64f1640 1 -- 172.21.15.44:0/746055267 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab01a5a10 con 0x7fbab010a630 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.259+0000 7fbab88ea740 1 Processor -- start 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.259+0000 7fbaad7fa640 1 -- 172.21.15.44:0/746055267 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fbaa4042030 con 0x7fbab010a630 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.259+0000 7fbab88ea740 1 -- start start 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.259+0000 7fbab88ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fb9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.259+0000 7fbab88ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fba050 0x5622c6fc5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.259+0000 7fbab88ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fc5fe0 0x5622c6fc83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.259+0000 7fbab88ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5622c6f30930 con 0x5622c6fb9730 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.259+0000 7fbab88ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5622c6f1fe30 con 0x5622c6fc5fe0 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.259+0000 7fbab88ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5622c6f23e70 con 0x5622c6fba050 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbab4a8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fb9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbab4a8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fb9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39060/0 (socket says 172.21.15.44:39060) 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbab4a8e640 1 -- 172.21.15.44:0/4043585560 learned_addr learned my addr 172.21.15.44:0/4043585560 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbaaffff640 1 --2- 172.21.15.44:0/4043585560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fba050 0x5622c6fc5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/4043585560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fc5fe0 0x5622c6fc83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbab4a8e640 1 -- 172.21.15.44:0/4043585560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fba050 msgr2=0x5622c6fc5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbab4a8e640 1 --2- 172.21.15.44:0/4043585560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fba050 0x5622c6fc5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbab4a8e640 1 -- 172.21.15.44:0/4043585560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fc5fe0 msgr2=0x5622c6fc83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbab4a8e640 1 --2- 172.21.15.44:0/4043585560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fc5fe0 0x5622c6fc83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbab4a8e640 1 -- 172.21.15.44:0/4043585560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622c6c7f650 con 0x5622c6fb9730 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/4043585560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fc5fe0 0x5622c6fc83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbaad7fa640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fba980b3d40 0x7fba980b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbab4a8e640 1 --2- 172.21.15.44:0/4043585560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fb9b10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80103a0 tx=0x7fbaa8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b034eea60571c3c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbaaffff640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fba980b3d40 0x7fba980b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fbaad7fa640 1 -- 172.21.15.44:0/746055267 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7fbaa41045f0 con 0x7fbab010a630 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fba967fc640 1 -- 172.21.15.44:0/4043585560 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8016070 con 0x5622c6fb9730 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fba967fc640 1 -- 172.21.15.44:0/4043585560 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa8004d20 con 0x5622c6fb9730 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.260+0000 7fba967fc640 1 -- 172.21.15.44:0/4043585560 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8005350 con 0x5622c6fb9730 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.261+0000 7fbab88ea740 1 -- 172.21.15.44:0/4043585560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 msgr2=0x5622c6fb9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.261+0000 7fbaaffff640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fba980b3d40 0x7fba980b6200 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5622c6fbc8b0 tx=0x7fbaa0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.261+0000 7fbab88ea740 1 --2- 172.21.15.44:0/4043585560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fb9b10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80103a0 tx=0x7fbaa8010860 comp rx=0 tx=0).stop 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.261+0000 7fbab88ea740 1 -- 172.21.15.44:0/4043585560 shutdown_connections 2024-08-20T03:11:34.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.261+0000 7fbab88ea740 1 --2- 172.21.15.44:0/4043585560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fc5fe0 0x5622c6fc83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.261+0000 7fbab88ea740 1 --2- 172.21.15.44:0/4043585560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fba050 0x5622c6fc5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.261+0000 7fbab88ea740 1 --2- 172.21.15.44:0/4043585560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fb9b10 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.261+0000 7fbab88ea740 1 -- 172.21.15.44:0/4043585560 >> 172.21.15.44:0/4043585560 conn(0x5622c6fb7560 msgr2=0x5622c6fb8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.261+0000 7fbab88ea740 1 -- 172.21.15.44:0/4043585560 shutdown_connections 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.261+0000 7fbab88ea740 1 -- 172.21.15.44:0/4043585560 wait complete. 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.261+0000 7fbab88ea740 1 -- start start 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbab88ea740 1 Processor -- start 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbab88ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fb9730 0x5622c6fc3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbab88ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fba050 0x5622c6fc3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbab88ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fc5fe0 0x5622c6fc3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbab88ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5622c6d82700 con 0x5622c6fba050 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbab88ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5622c6ea6810 con 0x5622c6fc5fe0 2024-08-20T03:11:34.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbab88ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x5622c6fc8c90 con 0x5622c6fb9730 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbab88ea740 10 client.0 _ll_register_callbacks cb 0x5622c6f19700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbaaf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fba050 0x5622c6fc3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbaaf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fba050 0x5622c6fc3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39062/0 (socket says 172.21.15.44:39062) 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbaaf7fe640 1 -- 172.21.15.44:0/1283651075 learned_addr learned my addr 172.21.15.44:0/1283651075 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbab4a8e640 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fc5fe0 0x5622c6fc3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbaaffff640 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fb9730 0x5622c6fc3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbaaf7fe640 1 -- 172.21.15.44:0/1283651075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fb9730 msgr2=0x5622c6fc3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fb9730 0x5622c6fc3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbaaf7fe640 1 -- 172.21.15.44:0/1283651075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fc5fe0 msgr2=0x5622c6fc3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fc5fe0 0x5622c6fc3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbaaf7fe640 1 -- 172.21.15.44:0/1283651075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622c6fd20f0 con 0x5622c6fba050 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.262+0000 7fbaaffff640 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fb9730 0x5622c6fc3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fba050 0x5622c6fc3560 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40c8560 tx=0x7fbaa403bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd843120ad5d8a40 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:34.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fba8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fba8f7fe640 1 -- 172.21.15.44:0/1283651075 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4049040 con 0x5622c6fba050 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fba8f7fe640 1 -- 172.21.15.44:0/1283651075 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7fbaa40c7580 con 0x5622c6fba050 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fbab88ea740 1 -- 172.21.15.44:0/1283651075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5622c6ecf150 con 0x5622c6fba050 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fba8f7fe640 1 -- 172.21.15.44:0/1283651075 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbaa40caa60 con 0x5622c6fba050 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fbab88ea740 10 client.17403 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fbab88ea740 1 -- 172.21.15.44:0/1283651075 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5622c6fd4a10 con 0x5622c6fba050 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fba8f7fe640 1 -- 172.21.15.44:0/1283651075 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5622c6ecf150 con 0x5622c6fba050 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fba8f7fe640 1 -- 172.21.15.44:0/1283651075 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7fbaa40cb550 con 0x5622c6fba050 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fbab88ea740 10 client.17403 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fbab88ea740 10 client.17403 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fbab88ea740 20 client.17403 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fbab88ea740 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5622c6fd5d60 0x5622c6fd8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fbab88ea740 4 client.17403 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.263+0000 7fbab88ea740 1 -- 172.21.15.44:0/1283651075 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622c6f32880 con 0x5622c6fd5d60 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.264+0000 7fbaaffff640 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5622c6fd5d60 0x5622c6fd8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.264+0000 7fbaaffff640 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5622c6fd5d60 0x5622c6fd8190 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ae1a43b1c46040e server_cookie=400751d640ff7222 in_seq=0 out_seq=0 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.264+0000 7fba8f7fe640 10 client.17403 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fba8f7fe640 1 -- 172.21.15.44:0/1283651075 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622c6f32880 con 0x5622c6fd5d60 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fba8f7fe640 10 client.17403 handle_command_reply: tid=0 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fbab88ea740 1 client.17403 shutdown 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fbab88ea740 1 -- 172.21.15.44:0/1283651075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fba050 msgr2=0x5622c6fc3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fbab88ea740 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fba050 0x5622c6fc3560 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40c8560 tx=0x7fbaa403bf60 comp rx=0 tx=0).stop 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fbab88ea740 1 -- 172.21.15.44:0/1283651075 shutdown_connections 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fbab88ea740 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5622c6fd5d60 0x5622c6fd8190 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fbab88ea740 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fc5fe0 0x5622c6fc3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fbab88ea740 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fba050 0x5622c6fc3560 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fba8f7fe640 0 client.17403 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:11:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fbab88ea740 1 --2- 172.21.15.44:0/1283651075 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fb9730 0x5622c6fc3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fbab88ea740 1 -- 172.21.15.44:0/1283651075 >> 172.21.15.44:0/1283651075 conn(0x5622c6fb7560 msgr2=0x5622c6fbd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fbab88ea740 1 -- 172.21.15.44:0/1283651075 shutdown_connections 2024-08-20T03:11:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fbab88ea740 1 -- 172.21.15.44:0/1283651075 wait complete. 2024-08-20T03:11:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.265+0000 7fbab88ea740 20 client.17403 trim_cache size 0 max 16384 2024-08-20T03:11:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbab88ea740 1 Processor -- start 2024-08-20T03:11:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbab88ea740 1 -- start start 2024-08-20T03:11:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbab88ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fbd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbab88ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fba050 0x5622c6fdda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbab88ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fc5fe0 0x5622c6fddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbab88ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbab01ae260 con 0x5622c6fb9730 2024-08-20T03:11:34.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbab88ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbab01afef0 con 0x5622c6fba050 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbab88ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbab01ae9c0 con 0x5622c6fc5fe0 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbaaffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fbd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbaaffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fbd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39066/0 (socket says 172.21.15.44:39066) 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbaaffff640 1 -- 172.21.15.44:0/3291880380 learned_addr learned my addr 172.21.15.44:0/3291880380 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/3291880380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fba050 0x5622c6fdda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.278+0000 7fbab4a8e640 1 --2- 172.21.15.44:0/3291880380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fc5fe0 0x5622c6fddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbaaffff640 1 -- 172.21.15.44:0/3291880380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fc5fe0 msgr2=0x5622c6fddfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbaaffff640 1 --2- 172.21.15.44:0/3291880380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fc5fe0 0x5622c6fddfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbaaffff640 1 -- 172.21.15.44:0/3291880380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fba050 msgr2=0x5622c6fdda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbaaffff640 1 --2- 172.21.15.44:0/3291880380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fba050 0x5622c6fdda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbaaffff640 1 -- 172.21.15.44:0/3291880380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622c6fb9e80 con 0x5622c6fb9730 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbab4a8e640 1 --2- 172.21.15.44:0/3291880380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fc5fe0 0x5622c6fddfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/3291880380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fba050 0x5622c6fdda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:34.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbaaffff640 1 --2- 172.21.15.44:0/3291880380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fbd8f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x5622c6fd89d0 tx=0x7fbaa001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40555af44675fac server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fba95ffb640 1 -- 172.21.15.44:0/3291880380 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0036070 con 0x5622c6fb9730 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fba95ffb640 1 -- 172.21.15.44:0/3291880380 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa001dd10 con 0x5622c6fb9730 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fba95ffb640 1 -- 172.21.15.44:0/3291880380 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0029650 con 0x5622c6fb9730 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbab88ea740 1 -- 172.21.15.44:0/3291880380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 msgr2=0x5622c6fbd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbab88ea740 1 --2- 172.21.15.44:0/3291880380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fbd8f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x5622c6fd89d0 tx=0x7fbaa001b000 comp rx=0 tx=0).stop 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbab88ea740 1 -- 172.21.15.44:0/3291880380 shutdown_connections 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbab88ea740 1 --2- 172.21.15.44:0/3291880380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fc5fe0 0x5622c6fddfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbab88ea740 1 --2- 172.21.15.44:0/3291880380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fba050 0x5622c6fdda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.279+0000 7fbab88ea740 1 --2- 172.21.15.44:0/3291880380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fbd8f0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbab88ea740 1 -- 172.21.15.44:0/3291880380 >> 172.21.15.44:0/3291880380 conn(0x5622c6fb7560 msgr2=0x5622c6fc4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbab88ea740 1 -- 172.21.15.44:0/3291880380 shutdown_connections 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbab88ea740 1 -- 172.21.15.44:0/3291880380 wait complete. 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbab88ea740 1 -- start start 2024-08-20T03:11:34.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbab88ea740 1 Processor -- start 2024-08-20T03:11:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbab88ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbab88ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fba050 0x5622c6fce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbab88ea740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fc5fe0 0x5622c6fbf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbab88ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbab01af820 con 0x5622c6fb9730 2024-08-20T03:11:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbab88ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbab01af0f0 con 0x5622c6fba050 2024-08-20T03:11:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbab88ea740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fbab01ac260 con 0x5622c6fc5fe0 2024-08-20T03:11:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbab88ea740 10 client.0 _ll_register_callbacks cb 0x7fbaa4005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbaaffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbaaffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39078/0 (socket says 172.21.15.44:39078) 2024-08-20T03:11:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.280+0000 7fbaaffff640 1 -- 172.21.15.44:0/1816234018 learned_addr learned my addr 172.21.15.44:0/1816234018 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fba050 0x5622c6fce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbab4a8e640 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fc5fe0 0x5622c6fbf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbaaffff640 1 -- 172.21.15.44:0/1816234018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fc5fe0 msgr2=0x5622c6fbf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbaaffff640 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fc5fe0 0x5622c6fbf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbaaffff640 1 -- 172.21.15.44:0/1816234018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fba050 msgr2=0x5622c6fce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbaaffff640 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fba050 0x5622c6fce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbaaffff640 1 -- 172.21.15.44:0/1816234018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622c6fc0060 con 0x5622c6fb9730 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fba050 0x5622c6fce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbab4a8e640 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fc5fe0 0x5622c6fbf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbaaffff640 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fce100 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0005bc0 tx=0x7fbaa001df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdfa0c19de93354 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fba8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fba8f7fe640 1 -- 172.21.15.44:0/1816234018 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0041060 con 0x5622c6fb9730 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fba8f7fe640 1 -- 172.21.15.44:0/1816234018 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7fbaa0005100 con 0x5622c6fb9730 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbab88ea740 1 -- 172.21.15.44:0/1816234018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaa001dd10 con 0x5622c6fb9730 2024-08-20T03:11:34.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fba8f7fe640 1 -- 172.21.15.44:0/1816234018 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbaa0036030 con 0x5622c6fb9730 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbab88ea740 10 client.17409 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fbab88ea740 1 -- 172.21.15.44:0/1816234018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5622c7040460 con 0x5622c6fb9730 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.281+0000 7fba8f7fe640 1 -- 172.21.15.44:0/1816234018 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa001dd10 con 0x5622c6fb9730 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fba8f7fe640 1 -- 172.21.15.44:0/1816234018 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7fbaa0030030 con 0x5622c6fb9730 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fbab88ea740 10 client.17409 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fbab88ea740 10 client.17409 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fbab88ea740 20 client.17409 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fbab88ea740 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5622c7040c40 0x5622c7043060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fbab88ea740 4 client.17409 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fbab88ea740 1 -- 172.21.15.44:0/1816234018 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622c6f2e320 con 0x5622c7040c40 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5622c7040c40 0x5622c7043060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fbaaf7fe640 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5622c7040c40 0x5622c7043060 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f9e343018715033 server_cookie=a2fade627c536457 in_seq=0 out_seq=0 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fba8f7fe640 10 client.17409 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:11:34.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fba8f7fe640 1 -- 172.21.15.44:0/1816234018 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622c6f2e320 con 0x5622c7040c40 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fba8f7fe640 10 client.17409 handle_command_reply: tid=0 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.282+0000 7fbab88ea740 1 client.17409 shutdown 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.283+0000 7fbab88ea740 1 -- 172.21.15.44:0/1816234018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 msgr2=0x5622c6fce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.283+0000 7fbab88ea740 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fce100 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0005bc0 tx=0x7fbaa001df50 comp rx=0 tx=0).stop 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.283+0000 7fbab88ea740 1 -- 172.21.15.44:0/1816234018 shutdown_connections 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.283+0000 7fbab88ea740 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x5622c7040c40 0x5622c7043060 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.283+0000 7fbab88ea740 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c6fc5fe0 0x5622c6fbf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.283+0000 7fba8f7fe640 0 client.17409 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.283+0000 7fbab88ea740 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5622c6fba050 0x5622c6fce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.283+0000 7fbab88ea740 1 --2- 172.21.15.44:0/1816234018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5622c6fb9730 0x5622c6fce100 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.283+0000 7fbab88ea740 1 -- 172.21.15.44:0/1816234018 >> 172.21.15.44:0/1816234018 conn(0x5622c6fb7560 msgr2=0x5622c6fc2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.283+0000 7fbab88ea740 1 -- 172.21.15.44:0/1816234018 shutdown_connections 2024-08-20T03:11:34.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.283+0000 7fbab88ea740 1 -- 172.21.15.44:0/1816234018 wait complete. 2024-08-20T03:11:34.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.283+0000 7fbab88ea740 20 client.17409 trim_cache size 0 max 16384 2024-08-20T03:11:34.302 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:11:34.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.284+0000 7fbab64f1640 1 -- 172.21.15.44:0/746055267 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fba980b3d40 msgr2=0x7fba980b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:34.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.284+0000 7fbab64f1640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fba980b3d40 0x7fba980b6200 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5622c6fbc8b0 tx=0x7fbaa0005f90 comp rx=0 tx=0).stop 2024-08-20T03:11:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.284+0000 7fbab64f1640 1 -- 172.21.15.44:0/746055267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbab010a630 msgr2=0x7fbab01994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.284+0000 7fbab64f1640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbab010a630 0x7fbab01994f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4005940 tx=0x7fbaa4004270 comp rx=0 tx=0).stop 2024-08-20T03:11:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.284+0000 7fbab64f1640 1 -- 172.21.15.44:0/746055267 shutdown_connections 2024-08-20T03:11:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.284+0000 7fbab64f1640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fba980b3d40 0x7fba980b6200 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.284+0000 7fbab64f1640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbab010cf60 0x7fbab019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.284+0000 7fbab64f1640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbab010a630 0x7fbab01994f0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.284+0000 7fbab64f1640 1 --2- 172.21.15.44:0/746055267 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbab0101520 0x7fbab0198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.284+0000 7fbab64f1640 1 -- 172.21.15.44:0/746055267 >> 172.21.15.44:0/746055267 conn(0x7fbab0076c80 msgr2=0x7fbab01002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.284+0000 7fbab64f1640 1 -- 172.21.15.44:0/746055267 shutdown_connections 2024-08-20T03:11:34.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:34.284+0000 7fbab64f1640 1 -- 172.21.15.44:0/746055267 wait complete. 2024-08-20T03:11:34.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:33 smithi165 ceph-mon[22357]: pgmap v598: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:34.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:33 smithi099 ceph-mon[21984]: pgmap v598: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:35.012 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T03:11:35.266 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:11:35.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.716+0000 7f67e2913640 1 -- 172.21.15.44:0/4146498307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc1084c0 msgr2=0x7f67dc108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:35.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.716+0000 7f67e2913640 1 --2- 172.21.15.44:0/4146498307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc1084c0 0x7f67dc108900 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009900 tx=0x7f67d002a410 comp rx=0 tx=0).stop 2024-08-20T03:11:35.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.716+0000 7f67e2913640 1 -- 172.21.15.44:0/4146498307 shutdown_connections 2024-08-20T03:11:35.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.716+0000 7f67e2913640 1 --2- 172.21.15.44:0/4146498307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc108e40 0x7f67dc10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.716+0000 7f67e2913640 1 --2- 172.21.15.44:0/4146498307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc1084c0 0x7f67dc108900 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.716+0000 7f67e2913640 1 --2- 172.21.15.44:0/4146498307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67dc102ab0 0x7f67dc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.716+0000 7f67e2913640 1 -- 172.21.15.44:0/4146498307 >> 172.21.15.44:0/4146498307 conn(0x7f67dc0fe580 msgr2=0x7f67dc1009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:35.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.716+0000 7f67e2913640 1 -- 172.21.15.44:0/4146498307 shutdown_connections 2024-08-20T03:11:35.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.716+0000 7f67e2913640 1 -- 172.21.15.44:0/4146498307 wait complete. 2024-08-20T03:11:35.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.716+0000 7f67e2913640 1 Processor -- start 2024-08-20T03:11:35.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.716+0000 7f67e2913640 1 -- start start 2024-08-20T03:11:35.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67e2913640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67dc102ab0 0x7f67dc198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67e2913640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc1084c0 0x7f67dc199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67e2913640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc108e40 0x7f67dc19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67e2913640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc112b00 con 0x7f67dc1084c0 2024-08-20T03:11:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67e2913640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc112980 con 0x7f67dc102ab0 2024-08-20T03:11:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67e2913640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc112c80 con 0x7f67dc108e40 2024-08-20T03:11:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67db7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc1084c0 0x7f67dc199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67db7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc1084c0 0x7f67dc199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39104/0 (socket says 172.21.15.44:39104) 2024-08-20T03:11:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67db7fe640 1 -- 172.21.15.44:0/1997667163 learned_addr learned my addr 172.21.15.44:0/1997667163 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67e0eb0640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc108e40 0x7f67dc19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67dbfff640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67dc102ab0 0x7f67dc198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67db7fe640 1 -- 172.21.15.44:0/1997667163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc108e40 msgr2=0x7f67dc19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:35.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67db7fe640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc108e40 0x7f67dc19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67db7fe640 1 -- 172.21.15.44:0/1997667163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67dc102ab0 msgr2=0x7f67dc198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67db7fe640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67dc102ab0 0x7f67dc198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67db7fe640 1 -- 172.21.15.44:0/1997667163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc19f0a0 con 0x7f67dc1084c0 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67e0eb0640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc108e40 0x7f67dc19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67db7fe640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc1084c0 0x7f67dc199530 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f67d0005940 tx=0x7f67d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67d97fa640 1 -- 172.21.15.44:0/1997667163 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d003f030 con 0x7f67dc1084c0 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67e2913640 1 -- 172.21.15.44:0/1997667163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67dc19f330 con 0x7f67dc1084c0 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67d97fa640 1 -- 172.21.15.44:0/1997667163 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d0003cb0 con 0x7f67dc1084c0 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67d97fa640 1 -- 172.21.15.44:0/1997667163 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0035070 con 0x7f67dc1084c0 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.717+0000 7f67e2913640 1 -- 172.21.15.44:0/1997667163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67dc1a5a50 con 0x7f67dc1084c0 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67d97fa640 1 -- 172.21.15.44:0/1997667163 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f67d0042030 con 0x7f67dc1084c0 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e4d0c740 1 Processor -- start 2024-08-20T03:11:35.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e4d0c740 1 -- start start 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e4d0c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a30880 0x563048a30c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e4d0c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a311a0 0x563048a3cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e4d0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a3d130 0x563048a3f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e4d0c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5630489a7930 con 0x563048a30880 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e4d0c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563048996e30 con 0x563048a311a0 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e4d0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x56304899ae70 con 0x563048a3d130 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e0eb0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a30880 0x563048a30c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e0eb0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a30880 0x563048a30c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39106/0 (socket says 172.21.15.44:39106) 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e0eb0640 1 -- 172.21.15.44:0/1892319699 learned_addr learned my addr 172.21.15.44:0/1892319699 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67dbfff640 1 --2- 172.21.15.44:0/1892319699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a311a0 0x563048a3cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e0eb0640 1 -- 172.21.15.44:0/1892319699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a3d130 msgr2=0x563048a3f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67db7fe640 1 --2- 172.21.15.44:0/1892319699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a3d130 0x563048a3f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.720+0000 7f67e0eb0640 1 --2- 172.21.15.44:0/1892319699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a3d130 0x563048a3f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.721+0000 7f67e0eb0640 1 -- 172.21.15.44:0/1892319699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a311a0 msgr2=0x563048a3cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.721+0000 7f67e0eb0640 1 --2- 172.21.15.44:0/1892319699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a311a0 0x563048a3cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.721+0000 7f67e0eb0640 1 -- 172.21.15.44:0/1892319699 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630486f6650 con 0x563048a30880 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.721+0000 7f67db7fe640 1 --2- 172.21.15.44:0/1892319699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a3d130 0x563048a3f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.721+0000 7f67dbfff640 1 --2- 172.21.15.44:0/1892319699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a311a0 0x563048a3cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.721+0000 7f67e0eb0640 1 --2- 172.21.15.44:0/1892319699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a30880 0x563048a30c60 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f67d400c450 tx=0x7f67d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71888dafb1fb7fee server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.721+0000 7f67d97fa640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f67c40b3d40 0x7f67c40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.721+0000 7f67dbfff640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f67c40b3d40 0x7f67c40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.721+0000 7f67c27fc640 1 -- 172.21.15.44:0/1892319699 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4007430 con 0x563048a30880 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.721+0000 7f67c27fc640 1 -- 172.21.15.44:0/1892319699 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d4007a80 con 0x563048a30880 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.721+0000 7f67c27fc640 1 -- 172.21.15.44:0/1892319699 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d400aa00 con 0x563048a30880 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.721+0000 7f67d97fa640 1 -- 172.21.15.44:0/1997667163 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f67d0104940 con 0x7f67dc1084c0 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1892319699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a30880 msgr2=0x563048a30c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:35.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1892319699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a30880 0x563048a30c60 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f67d400c450 tx=0x7f67d400c910 comp rx=0 tx=0).stop 2024-08-20T03:11:35.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67dbfff640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f67c40b3d40 0x7f67c40b6200 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x563048a33a00 tx=0x7f67cc00b710 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:35.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1892319699 shutdown_connections 2024-08-20T03:11:35.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1892319699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a3d130 0x563048a3f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1892319699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a311a0 0x563048a3cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1892319699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a30880 0x563048a30c60 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1892319699 >> 172.21.15.44:0/1892319699 conn(0x563048a2e6b0 msgr2=0x563048a2f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:35.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1892319699 shutdown_connections 2024-08-20T03:11:35.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1892319699 wait complete. 2024-08-20T03:11:35.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 -- start start 2024-08-20T03:11:35.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 Processor -- start 2024-08-20T03:11:35.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a30880 0x563048a3a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a311a0 0x563048a3a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.722+0000 7f67e4d0c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 0x563048a3ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e4d0c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5630487f9700 con 0x563048a3d130 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e4d0c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56304891d810 con 0x563048a30880 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e4d0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563048a3fde0 con 0x563048a311a0 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e4d0c740 10 client.0 _ll_register_callbacks cb 0x563048990700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e0eb0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 0x563048a3ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e0eb0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 0x563048a3ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39122/0 (socket says 172.21.15.44:39122) 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e0eb0640 1 -- 172.21.15.44:0/1271497867 learned_addr learned my addr 172.21.15.44:0/1271497867 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67db7fe640 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a311a0 0x563048a3a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67dbfff640 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a30880 0x563048a3a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e0eb0640 1 -- 172.21.15.44:0/1271497867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a311a0 msgr2=0x563048a3a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e0eb0640 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a311a0 0x563048a3a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e0eb0640 1 -- 172.21.15.44:0/1271497867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a30880 msgr2=0x563048a3a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e0eb0640 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a30880 0x563048a3a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e0eb0640 1 -- 172.21.15.44:0/1271497867 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563048a491c0 con 0x563048a3d130 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e0eb0640 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 0x563048a3ab60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009ba0 tx=0x7f67d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66fd506f658f159 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67bb7fe640 1 -- 172.21.15.44:0/1271497867 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40094f0 con 0x563048a3d130 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67bb7fe640 1 -- 172.21.15.44:0/1271497867 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f67d4012040 con 0x563048a3d130 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1271497867 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563048946150 con 0x563048a3d130 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.723+0000 7f67bb7fe640 1 -- 172.21.15.44:0/1271497867 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f67d402c050 con 0x563048a3d130 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.724+0000 7f67e4d0c740 10 client.17421 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.724+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1271497867 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563048a4bac0 con 0x563048a3d130 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.724+0000 7f67bb7fe640 1 -- 172.21.15.44:0/1271497867 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563048946150 con 0x563048a3d130 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.724+0000 7f67bb7fe640 1 -- 172.21.15.44:0/1271497867 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f67d401e410 con 0x563048a3d130 2024-08-20T03:11:35.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.724+0000 7f67e4d0c740 10 client.17421 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.724+0000 7f67e4d0c740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.724+0000 7f67e4d0c740 20 client.17421 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.724+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563048a4cf20 0x563048a4f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.724+0000 7f67e4d0c740 4 client.17421 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.724+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1271497867 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630489a9880 con 0x563048a4cf20 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.724+0000 7f67dbfff640 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563048a4cf20 0x563048a4f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.725+0000 7f67bb7fe640 10 client.17421 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.725+0000 7f67dbfff640 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563048a4cf20 0x563048a4f310 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e34ec99905f34c86 server_cookie=661aacc8aa223c6f in_seq=0 out_seq=0 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.725+0000 7f67bb7fe640 1 -- 172.21.15.44:0/1271497867 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630489a9880 con 0x563048a4cf20 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.725+0000 7f67bb7fe640 10 client.17421 handle_command_reply: tid=0 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.725+0000 7f67e4d0c740 1 client.17421 shutdown 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.726+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1271497867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 msgr2=0x563048a3ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:35.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.726+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 0x563048a3ab60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009ba0 tx=0x7f67d4009bd0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.726+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1271497867 shutdown_connections 2024-08-20T03:11:35.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.726+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563048a4cf20 0x563048a4f310 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.726+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 0x563048a3ab60 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.726+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a311a0 0x563048a3a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.726+0000 7f67bb7fe640 0 client.17421 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:35.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.726+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1271497867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a30880 0x563048a3a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.726+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1271497867 >> 172.21.15.44:0/1271497867 conn(0x563048a2e6b0 msgr2=0x563048a34250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:35.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.726+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1271497867 shutdown_connections 2024-08-20T03:11:35.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.726+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1271497867 wait complete. 2024-08-20T03:11:35.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.726+0000 7f67e4d0c740 20 client.17421 trim_cache size 0 max 16384 2024-08-20T03:11:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67e4d0c740 1 Processor -- start 2024-08-20T03:11:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67e4d0c740 1 -- start start 2024-08-20T03:11:35.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67e4d0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a30880 0x563048a3a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67e4d0c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a311a0 0x563048a549c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67e4d0c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 0x563048a54f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67e4d0c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1ae400 con 0x563048a3d130 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67e4d0c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1b0090 con 0x563048a311a0 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67e4d0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1aeb60 con 0x563048a30880 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67e0eb0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 0x563048a54f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67e0eb0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 0x563048a54f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39138/0 (socket says 172.21.15.44:39138) 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67e0eb0640 1 -- 172.21.15.44:0/1951588666 learned_addr learned my addr 172.21.15.44:0/1951588666 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67dbfff640 1 --2- 172.21.15.44:0/1951588666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a30880 0x563048a3a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67db7fe640 1 --2- 172.21.15.44:0/1951588666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a311a0 0x563048a549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.739+0000 7f67e0eb0640 1 -- 172.21.15.44:0/1951588666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a30880 msgr2=0x563048a3a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67e0eb0640 1 --2- 172.21.15.44:0/1951588666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a30880 0x563048a3a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67e0eb0640 1 -- 172.21.15.44:0/1951588666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a311a0 msgr2=0x563048a549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67e0eb0640 1 --2- 172.21.15.44:0/1951588666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a311a0 0x563048a549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67e0eb0640 1 -- 172.21.15.44:0/1951588666 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563048a2e4e0 con 0x563048a3d130 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67dbfff640 1 --2- 172.21.15.44:0/1951588666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a30880 0x563048a3a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67db7fe640 1 --2- 172.21.15.44:0/1951588666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a311a0 0x563048a549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67e0eb0640 1 --2- 172.21.15.44:0/1951588666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 0x563048a54f00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f67d400c950 tx=0x7f67d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f223595814f3a5f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67c1ffb640 1 -- 172.21.15.44:0/1951588666 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4035070 con 0x563048a3d130 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67c1ffb640 1 -- 172.21.15.44:0/1951588666 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d4023070 con 0x563048a3d130 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67c1ffb640 1 -- 172.21.15.44:0/1951588666 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d400a5a0 con 0x563048a3d130 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1951588666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 msgr2=0x563048a54f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1951588666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 0x563048a54f00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f67d400c950 tx=0x7f67d400ce30 comp rx=0 tx=0).stop 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1951588666 shutdown_connections 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1951588666 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a3d130 0x563048a54f00 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1951588666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a311a0 0x563048a549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.740+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/1951588666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a30880 0x563048a3a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1951588666 >> 172.21.15.44:0/1951588666 conn(0x563048a2e6b0 msgr2=0x563048a3bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1951588666 shutdown_connections 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67e4d0c740 1 -- 172.21.15.44:0/1951588666 wait complete. 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67e4d0c740 1 -- start start 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67e4d0c740 1 Processor -- start 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67e4d0c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a30880 0x563048aafc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67e4d0c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a311a0 0x563048ab01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67e4d0c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a3d130 0x563048a426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67e4d0c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1af9c0 con 0x563048a311a0 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67e4d0c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1af290 con 0x563048a30880 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67e4d0c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1ac3e0 con 0x563048a3d130 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67e4d0c740 10 client.0 _ll_register_callbacks cb 0x7f67d4009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:35.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.741+0000 7f67db7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a311a0 0x563048ab01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67db7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a311a0 0x563048ab01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39148/0 (socket says 172.21.15.44:39148) 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67db7fe640 1 -- 172.21.15.44:0/2548269773 learned_addr learned my addr 172.21.15.44:0/2548269773 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67e0eb0640 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a3d130 0x563048a426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67dbfff640 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a30880 0x563048aafc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67db7fe640 1 -- 172.21.15.44:0/2548269773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a3d130 msgr2=0x563048a426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67db7fe640 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a3d130 0x563048a426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67db7fe640 1 -- 172.21.15.44:0/2548269773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a30880 msgr2=0x563048aafc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67db7fe640 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a30880 0x563048aafc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67db7fe640 1 -- 172.21.15.44:0/2548269773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563048a2f130 con 0x563048a311a0 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67dbfff640 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a30880 0x563048aafc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67db7fe640 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a311a0 0x563048ab01e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f67d0006010 tx=0x7f67d00c7510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a0dc3347306563 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:35.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67bb7fe640 1 -- 172.21.15.44:0/2548269773 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0049030 con 0x563048a311a0 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67bb7fe640 1 -- 172.21.15.44:0/2548269773 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f67d00c7820 con 0x563048a311a0 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67e4d0c740 1 -- 172.21.15.44:0/2548269773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d4023070 con 0x563048a311a0 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67bb7fe640 1 -- 172.21.15.44:0/2548269773 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f67d0005d50 con 0x563048a311a0 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67e4d0c740 10 client.17427 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67e4d0c740 1 -- 172.21.15.44:0/2548269773 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563048ab73b0 con 0x563048a311a0 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.742+0000 7f67bb7fe640 1 -- 172.21.15.44:0/2548269773 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d4023070 con 0x563048a311a0 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.743+0000 7f67bb7fe640 1 -- 172.21.15.44:0/2548269773 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f67d00cb7f0 con 0x563048a311a0 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.743+0000 7f67e4d0c740 10 client.17427 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.743+0000 7f67e4d0c740 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.743+0000 7f67e4d0c740 20 client.17427 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.743+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563048ab7ae0 0x563048ab9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.743+0000 7f67e4d0c740 4 client.17427 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.743+0000 7f67e4d0c740 1 -- 172.21.15.44:0/2548269773 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630489a5320 con 0x563048ab7ae0 2024-08-20T03:11:35.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.743+0000 7f67dbfff640 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563048ab7ae0 0x563048ab9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.743+0000 7f67dbfff640 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563048ab7ae0 0x563048ab9f00 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94e2a23502bc827 server_cookie=6af861ac85c93755 in_seq=0 out_seq=0 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.743+0000 7f67bb7fe640 10 client.17427 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67bb7fe640 1 -- 172.21.15.44:0/2548269773 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630489a5320 con 0x563048ab7ae0 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67bb7fe640 10 client.17427 handle_command_reply: tid=0 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67e4d0c740 1 client.17427 shutdown 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67e4d0c740 1 -- 172.21.15.44:0/2548269773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a311a0 msgr2=0x563048ab01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a311a0 0x563048ab01e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f67d0006010 tx=0x7f67d00c7510 comp rx=0 tx=0).stop 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67e4d0c740 1 -- 172.21.15.44:0/2548269773 shutdown_connections 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x563048ab7ae0 0x563048ab9f00 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563048a3d130 0x563048a426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67bb7fe640 0 client.17427 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563048a311a0 0x563048ab01e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67e4d0c740 1 --2- 172.21.15.44:0/2548269773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563048a30880 0x563048aafc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67e4d0c740 1 -- 172.21.15.44:0/2548269773 >> 172.21.15.44:0/2548269773 conn(0x563048a2e6b0 msgr2=0x563048a33a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:35.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67e4d0c740 1 -- 172.21.15.44:0/2548269773 shutdown_connections 2024-08-20T03:11:35.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67e4d0c740 1 -- 172.21.15.44:0/2548269773 wait complete. 2024-08-20T03:11:35.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.744+0000 7f67e4d0c740 20 client.17427 trim_cache size 0 max 16384 2024-08-20T03:11:35.757 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:11:35.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.745+0000 7f67e2913640 1 -- 172.21.15.44:0/1997667163 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f67c40b3d40 msgr2=0x7f67c40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:35.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.745+0000 7f67e2913640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f67c40b3d40 0x7f67c40b6200 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x563048a33a00 tx=0x7f67cc00b710 comp rx=0 tx=0).stop 2024-08-20T03:11:35.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.745+0000 7f67e2913640 1 -- 172.21.15.44:0/1997667163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc1084c0 msgr2=0x7f67dc199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:35.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.745+0000 7f67e2913640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc1084c0 0x7f67dc199530 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f67d0005940 tx=0x7f67d00042f0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.745+0000 7f67e2913640 1 -- 172.21.15.44:0/1997667163 shutdown_connections 2024-08-20T03:11:35.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.745+0000 7f67e2913640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f67c40b3d40 0x7f67c40b6200 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.745+0000 7f67e2913640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67dc108e40 0x7f67dc19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.745+0000 7f67e2913640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67dc1084c0 0x7f67dc199530 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.745+0000 7f67e2913640 1 --2- 172.21.15.44:0/1997667163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67dc102ab0 0x7f67dc198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:35.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.745+0000 7f67e2913640 1 -- 172.21.15.44:0/1997667163 >> 172.21.15.44:0/1997667163 conn(0x7f67dc0fe580 msgr2=0x7f67dc100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:35.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.745+0000 7f67e2913640 1 -- 172.21.15.44:0/1997667163 shutdown_connections 2024-08-20T03:11:35.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:35.745+0000 7f67e2913640 1 -- 172.21.15.44:0/1997667163 wait complete. 2024-08-20T03:11:36.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:35 smithi044 ceph-mon[18790]: pgmap v599: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:36.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:35 smithi165 ceph-mon[22357]: pgmap v599: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:35 smithi099 ceph-mon[21984]: pgmap v599: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:37.456 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T03:11:37.457 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T03:11:37.726 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:11:38.113 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:37 smithi044 ceph-mon[18790]: pgmap v600: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:38.114 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:11:38.114 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:11:38.114 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:11:38.114 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:37 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:11:38.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.197+0000 7f0d45ebc640 1 -- 172.21.15.44:0/3332591963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d40108e40 msgr2=0x7f0d4010f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:38.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.197+0000 7f0d45ebc640 1 --2- 172.21.15.44:0/3332591963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d40108e40 0x7f0d4010f640 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009ca0 tx=0x7f0d3802a8c0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.197+0000 7f0d45ebc640 1 -- 172.21.15.44:0/3332591963 shutdown_connections 2024-08-20T03:11:38.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.197+0000 7f0d45ebc640 1 --2- 172.21.15.44:0/3332591963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d40108e40 0x7f0d4010f640 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.197+0000 7f0d45ebc640 1 --2- 172.21.15.44:0/3332591963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d401084c0 0x7f0d40108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.197+0000 7f0d45ebc640 1 --2- 172.21.15.44:0/3332591963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d40102ab0 0x7f0d40102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.197+0000 7f0d45ebc640 1 -- 172.21.15.44:0/3332591963 >> 172.21.15.44:0/3332591963 conn(0x7f0d400fe500 msgr2=0x7f0d40100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:38.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.197+0000 7f0d45ebc640 1 -- 172.21.15.44:0/3332591963 shutdown_connections 2024-08-20T03:11:38.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.197+0000 7f0d45ebc640 1 -- 172.21.15.44:0/3332591963 wait complete. 2024-08-20T03:11:38.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.197+0000 7f0d45ebc640 1 Processor -- start 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.197+0000 7f0d45ebc640 1 -- start start 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d45ebc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d40102ab0 0x7f0d400751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d45ebc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d401084c0 0x7f0d40075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d45ebc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d40108e40 0x7f0d40075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d45ebc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d40112ac0 con 0x7f0d40102ab0 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d45ebc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0d40112940 con 0x7f0d401084c0 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d45ebc640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d40112c40 con 0x7f0d40108e40 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d3f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d40102ab0 0x7f0d400751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d3f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d40102ab0 0x7f0d400751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37946/0 (socket says 172.21.15.44:37946) 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d3f7fe640 1 -- 172.21.15.44:0/2939301127 learned_addr learned my addr 172.21.15.44:0/2939301127 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d36ffd640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d401084c0 0x7f0d40075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d40108e40 0x7f0d40075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d3f7fe640 1 -- 172.21.15.44:0/2939301127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d40108e40 msgr2=0x7f0d40075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:38.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d40108e40 0x7f0d40075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d3f7fe640 1 -- 172.21.15.44:0/2939301127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d401084c0 msgr2=0x7f0d40075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:38.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d401084c0 0x7f0d40075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d3f7fe640 1 -- 172.21.15.44:0/2939301127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d401a7330 con 0x7f0d40102ab0 2024-08-20T03:11:38.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d40108e40 0x7f0d40075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:38.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.198+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d40102ab0 0x7f0d400751d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000c920 tx=0x7f0d3000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:38.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.199+0000 7f0d3d7fa640 1 -- 172.21.15.44:0/2939301127 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30007c70 con 0x7f0d40102ab0 2024-08-20T03:11:38.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.199+0000 7f0d3d7fa640 1 -- 172.21.15.44:0/2939301127 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d30007e10 con 0x7f0d40102ab0 2024-08-20T03:11:38.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.199+0000 7f0d45ebc640 1 -- 172.21.15.44:0/2939301127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d401a7620 con 0x7f0d40102ab0 2024-08-20T03:11:38.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.199+0000 7f0d3d7fa640 1 -- 172.21.15.44:0/2939301127 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30018820 con 0x7f0d40102ab0 2024-08-20T03:11:38.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.199+0000 7f0d45ebc640 1 -- 172.21.15.44:0/2939301127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d401a7b60 con 0x7f0d40102ab0 2024-08-20T03:11:38.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.201+0000 7f0d3d7fa640 1 -- 172.21.15.44:0/2939301127 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0d3000f450 con 0x7f0d40102ab0 2024-08-20T03:11:38.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.201+0000 7f0d482b5740 1 Processor -- start 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.201+0000 7f0d482b5740 1 -- start start 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.201+0000 7f0d482b5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03e3b0 0x564bfc03e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.201+0000 7f0d482b5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc04a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.201+0000 7f0d482b5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc04ac60 0x564bfc04d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.201+0000 7f0d482b5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564bfbfb4940 con 0x564bfc03e3b0 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.201+0000 7f0d482b5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564bfbfa3e40 con 0x564bfc03ecd0 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.201+0000 7f0d482b5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564bfbfa7e80 con 0x564bfc04ac60 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03e3b0 0x564bfc03e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03e3b0 0x564bfc03e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37952/0 (socket says 172.21.15.44:37952) 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3ffff640 1 -- 172.21.15.44:0/175062429 learned_addr learned my addr 172.21.15.44:0/175062429 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/175062429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc04a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3d7fa640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0d2c0b3d40 0x7f0d2c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d36ffd640 1 --2- 172.21.15.44:0/175062429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc04ac60 0x564bfc04d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3ffff640 1 -- 172.21.15.44:0/175062429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc04ac60 msgr2=0x564bfc04d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/175062429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc04ac60 0x564bfc04d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3ffff640 1 -- 172.21.15.44:0/175062429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 msgr2=0x564bfc04a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/175062429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc04a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3ffff640 1 -- 172.21.15.44:0/175062429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bfbf40520 con 0x564bfc03e3b0 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/175062429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc04a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3d7fa640 1 -- 172.21.15.44:0/2939301127 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f0d300e4080 con 0x7f0d40102ab0 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0d2c0b3d40 0x7f0d2c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/175062429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03e3b0 0x564bfc03e790 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0d380098e0 tx=0x7f0d38009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718025f06276c5b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d35ffb640 1 -- 172.21.15.44:0/175062429 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38003e40 con 0x564bfc03e3b0 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.202+0000 7f0d35ffb640 1 -- 172.21.15.44:0/175062429 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d38036070 con 0x564bfc03e3b0 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.203+0000 7f0d35ffb640 1 -- 172.21.15.44:0/175062429 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3802fa20 con 0x564bfc03e3b0 2024-08-20T03:11:38.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.203+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0d2c0b3d40 0x7f0d2c0b6200 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0d3804b000 tx=0x7f0d38048000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.203+0000 7f0d482b5740 1 -- 172.21.15.44:0/175062429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03e3b0 msgr2=0x564bfc03e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.203+0000 7f0d482b5740 1 --2- 172.21.15.44:0/175062429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03e3b0 0x564bfc03e790 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0d380098e0 tx=0x7f0d38009910 comp rx=0 tx=0).stop 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.203+0000 7f0d482b5740 1 -- 172.21.15.44:0/175062429 shutdown_connections 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.203+0000 7f0d482b5740 1 --2- 172.21.15.44:0/175062429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc04ac60 0x564bfc04d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.203+0000 7f0d482b5740 1 --2- 172.21.15.44:0/175062429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc04a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.203+0000 7f0d482b5740 1 --2- 172.21.15.44:0/175062429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03e3b0 0x564bfc03e790 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.203+0000 7f0d482b5740 1 -- 172.21.15.44:0/175062429 >> 172.21.15.44:0/175062429 conn(0x564bfc03c1e0 msgr2=0x564bfc03d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.203+0000 7f0d482b5740 1 -- 172.21.15.44:0/175062429 shutdown_connections 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.203+0000 7f0d482b5740 1 -- 172.21.15.44:0/175062429 wait complete. 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.203+0000 7f0d482b5740 1 -- start start 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d482b5740 1 Processor -- start 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d482b5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc047c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d482b5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc0481b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d482b5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 0x564bfc0486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d482b5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x564bfbe06890 con 0x564bfc04ac60 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d482b5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564bfbf2a700 con 0x564bfc03ecd0 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d482b5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x564bfc04d830 con 0x564bfc03e3b0 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d482b5740 10 client.0 _ll_register_callbacks cb 0x564bfbf9f6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 0x564bfc0486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 0x564bfc0486f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37954/0 (socket says 172.21.15.44:37954) 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d3ffff640 1 -- 172.21.15.44:0/3248577784 learned_addr learned my addr 172.21.15.44:0/3248577784 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc047c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d36ffd640 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc0481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d3ffff640 1 -- 172.21.15.44:0/3248577784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 msgr2=0x564bfc047c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.204+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc047c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d3ffff640 1 -- 172.21.15.44:0/3248577784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 msgr2=0x564bfc0481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc0481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d3ffff640 1 -- 172.21.15.44:0/3248577784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bfbd03930 con 0x564bfc04ac60 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d36ffd640 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc0481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc047c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 0x564bfc0486f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0d38004af0 tx=0x7f0d380093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cfab025a89abc5d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d1effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d1effd640 1 -- 172.21.15.44:0/3248577784 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38004040 con 0x564bfc04ac60 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d1effd640 1 -- 172.21.15.44:0/3248577784 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f0d3804d050 con 0x564bfc04ac60 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d482b5740 1 -- 172.21.15.44:0/3248577784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d38036070 con 0x564bfc04ac60 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d1effd640 1 -- 172.21.15.44:0/3248577784 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0d380041e0 con 0x564bfc04ac60 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d482b5740 10 client.17439 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d482b5740 1 -- 172.21.15.44:0/3248577784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564bfc059440 con 0x564bfc04ac60 2024-08-20T03:11:38.230 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d1effd640 1 -- 172.21.15.44:0/3248577784 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d38036070 con 0x564bfc04ac60 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.205+0000 7f0d1effd640 1 -- 172.21.15.44:0/3248577784 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f0d38030b90 con 0x564bfc04ac60 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.206+0000 7f0d482b5740 10 client.17439 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.206+0000 7f0d482b5740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.206+0000 7f0d482b5740 20 client.17439 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.206+0000 7f0d482b5740 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x564bfc05a790 0x564bfc05cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.206+0000 7f0d482b5740 4 client.17439 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.206+0000 7f0d482b5740 1 -- 172.21.15.44:0/3248577784 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bfbfb6890 con 0x564bfc05a790 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.206+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x564bfc05a790 0x564bfc05cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.206+0000 7f0d1effd640 10 client.17439 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.206+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x564bfc05a790 0x564bfc05cb80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34d2edfb34c136af server_cookie=93192a24e5d98020 in_seq=0 out_seq=0 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d1effd640 1 -- 172.21.15.44:0/3248577784 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bfbfb6890 con 0x564bfc05a790 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d1effd640 10 client.17439 handle_command_reply: tid=0 2024-08-20T03:11:38.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d482b5740 1 client.17439 shutdown 2024-08-20T03:11:38.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d482b5740 1 -- 172.21.15.44:0/3248577784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 msgr2=0x564bfc0486f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:38.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d482b5740 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 0x564bfc0486f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0d38004af0 tx=0x7f0d380093a0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d482b5740 1 -- 172.21.15.44:0/3248577784 shutdown_connections 2024-08-20T03:11:38.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d482b5740 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x564bfc05a790 0x564bfc05cb80 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d482b5740 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 0x564bfc0486f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d482b5740 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc0481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d1effd640 0 client.17439 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:38.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d482b5740 1 --2- 172.21.15.44:0/3248577784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc047c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d482b5740 1 -- 172.21.15.44:0/3248577784 >> 172.21.15.44:0/3248577784 conn(0x564bfc03c1e0 msgr2=0x564bfc041d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:38.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d482b5740 1 -- 172.21.15.44:0/3248577784 shutdown_connections 2024-08-20T03:11:38.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d482b5740 1 -- 172.21.15.44:0/3248577784 wait complete. 2024-08-20T03:11:38.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.207+0000 7f0d482b5740 20 client.17439 trim_cache size 0 max 16384 2024-08-20T03:11:38.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d482b5740 1 Processor -- start 2024-08-20T03:11:38.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d482b5740 1 -- start start 2024-08-20T03:11:38.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d482b5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc042420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d482b5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03ecd0 0x564bfc062270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d482b5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc04ac60 0x564bfc0627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d482b5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401b0480 con 0x564bfc03ecd0 2024-08-20T03:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d482b5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401b2110 con 0x564bfc04ac60 2024-08-20T03:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d482b5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401b0be0 con 0x564bfc03e3b0 2024-08-20T03:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d36ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03ecd0 0x564bfc062270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d36ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03ecd0 0x564bfc062270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37956/0 (socket says 172.21.15.44:37956) 2024-08-20T03:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d36ffd640 1 -- 172.21.15.44:0/1762798950 learned_addr learned my addr 172.21.15.44:0/1762798950 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/1762798950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc042420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/1762798950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc04ac60 0x564bfc0627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d36ffd640 1 -- 172.21.15.44:0/1762798950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 msgr2=0x564bfc042420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d36ffd640 1 --2- 172.21.15.44:0/1762798950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc042420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d36ffd640 1 -- 172.21.15.44:0/1762798950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc04ac60 msgr2=0x564bfc0627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:38.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d36ffd640 1 --2- 172.21.15.44:0/1762798950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc04ac60 0x564bfc0627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.225+0000 7f0d36ffd640 1 -- 172.21.15.44:0/1762798950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bfc048b10 con 0x564bfc03ecd0 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.226+0000 7f0d36ffd640 1 --2- 172.21.15.44:0/1762798950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03ecd0 0x564bfc062270 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800bc40 tx=0x7f0d28012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdbd74c74ba2d02 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.226+0000 7f0d357fa640 1 -- 172.21.15.44:0/1762798950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800bdf0 con 0x564bfc03ecd0 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.226+0000 7f0d357fa640 1 -- 172.21.15.44:0/1762798950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d28005020 con 0x564bfc03ecd0 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.226+0000 7f0d357fa640 1 -- 172.21.15.44:0/1762798950 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28004a20 con 0x564bfc03ecd0 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 -- 172.21.15.44:0/1762798950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03ecd0 msgr2=0x564bfc062270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 --2- 172.21.15.44:0/1762798950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03ecd0 0x564bfc062270 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800bc40 tx=0x7f0d28012820 comp rx=0 tx=0).stop 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 -- 172.21.15.44:0/1762798950 shutdown_connections 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 --2- 172.21.15.44:0/1762798950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc04ac60 0x564bfc0627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 --2- 172.21.15.44:0/1762798950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc03ecd0 0x564bfc062270 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 --2- 172.21.15.44:0/1762798950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc042420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 -- 172.21.15.44:0/1762798950 >> 172.21.15.44:0/1762798950 conn(0x564bfc03c1e0 msgr2=0x564bfc048400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:38.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 -- 172.21.15.44:0/1762798950 shutdown_connections 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 -- 172.21.15.44:0/1762798950 wait complete. 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 -- start start 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 Processor -- start 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc052ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc04e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 0x564bfc04e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401b1a40 con 0x564bfc04ac60 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401b1310 con 0x564bfc03ecd0 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401ae480 con 0x564bfc03e3b0 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.227+0000 7f0d482b5740 10 client.0 _ll_register_callbacks cb 0x7f0d3804a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 0x564bfc04e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 0x564bfc04e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37960/0 (socket says 172.21.15.44:37960) 2024-08-20T03:11:38.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d3ffff640 1 -- 172.21.15.44:0/2956870771 learned_addr learned my addr 172.21.15.44:0/2956870771 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d36ffd640 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc04e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc052ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d3ffff640 1 -- 172.21.15.44:0/2956870771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 msgr2=0x564bfc052ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc052ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d3ffff640 1 -- 172.21.15.44:0/2956870771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 msgr2=0x564bfc04e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc04e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d3ffff640 1 -- 172.21.15.44:0/2956870771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bfc04eeb0 con 0x564bfc04ac60 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d36ffd640 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc04e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d3ffff640 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 0x564bfc04e8f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0d38031c60 tx=0x7f0d3802faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08c363ee84cff8f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d1effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d1effd640 1 -- 172.21.15.44:0/2956870771 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38069030 con 0x564bfc04ac60 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d1effd640 1 -- 172.21.15.44:0/2956870771 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f0d38057070 con 0x564bfc04ac60 2024-08-20T03:11:38.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d482b5740 1 -- 172.21.15.44:0/2956870771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d28005020 con 0x564bfc04ac60 2024-08-20T03:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d1effd640 1 -- 172.21.15.44:0/2956870771 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0d38073050 con 0x564bfc04ac60 2024-08-20T03:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d482b5740 10 client.17445 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.228+0000 7f0d482b5740 1 -- 172.21.15.44:0/2956870771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564bfc0c5070 con 0x564bfc04ac60 2024-08-20T03:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.229+0000 7f0d1effd640 1 -- 172.21.15.44:0/2956870771 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d28005020 con 0x564bfc04ac60 2024-08-20T03:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.229+0000 7f0d1effd640 1 -- 172.21.15.44:0/2956870771 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f0d380305c0 con 0x564bfc04ac60 2024-08-20T03:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.229+0000 7f0d482b5740 10 client.17445 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.229+0000 7f0d482b5740 10 client.17445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.229+0000 7f0d482b5740 20 client.17445 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.229+0000 7f0d482b5740 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x564bfc0c5870 0x564bfc0c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.229+0000 7f0d482b5740 4 client.17445 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T03:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.229+0000 7f0d482b5740 1 -- 172.21.15.44:0/2956870771 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564bfbfb2330 con 0x564bfc0c5870 2024-08-20T03:11:38.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.229+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x564bfc0c5870 0x564bfc0c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.229+0000 7f0d3f7fe640 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x564bfc0c5870 0x564bfc0c7c90 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea75386541c1bed3 server_cookie=1ac166903363f45e in_seq=0 out_seq=0 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.229+0000 7f0d1effd640 10 client.17445 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.230+0000 7f0d1effd640 1 -- 172.21.15.44:0/2956870771 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564bfbfb2330 con 0x564bfc0c5870 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.230+0000 7f0d1effd640 10 client.17445 handle_command_reply: tid=0 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.230+0000 7f0d482b5740 1 client.17445 shutdown 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.230+0000 7f0d482b5740 1 -- 172.21.15.44:0/2956870771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 msgr2=0x564bfc04e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.230+0000 7f0d482b5740 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 0x564bfc04e8f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f0d38031c60 tx=0x7f0d3802faf0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d482b5740 1 -- 172.21.15.44:0/2956870771 shutdown_connections 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d482b5740 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x564bfc0c5870 0x564bfc0c7c90 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d1effd640 0 client.17445 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d482b5740 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bfc04ac60 0x564bfc04e8f0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d482b5740 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564bfc03ecd0 0x564bfc04e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d482b5740 1 --2- 172.21.15.44:0/2956870771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564bfc03e3b0 0x564bfc052ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d482b5740 1 -- 172.21.15.44:0/2956870771 >> 172.21.15.44:0/2956870771 conn(0x564bfc03c1e0 msgr2=0x564bfc055940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d482b5740 1 -- 172.21.15.44:0/2956870771 shutdown_connections 2024-08-20T03:11:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d482b5740 1 -- 172.21.15.44:0/2956870771 wait complete. 2024-08-20T03:11:38.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d482b5740 20 client.17445 trim_cache size 0 max 16384 2024-08-20T03:11:38.244 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:11:38.244 INFO:teuthology.orchestra.run.smithi044.stdout: "return_code": 0, 2024-08-20T03:11:38.244 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_tag": "9a8531d2-69f6-4627-a210-c23114712d65", 2024-08-20T03:11:38.244 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": "asynchronous" 2024-08-20T03:11:38.244 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:11:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d45ebc640 1 -- 172.21.15.44:0/2939301127 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0d2c0b3d40 msgr2=0x7f0d2c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d45ebc640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0d2c0b3d40 0x7f0d2c0b6200 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0d3804b000 tx=0x7f0d38048000 comp rx=0 tx=0).stop 2024-08-20T03:11:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d45ebc640 1 -- 172.21.15.44:0/2939301127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d40102ab0 msgr2=0x7f0d400751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d45ebc640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d40102ab0 0x7f0d400751d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000c920 tx=0x7f0d3000cde0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d45ebc640 1 -- 172.21.15.44:0/2939301127 shutdown_connections 2024-08-20T03:11:38.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d45ebc640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f0d2c0b3d40 0x7f0d2c0b6200 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d45ebc640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d40108e40 0x7f0d40075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d45ebc640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d401084c0 0x7f0d40075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.231+0000 7f0d45ebc640 1 --2- 172.21.15.44:0/2939301127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d40102ab0 0x7f0d400751d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.232+0000 7f0d45ebc640 1 -- 172.21.15.44:0/2939301127 >> 172.21.15.44:0/2939301127 conn(0x7f0d400fe500 msgr2=0x7f0d400fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.232+0000 7f0d45ebc640 1 -- 172.21.15.44:0/2939301127 shutdown_connections 2024-08-20T03:11:38.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:38.232+0000 7f0d45ebc640 1 -- 172.21.15.44:0/2939301127 wait complete. 2024-08-20T03:11:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:37 smithi165 ceph-mon[22357]: pgmap v600: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:11:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:11:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:11:38.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:37 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:11:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:37 smithi099 ceph-mon[21984]: pgmap v600: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:11:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:11:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:11:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:37 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:11:39.092 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:11:39.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:38 smithi044 ceph-mon[18790]: scrub queued for path: / 2024-08-20T03:11:39.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:38 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:11:39.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:38 smithi044 ceph-mon[18790]: scrub summary: active paths [/] 2024-08-20T03:11:39.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:38 smithi044 ceph-mon[18790]: scrub summary: idle+waiting paths [/] 2024-08-20T03:11:39.346 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:11:39.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:38 smithi165 ceph-mon[22357]: scrub queued for path: / 2024-08-20T03:11:39.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:38 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:11:39.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:38 smithi165 ceph-mon[22357]: scrub summary: active paths [/] 2024-08-20T03:11:39.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:38 smithi165 ceph-mon[22357]: scrub summary: idle+waiting paths [/] 2024-08-20T03:11:39.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:38 smithi099 ceph-mon[21984]: scrub queued for path: / 2024-08-20T03:11:39.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:38 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:11:39.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:38 smithi099 ceph-mon[21984]: scrub summary: active paths [/] 2024-08-20T03:11:39.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:38 smithi099 ceph-mon[21984]: scrub summary: idle+waiting paths [/] 2024-08-20T03:11:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.856+0000 7f13773c8640 1 -- 172.21.15.44:0/2119130380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13701084c0 msgr2=0x7f1370108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.856+0000 7f13773c8640 1 --2- 172.21.15.44:0/2119130380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13701084c0 0x7f1370108900 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1360009900 tx=0x7f136002a410 comp rx=0 tx=0).stop 2024-08-20T03:11:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 -- 172.21.15.44:0/2119130380 shutdown_connections 2024-08-20T03:11:39.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 --2- 172.21.15.44:0/2119130380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1370108e40 0x7f137010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 --2- 172.21.15.44:0/2119130380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13701084c0 0x7f1370108900 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 --2- 172.21.15.44:0/2119130380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1370102ab0 0x7f1370102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 -- 172.21.15.44:0/2119130380 >> 172.21.15.44:0/2119130380 conn(0x7f13700fe500 msgr2=0x7f1370100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 -- 172.21.15.44:0/2119130380 shutdown_connections 2024-08-20T03:11:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 -- 172.21.15.44:0/2119130380 wait complete. 2024-08-20T03:11:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 Processor -- start 2024-08-20T03:11:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 -- start start 2024-08-20T03:11:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1370102ab0 0x7f1370198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13701084c0 0x7f1370199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1370108e40 0x7f137019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1370112b10 con 0x7f1370108e40 2024-08-20T03:11:39.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1370112990 con 0x7f1370102ab0 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.857+0000 7f13773c8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1370112c90 con 0x7f13701084c0 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f1375965640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1370108e40 0x7f137019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f1375965640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1370108e40 0x7f137019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37978/0 (socket says 172.21.15.44:37978) 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f1374963640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13701084c0 0x7f1370199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f1375965640 1 -- 172.21.15.44:0/4288803974 learned_addr learned my addr 172.21.15.44:0/4288803974 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f1375164640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1370102ab0 0x7f1370198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f1375965640 1 -- 172.21.15.44:0/4288803974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13701084c0 msgr2=0x7f1370199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f1375965640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13701084c0 0x7f1370199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f1375965640 1 -- 172.21.15.44:0/4288803974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1370102ab0 msgr2=0x7f1370198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f1375965640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1370102ab0 0x7f1370198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f1375965640 1 -- 172.21.15.44:0/4288803974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137019efd0 con 0x7f1370108e40 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f1375965640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1370108e40 0x7f137019e800 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f136800cad0 tx=0x7f136800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:39.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f13667fc640 1 -- 172.21.15.44:0/4288803974 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136800ad60 con 0x7f1370108e40 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f13667fc640 1 -- 172.21.15.44:0/4288803974 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1368005020 con 0x7f1370108e40 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f13773c8640 1 -- 172.21.15.44:0/4288803974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137019f2c0 con 0x7f1370108e40 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f13667fc640 1 -- 172.21.15.44:0/4288803974 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136800f6b0 con 0x7f1370108e40 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.858+0000 7f13773c8640 1 -- 172.21.15.44:0/4288803974 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13701a5980 con 0x7f1370108e40 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.861+0000 7f13667fc640 1 -- 172.21.15.44:0/4288803974 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f13680051d0 con 0x7f1370108e40 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.861+0000 7f13797c1740 1 Processor -- start 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.861+0000 7f13797c1740 1 -- start start 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.861+0000 7f13797c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77db830 0x55b6e77dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.861+0000 7f13797c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77dc150 0x55b6e77e7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.861+0000 7f13797c1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77e80e0 0x55b6e77ea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.861+0000 7f13797c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b6e77528e0 con 0x55b6e77db830 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.861+0000 7f13797c1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b6e7741de0 con 0x55b6e77e80e0 2024-08-20T03:11:39.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.861+0000 7f13797c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6e7745e20 con 0x55b6e77dc150 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.861+0000 7f1374963640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77db830 0x55b6e77dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.861+0000 7f1374963640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77db830 0x55b6e77dbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37980/0 (socket says 172.21.15.44:37980) 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.861+0000 7f1374963640 1 -- 172.21.15.44:0/3864800512 learned_addr learned my addr 172.21.15.44:0/3864800512 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f1375164640 1 --2- 172.21.15.44:0/3864800512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77dc150 0x55b6e77e7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f1374963640 1 --2- 172.21.15.44:0/3864800512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77e80e0 0x55b6e77ea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f1374963640 1 -- 172.21.15.44:0/3864800512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77dc150 msgr2=0x55b6e77e7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f1374963640 1 --2- 172.21.15.44:0/3864800512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77dc150 0x55b6e77e7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f1374963640 1 -- 172.21.15.44:0/3864800512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77e80e0 msgr2=0x55b6e77ea4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f1374963640 1 --2- 172.21.15.44:0/3864800512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77e80e0 0x55b6e77ea4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f1374963640 1 -- 172.21.15.44:0/3864800512 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6e74a1650 con 0x55b6e77db830 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f13667fc640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f135c0b3d40 0x7f135c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f1374963640 1 --2- 172.21.15.44:0/3864800512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77db830 0x55b6e77dbc10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1360033000 tx=0x7f1360000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c574254082a6212c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:39.880 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f1375164640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f135c0b3d40 0x7f135c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f13667fc640 1 -- 172.21.15.44:0/4288803974 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f13680e3ed0 con 0x7f1370108e40 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f135b7fe640 1 -- 172.21.15.44:0/3864800512 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360044030 con 0x55b6e77db830 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f135b7fe640 1 -- 172.21.15.44:0/3864800512 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13600054d0 con 0x55b6e77db830 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.862+0000 7f135b7fe640 1 -- 172.21.15.44:0/3864800512 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360035040 con 0x55b6e77db830 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.863+0000 7f13797c1740 1 -- 172.21.15.44:0/3864800512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77db830 msgr2=0x55b6e77dbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.863+0000 7f13797c1740 1 --2- 172.21.15.44:0/3864800512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77db830 0x55b6e77dbc10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1360033000 tx=0x7f1360000bc0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.863+0000 7f1375164640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f135c0b3d40 0x7f135c0b6200 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55b6e77de9b0 tx=0x7f136c00a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.863+0000 7f13797c1740 1 -- 172.21.15.44:0/3864800512 shutdown_connections 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.863+0000 7f13797c1740 1 --2- 172.21.15.44:0/3864800512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77e80e0 0x55b6e77ea4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.863+0000 7f13797c1740 1 --2- 172.21.15.44:0/3864800512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77dc150 0x55b6e77e7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.863+0000 7f13797c1740 1 --2- 172.21.15.44:0/3864800512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77db830 0x55b6e77dbc10 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.863+0000 7f13797c1740 1 -- 172.21.15.44:0/3864800512 >> 172.21.15.44:0/3864800512 conn(0x55b6e77d9660 msgr2=0x55b6e77da800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.863+0000 7f13797c1740 1 -- 172.21.15.44:0/3864800512 shutdown_connections 2024-08-20T03:11:39.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.863+0000 7f13797c1740 1 -- 172.21.15.44:0/3864800512 wait complete. 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.863+0000 7f13797c1740 1 -- start start 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.863+0000 7f13797c1740 1 Processor -- start 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f13797c1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77db830 0x55b6e77e5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f13797c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 0x55b6e77e55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f13797c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77e80e0 0x55b6e77e5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f13797c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55b6e75a4700 con 0x55b6e77dc150 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f13797c1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b6e76c87c0 con 0x55b6e77db830 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f13797c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55b6e77ead30 con 0x55b6e77e80e0 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f13797c1740 10 client.0 _ll_register_callbacks cb 0x55b6e773b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1374963640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 0x55b6e77e55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1374963640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 0x55b6e77e55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37996/0 (socket says 172.21.15.44:37996) 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1374963640 1 -- 172.21.15.44:0/4294117381 learned_addr learned my addr 172.21.15.44:0/4294117381 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:39.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1375164640 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77db830 0x55b6e77e5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1375965640 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77e80e0 0x55b6e77e5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1374963640 1 -- 172.21.15.44:0/4294117381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77e80e0 msgr2=0x55b6e77e5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1374963640 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77e80e0 0x55b6e77e5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1374963640 1 -- 172.21.15.44:0/4294117381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77db830 msgr2=0x55b6e77e5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1374963640 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77db830 0x55b6e77e5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1374963640 1 -- 172.21.15.44:0/4294117381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6e77f4140 con 0x55b6e77dc150 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1375965640 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77e80e0 0x55b6e77e5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1374963640 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 0x55b6e77e55a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1360009650 tx=0x7f13600044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5748611945e705 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.864+0000 7f1358ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f1358ff9640 1 -- 172.21.15.44:0/4294117381 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136004d060 con 0x55b6e77dc150 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f1358ff9640 1 -- 172.21.15.44:0/4294117381 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f1360005200 con 0x55b6e77dc150 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f13797c1740 1 -- 172.21.15.44:0/4294117381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6e76f1100 con 0x55b6e77dc150 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f1358ff9640 1 -- 172.21.15.44:0/4294117381 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1360035030 con 0x55b6e77dc150 2024-08-20T03:11:39.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f13797c1740 10 client.17457 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f13797c1740 1 -- 172.21.15.44:0/4294117381 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b6e77f6ad0 con 0x55b6e77dc150 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f1358ff9640 1 -- 172.21.15.44:0/4294117381 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b6e76f1100 con 0x55b6e77dc150 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f1358ff9640 1 -- 172.21.15.44:0/4294117381 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f13600314a0 con 0x55b6e77dc150 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f13797c1740 10 client.17457 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f13797c1740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f13797c1740 20 client.17457 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f13797c1740 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b6e77f7e20 0x55b6e77fa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f13797c1740 4 client.17457 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.865+0000 7f13797c1740 1 -- 172.21.15.44:0/4294117381 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6e7754830 con 0x55b6e77f7e20 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.866+0000 7f1375164640 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b6e77f7e20 0x55b6e77fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.866+0000 7f1375164640 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b6e77f7e20 0x55b6e77fa210 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24c4532a974f72cc server_cookie=64b7ce74abc8fabb in_seq=0 out_seq=0 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.866+0000 7f1358ff9640 10 client.17457 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f1358ff9640 1 -- 172.21.15.44:0/4294117381 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6e7754830 con 0x55b6e77f7e20 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f1358ff9640 10 client.17457 handle_command_reply: tid=0 2024-08-20T03:11:39.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f13797c1740 1 client.17457 shutdown 2024-08-20T03:11:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f13797c1740 1 -- 172.21.15.44:0/4294117381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 msgr2=0x55b6e77e55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f13797c1740 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 0x55b6e77e55a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1360009650 tx=0x7f13600044d0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f13797c1740 1 -- 172.21.15.44:0/4294117381 shutdown_connections 2024-08-20T03:11:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f13797c1740 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b6e77f7e20 0x55b6e77fa210 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f13797c1740 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77e80e0 0x55b6e77e5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f13797c1740 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 0x55b6e77e55a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f13797c1740 1 --2- 172.21.15.44:0/4294117381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77db830 0x55b6e77e5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f13797c1740 1 -- 172.21.15.44:0/4294117381 >> 172.21.15.44:0/4294117381 conn(0x55b6e77d9660 msgr2=0x55b6e77df200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f1358ff9640 0 client.17457 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f13797c1740 1 -- 172.21.15.44:0/4294117381 shutdown_connections 2024-08-20T03:11:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f13797c1740 1 -- 172.21.15.44:0/4294117381 wait complete. 2024-08-20T03:11:39.885 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.867+0000 7f13797c1740 20 client.17457 trim_cache size 0 max 16384 2024-08-20T03:11:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.885+0000 7f13797c1740 1 Processor -- start 2024-08-20T03:11:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.885+0000 7f13797c1740 1 -- start start 2024-08-20T03:11:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.885+0000 7f13797c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77db830 0x55b6e77f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.885+0000 7f13797c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 0x55b6e77e2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.885+0000 7f13797c1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77e80e0 0x55b6e77e27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.885+0000 7f13797c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f13701ae1b0 con 0x55b6e77dc150 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.885+0000 7f13797c1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f13701afe40 con 0x55b6e77e80e0 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.885+0000 7f13797c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f13701ae910 con 0x55b6e77db830 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.885+0000 7f1374963640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 0x55b6e77e2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.885+0000 7f1374963640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 0x55b6e77e2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37998/0 (socket says 172.21.15.44:37998) 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.885+0000 7f1374963640 1 -- 172.21.15.44:0/1985477482 learned_addr learned my addr 172.21.15.44:0/1985477482 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f1375164640 1 --2- 172.21.15.44:0/1985477482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77db830 0x55b6e77f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f1375965640 1 --2- 172.21.15.44:0/1985477482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77e80e0 0x55b6e77e27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f1374963640 1 -- 172.21.15.44:0/1985477482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77db830 msgr2=0x55b6e77f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f1374963640 1 --2- 172.21.15.44:0/1985477482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77db830 0x55b6e77f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f1374963640 1 -- 172.21.15.44:0/1985477482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77e80e0 msgr2=0x55b6e77e27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f1374963640 1 --2- 172.21.15.44:0/1985477482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77e80e0 0x55b6e77e27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f1374963640 1 -- 172.21.15.44:0/1985477482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6e77dbf80 con 0x55b6e77dc150 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f1375164640 1 --2- 172.21.15.44:0/1985477482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77db830 0x55b6e77f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f1375965640 1 --2- 172.21.15.44:0/1985477482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77e80e0 0x55b6e77e27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f1374963640 1 --2- 172.21.15.44:0/1985477482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 0x55b6e77e2270 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f13600092a0 tx=0x7f1360000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8290ea31e5bd5847 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f135affd640 1 -- 172.21.15.44:0/1985477482 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360044030 con 0x55b6e77dc150 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f135affd640 1 -- 172.21.15.44:0/1985477482 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1360031a90 con 0x55b6e77dc150 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.886+0000 7f135affd640 1 -- 172.21.15.44:0/1985477482 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136004d070 con 0x55b6e77dc150 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.887+0000 7f13797c1740 1 -- 172.21.15.44:0/1985477482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 msgr2=0x55b6e77e2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.887+0000 7f13797c1740 1 --2- 172.21.15.44:0/1985477482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 0x55b6e77e2270 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f13600092a0 tx=0x7f1360000bc0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.887+0000 7f13797c1740 1 -- 172.21.15.44:0/1985477482 shutdown_connections 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.887+0000 7f13797c1740 1 --2- 172.21.15.44:0/1985477482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77e80e0 0x55b6e77e27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.887+0000 7f13797c1740 1 --2- 172.21.15.44:0/1985477482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77dc150 0x55b6e77e2270 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.887+0000 7f13797c1740 1 --2- 172.21.15.44:0/1985477482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77db830 0x55b6e77f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.887+0000 7f13797c1740 1 -- 172.21.15.44:0/1985477482 >> 172.21.15.44:0/1985477482 conn(0x55b6e77d9660 msgr2=0x55b6e77f3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:39.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.887+0000 7f13797c1740 1 -- 172.21.15.44:0/1985477482 shutdown_connections 2024-08-20T03:11:39.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.887+0000 7f13797c1740 1 -- 172.21.15.44:0/1985477482 wait complete. 2024-08-20T03:11:39.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.887+0000 7f13797c1740 1 -- start start 2024-08-20T03:11:39.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.887+0000 7f13797c1740 1 Processor -- start 2024-08-20T03:11:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f13797c1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77db830 0x55b6e77fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f13797c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77dc150 0x55b6e77fc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f13797c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77e80e0 0x55b6e77fc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f13797c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f13701af770 con 0x55b6e77e80e0 2024-08-20T03:11:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f13797c1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f13701af040 con 0x55b6e77db830 2024-08-20T03:11:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f13797c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f13701ac1b0 con 0x55b6e77dc150 2024-08-20T03:11:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f13797c1740 10 client.0 _ll_register_callbacks cb 0x7f1360004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:11:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1375965640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77e80e0 0x55b6e77fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1375965640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77e80e0 0x55b6e77fc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38012/0 (socket says 172.21.15.44:38012) 2024-08-20T03:11:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1375965640 1 -- 172.21.15.44:0/2594924828 learned_addr learned my addr 172.21.15.44:0/2594924828 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:11:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1374963640 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77dc150 0x55b6e77fc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1375164640 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77db830 0x55b6e77fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1375965640 1 -- 172.21.15.44:0/2594924828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77dc150 msgr2=0x55b6e77fc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1375965640 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77dc150 0x55b6e77fc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1375965640 1 -- 172.21.15.44:0/2594924828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77db830 msgr2=0x55b6e77fbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:11:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1375965640 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77db830 0x55b6e77fbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1375965640 1 -- 172.21.15.44:0/2594924828 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6e7860860 con 0x55b6e77e80e0 2024-08-20T03:11:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1375164640 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77db830 0x55b6e77fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:11:39.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1375965640 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77e80e0 0x55b6e77fc810 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f136800caa0 tx=0x7f13680a62b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae3c31972b0350 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1358ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1358ff9640 1 -- 172.21.15.44:0/2594924828 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136801d030 con 0x55b6e77e80e0 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f1358ff9640 1 -- 172.21.15.44:0/2594924828 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8449+0+0 (secure 0 0 0) 0x7f13680c53f0 con 0x55b6e77e80e0 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.888+0000 7f13797c1740 1 -- 172.21.15.44:0/2594924828 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1360031a90 con 0x55b6e77e80e0 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.889+0000 7f1358ff9640 1 -- 172.21.15.44:0/2594924828 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f13680aab20 con 0x55b6e77e80e0 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.889+0000 7f13797c1740 10 client.17463 fetch_fsmap learned FSMap version 42 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.889+0000 7f13797c1740 1 -- 172.21.15.44:0/2594924828 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b6e7861b40 con 0x55b6e77e80e0 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.889+0000 7f1358ff9640 1 -- 172.21.15.44:0/2594924828 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1360031a90 con 0x55b6e77e80e0 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.889+0000 7f1358ff9640 1 -- 172.21.15.44:0/2594924828 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 42) ==== 8394+0+0 (secure 0 0 0) 0x7f13680a6490 con 0x55b6e77e80e0 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.889+0000 7f13797c1740 10 client.17463 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.889+0000 7f13797c1740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.889+0000 7f13797c1740 20 client.17463 populate_metadata read hostname 'smithi044' 2024-08-20T03:11:39.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.889+0000 7f13797c1740 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b6e78622c0 0x55b6e78646e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.889+0000 7f13797c1740 4 client.17463 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.889+0000 7f13797c1740 1 -- 172.21.15.44:0/2594924828 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6e77502d0 con 0x55b6e78622c0 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.889+0000 7f1375164640 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b6e78622c0 0x55b6e78646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f1375164640 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b6e78622c0 0x55b6e78646e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b22c3a9022c36193 server_cookie=21b158827d8f0315 in_seq=0 out_seq=0 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f1358ff9640 10 client.17463 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f1358ff9640 1 -- 172.21.15.44:0/2594924828 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b6e77502d0 con 0x55b6e78622c0 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f1358ff9640 10 client.17463 handle_command_reply: tid=0 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f13797c1740 1 client.17463 shutdown 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f13797c1740 1 -- 172.21.15.44:0/2594924828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77e80e0 msgr2=0x55b6e77fc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f13797c1740 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77e80e0 0x55b6e77fc810 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f136800caa0 tx=0x7f13680a62b0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f13797c1740 1 -- 172.21.15.44:0/2594924828 shutdown_connections 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f13797c1740 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55b6e78622c0 0x55b6e78646e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f13797c1740 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b6e77e80e0 0x55b6e77fc810 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f13797c1740 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b6e77dc150 0x55b6e77fc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f13797c1740 1 --2- 172.21.15.44:0/2594924828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b6e77db830 0x55b6e77fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f13797c1740 1 -- 172.21.15.44:0/2594924828 >> 172.21.15.44:0/2594924828 conn(0x55b6e77d9660 msgr2=0x55b6e77f3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f1358ff9640 0 client.17463 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:11:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f13797c1740 1 -- 172.21.15.44:0/2594924828 shutdown_connections 2024-08-20T03:11:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f13797c1740 1 -- 172.21.15.44:0/2594924828 wait complete. 2024-08-20T03:11:39.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.890+0000 7f13797c1740 20 client.17463 trim_cache size 0 max 16384 2024-08-20T03:11:39.899 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:11:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-20T03:11:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": { 2024-08-20T03:11:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: "9a8531d2-69f6-4627-a210-c23114712d65": { 2024-08-20T03:11:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: "path": "/", 2024-08-20T03:11:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: "tag": "9a8531d2-69f6-4627-a210-c23114712d65", 2024-08-20T03:11:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: "options": "recursive,force" 2024-08-20T03:11:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:11:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:11:39.900 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:11:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.891+0000 7f13773c8640 1 -- 172.21.15.44:0/4288803974 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f135c0b3d40 msgr2=0x7f135c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.891+0000 7f13773c8640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f135c0b3d40 0x7f135c0b6200 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55b6e77de9b0 tx=0x7f136c00a2d0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.891+0000 7f13773c8640 1 -- 172.21.15.44:0/4288803974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1370108e40 msgr2=0x7f137019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:11:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.891+0000 7f13773c8640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1370108e40 0x7f137019e800 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f136800cad0 tx=0x7f136800ced0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.891+0000 7f13773c8640 1 -- 172.21.15.44:0/4288803974 shutdown_connections 2024-08-20T03:11:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.891+0000 7f13773c8640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f135c0b3d40 0x7f135c0b6200 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.891+0000 7f13773c8640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1370108e40 0x7f137019e800 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.891+0000 7f13773c8640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13701084c0 0x7f1370199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.891+0000 7f13773c8640 1 --2- 172.21.15.44:0/4288803974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1370102ab0 0x7f1370198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:11:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.891+0000 7f13773c8640 1 -- 172.21.15.44:0/4288803974 >> 172.21.15.44:0/4288803974 conn(0x7f13700fe500 msgr2=0x7f13701008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:11:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.891+0000 7f13773c8640 1 -- 172.21.15.44:0/4288803974 shutdown_connections 2024-08-20T03:11:39.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:11:39.891+0000 7f13773c8640 1 -- 172.21.15.44:0/4288803974 wait complete. 2024-08-20T03:11:40.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:39 smithi044 ceph-mon[18790]: pgmap v601: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:40.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:39 smithi165 ceph-mon[22357]: pgmap v601: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:40.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:39 smithi099 ceph-mon[21984]: pgmap v601: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:40.545 INFO:tasks.cephfs.filesystem:scrub status for tag:9a8531d2-69f6-4627-a210-c23114712d65 - {'path': '/', 'tag': '9a8531d2-69f6-4627-a210-c23114712d65', 'options': 'recursive,force'} 2024-08-20T03:11:41.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:40 smithi044 ceph-mon[18790]: scrub summary: idle 2024-08-20T03:11:41.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:40 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:11:41.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:40 smithi165 ceph-mon[22357]: scrub summary: idle 2024-08-20T03:11:41.387 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:40 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:11:41.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:40 smithi099 ceph-mon[21984]: scrub summary: idle 2024-08-20T03:11:41.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:40 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:11:42.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:41 smithi044 ceph-mon[18790]: pgmap v602: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:42.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:41 smithi165 ceph-mon[22357]: pgmap v602: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:42.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:41 smithi099 ceph-mon[21984]: pgmap v602: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-08-20T03:11:44.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:43 smithi044 ceph-mon[18790]: pgmap v603: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 474 B/s wr, 10 op/s 2024-08-20T03:11:44.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:43 smithi165 ceph-mon[22357]: pgmap v603: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 474 B/s wr, 10 op/s 2024-08-20T03:11:44.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:43 smithi099 ceph-mon[21984]: pgmap v603: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 474 B/s wr, 10 op/s 2024-08-20T03:11:45.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:44 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:11:45.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:44 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:11:45.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:44 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:11:46.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:45 smithi044 ceph-mon[18790]: pgmap v604: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-08-20T03:11:46.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:45 smithi165 ceph-mon[22357]: pgmap v604: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-08-20T03:11:46.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:45 smithi099 ceph-mon[21984]: pgmap v604: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-08-20T03:11:48.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:47 smithi044 ceph-mon[18790]: pgmap v605: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 426 B/s wr, 11 op/s 2024-08-20T03:11:48.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:47 smithi165 ceph-mon[22357]: pgmap v605: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 426 B/s wr, 11 op/s 2024-08-20T03:11:48.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:47 smithi099 ceph-mon[21984]: pgmap v605: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 426 B/s wr, 11 op/s 2024-08-20T03:11:50.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:50 smithi044 ceph-mon[18790]: pgmap v606: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-08-20T03:11:50.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:50 smithi165 ceph-mon[22357]: pgmap v606: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-08-20T03:11:50.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:50 smithi099 ceph-mon[21984]: pgmap v606: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-08-20T03:11:52.287 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:52 smithi044 ceph-mon[18790]: pgmap v607: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-08-20T03:11:52.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:52 smithi165 ceph-mon[22357]: pgmap v607: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-08-20T03:11:52.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:52 smithi099 ceph-mon[21984]: pgmap v607: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-08-20T03:11:54.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:54 smithi044 ceph-mon[18790]: pgmap v608: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-08-20T03:11:54.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:54 smithi165 ceph-mon[22357]: pgmap v608: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-08-20T03:11:54.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:54 smithi099 ceph-mon[21984]: pgmap v608: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 11 op/s 2024-08-20T03:11:56.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:56 smithi044 ceph-mon[18790]: pgmap v609: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-08-20T03:11:56.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:56 smithi165 ceph-mon[22357]: pgmap v609: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-08-20T03:11:56.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:56 smithi099 ceph-mon[21984]: pgmap v609: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-08-20T03:11:58.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:11:58 smithi044 ceph-mon[18790]: pgmap v610: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:11:58.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:11:58 smithi165 ceph-mon[22357]: pgmap v610: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:11:58.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:11:58 smithi099 ceph-mon[21984]: pgmap v610: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-08-20T03:12:00.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:00 smithi044 ceph-mon[18790]: pgmap v611: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:00.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:00 smithi165 ceph-mon[22357]: pgmap v611: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:00.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:00 smithi099 ceph-mon[21984]: pgmap v611: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:02.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:02 smithi044 ceph-mon[18790]: pgmap v612: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:12:02.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:02 smithi165 ceph-mon[22357]: pgmap v612: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:12:02.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:02 smithi099 ceph-mon[21984]: pgmap v612: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:12:04.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:04 smithi044 ceph-mon[18790]: pgmap v613: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-08-20T03:12:04.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:04 smithi165 ceph-mon[22357]: pgmap v613: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-08-20T03:12:04.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:04 smithi099 ceph-mon[21984]: pgmap v613: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-08-20T03:12:06.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:06 smithi044 ceph-mon[18790]: pgmap v614: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-08-20T03:12:06.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:06 smithi165 ceph-mon[22357]: pgmap v614: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-08-20T03:12:06.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:06 smithi099 ceph-mon[21984]: pgmap v614: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-08-20T03:12:08.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:08 smithi044 ceph-mon[18790]: pgmap v615: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-08-20T03:12:08.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:08 smithi165 ceph-mon[22357]: pgmap v615: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-08-20T03:12:08.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:08 smithi099 ceph-mon[21984]: pgmap v615: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-08-20T03:12:10.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:10 smithi165 ceph-mon[22357]: pgmap v616: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-08-20T03:12:10.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:10 smithi099 ceph-mon[21984]: pgmap v616: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-08-20T03:12:10.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:10 smithi044 ceph-mon[18790]: pgmap v616: 97 pgs: 97 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-08-20T03:12:10.546 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T03:12:11.038 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:11.332 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:11 smithi044 ceph-mon[18790]: osdmap e84: 12 total, 12 up, 12 in 2024-08-20T03:12:11.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:11 smithi165 ceph-mon[22357]: osdmap e84: 12 total, 12 up, 12 in 2024-08-20T03:12:11.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:11 smithi099 ceph-mon[21984]: osdmap e84: 12 total, 12 up, 12 in 2024-08-20T03:12:11.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.637+0000 7fed07562640 1 -- 172.21.15.44:0/2915372788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed00107b90 msgr2=0x7fed0010e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:11.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.637+0000 7fed07562640 1 --2- 172.21.15.44:0/2915372788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed00107b90 0x7fed0010e390 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009ca0 tx=0x7fecf802a5c0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.637+0000 7fed07562640 1 -- 172.21.15.44:0/2915372788 shutdown_connections 2024-08-20T03:12:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.637+0000 7fed07562640 1 --2- 172.21.15.44:0/2915372788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed00107b90 0x7fed0010e390 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.637+0000 7fed07562640 1 --2- 172.21.15.44:0/2915372788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed00107210 0x7fed00107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.637+0000 7fed07562640 1 --2- 172.21.15.44:0/2915372788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed00101800 0x7fed00101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.637+0000 7fed07562640 1 -- 172.21.15.44:0/2915372788 >> 172.21.15.44:0/2915372788 conn(0x7fed000fd270 msgr2=0x7fed000ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:11.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.641+0000 7fed07562640 1 -- 172.21.15.44:0/2915372788 shutdown_connections 2024-08-20T03:12:11.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.645+0000 7fed07562640 1 -- 172.21.15.44:0/2915372788 wait complete. 2024-08-20T03:12:11.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.645+0000 7fed07562640 1 Processor -- start 2024-08-20T03:12:11.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.648+0000 7fed07562640 1 -- start start 2024-08-20T03:12:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.648+0000 7fed07562640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed00101800 0x7fed0006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.648+0000 7fed07562640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed00107210 0x7fed0006e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.648+0000 7fed07562640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed00107b90 0x7fed00073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.648+0000 7fed07562640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fed0006cd50 con 0x7fed00101800 2024-08-20T03:12:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.648+0000 7fed07562640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fed0006cbd0 con 0x7fed00107210 2024-08-20T03:12:11.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.648+0000 7fed07562640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fed0006ced0 con 0x7fed00107b90 2024-08-20T03:12:11.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.651+0000 7fed06560640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed00101800 0x7fed0006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.651+0000 7fed06560640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed00101800 0x7fed0006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41068/0 (socket says 172.21.15.44:41068) 2024-08-20T03:12:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.651+0000 7fed06560640 1 -- 172.21.15.44:0/3807383308 learned_addr learned my addr 172.21.15.44:0/3807383308 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.652+0000 7fed05d5f640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed00107210 0x7fed0006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.652+0000 7fed06560640 1 -- 172.21.15.44:0/3807383308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed00107b90 msgr2=0x7fed00073e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T03:12:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.652+0000 7fed06560640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed00107b90 0x7fed00073e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.652+0000 7fed06560640 1 -- 172.21.15.44:0/3807383308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed00107210 msgr2=0x7fed0006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.652+0000 7fed06560640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed00107210 0x7fed0006e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.652+0000 7fed06560640 1 -- 172.21.15.44:0/3807383308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed000743c0 con 0x7fed00101800 2024-08-20T03:12:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.652+0000 7fed05d5f640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed00107210 0x7fed0006e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:12:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.653+0000 7fed06560640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed00101800 0x7fed0006e440 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00daf0 tx=0x7fecfc00af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.653+0000 7fecf77fe640 1 -- 172.21.15.44:0/3807383308 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc01a070 con 0x7fed00101800 2024-08-20T03:12:11.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.653+0000 7fed07562640 1 -- 172.21.15.44:0/3807383308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed000746b0 con 0x7fed00101800 2024-08-20T03:12:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.653+0000 7fed07562640 1 -- 172.21.15.44:0/3807383308 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed00074c40 con 0x7fed00101800 2024-08-20T03:12:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.655+0000 7fecf77fe640 1 -- 172.21.15.44:0/3807383308 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecfc00cc70 con 0x7fed00101800 2024-08-20T03:12:11.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.655+0000 7fecf77fe640 1 -- 172.21.15.44:0/3807383308 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc01a070 con 0x7fed00101800 2024-08-20T03:12:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.658+0000 7fed0abbd740 1 Processor -- start 2024-08-20T03:12:11.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.658+0000 7fecf77fe640 1 -- 172.21.15.44:0/3807383308 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fecfc01d070 con 0x7fed00101800 2024-08-20T03:12:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.660+0000 7fecf77fe640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fecec0b40b0 0x7fecec0b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.660+0000 7fecf77fe640 1 -- 172.21.15.44:0/3807383308 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fecfc0e7060 con 0x7fed00101800 2024-08-20T03:12:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.662+0000 7fed0abbd740 1 -- start start 2024-08-20T03:12:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.663+0000 7fed0abbd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd748a10 0x55a8cd748df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.663+0000 7fed0abbd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 0x55a8cd754d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.663+0000 7fed0abbd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 0x55a8cd7576b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.663+0000 7fed0abbd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a8cd6bf8e0 con 0x55a8cd748a10 2024-08-20T03:12:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.663+0000 7fed0abbd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a8cd6aede0 con 0x55a8cd7552c0 2024-08-20T03:12:11.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.663+0000 7fed0abbd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a8cd6b2e20 con 0x55a8cd749330 2024-08-20T03:12:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.663+0000 7fed05d5f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 0x55a8cd754d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.663+0000 7fed05d5f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 0x55a8cd754d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45868/0 (socket says 172.21.15.44:45868) 2024-08-20T03:12:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.663+0000 7fed05d5f640 1 -- 172.21.15.44:0/648392672 learned_addr learned my addr 172.21.15.44:0/648392672 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.663+0000 7fed06560640 1 --2- 172.21.15.44:0/648392672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd748a10 0x55a8cd748df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.663+0000 7fed05d5f640 1 -- 172.21.15.44:0/648392672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 msgr2=0x55a8cd7576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.663+0000 7fed05d5f640 1 --2- 172.21.15.44:0/648392672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 0x55a8cd7576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.664+0000 7fed05d5f640 1 -- 172.21.15.44:0/648392672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd748a10 msgr2=0x55a8cd748df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.664+0000 7fed05d5f640 1 --2- 172.21.15.44:0/648392672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd748a10 0x55a8cd748df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.664+0000 7fed05d5f640 1 -- 172.21.15.44:0/648392672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8cd40e650 con 0x55a8cd749330 2024-08-20T03:12:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.664+0000 7fed05d5f640 1 --2- 172.21.15.44:0/648392672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 0x55a8cd754d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fecf000a900 tx=0x7fecf000adc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d66a87a07ed816 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:11.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.664+0000 7fed06d61640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fecec0b40b0 0x7fecec0b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.664+0000 7fecf4ff9640 1 -- 172.21.15.44:0/648392672 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0015070 con 0x55a8cd749330 2024-08-20T03:12:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.665+0000 7fecf4ff9640 1 -- 172.21.15.44:0/648392672 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecf000cc00 con 0x55a8cd749330 2024-08-20T03:12:11.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.665+0000 7fecf4ff9640 1 -- 172.21.15.44:0/648392672 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf00185c0 con 0x55a8cd749330 2024-08-20T03:12:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.665+0000 7fed0abbd740 1 -- 172.21.15.44:0/648392672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 msgr2=0x55a8cd754d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:11.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.665+0000 7fed0abbd740 1 --2- 172.21.15.44:0/648392672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 0x55a8cd754d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fecf000a900 tx=0x7fecf000adc0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.665+0000 7fed06d61640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fecec0b40b0 0x7fecec0b6570 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55a8cd7489c0 tx=0x7fecf8005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.666+0000 7fed0abbd740 1 -- 172.21.15.44:0/648392672 shutdown_connections 2024-08-20T03:12:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.666+0000 7fed0abbd740 1 --2- 172.21.15.44:0/648392672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 0x55a8cd7576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.666+0000 7fed0abbd740 1 --2- 172.21.15.44:0/648392672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 0x55a8cd754d80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.666+0000 7fed0abbd740 1 --2- 172.21.15.44:0/648392672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd748a10 0x55a8cd748df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.666+0000 7fed0abbd740 1 -- 172.21.15.44:0/648392672 >> 172.21.15.44:0/648392672 conn(0x55a8cd746840 msgr2=0x55a8cd7479e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.666+0000 7fed0abbd740 1 -- 172.21.15.44:0/648392672 shutdown_connections 2024-08-20T03:12:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.666+0000 7fed0abbd740 1 -- 172.21.15.44:0/648392672 wait complete. 2024-08-20T03:12:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.666+0000 7fed0abbd740 1 -- start start 2024-08-20T03:12:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.666+0000 7fed0abbd740 1 Processor -- start 2024-08-20T03:12:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.667+0000 7fed0abbd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd748a10 0x55a8cd754cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.667+0000 7fed0abbd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 0x55a8cd752100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.667+0000 7fed0abbd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 0x55a8cd752890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.667+0000 7fed0abbd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a8cd511700 con 0x55a8cd748a10 2024-08-20T03:12:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.667+0000 7fed0abbd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a8cd6357c0 con 0x55a8cd7552c0 2024-08-20T03:12:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.667+0000 7fed0abbd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a8cd757fc0 con 0x55a8cd749330 2024-08-20T03:12:11.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.668+0000 7fed0abbd740 10 client.0 _ll_register_callbacks cb 0x55a8cd6a86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:12:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.668+0000 7fed05d5f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 0x55a8cd752100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.668+0000 7fed05d5f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 0x55a8cd752100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45876/0 (socket says 172.21.15.44:45876) 2024-08-20T03:12:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.668+0000 7fed05d5f640 1 -- 172.21.15.44:0/1630573941 learned_addr learned my addr 172.21.15.44:0/1630573941 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.668+0000 7fed06560640 1 --2- 172.21.15.44:0/1630573941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd748a10 0x55a8cd754cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.669+0000 7fed06560640 1 -- 172.21.15.44:0/1630573941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 msgr2=0x55a8cd752100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.669+0000 7fed06560640 1 --2- 172.21.15.44:0/1630573941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 0x55a8cd752100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.669+0000 7fed06560640 1 -- 172.21.15.44:0/1630573941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 msgr2=0x55a8cd752890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:12:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.669+0000 7fed06560640 1 --2- 172.21.15.44:0/1630573941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 0x55a8cd752890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.669+0000 7fed06560640 1 -- 172.21.15.44:0/1630573941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8cd7613d0 con 0x55a8cd748a10 2024-08-20T03:12:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.670+0000 7fed06560640 1 --2- 172.21.15.44:0/1630573941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd748a10 0x55a8cd754cb0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0e0540 tx=0x7fecfc0aa870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5839f7732a8b45d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.670+0000 7fece9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:12:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.670+0000 7fece9ffb640 1 -- 172.21.15.44:0/1630573941 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0af220 con 0x55a8cd748a10 2024-08-20T03:12:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.670+0000 7fed0abbd740 1 -- 172.21.15.44:0/1630573941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8cd65e100 con 0x55a8cd748a10 2024-08-20T03:12:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.670+0000 7fece9ffb640 1 -- 172.21.15.44:0/1630573941 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fecfc0e7060 con 0x55a8cd748a10 2024-08-20T03:12:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.671+0000 7fece9ffb640 1 -- 172.21.15.44:0/1630573941 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fecfc0b3cd0 con 0x55a8cd748a10 2024-08-20T03:12:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.671+0000 7fed0abbd740 10 client.17475 fetch_fsmap learned FSMap version 44 2024-08-20T03:12:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.671+0000 7fed0abbd740 1 -- 172.21.15.44:0/1630573941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a8cd763ca0 con 0x55a8cd748a10 2024-08-20T03:12:11.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.672+0000 7fece9ffb640 1 -- 172.21.15.44:0/1630573941 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a8cd65e100 con 0x55a8cd748a10 2024-08-20T03:12:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.673+0000 7fece9ffb640 1 -- 172.21.15.44:0/1630573941 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7fecfc0a66a0 con 0x55a8cd748a10 2024-08-20T03:12:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.673+0000 7fed0abbd740 10 client.17475 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T03:12:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.673+0000 7fed0abbd740 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:12:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.673+0000 7fed0abbd740 20 client.17475 populate_metadata read hostname 'smithi044' 2024-08-20T03:12:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.673+0000 7fed0abbd740 1 --2- 172.21.15.44:0/1630573941 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a8cd765640 0x55a8cd767aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.673+0000 7fed0abbd740 4 client.17475 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:12:11.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.673+0000 7fed0abbd740 1 -- 172.21.15.44:0/1630573941 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8cd6c1830 con 0x55a8cd765640 2024-08-20T03:12:11.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.673+0000 7fed05d5f640 1 --2- 172.21.15.44:0/1630573941 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a8cd765640 0x55a8cd767aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.675+0000 7fed05d5f640 1 --2- 172.21.15.44:0/1630573941 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a8cd765640 0x55a8cd767aa0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e580a2a12a879a1 server_cookie=a6423ce0f44f73c5 in_seq=0 out_seq=0 2024-08-20T03:12:11.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.675+0000 7fece9ffb640 10 client.17475 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:12:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.676+0000 7fece9ffb640 1 -- 172.21.15.44:0/1630573941 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8cd6c1830 con 0x55a8cd765640 2024-08-20T03:12:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.676+0000 7fece9ffb640 10 client.17475 handle_command_reply: tid=0 2024-08-20T03:12:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.676+0000 7fed0abbd740 1 client.17475 shutdown 2024-08-20T03:12:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.677+0000 7fed0abbd740 1 -- 172.21.15.44:0/1630573941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd748a10 msgr2=0x55a8cd754cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.677+0000 7fed0abbd740 1 --2- 172.21.15.44:0/1630573941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd748a10 0x55a8cd754cb0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0e0540 tx=0x7fecfc0aa870 comp rx=0 tx=0).stop 2024-08-20T03:12:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.677+0000 7fed0abbd740 1 -- 172.21.15.44:0/1630573941 shutdown_connections 2024-08-20T03:12:11.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.677+0000 7fed0abbd740 1 --2- 172.21.15.44:0/1630573941 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a8cd765640 0x55a8cd767aa0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.677+0000 7fed0abbd740 1 --2- 172.21.15.44:0/1630573941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 0x55a8cd752890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.677+0000 7fed0abbd740 1 --2- 172.21.15.44:0/1630573941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd749330 0x55a8cd752100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.677+0000 7fed0abbd740 1 --2- 172.21.15.44:0/1630573941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd748a10 0x55a8cd754cb0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.677+0000 7fed0abbd740 1 -- 172.21.15.44:0/1630573941 >> 172.21.15.44:0/1630573941 conn(0x55a8cd746840 msgr2=0x55a8cd74c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:11.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.677+0000 7fece9ffb640 0 client.17475 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:12:11.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.680+0000 7fed0abbd740 1 -- 172.21.15.44:0/1630573941 shutdown_connections 2024-08-20T03:12:11.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.680+0000 7fed0abbd740 1 -- 172.21.15.44:0/1630573941 wait complete. 2024-08-20T03:12:11.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.680+0000 7fed0abbd740 20 client.17475 trim_cache size 0 max 16384 2024-08-20T03:12:11.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.714+0000 7fed0abbd740 1 Processor -- start 2024-08-20T03:12:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.714+0000 7fed0abbd740 1 -- start start 2024-08-20T03:12:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.714+0000 7fed0abbd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd748a10 0x55a8cd74ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.714+0000 7fed0abbd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd749330 0x55a8cd7515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.714+0000 7fed0abbd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 0x55a8cd76d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.714+0000 7fed0abbd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fece400e7e0 con 0x55a8cd749330 2024-08-20T03:12:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.714+0000 7fed0abbd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fece4010410 con 0x55a8cd7552c0 2024-08-20T03:12:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.714+0000 7fed0abbd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fece400ef40 con 0x55a8cd748a10 2024-08-20T03:12:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.714+0000 7fed05d5f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd749330 0x55a8cd7515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.714+0000 7fed05d5f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd749330 0x55a8cd7515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41106/0 (socket says 172.21.15.44:41106) 2024-08-20T03:12:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.714+0000 7fed05d5f640 1 -- 172.21.15.44:0/492334061 learned_addr learned my addr 172.21.15.44:0/492334061 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.715+0000 7fed06d61640 1 --2- 172.21.15.44:0/492334061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 0x55a8cd76d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.715+0000 7fed06560640 1 --2- 172.21.15.44:0/492334061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd748a10 0x55a8cd74ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.715+0000 7fed05d5f640 1 -- 172.21.15.44:0/492334061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd748a10 msgr2=0x55a8cd74ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:11.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.715+0000 7fed05d5f640 1 --2- 172.21.15.44:0/492334061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd748a10 0x55a8cd74ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.715+0000 7fed05d5f640 1 -- 172.21.15.44:0/492334061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 msgr2=0x55a8cd76d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:11.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.715+0000 7fed05d5f640 1 --2- 172.21.15.44:0/492334061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 0x55a8cd76d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.715+0000 7fed05d5f640 1 -- 172.21.15.44:0/492334061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8cd752c00 con 0x55a8cd749330 2024-08-20T03:12:11.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.716+0000 7fed05d5f640 1 --2- 172.21.15.44:0/492334061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd749330 0x55a8cd7515f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fecf000a120 tx=0x7fecf0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322a4b74a4d723ee server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:11.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.717+0000 7fecebfff640 1 -- 172.21.15.44:0/492334061 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0015040 con 0x55a8cd749330 2024-08-20T03:12:11.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.717+0000 7fecebfff640 1 -- 172.21.15.44:0/492334061 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecf000a4e0 con 0x55a8cd749330 2024-08-20T03:12:11.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.717+0000 7fecebfff640 1 -- 172.21.15.44:0/492334061 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf00116e0 con 0x55a8cd749330 2024-08-20T03:12:11.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.718+0000 7fed0abbd740 1 -- 172.21.15.44:0/492334061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd749330 msgr2=0x55a8cd7515f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:11.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.718+0000 7fed0abbd740 1 --2- 172.21.15.44:0/492334061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd749330 0x55a8cd7515f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fecf000a120 tx=0x7fecf0008040 comp rx=0 tx=0).stop 2024-08-20T03:12:11.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.721+0000 7fed0abbd740 1 -- 172.21.15.44:0/492334061 shutdown_connections 2024-08-20T03:12:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.721+0000 7fed0abbd740 1 --2- 172.21.15.44:0/492334061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd7552c0 0x55a8cd76d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.721+0000 7fed0abbd740 1 --2- 172.21.15.44:0/492334061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd749330 0x55a8cd7515f0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.721+0000 7fed0abbd740 1 --2- 172.21.15.44:0/492334061 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd748a10 0x55a8cd74ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.721+0000 7fed0abbd740 1 -- 172.21.15.44:0/492334061 >> 172.21.15.44:0/492334061 conn(0x55a8cd746840 msgr2=0x55a8cd750980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:11.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.721+0000 7fed0abbd740 1 -- 172.21.15.44:0/492334061 shutdown_connections 2024-08-20T03:12:11.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.724+0000 7fed0abbd740 1 -- 172.21.15.44:0/492334061 wait complete. 2024-08-20T03:12:11.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.724+0000 7fed0abbd740 1 -- start start 2024-08-20T03:12:11.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.724+0000 7fed0abbd740 1 Processor -- start 2024-08-20T03:12:11.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.725+0000 7fed0abbd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd748a10 0x55a8cd7582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.725+0000 7fed0abbd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd749330 0x55a8cd758830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.725+0000 7fed0abbd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd7552c0 0x55a8cd758d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.725+0000 7fed0abbd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fece400fd40 con 0x55a8cd749330 2024-08-20T03:12:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.725+0000 7fed0abbd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fece400f640 con 0x55a8cd748a10 2024-08-20T03:12:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.725+0000 7fed0abbd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fece400c7e0 con 0x55a8cd7552c0 2024-08-20T03:12:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.725+0000 7fed0abbd740 10 client.0 _ll_register_callbacks cb 0x7fecfc0a63b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:12:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.725+0000 7fed06560640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd748a10 0x55a8cd7582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.725+0000 7fed06560640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd748a10 0x55a8cd7582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:39820/0 (socket says 172.21.15.44:39820) 2024-08-20T03:12:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.725+0000 7fed06560640 1 -- 172.21.15.44:0/3269926490 learned_addr learned my addr 172.21.15.44:0/3269926490 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:11.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.725+0000 7fed06d61640 1 --2- 172.21.15.44:0/3269926490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd7552c0 0x55a8cd758d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.726+0000 7fed06560640 1 -- 172.21.15.44:0/3269926490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd7552c0 msgr2=0x55a8cd758d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.726+0000 7fed06560640 1 --2- 172.21.15.44:0/3269926490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd7552c0 0x55a8cd758d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.726+0000 7fed06560640 1 -- 172.21.15.44:0/3269926490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd749330 msgr2=0x55a8cd758830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.726+0000 7fed06560640 1 --2- 172.21.15.44:0/3269926490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd749330 0x55a8cd758830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.726+0000 7fed06560640 1 -- 172.21.15.44:0/3269926490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8cd755020 con 0x55a8cd748a10 2024-08-20T03:12:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.726+0000 7fed06560640 1 --2- 172.21.15.44:0/3269926490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd748a10 0x55a8cd7582f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0b3e90 tx=0x7fecfc092300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9a14b92724f3f92 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.726+0000 7fece9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-08-20T03:12:11.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.726+0000 7fece9ffb640 1 -- 172.21.15.44:0/3269926490 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc084e60 con 0x55a8cd748a10 2024-08-20T03:12:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.727+0000 7fed0abbd740 1 -- 172.21.15.44:0/3269926490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecf000a4e0 con 0x55a8cd748a10 2024-08-20T03:12:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.727+0000 7fece9ffb640 1 -- 172.21.15.44:0/3269926490 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fecfc0a7210 con 0x55a8cd748a10 2024-08-20T03:12:11.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.728+0000 7fece9ffb640 1 -- 172.21.15.44:0/3269926490 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fecfc0af220 con 0x55a8cd748a10 2024-08-20T03:12:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.729+0000 7fed0abbd740 10 client.26855 fetch_fsmap learned FSMap version 44 2024-08-20T03:12:11.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.729+0000 7fed0abbd740 1 -- 172.21.15.44:0/3269926490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a8cd7cf660 con 0x55a8cd748a10 2024-08-20T03:12:11.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.732+0000 7fece9ffb640 1 -- 172.21.15.44:0/3269926490 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecf000a4e0 con 0x55a8cd748a10 2024-08-20T03:12:11.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.732+0000 7fece9ffb640 1 -- 172.21.15.44:0/3269926490 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7fecfc004030 con 0x55a8cd748a10 2024-08-20T03:12:11.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.734+0000 7fed0abbd740 10 client.26855 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T03:12:11.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.734+0000 7fed0abbd740 10 client.26855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:12:11.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.734+0000 7fed0abbd740 20 client.26855 populate_metadata read hostname 'smithi044' 2024-08-20T03:12:11.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.734+0000 7fed0abbd740 1 --2- 172.21.15.44:0/3269926490 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a8cd7cfc00 0x55a8cd7d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:11.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.734+0000 7fed0abbd740 4 client.26855 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T03:12:11.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.734+0000 7fed0abbd740 1 -- 172.21.15.44:0/3269926490 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a8cd6bd2d0 con 0x55a8cd7cfc00 2024-08-20T03:12:11.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.737+0000 7fed05d5f640 1 --2- 172.21.15.44:0/3269926490 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a8cd7cfc00 0x55a8cd7d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.742+0000 7fed05d5f640 1 --2- 172.21.15.44:0/3269926490 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a8cd7cfc00 0x55a8cd7d2020 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17731d66af528e7b server_cookie=5462bafcea3ae14a in_seq=0 out_seq=0 2024-08-20T03:12:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.742+0000 7fece9ffb640 10 client.26855 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:12:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.743+0000 7fece9ffb640 1 -- 172.21.15.44:0/3269926490 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a8cd6bd2d0 con 0x55a8cd7cfc00 2024-08-20T03:12:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.743+0000 7fece9ffb640 10 client.26855 handle_command_reply: tid=0 2024-08-20T03:12:11.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.743+0000 7fed0abbd740 1 client.26855 shutdown 2024-08-20T03:12:11.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.744+0000 7fed0abbd740 1 -- 172.21.15.44:0/3269926490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd748a10 msgr2=0x55a8cd7582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:11.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.744+0000 7fed0abbd740 1 --2- 172.21.15.44:0/3269926490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd748a10 0x55a8cd7582f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0b3e90 tx=0x7fecfc092300 comp rx=0 tx=0).stop 2024-08-20T03:12:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.744+0000 7fed0abbd740 1 -- 172.21.15.44:0/3269926490 shutdown_connections 2024-08-20T03:12:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.745+0000 7fed0abbd740 1 --2- 172.21.15.44:0/3269926490 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55a8cd7cfc00 0x55a8cd7d2020 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.745+0000 7fed0abbd740 1 --2- 172.21.15.44:0/3269926490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8cd7552c0 0x55a8cd758d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.745+0000 7fed0abbd740 1 --2- 172.21.15.44:0/3269926490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a8cd749330 0x55a8cd758830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.745+0000 7fed0abbd740 1 --2- 172.21.15.44:0/3269926490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8cd748a10 0x55a8cd7582f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.745+0000 7fed0abbd740 1 -- 172.21.15.44:0/3269926490 >> 172.21.15.44:0/3269926490 conn(0x55a8cd746840 msgr2=0x55a8cd745900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:11.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.745+0000 7fece9ffb640 0 client.26855 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:12:11.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.745+0000 7fed0abbd740 1 -- 172.21.15.44:0/3269926490 shutdown_connections 2024-08-20T03:12:11.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.745+0000 7fed0abbd740 1 -- 172.21.15.44:0/3269926490 wait complete. 2024-08-20T03:12:11.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.745+0000 7fed0abbd740 20 client.26855 trim_cache size 0 max 16384 2024-08-20T03:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: "status": "no active scrubs running", 2024-08-20T03:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: "scrubs": {} 2024-08-20T03:12:11.748 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:12:11.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.747+0000 7fed07562640 1 -- 172.21.15.44:0/3807383308 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fecec0b40b0 msgr2=0x7fecec0b6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:11.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.747+0000 7fed07562640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fecec0b40b0 0x7fecec0b6570 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55a8cd7489c0 tx=0x7fecf8005d80 comp rx=0 tx=0).stop 2024-08-20T03:12:11.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.747+0000 7fed07562640 1 -- 172.21.15.44:0/3807383308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed00101800 msgr2=0x7fed0006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:11.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.747+0000 7fed07562640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed00101800 0x7fed0006e440 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00daf0 tx=0x7fecfc00af60 comp rx=0 tx=0).stop 2024-08-20T03:12:11.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.747+0000 7fed07562640 1 -- 172.21.15.44:0/3807383308 shutdown_connections 2024-08-20T03:12:11.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.747+0000 7fed07562640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fecec0b40b0 0x7fecec0b6570 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.747+0000 7fed07562640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed00107b90 0x7fed00073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.747+0000 7fed07562640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed00107210 0x7fed0006e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.747+0000 7fed07562640 1 --2- 172.21.15.44:0/3807383308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fed00101800 0x7fed0006e440 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:11.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.747+0000 7fed07562640 1 -- 172.21.15.44:0/3807383308 >> 172.21.15.44:0/3807383308 conn(0x7fed000fd270 msgr2=0x7fed000ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:11.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.747+0000 7fed07562640 1 -- 172.21.15.44:0/3807383308 shutdown_connections 2024-08-20T03:12:11.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:11.747+0000 7fed07562640 1 -- 172.21.15.44:0/3807383308 wait complete. 2024-08-20T03:12:12.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:12 smithi165 ceph-mon[22357]: pgmap v618: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-08-20T03:12:12.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:12 smithi099 ceph-mon[21984]: pgmap v618: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-08-20T03:12:12.482 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:12 smithi044 ceph-mon[18790]: pgmap v618: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 815 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-08-20T03:12:12.614 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T03:12:12.614 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:12:13.100 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.720+0000 7f2fe5fa3640 1 -- 172.21.15.44:0/2811955069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe0109e80 msgr2=0x7f2fe0110ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:13.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.720+0000 7f2fe5fa3640 1 --2- 172.21.15.44:0/2811955069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe0109e80 0x7f2fe0110ee0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009900 tx=0x7f2fc802a440 comp rx=0 tx=0).stop 2024-08-20T03:12:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.724+0000 7f2fe5fa3640 1 -- 172.21.15.44:0/2811955069 shutdown_connections 2024-08-20T03:12:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.724+0000 7f2fe5fa3640 1 --2- 172.21.15.44:0/2811955069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe011bdd0 0x7f2fe011e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.724+0000 7f2fe5fa3640 1 --2- 172.21.15.44:0/2811955069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fe010a260 0x7f2fe010a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.724+0000 7f2fe5fa3640 1 --2- 172.21.15.44:0/2811955069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe0109e80 0x7f2fe0110ee0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.724+0000 7f2fe5fa3640 1 -- 172.21.15.44:0/2811955069 >> 172.21.15.44:0/2811955069 conn(0x7f2fe006c840 msgr2=0x7f2fe006cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.726+0000 7f2fe5fa3640 1 -- 172.21.15.44:0/2811955069 shutdown_connections 2024-08-20T03:12:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.729+0000 7f2fe5fa3640 1 -- 172.21.15.44:0/2811955069 wait complete. 2024-08-20T03:12:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.730+0000 7f2fe5fa3640 1 Processor -- start 2024-08-20T03:12:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.732+0000 7f2fe5fa3640 1 -- start start 2024-08-20T03:12:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.733+0000 7f2fe5fa3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe0109e80 0x7f2fe01a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:13.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.733+0000 7f2fe5fa3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fe010a260 0x7f2fe01a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.733+0000 7f2fe5fa3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe011bdd0 0x7f2fe01ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.733+0000 7f2fe5fa3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe0121740 con 0x7f2fe011bdd0 2024-08-20T03:12:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.733+0000 7f2fe5fa3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe01215c0 con 0x7f2fe0109e80 2024-08-20T03:12:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.733+0000 7f2fe5fa3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe01218c0 con 0x7f2fe010a260 2024-08-20T03:12:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.733+0000 7f2fdf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe0109e80 0x7f2fe01a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.734+0000 7f2fd6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fe010a260 0x7f2fe01a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.734+0000 7f2fd6ffd640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fe010a260 0x7f2fe01a7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45918/0 (socket says 172.21.15.44:45918) 2024-08-20T03:12:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.734+0000 7f2fd6ffd640 1 -- 172.21.15.44:0/711305390 learned_addr learned my addr 172.21.15.44:0/711305390 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.734+0000 7f2fd6ffd640 1 -- 172.21.15.44:0/711305390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe0109e80 msgr2=0x7f2fe01a79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.734+0000 7f2fd6ffd640 1 --2- 172.21.15.44:0/711305390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe0109e80 0x7f2fe01a79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.734+0000 7f2fd6ffd640 1 -- 172.21.15.44:0/711305390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe011bdd0 msgr2=0x7f2fe01ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:13.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.734+0000 7f2fd6ffd640 1 --2- 172.21.15.44:0/711305390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe011bdd0 0x7f2fe01ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.734+0000 7f2fd6ffd640 1 -- 172.21.15.44:0/711305390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe01adaa0 con 0x7f2fe010a260 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.735+0000 7f2fd6ffd640 1 --2- 172.21.15.44:0/711305390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fe010a260 0x7f2fe01a7f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000b9e0 tx=0x7f2fd000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.736+0000 7f2fdd7fa640 1 -- 172.21.15.44:0/711305390 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0019070 con 0x7f2fe010a260 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.736+0000 7f2fdd7fa640 1 -- 172.21.15.44:0/711305390 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fd0014440 con 0x7f2fe010a260 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.736+0000 7f2fdd7fa640 1 -- 172.21.15.44:0/711305390 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00075d0 con 0x7f2fe010a260 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.736+0000 7f2fe5fa3640 1 -- 172.21.15.44:0/711305390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe01add90 con 0x7f2fe010a260 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.736+0000 7f2fe5fa3640 1 -- 172.21.15.44:0/711305390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe01b4450 con 0x7f2fe010a260 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.739+0000 7f2fe5fa3640 1 -- 172.21.15.44:0/711305390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc00052c0 con 0x7f2fe010a260 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.742+0000 7f2fdd7fa640 1 -- 172.21.15.44:0/711305390 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2fd001c070 con 0x7f2fe010a260 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.743+0000 7f2fdd7fa640 1 --2- 172.21.15.44:0/711305390 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2fcc0b3fc0 0x7f2fcc0b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.743+0000 7f2fdd7fa640 1 -- 172.21.15.44:0/711305390 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f2fd00e3410 con 0x7f2fe010a260 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.749+0000 7f2fdd7fa640 1 -- 172.21.15.44:0/711305390 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2fd00aa7d0 con 0x7f2fe010a260 2024-08-20T03:12:13.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.753+0000 7f2fdf7fe640 1 --2- 172.21.15.44:0/711305390 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2fcc0b3fc0 0x7f2fcc0b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:13.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:13.753+0000 7f2fdf7fe640 1 --2- 172.21.15.44:0/711305390 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2fcc0b3fc0 0x7f2fcc0b6480 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8005ab0 tx=0x7f2fc8000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:14.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.187+0000 7f2fe5fa3640 1 -- 172.21.15.44:0/711305390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fc00055b0 con 0x7f2fe010a260 2024-08-20T03:12:14.192 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.190+0000 7f2fdd7fa640 1 -- 172.21.15.44:0/711305390 <== mon.1 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11973 (secure 0 0 0) 0x7f2fd00aa0d0 con 0x7f2fe010a260 2024-08-20T03:12:14.193 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:12:14.193 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":44,"btime":"2024-08-20T03:11:43:976127+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:11:43.142495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24569,24583,24596,24589,24604]},"id":1}]} 2024-08-20T03:12:14.193 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 44 2024-08-20T03:12:14.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.197+0000 7f2fd67fc640 1 -- 172.21.15.44:0/711305390 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2fcc0b3fc0 msgr2=0x7f2fcc0b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:14.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.197+0000 7f2fd67fc640 1 --2- 172.21.15.44:0/711305390 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2fcc0b3fc0 0x7f2fcc0b6480 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8005ab0 tx=0x7f2fc8000bc0 comp rx=0 tx=0).stop 2024-08-20T03:12:14.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.197+0000 7f2fd67fc640 1 -- 172.21.15.44:0/711305390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fe010a260 msgr2=0x7f2fe01a7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:14.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.197+0000 7f2fd67fc640 1 --2- 172.21.15.44:0/711305390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fe010a260 0x7f2fe01a7f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000b9e0 tx=0x7f2fd000bea0 comp rx=0 tx=0).stop 2024-08-20T03:12:14.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.197+0000 7f2fd67fc640 1 -- 172.21.15.44:0/711305390 shutdown_connections 2024-08-20T03:12:14.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.197+0000 7f2fd67fc640 1 --2- 172.21.15.44:0/711305390 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2fcc0b3fc0 0x7f2fcc0b6480 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:14.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.197+0000 7f2fd67fc640 1 --2- 172.21.15.44:0/711305390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fe011bdd0 0x7f2fe01ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:14.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.198+0000 7f2fd67fc640 1 --2- 172.21.15.44:0/711305390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fe010a260 0x7f2fe01a7f30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:14.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.198+0000 7f2fd67fc640 1 --2- 172.21.15.44:0/711305390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fe0109e80 0x7f2fe01a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:14.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.198+0000 7f2fd67fc640 1 -- 172.21.15.44:0/711305390 >> 172.21.15.44:0/711305390 conn(0x7f2fe006c840 msgr2=0x7f2fe011c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:14.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.200+0000 7f2fd67fc640 1 -- 172.21.15.44:0/711305390 shutdown_connections 2024-08-20T03:12:14.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:14.200+0000 7f2fd67fc640 1 -- 172.21.15.44:0/711305390 wait complete. 2024-08-20T03:12:14.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:14 smithi044 ceph-mon[18790]: pgmap v619: 97 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 81 active+clean; 814 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-08-20T03:12:14.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:14 smithi165 ceph-mon[22357]: pgmap v619: 97 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 81 active+clean; 814 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-08-20T03:12:14.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:14 smithi099 ceph-mon[21984]: pgmap v619: 97 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 81 active+clean; 814 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-08-20T03:12:14.921 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-08-20T03:12:15.286 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:15 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/711305390' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:12:15.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:15 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/711305390' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:12:15.402 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:15.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:15 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/711305390' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:12:16.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.206+0000 7f07633e3640 1 -- 172.21.15.44:0/2354311989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c101f10 msgr2=0x7f075c1063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.206+0000 7f07633e3640 1 --2- 172.21.15.44:0/2354311989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c101f10 0x7f075c1063e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f074c01a910 tx=0x7f074c03b430 comp rx=0 tx=0).stop 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.206+0000 7f07633e3640 1 -- 172.21.15.44:0/2354311989 shutdown_connections 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.206+0000 7f07633e3640 1 --2- 172.21.15.44:0/2354311989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075c106a30 0x7f075c106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.206+0000 7f07633e3640 1 --2- 172.21.15.44:0/2354311989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c101f10 0x7f075c1063e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.206+0000 7f07633e3640 1 --2- 172.21.15.44:0/2354311989 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f075c1015f0 0x7f075c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.206+0000 7f07633e3640 1 -- 172.21.15.44:0/2354311989 >> 172.21.15.44:0/2354311989 conn(0x7f075c0fd2a0 msgr2=0x7f075c0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.209+0000 7f07633e3640 1 -- 172.21.15.44:0/2354311989 shutdown_connections 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.210+0000 7f07633e3640 1 -- 172.21.15.44:0/2354311989 wait complete. 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.211+0000 7f07633e3640 1 Processor -- start 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.216+0000 7f07633e3640 1 -- start start 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.216+0000 7f07633e3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075c1015f0 0x7f075c111ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.216+0000 7f07633e3640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f075c101f10 0x7f075c112020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.216+0000 7f07633e3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c106a30 0x7f075c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.216+0000 7f07633e3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f075c10a260 con 0x7f075c106a30 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.216+0000 7f07633e3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f075c10a0e0 con 0x7f075c1015f0 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.216+0000 7f07633e3640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f075c10a3e0 con 0x7f075c101f10 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.217+0000 7f0762be2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c106a30 0x7f075c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.217+0000 7f0762be2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c106a30 0x7f075c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41144/0 (socket says 172.21.15.44:41144) 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.217+0000 7f0762be2640 1 -- 172.21.15.44:0/992873164 learned_addr learned my addr 172.21.15.44:0/992873164 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.218+0000 7f0761be0640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f075c101f10 0x7f075c112020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.218+0000 7f07623e1640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075c1015f0 0x7f075c111ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.218+0000 7f0762be2640 1 -- 172.21.15.44:0/992873164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f075c101f10 msgr2=0x7f075c112020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.218+0000 7f0762be2640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f075c101f10 0x7f075c112020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.218+0000 7f0762be2640 1 -- 172.21.15.44:0/992873164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075c1015f0 msgr2=0x7f075c111ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.218+0000 7f0762be2640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075c1015f0 0x7f075c111ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.218+0000 7f0762be2640 1 -- 172.21.15.44:0/992873164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c19cca0 con 0x7f075c106a30 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.218+0000 7f07623e1640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075c1015f0 0x7f075c111ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:12:16.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.219+0000 7f0762be2640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c106a30 0x7f075c19c520 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f075400b970 tx=0x7f075400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.224+0000 7f07537fe640 1 -- 172.21.15.44:0/992873164 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754019070 con 0x7f075c106a30 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.224+0000 7f07537fe640 1 -- 172.21.15.44:0/992873164 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0754014440 con 0x7f075c106a30 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.224+0000 7f07537fe640 1 -- 172.21.15.44:0/992873164 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754019070 con 0x7f075c106a30 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.224+0000 7f07633e3640 1 -- 172.21.15.44:0/992873164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c19cf90 con 0x7f075c106a30 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.224+0000 7f07633e3640 1 -- 172.21.15.44:0/992873164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c06c8a0 con 0x7f075c106a30 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.229+0000 7f0766a3e740 1 Processor -- start 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.230+0000 7f07537fe640 1 -- 172.21.15.44:0/992873164 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f075401c070 con 0x7f075c106a30 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.231+0000 7f07537fe640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f07480b41e0 0x7f07480b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.231+0000 7f07537fe640 1 -- 172.21.15.44:0/992873164 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f075400c940 con 0x7f075c106a30 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.232+0000 7f0766a3e740 1 -- start start 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.232+0000 7f0766a3e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef88a30 0x55a00ef88e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.232+0000 7f0766a3e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef89350 0x55a00ef94da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.232+0000 7f0766a3e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef952e0 0x55a00ef976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.232+0000 7f0766a3e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a00eeff930 con 0x55a00ef952e0 2024-08-20T03:12:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.232+0000 7f0766a3e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a00eeeee30 con 0x55a00ef88a30 2024-08-20T03:12:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.232+0000 7f0766a3e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a00eef2e70 con 0x55a00ef89350 2024-08-20T03:12:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.232+0000 7f0761be0640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f07480b41e0 0x7f07480b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.232+0000 7f0761be0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef89350 0x55a00ef94da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.232+0000 7f0761be0640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef89350 0x55a00ef94da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45948/0 (socket says 172.21.15.44:45948) 2024-08-20T03:12:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.232+0000 7f0761be0640 1 -- 172.21.15.44:0/4177885932 learned_addr learned my addr 172.21.15.44:0/4177885932 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.233+0000 7f0761be0640 1 -- 172.21.15.44:0/4177885932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef88a30 msgr2=0x55a00ef88e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:12:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.233+0000 7f0761be0640 1 --2- 172.21.15.44:0/4177885932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef88a30 0x55a00ef88e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.233+0000 7f0761be0640 1 -- 172.21.15.44:0/4177885932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef952e0 msgr2=0x55a00ef976d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:12:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.233+0000 7f0761be0640 1 --2- 172.21.15.44:0/4177885932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef952e0 0x55a00ef976d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.233+0000 7f0761be0640 1 -- 172.21.15.44:0/4177885932 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a00ec4e650 con 0x55a00ef89350 2024-08-20T03:12:16.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.233+0000 7f0761be0640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f07480b41e0 0x7f07480b66a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f074c03b950 tx=0x7f074c0047d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.234+0000 7f0761be0640 1 --2- 172.21.15.44:0/4177885932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef89350 0x55a00ef94da0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f074c01a6d0 tx=0x7f074c019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ae8a5c834179dd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.234+0000 7f0750ff9640 1 -- 172.21.15.44:0/4177885932 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c046040 con 0x55a00ef89350 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.235+0000 7f0750ff9640 1 -- 172.21.15.44:0/4177885932 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074c0409c0 con 0x55a00ef89350 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.236+0000 7f0766a3e740 1 -- 172.21.15.44:0/4177885932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef89350 msgr2=0x55a00ef94da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.236+0000 7f0766a3e740 1 --2- 172.21.15.44:0/4177885932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef89350 0x55a00ef94da0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f074c01a6d0 tx=0x7f074c019000 comp rx=0 tx=0).stop 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.236+0000 7f0766a3e740 1 -- 172.21.15.44:0/4177885932 shutdown_connections 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.236+0000 7f0766a3e740 1 --2- 172.21.15.44:0/4177885932 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef952e0 0x55a00ef976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.236+0000 7f0766a3e740 1 --2- 172.21.15.44:0/4177885932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef89350 0x55a00ef94da0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.236+0000 7f0766a3e740 1 --2- 172.21.15.44:0/4177885932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef88a30 0x55a00ef88e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.236+0000 7f0766a3e740 1 -- 172.21.15.44:0/4177885932 >> 172.21.15.44:0/4177885932 conn(0x55a00ef86860 msgr2=0x55a00ef87a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.241+0000 7f0766a3e740 1 -- 172.21.15.44:0/4177885932 shutdown_connections 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.241+0000 7f0766a3e740 1 -- 172.21.15.44:0/4177885932 wait complete. 2024-08-20T03:12:16.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.241+0000 7f0766a3e740 1 -- start start 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.242+0000 7f0766a3e740 1 Processor -- start 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.242+0000 7f0766a3e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef88a30 0x55a00ef92340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.242+0000 7f0766a3e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef89350 0x55a00ef92880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.242+0000 7f0766a3e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef952e0 0x55a00ef92dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.242+0000 7f0766a3e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55a00ed51700 con 0x55a00ef89350 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.242+0000 7f0766a3e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a00ee75810 con 0x55a00ef952e0 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.242+0000 7f0766a3e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55a00efb9f90 con 0x55a00ef88a30 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.242+0000 7f0766a3e740 10 client.0 _ll_register_callbacks cb 0x55a00eee8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.243+0000 7f0761be0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef89350 0x55a00ef92880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.243+0000 7f0761be0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef89350 0x55a00ef92880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41146/0 (socket says 172.21.15.44:41146) 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.243+0000 7f0761be0640 1 -- 172.21.15.44:0/3694821628 learned_addr learned my addr 172.21.15.44:0/3694821628 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.244+0000 7f07623e1640 1 --2- 172.21.15.44:0/3694821628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef88a30 0x55a00ef92340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.244+0000 7f0761be0640 1 -- 172.21.15.44:0/3694821628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef88a30 msgr2=0x55a00ef92340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:16.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.244+0000 7f0761be0640 1 --2- 172.21.15.44:0/3694821628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef88a30 0x55a00ef92340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.244+0000 7f0761be0640 1 -- 172.21.15.44:0/3694821628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef952e0 msgr2=0x55a00ef92dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:12:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.244+0000 7f0761be0640 1 --2- 172.21.15.44:0/3694821628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef952e0 0x55a00ef92dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.244+0000 7f0761be0640 1 -- 172.21.15.44:0/3694821628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00efc34c0 con 0x55a00ef89350 2024-08-20T03:12:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.246+0000 7f0761be0640 1 --2- 172.21.15.44:0/3694821628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef89350 0x55a00ef92880 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f074c03ba20 tx=0x7f074c013db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e620268e7909cee1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.246+0000 7f0745ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:12:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.248+0000 7f0745ffb640 1 -- 172.21.15.44:0/3694821628 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c046040 con 0x55a00ef89350 2024-08-20T03:12:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.248+0000 7f0766a3e740 1 -- 172.21.15.44:0/3694821628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a00ee9e150 con 0x55a00ef89350 2024-08-20T03:12:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.248+0000 7f0745ffb640 1 -- 172.21.15.44:0/3694821628 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f074c040510 con 0x55a00ef89350 2024-08-20T03:12:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.249+0000 7f0745ffb640 1 -- 172.21.15.44:0/3694821628 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f074c0056b0 con 0x55a00ef89350 2024-08-20T03:12:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.250+0000 7f0766a3e740 10 client.17490 fetch_fsmap learned FSMap version 44 2024-08-20T03:12:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.250+0000 7f0766a3e740 1 -- 172.21.15.44:0/3694821628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a00efc5ee0 con 0x55a00ef89350 2024-08-20T03:12:16.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.251+0000 7f0745ffb640 1 -- 172.21.15.44:0/3694821628 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a00ee9e150 con 0x55a00ef89350 2024-08-20T03:12:16.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.252+0000 7f0745ffb640 1 -- 172.21.15.44:0/3694821628 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7f074c059020 con 0x55a00ef89350 2024-08-20T03:12:16.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.252+0000 7f0766a3e740 10 client.17490 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T03:12:16.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.252+0000 7f0766a3e740 10 client.17490 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:12:16.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.252+0000 7f0766a3e740 20 client.17490 populate_metadata read hostname 'smithi044' 2024-08-20T03:12:16.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.253+0000 7f0766a3e740 1 --2- 172.21.15.44:0/3694821628 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55a00efc7340 0x55a00efc9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.253+0000 7f0766a3e740 4 client.17490 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:12:16.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.253+0000 7f0766a3e740 1 -- 172.21.15.44:0/3694821628 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a00ef01880 con 0x55a00efc7340 2024-08-20T03:12:16.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.254+0000 7f07623e1640 1 --2- 172.21.15.44:0/3694821628 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55a00efc7340 0x55a00efc9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.263+0000 7f07623e1640 1 --2- 172.21.15.44:0/3694821628 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55a00efc7340 0x55a00efc9780 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=403d81fe8b4e178f server_cookie=f34e9a680dbfa5c5 in_seq=0 out_seq=0 2024-08-20T03:12:16.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.264+0000 7f0745ffb640 10 client.17490 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:12:16.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.267+0000 7f0745ffb640 1 -- 172.21.15.44:0/3694821628 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a00ef01880 con 0x55a00efc7340 2024-08-20T03:12:16.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.267+0000 7f0745ffb640 10 client.17490 handle_command_reply: tid=0 2024-08-20T03:12:16.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.267+0000 7f0766a3e740 1 client.17490 shutdown 2024-08-20T03:12:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.270+0000 7f0766a3e740 1 -- 172.21.15.44:0/3694821628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef89350 msgr2=0x55a00ef92880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.270+0000 7f0766a3e740 1 --2- 172.21.15.44:0/3694821628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef89350 0x55a00ef92880 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f074c03ba20 tx=0x7f074c013db0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.273+0000 7f0766a3e740 1 -- 172.21.15.44:0/3694821628 shutdown_connections 2024-08-20T03:12:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.273+0000 7f0766a3e740 1 --2- 172.21.15.44:0/3694821628 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55a00efc7340 0x55a00efc9780 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.273+0000 7f0766a3e740 1 --2- 172.21.15.44:0/3694821628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef952e0 0x55a00ef92dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.273+0000 7f0766a3e740 1 --2- 172.21.15.44:0/3694821628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef89350 0x55a00ef92880 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.273+0000 7f0766a3e740 1 --2- 172.21.15.44:0/3694821628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef88a30 0x55a00ef92340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.273+0000 7f0766a3e740 1 -- 172.21.15.44:0/3694821628 >> 172.21.15.44:0/3694821628 conn(0x55a00ef86860 msgr2=0x55a00ef8c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.273+0000 7f0745ffb640 0 client.17490 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:12:16.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.276+0000 7f0766a3e740 1 -- 172.21.15.44:0/3694821628 shutdown_connections 2024-08-20T03:12:16.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.276+0000 7f0766a3e740 1 -- 172.21.15.44:0/3694821628 wait complete. 2024-08-20T03:12:16.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.279+0000 7f0766a3e740 20 client.17490 trim_cache size 0 max 16384 2024-08-20T03:12:16.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.305+0000 7f0766a3e740 1 Processor -- start 2024-08-20T03:12:16.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.306+0000 7f0766a3e740 1 -- start start 2024-08-20T03:12:16.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.306+0000 7f0766a3e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef88a30 0x55a00ef91bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.306+0000 7f0766a3e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef89350 0x55a00ef8caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.306+0000 7f0766a3e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 0x55a00efcf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.306+0000 7f0766a3e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f074000e740 con 0x55a00ef88a30 2024-08-20T03:12:16.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.306+0000 7f0766a3e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0740010370 con 0x55a00ef89350 2024-08-20T03:12:16.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.306+0000 7f0766a3e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f074000eea0 con 0x55a00ef952e0 2024-08-20T03:12:16.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.307+0000 7f07623e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef88a30 0x55a00ef91bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.307+0000 7f0762be2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 0x55a00efcf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.307+0000 7f0762be2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 0x55a00efcf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45954/0 (socket says 172.21.15.44:45954) 2024-08-20T03:12:16.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.307+0000 7f0762be2640 1 -- 172.21.15.44:0/786963977 learned_addr learned my addr 172.21.15.44:0/786963977 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:16.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.308+0000 7f0762be2640 1 -- 172.21.15.44:0/786963977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef89350 msgr2=0x55a00ef8caa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:12:16.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.308+0000 7f0762be2640 1 --2- 172.21.15.44:0/786963977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef89350 0x55a00ef8caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.308+0000 7f0762be2640 1 -- 172.21.15.44:0/786963977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef88a30 msgr2=0x55a00ef91bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:16.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.308+0000 7f0762be2640 1 --2- 172.21.15.44:0/786963977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef88a30 0x55a00ef91bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.308+0000 7f0762be2640 1 -- 172.21.15.44:0/786963977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a00ef87f40 con 0x55a00ef952e0 2024-08-20T03:12:16.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.309+0000 7f0762be2640 1 --2- 172.21.15.44:0/786963977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 0x55a00efcf090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f07540e0ba0 tx=0x7f07540a9a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=791403423d75bd2a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:16.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.310+0000 7f0747fff640 1 -- 172.21.15.44:0/786963977 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07540a9b70 con 0x55a00ef952e0 2024-08-20T03:12:16.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.310+0000 7f0747fff640 1 -- 172.21.15.44:0/786963977 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07540ae170 con 0x55a00ef952e0 2024-08-20T03:12:16.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.311+0000 7f0766a3e740 1 -- 172.21.15.44:0/786963977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 msgr2=0x55a00efcf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:16.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.311+0000 7f0766a3e740 1 --2- 172.21.15.44:0/786963977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 0x55a00efcf090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f07540e0ba0 tx=0x7f07540a9a70 comp rx=0 tx=0).stop 2024-08-20T03:12:16.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.319+0000 7f0766a3e740 1 -- 172.21.15.44:0/786963977 shutdown_connections 2024-08-20T03:12:16.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.319+0000 7f0766a3e740 1 --2- 172.21.15.44:0/786963977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 0x55a00efcf090 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.319+0000 7f0766a3e740 1 --2- 172.21.15.44:0/786963977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef89350 0x55a00ef8caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.319+0000 7f0766a3e740 1 --2- 172.21.15.44:0/786963977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef88a30 0x55a00ef91bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.319+0000 7f0766a3e740 1 -- 172.21.15.44:0/786963977 >> 172.21.15.44:0/786963977 conn(0x55a00ef86860 msgr2=0x55a00ef94190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:16.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.321+0000 7f0766a3e740 1 -- 172.21.15.44:0/786963977 shutdown_connections 2024-08-20T03:12:16.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.321+0000 7f0766a3e740 1 -- 172.21.15.44:0/786963977 wait complete. 2024-08-20T03:12:16.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.322+0000 7f0766a3e740 1 -- start start 2024-08-20T03:12:16.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.322+0000 7f0766a3e740 1 Processor -- start 2024-08-20T03:12:16.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.322+0000 7f0766a3e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef88a30 0x55a00efbf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.322+0000 7f0766a3e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef89350 0x55a00efba280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.322+0000 7f0766a3e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 0x55a00efba7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.322+0000 7f0766a3e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f074000fca0 con 0x55a00ef89350 2024-08-20T03:12:16.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.322+0000 7f0766a3e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f074000f5a0 con 0x55a00ef88a30 2024-08-20T03:12:16.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.322+0000 7f0766a3e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f074000c7d0 con 0x55a00ef952e0 2024-08-20T03:12:16.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.323+0000 7f0766a3e740 10 client.0 _ll_register_callbacks cb 0x7f0758005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:12:16.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.323+0000 7f0762be2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 0x55a00efba7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.323+0000 7f0762be2640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 0x55a00efba7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:45970/0 (socket says 172.21.15.44:45970) 2024-08-20T03:12:16.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.323+0000 7f0762be2640 1 -- 172.21.15.44:0/1902153129 learned_addr learned my addr 172.21.15.44:0/1902153129 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:16.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.324+0000 7f07623e1640 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef88a30 0x55a00efbf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.324+0000 7f0761be0640 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef89350 0x55a00efba280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.324+0000 7f0762be2640 1 -- 172.21.15.44:0/1902153129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef88a30 msgr2=0x55a00efbf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:16.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.324+0000 7f0762be2640 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef88a30 0x55a00efbf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.324+0000 7f0762be2640 1 -- 172.21.15.44:0/1902153129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef89350 msgr2=0x55a00efba280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:16.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.324+0000 7f0762be2640 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef89350 0x55a00efba280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.324+0000 7f0762be2640 1 -- 172.21.15.44:0/1902153129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00efbaf30 con 0x55a00ef952e0 2024-08-20T03:12:16.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.325+0000 7f0762be2640 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 0x55a00efba7c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f07540910b0 tx=0x7f07540a6030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2b9180780d020e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:16.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.325+0000 7f0745ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:12:16.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.326+0000 7f0745ffb640 1 -- 172.21.15.44:0/1902153129 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07540a9b70 con 0x55a00ef952e0 2024-08-20T03:12:16.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.326+0000 7f0745ffb640 1 -- 172.21.15.44:0/1902153129 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f07540a6160 con 0x55a00ef952e0 2024-08-20T03:12:16.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.329+0000 7f0766a3e740 1 -- 172.21.15.44:0/1902153129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07540ae170 con 0x55a00ef952e0 2024-08-20T03:12:16.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.332+0000 7f0745ffb640 1 -- 172.21.15.44:0/1902153129 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f07540b2c20 con 0x55a00ef952e0 2024-08-20T03:12:16.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.334+0000 7f0766a3e740 10 client.26821 fetch_fsmap learned FSMap version 44 2024-08-20T03:12:16.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.334+0000 7f0766a3e740 1 -- 172.21.15.44:0/1902153129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a00f030ff0 con 0x55a00ef952e0 2024-08-20T03:12:16.336 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:16 smithi044 ceph-mon[18790]: pgmap v620: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 810 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-08-20T03:12:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.337+0000 7f0745ffb640 1 -- 172.21.15.44:0/1902153129 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07540ae170 con 0x55a00ef952e0 2024-08-20T03:12:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.337+0000 7f0745ffb640 1 -- 172.21.15.44:0/1902153129 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7f07540a8dd0 con 0x55a00ef952e0 2024-08-20T03:12:16.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.338+0000 7f0766a3e740 10 client.26821 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T03:12:16.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.338+0000 7f0766a3e740 10 client.26821 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-08-20T03:12:16.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.338+0000 7f0766a3e740 20 client.26821 populate_metadata read hostname 'smithi044' 2024-08-20T03:12:16.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.338+0000 7f0766a3e740 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55a00f0316a0 0x55a00f033ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:16.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.338+0000 7f0766a3e740 4 client.26821 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:12:16.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.338+0000 7f0766a3e740 1 -- 172.21.15.44:0/1902153129 --> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a00eefd320 con 0x55a00f0316a0 2024-08-20T03:12:16.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.339+0000 7f07623e1640 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55a00f0316a0 0x55a00f033ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.356+0000 7f07623e1640 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55a00f0316a0 0x55a00f033ac0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96615897c87bc9bc server_cookie=f9810f8c2a47362e in_seq=0 out_seq=0 2024-08-20T03:12:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.356+0000 7f0745ffb640 10 client.26821 ms_handle_connect on v2:172.21.15.99:6836/728486889 2024-08-20T03:12:16.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.356+0000 7f0745ffb640 1 -- 172.21.15.44:0/1902153129 <== mds.4 v2:172.21.15.99:6836/728486889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a00eefd320 con 0x55a00f0316a0 2024-08-20T03:12:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.356+0000 7f0745ffb640 10 client.26821 handle_command_reply: tid=0 2024-08-20T03:12:16.359 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.356+0000 7f0766a3e740 1 client.26821 shutdown 2024-08-20T03:12:16.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.358+0000 7f0766a3e740 1 -- 172.21.15.44:0/1902153129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 msgr2=0x55a00efba7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:16.360 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.358+0000 7f0766a3e740 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 0x55a00efba7c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f07540910b0 tx=0x7f07540a6030 comp rx=0 tx=0).stop 2024-08-20T03:12:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.358+0000 7f0766a3e740 1 -- 172.21.15.44:0/1902153129 shutdown_connections 2024-08-20T03:12:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.358+0000 7f0766a3e740 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a00ef952e0 0x55a00efba7c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.358+0000 7f0766a3e740 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a00ef89350 0x55a00efba280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.358+0000 7f0766a3e740 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.99:6836/728486889,v1:172.21.15.99:6837/728486889] conn(0x55a00f0316a0 0x55a00f033ac0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.358+0000 7f0766a3e740 1 --2- 172.21.15.44:0/1902153129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a00ef88a30 0x55a00efbf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.358+0000 7f0766a3e740 1 -- 172.21.15.44:0/1902153129 >> 172.21.15.44:0/1902153129 conn(0x55a00ef86860 msgr2=0x55a00ef84fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.358+0000 7f0745ffb640 0 client.26821 ms_handle_reset on v2:172.21.15.99:6836/728486889 2024-08-20T03:12:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.358+0000 7f0766a3e740 1 -- 172.21.15.44:0/1902153129 shutdown_connections 2024-08-20T03:12:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.358+0000 7f0766a3e740 1 -- 172.21.15.44:0/1902153129 wait complete. 2024-08-20T03:12:16.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.358+0000 7f0766a3e740 20 client.26821 trim_cache size 0 max 16384 2024-08-20T03:12:16.362 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:12:16.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.362+0000 7f07633e3640 1 -- 172.21.15.44:0/992873164 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f07480b41e0 msgr2=0x7f07480b66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:16.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.362+0000 7f07633e3640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f07480b41e0 0x7f07480b66a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f074c03b950 tx=0x7f074c0047d0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.362+0000 7f07633e3640 1 -- 172.21.15.44:0/992873164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c106a30 msgr2=0x7f075c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:16.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.362+0000 7f07633e3640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c106a30 0x7f075c19c520 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f075400b970 tx=0x7f075400be30 comp rx=0 tx=0).stop 2024-08-20T03:12:16.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.362+0000 7f07633e3640 1 -- 172.21.15.44:0/992873164 shutdown_connections 2024-08-20T03:12:16.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.362+0000 7f07633e3640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f07480b41e0 0x7f07480b66a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.362+0000 7f07633e3640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f075c106a30 0x7f075c19c520 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.362+0000 7f07633e3640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f075c101f10 0x7f075c112020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.362+0000 7f07633e3640 1 --2- 172.21.15.44:0/992873164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f075c1015f0 0x7f075c111ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:16.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.362+0000 7f07633e3640 1 -- 172.21.15.44:0/992873164 >> 172.21.15.44:0/992873164 conn(0x7f075c0fd2a0 msgr2=0x7f075c1043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:16.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.363+0000 7f07633e3640 1 -- 172.21.15.44:0/992873164 shutdown_connections 2024-08-20T03:12:16.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:16.363+0000 7f07633e3640 1 -- 172.21.15.44:0/992873164 wait complete. 2024-08-20T03:12:16.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:16 smithi165 ceph-mon[22357]: pgmap v620: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 810 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-08-20T03:12:16.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:16 smithi099 ceph-mon[21984]: pgmap v620: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 810 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-08-20T03:12:17.250 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-08-20T03:12:17.702 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:18.194 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:18 smithi044 ceph-mon[18790]: pgmap v621: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 808 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-08-20T03:12:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.350+0000 7f9245904640 1 -- 172.21.15.44:0/2669507071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9240109e80 msgr2=0x7f9240110ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.350+0000 7f9245904640 1 --2- 172.21.15.44:0/2669507071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9240109e80 0x7f9240110ee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9230009930 tx=0x7f923002a420 comp rx=0 tx=0).stop 2024-08-20T03:12:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.350+0000 7f9245904640 1 -- 172.21.15.44:0/2669507071 shutdown_connections 2024-08-20T03:12:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.350+0000 7f9245904640 1 --2- 172.21.15.44:0/2669507071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924011bdd0 0x7f924011e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.350+0000 7f9245904640 1 --2- 172.21.15.44:0/2669507071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f924010a260 0x7f924010a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.350+0000 7f9245904640 1 --2- 172.21.15.44:0/2669507071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9240109e80 0x7f9240110ee0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.350+0000 7f9245904640 1 -- 172.21.15.44:0/2669507071 >> 172.21.15.44:0/2669507071 conn(0x7f924006c840 msgr2=0x7f924006cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:18.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.351+0000 7f9245904640 1 -- 172.21.15.44:0/2669507071 shutdown_connections 2024-08-20T03:12:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.351+0000 7f9245904640 1 -- 172.21.15.44:0/2669507071 wait complete. 2024-08-20T03:12:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.351+0000 7f9245904640 1 Processor -- start 2024-08-20T03:12:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.352+0000 7f9245904640 1 -- start start 2024-08-20T03:12:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.352+0000 7f9245904640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9240109e80 0x7f92401a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.352+0000 7f9245904640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f924010a260 0x7f92401a8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.352+0000 7f9245904640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924011bdd0 0x7f92401ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.352+0000 7f9245904640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9240121720 con 0x7f9240109e80 2024-08-20T03:12:18.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.352+0000 7f9245904640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92401215a0 con 0x7f924010a260 2024-08-20T03:12:18.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.352+0000 7f9245904640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f92401218a0 con 0x7f924011bdd0 2024-08-20T03:12:18.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.353+0000 7f923e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f924010a260 0x7f92401a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.353+0000 7f923e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f924010a260 0x7f92401a8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60890/0 (socket says 172.21.15.44:60890) 2024-08-20T03:12:18.355 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.353+0000 7f923e7fc640 1 -- 172.21.15.44:0/3735656929 learned_addr learned my addr 172.21.15.44:0/3735656929 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.353+0000 7f923f7fe640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924011bdd0 0x7f92401ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.353+0000 7f923e7fc640 1 -- 172.21.15.44:0/3735656929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924011bdd0 msgr2=0x7f92401ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.353+0000 7f923e7fc640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924011bdd0 0x7f92401ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.353+0000 7f923e7fc640 1 -- 172.21.15.44:0/3735656929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9240109e80 msgr2=0x7f92401a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.353+0000 7f923e7fc640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9240109e80 0x7f92401a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.356 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.353+0000 7f923e7fc640 1 -- 172.21.15.44:0/3735656929 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92401adcc0 con 0x7f924010a260 2024-08-20T03:12:18.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.354+0000 7f923e7fc640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f924010a260 0x7f92401a8150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f923400b3f0 tx=0x7f923400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:18.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.363+0000 7f9244902640 1 -- 172.21.15.44:0/3735656929 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234015020 con 0x7f924010a260 2024-08-20T03:12:18.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.363+0000 7f9244902640 1 -- 172.21.15.44:0/3735656929 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9234011ce0 con 0x7f924010a260 2024-08-20T03:12:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.364+0000 7f9244902640 1 -- 172.21.15.44:0/3735656929 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234010de0 con 0x7f924010a260 2024-08-20T03:12:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.365+0000 7f9245904640 1 -- 172.21.15.44:0/3735656929 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92401adfb0 con 0x7f924010a260 2024-08-20T03:12:18.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.365+0000 7f9245904640 1 -- 172.21.15.44:0/3735656929 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92401b4670 con 0x7f924010a260 2024-08-20T03:12:18.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.374+0000 7f9247cfd740 1 Processor -- start 2024-08-20T03:12:18.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.374+0000 7f9247cfd740 1 -- start start 2024-08-20T03:12:18.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.374+0000 7f9247cfd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f06910 0x563d66f06cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.374+0000 7f9247cfd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f12c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.374+0000 7f9247cfd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f131c0 0x563d66f155b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.374+0000 7f9247cfd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563d66e7d930 con 0x563d66f06910 2024-08-20T03:12:18.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.374+0000 7f9247cfd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563d66e6ce30 con 0x563d66f131c0 2024-08-20T03:12:18.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.374+0000 7f9247cfd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563d66e70e70 con 0x563d66f07230 2024-08-20T03:12:18.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.374+0000 7f9244902640 1 -- 172.21.15.44:0/3735656929 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f9234018020 con 0x7f924010a260 2024-08-20T03:12:18.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.374+0000 7f923e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f131c0 0x563d66f155b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.374+0000 7f923e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f131c0 0x563d66f155b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60900/0 (socket says 172.21.15.44:60900) 2024-08-20T03:12:18.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.374+0000 7f923e7fc640 1 -- 172.21.15.44:0/2125865649 learned_addr learned my addr 172.21.15.44:0/2125865649 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:18.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.375+0000 7f923e7fc640 1 -- 172.21.15.44:0/2125865649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 msgr2=0x563d66f12c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:18.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.375+0000 7f923effd640 1 --2- 172.21.15.44:0/2125865649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f12c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.375+0000 7f923e7fc640 1 --2- 172.21.15.44:0/2125865649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f12c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.375+0000 7f923e7fc640 1 -- 172.21.15.44:0/2125865649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f06910 msgr2=0x563d66f06cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:18.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.375+0000 7f923f7fe640 1 --2- 172.21.15.44:0/2125865649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f06910 0x563d66f06cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.375+0000 7f923e7fc640 1 --2- 172.21.15.44:0/2125865649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f06910 0x563d66f06cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.375+0000 7f923e7fc640 1 -- 172.21.15.44:0/2125865649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d66bcc650 con 0x563d66f131c0 2024-08-20T03:12:18.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.375+0000 7f923f7fe640 1 --2- 172.21.15.44:0/2125865649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f06910 0x563d66f06cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:12:18.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.375+0000 7f923e7fc640 1 --2- 172.21.15.44:0/2125865649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f131c0 0x563d66f155b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f92340e0cc0 tx=0x7f92340e9ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44c744e9da07fe9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:18.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.376+0000 7f9244902640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92280b3ea0 0x7f92280b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.376+0000 7f9244902640 1 -- 172.21.15.44:0/3735656929 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f92340e3560 con 0x7f924010a260 2024-08-20T03:12:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.376+0000 7f923f7fe640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92280b3ea0 0x7f92280b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.377+0000 7f9225ffb640 1 -- 172.21.15.44:0/2125865649 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92340e3d20 con 0x563d66f131c0 2024-08-20T03:12:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.377+0000 7f9225ffb640 1 -- 172.21.15.44:0/2125865649 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92340f0070 con 0x563d66f131c0 2024-08-20T03:12:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.378+0000 7f9247cfd740 1 -- 172.21.15.44:0/2125865649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f131c0 msgr2=0x563d66f155b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.378+0000 7f9247cfd740 1 --2- 172.21.15.44:0/2125865649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f131c0 0x563d66f155b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f92340e0cc0 tx=0x7f92340e9ab0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.378+0000 7f923f7fe640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92280b3ea0 0x7f92280b6360 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f92380099a0 tx=0x7f9238009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.378+0000 7f9247cfd740 1 -- 172.21.15.44:0/2125865649 shutdown_connections 2024-08-20T03:12:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.378+0000 7f9247cfd740 1 --2- 172.21.15.44:0/2125865649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f131c0 0x563d66f155b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.378+0000 7f9247cfd740 1 --2- 172.21.15.44:0/2125865649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f12c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.378+0000 7f9247cfd740 1 --2- 172.21.15.44:0/2125865649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f06910 0x563d66f06cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.378+0000 7f9247cfd740 1 -- 172.21.15.44:0/2125865649 >> 172.21.15.44:0/2125865649 conn(0x563d66f04740 msgr2=0x563d66f058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:18.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.380+0000 7f9247cfd740 1 -- 172.21.15.44:0/2125865649 shutdown_connections 2024-08-20T03:12:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.382+0000 7f9247cfd740 1 -- 172.21.15.44:0/2125865649 wait complete. 2024-08-20T03:12:18.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.382+0000 7f9247cfd740 1 -- start start 2024-08-20T03:12:18.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.382+0000 7f9247cfd740 1 Processor -- start 2024-08-20T03:12:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.383+0000 7f9247cfd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f06910 0x563d66f100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.383+0000 7f9247cfd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f10620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.383+0000 7f9247cfd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f131c0 0x563d66f10b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.383+0000 7f9247cfd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563d66ccf700 con 0x563d66f131c0 2024-08-20T03:12:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.383+0000 7f9247cfd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563d66df3810 con 0x563d66f06910 2024-08-20T03:12:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.383+0000 7f9247cfd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x563d66f15cf0 con 0x563d66f07230 2024-08-20T03:12:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.383+0000 7f9247cfd740 10 client.0 _ll_register_callbacks cb 0x563d66e66700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:12:18.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.383+0000 7f923e7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f10620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.383+0000 7f923e7fc640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f10620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46026/0 (socket says 172.21.15.44:46026) 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.383+0000 7f923e7fc640 1 -- 172.21.15.44:0/2983246607 learned_addr learned my addr 172.21.15.44:0/2983246607 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.384+0000 7f923e7fc640 1 -- 172.21.15.44:0/2983246607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f06910 msgr2=0x563d66f100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.384+0000 7f923f7fe640 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f131c0 0x563d66f10b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.384+0000 7f923e7fc640 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f06910 0x563d66f100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.384+0000 7f923e7fc640 1 -- 172.21.15.44:0/2983246607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f131c0 msgr2=0x563d66f10b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.384+0000 7f923e7fc640 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f131c0 0x563d66f10b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.384+0000 7f923e7fc640 1 -- 172.21.15.44:0/2983246607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d66f1f0c0 con 0x563d66f07230 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.384+0000 7f923f7fe640 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f131c0 0x563d66f10b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.384+0000 7f923e7fc640 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f10620 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f92340a62c0 tx=0x7f92340eb3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28482c1d937085a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.385+0000 7f921effd640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.385+0000 7f921effd640 1 -- 172.21.15.44:0/2983246607 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92340a6100 con 0x563d66f07230 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.385+0000 7f9247cfd740 1 -- 172.21.15.44:0/2983246607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d66e1c150 con 0x563d66f07230 2024-08-20T03:12:18.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.385+0000 7f921effd640 1 -- 172.21.15.44:0/2983246607 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f92340e0ef0 con 0x563d66f07230 2024-08-20T03:12:18.389 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:18 smithi165 ceph-mon[22357]: pgmap v621: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 808 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-08-20T03:12:18.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.385+0000 7f921effd640 1 -- 172.21.15.44:0/2983246607 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f92340a99b0 con 0x563d66f07230 2024-08-20T03:12:18.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.386+0000 7f9247cfd740 10 client.26830 fetch_fsmap learned FSMap version 44 2024-08-20T03:12:18.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.386+0000 7f9247cfd740 1 -- 172.21.15.44:0/2983246607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563d66f219e0 con 0x563d66f07230 2024-08-20T03:12:18.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.389+0000 7f921effd640 1 -- 172.21.15.44:0/2983246607 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563d66e1c150 con 0x563d66f07230 2024-08-20T03:12:18.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.390+0000 7f921effd640 1 -- 172.21.15.44:0/2983246607 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7f9234022060 con 0x563d66f07230 2024-08-20T03:12:18.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.393+0000 7f9247cfd740 10 client.26830 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T03:12:18.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.393+0000 7f9247cfd740 10 client.26830 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:12:18.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.393+0000 7f9247cfd740 20 client.26830 populate_metadata read hostname 'smithi044' 2024-08-20T03:12:18.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.393+0000 7f9247cfd740 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x563d66f22d30 0x563d66f25120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.393+0000 7f9247cfd740 4 client.26830 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:12:18.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.393+0000 7f9247cfd740 1 -- 172.21.15.44:0/2983246607 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d66e7f880 con 0x563d66f22d30 2024-08-20T03:12:18.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.398+0000 7f923effd640 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x563d66f22d30 0x563d66f25120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.404+0000 7f923effd640 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x563d66f22d30 0x563d66f25120 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=802c47213534c18e server_cookie=c1916aa55da1ad9 in_seq=0 out_seq=0 2024-08-20T03:12:18.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.404+0000 7f921effd640 10 client.26830 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:12:18.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.405+0000 7f921effd640 1 -- 172.21.15.44:0/2983246607 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d66e7f880 con 0x563d66f22d30 2024-08-20T03:12:18.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.405+0000 7f921effd640 10 client.26830 handle_command_reply: tid=0 2024-08-20T03:12:18.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.405+0000 7f9247cfd740 1 client.26830 shutdown 2024-08-20T03:12:18.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.405+0000 7f9247cfd740 1 -- 172.21.15.44:0/2983246607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 msgr2=0x563d66f10620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:18.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.405+0000 7f9247cfd740 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f10620 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f92340a62c0 tx=0x7f92340eb3e0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.407+0000 7f9247cfd740 1 -- 172.21.15.44:0/2983246607 shutdown_connections 2024-08-20T03:12:18.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.407+0000 7f9247cfd740 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x563d66f22d30 0x563d66f25120 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.407+0000 7f9247cfd740 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f131c0 0x563d66f10b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.407+0000 7f9247cfd740 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f10620 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.407+0000 7f9247cfd740 1 --2- 172.21.15.44:0/2983246607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f06910 0x563d66f100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.407+0000 7f9247cfd740 1 -- 172.21.15.44:0/2983246607 >> 172.21.15.44:0/2983246607 conn(0x563d66f04740 msgr2=0x563d66f0a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:18.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.407+0000 7f921effd640 0 client.26830 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:12:18.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.407+0000 7f9247cfd740 1 -- 172.21.15.44:0/2983246607 shutdown_connections 2024-08-20T03:12:18.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.408+0000 7f9247cfd740 1 -- 172.21.15.44:0/2983246607 wait complete. 2024-08-20T03:12:18.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.408+0000 7f9247cfd740 20 client.26830 trim_cache size 0 max 16384 2024-08-20T03:12:18.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:18 smithi099 ceph-mon[21984]: pgmap v621: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 808 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-08-20T03:12:18.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.437+0000 7f9247cfd740 1 Processor -- start 2024-08-20T03:12:18.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.437+0000 7f9247cfd740 1 -- start start 2024-08-20T03:12:18.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.438+0000 7f9247cfd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f04740 0x563d66f03550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.438+0000 7f9247cfd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f06910 0x563d66f0d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.438+0000 7f9247cfd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f0d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.438+0000 7f9247cfd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f92401bce00 con 0x563d66f06910 2024-08-20T03:12:18.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.438+0000 7f9247cfd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92401bea30 con 0x563d66f04740 2024-08-20T03:12:18.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.438+0000 7f9247cfd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f92401bd560 con 0x563d66f07230 2024-08-20T03:12:18.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.438+0000 7f923effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f04740 0x563d66f03550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.438+0000 7f923effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f04740 0x563d66f03550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60908/0 (socket says 172.21.15.44:60908) 2024-08-20T03:12:18.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.438+0000 7f923effd640 1 -- 172.21.15.44:0/195421362 learned_addr learned my addr 172.21.15.44:0/195421362 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:18.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.438+0000 7f923e7fc640 1 --2- 172.21.15.44:0/195421362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f06910 0x563d66f0d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.439+0000 7f923effd640 1 -- 172.21.15.44:0/195421362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 msgr2=0x563d66f0d830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:12:18.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.439+0000 7f923effd640 1 --2- 172.21.15.44:0/195421362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f0d830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.439+0000 7f923effd640 1 -- 172.21.15.44:0/195421362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f06910 msgr2=0x563d66f0d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:18.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.439+0000 7f923effd640 1 --2- 172.21.15.44:0/195421362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f06910 0x563d66f0d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.439+0000 7f923effd640 1 -- 172.21.15.44:0/195421362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d66f0e5e0 con 0x563d66f04740 2024-08-20T03:12:18.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.440+0000 7f923effd640 1 --2- 172.21.15.44:0/195421362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f04740 0x563d66f03550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9230009930 tx=0x7f9230005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c18363882bffae5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:18.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.440+0000 7f92257fa640 1 -- 172.21.15.44:0/195421362 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230035070 con 0x563d66f04740 2024-08-20T03:12:18.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.440+0000 7f92257fa640 1 -- 172.21.15.44:0/195421362 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9230004030 con 0x563d66f04740 2024-08-20T03:12:18.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.440+0000 7f92257fa640 1 -- 172.21.15.44:0/195421362 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92300318a0 con 0x563d66f04740 2024-08-20T03:12:18.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.441+0000 7f9247cfd740 1 -- 172.21.15.44:0/195421362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f04740 msgr2=0x563d66f03550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:18.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.441+0000 7f9247cfd740 1 --2- 172.21.15.44:0/195421362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f04740 0x563d66f03550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9230009930 tx=0x7f9230005900 comp rx=0 tx=0).stop 2024-08-20T03:12:18.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.444+0000 7f9247cfd740 1 -- 172.21.15.44:0/195421362 shutdown_connections 2024-08-20T03:12:18.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.444+0000 7f9247cfd740 1 --2- 172.21.15.44:0/195421362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f07230 0x563d66f0d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.444+0000 7f9247cfd740 1 --2- 172.21.15.44:0/195421362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f06910 0x563d66f0d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.444+0000 7f9247cfd740 1 --2- 172.21.15.44:0/195421362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f04740 0x563d66f03550 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.444+0000 7f9247cfd740 1 -- 172.21.15.44:0/195421362 >> 172.21.15.44:0/195421362 conn(0x563d66eff8d0 msgr2=0x563d66f02e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:18.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.446+0000 7f9247cfd740 1 -- 172.21.15.44:0/195421362 shutdown_connections 2024-08-20T03:12:18.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.446+0000 7f9247cfd740 1 -- 172.21.15.44:0/195421362 wait complete. 2024-08-20T03:12:18.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.447+0000 7f9247cfd740 1 -- start start 2024-08-20T03:12:18.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.447+0000 7f9247cfd740 1 Processor -- start 2024-08-20T03:12:18.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.449+0000 7f9247cfd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f04740 0x563d66f26d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.449+0000 7f9247cfd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f06910 0x563d66f272a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.449+0000 7f9247cfd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f07230 0x563d66f277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.449+0000 7f9247cfd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f92401be360 con 0x563d66f04740 2024-08-20T03:12:18.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.449+0000 7f9247cfd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92401bdc60 con 0x563d66f07230 2024-08-20T03:12:18.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.449+0000 7f9247cfd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f92401bae90 con 0x563d66f06910 2024-08-20T03:12:18.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.450+0000 7f9247cfd740 10 client.0 _ll_register_callbacks cb 0x7f92340e94a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:12:18.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.450+0000 7f923effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f04740 0x563d66f26d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.450+0000 7f923effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f04740 0x563d66f26d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48094/0 (socket says 172.21.15.44:48094) 2024-08-20T03:12:18.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.450+0000 7f923effd640 1 -- 172.21.15.44:0/3662151413 learned_addr learned my addr 172.21.15.44:0/3662151413 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.451+0000 7f923effd640 1 -- 172.21.15.44:0/3662151413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f06910 msgr2=0x563d66f272a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:12:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.451+0000 7f923effd640 1 --2- 172.21.15.44:0/3662151413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f06910 0x563d66f272a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.451+0000 7f923effd640 1 -- 172.21.15.44:0/3662151413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f07230 msgr2=0x563d66f277e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:12:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.451+0000 7f923effd640 1 --2- 172.21.15.44:0/3662151413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f07230 0x563d66f277e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.451+0000 7f923effd640 1 -- 172.21.15.44:0/3662151413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d66f8b7d0 con 0x563d66f04740 2024-08-20T03:12:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.452+0000 7f923effd640 1 --2- 172.21.15.44:0/3662151413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f04740 0x563d66f26d60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9230005600 tx=0x7f9230000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c76b794fa5a884 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.452+0000 7f921effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:12:18.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.452+0000 7f921effd640 1 -- 172.21.15.44:0/3662151413 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230035070 con 0x563d66f04740 2024-08-20T03:12:18.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.452+0000 7f921effd640 1 -- 172.21.15.44:0/3662151413 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f92300043f0 con 0x563d66f04740 2024-08-20T03:12:18.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.454+0000 7f9247cfd740 1 -- 172.21.15.44:0/3662151413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9230004030 con 0x563d66f04740 2024-08-20T03:12:18.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.454+0000 7f921effd640 1 -- 172.21.15.44:0/3662151413 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9230031b00 con 0x563d66f04740 2024-08-20T03:12:18.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.455+0000 7f9247cfd740 10 client.17499 fetch_fsmap learned FSMap version 44 2024-08-20T03:12:18.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.455+0000 7f9247cfd740 1 -- 172.21.15.44:0/3662151413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563d66f8cae0 con 0x563d66f04740 2024-08-20T03:12:18.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.456+0000 7f921effd640 1 -- 172.21.15.44:0/3662151413 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9230031ca0 con 0x563d66f04740 2024-08-20T03:12:18.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.456+0000 7f921effd640 1 -- 172.21.15.44:0/3662151413 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7f923003b490 con 0x563d66f04740 2024-08-20T03:12:18.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.456+0000 7f9247cfd740 10 client.17499 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T03:12:18.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.456+0000 7f9247cfd740 10 client.17499 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-08-20T03:12:18.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.456+0000 7f9247cfd740 20 client.17499 populate_metadata read hostname 'smithi044' 2024-08-20T03:12:18.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.456+0000 7f9247cfd740 1 --2- 172.21.15.44:0/3662151413 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x563d66f131c0 0x563d66f8f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:18.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.456+0000 7f9247cfd740 4 client.17499 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:12:18.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.456+0000 7f9247cfd740 1 -- 172.21.15.44:0/3662151413 --> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9230004030 con 0x563d66f131c0 2024-08-20T03:12:18.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.457+0000 7f923e7fc640 1 --2- 172.21.15.44:0/3662151413 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x563d66f131c0 0x563d66f8f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:18.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.461+0000 7f923e7fc640 1 --2- 172.21.15.44:0/3662151413 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x563d66f131c0 0x563d66f8f260 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b0f528269a867b2 server_cookie=53a6add079214c2c in_seq=0 out_seq=0 2024-08-20T03:12:18.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.461+0000 7f921effd640 10 client.17499 ms_handle_connect on v2:172.21.15.165:6834/3019941967 2024-08-20T03:12:18.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.462+0000 7f921effd640 1 -- 172.21.15.44:0/3662151413 <== mds.3 v2:172.21.15.165:6834/3019941967 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9230004030 con 0x563d66f131c0 2024-08-20T03:12:18.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.462+0000 7f921effd640 10 client.17499 handle_command_reply: tid=0 2024-08-20T03:12:18.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.462+0000 7f9247cfd740 1 client.17499 shutdown 2024-08-20T03:12:18.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.462+0000 7f9247cfd740 1 -- 172.21.15.44:0/3662151413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f04740 msgr2=0x563d66f26d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:18.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.462+0000 7f9247cfd740 1 --2- 172.21.15.44:0/3662151413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f04740 0x563d66f26d60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9230005600 tx=0x7f9230000bc0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.462+0000 7f9247cfd740 1 -- 172.21.15.44:0/3662151413 shutdown_connections 2024-08-20T03:12:18.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.462+0000 7f9247cfd740 1 --2- 172.21.15.44:0/3662151413 >> [v2:172.21.15.165:6834/3019941967,v1:172.21.15.165:6835/3019941967] conn(0x563d66f131c0 0x563d66f8f260 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.462+0000 7f9247cfd740 1 --2- 172.21.15.44:0/3662151413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563d66f07230 0x563d66f277e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.463+0000 7f9247cfd740 1 --2- 172.21.15.44:0/3662151413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d66f06910 0x563d66f272a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.463+0000 7f9247cfd740 1 --2- 172.21.15.44:0/3662151413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d66f04740 0x563d66f26d60 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.463+0000 7f9247cfd740 1 -- 172.21.15.44:0/3662151413 >> 172.21.15.44:0/3662151413 conn(0x563d66eff8d0 msgr2=0x563d66f03660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:18.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.463+0000 7f921effd640 0 client.17499 ms_handle_reset on v2:172.21.15.165:6834/3019941967 2024-08-20T03:12:18.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.463+0000 7f9247cfd740 1 -- 172.21.15.44:0/3662151413 shutdown_connections 2024-08-20T03:12:18.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.463+0000 7f9247cfd740 1 -- 172.21.15.44:0/3662151413 wait complete. 2024-08-20T03:12:18.466 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:12:18.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.463+0000 7f9247cfd740 20 client.17499 trim_cache size 0 max 16384 2024-08-20T03:12:18.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.464+0000 7f9245904640 1 -- 172.21.15.44:0/3735656929 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92280b3ea0 msgr2=0x7f92280b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:18.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.464+0000 7f9245904640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92280b3ea0 0x7f92280b6360 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f92380099a0 tx=0x7f9238009500 comp rx=0 tx=0).stop 2024-08-20T03:12:18.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.464+0000 7f9245904640 1 -- 172.21.15.44:0/3735656929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f924010a260 msgr2=0x7f92401a8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:18.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.464+0000 7f9245904640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f924010a260 0x7f92401a8150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f923400b3f0 tx=0x7f923400b8b0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.464+0000 7f9245904640 1 -- 172.21.15.44:0/3735656929 shutdown_connections 2024-08-20T03:12:18.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.464+0000 7f9245904640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f92280b3ea0 0x7f92280b6360 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.464+0000 7f9245904640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924011bdd0 0x7f92401ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.464+0000 7f9245904640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f924010a260 0x7f92401a8150 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.464+0000 7f9245904640 1 --2- 172.21.15.44:0/3735656929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9240109e80 0x7f92401a7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:18.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.464+0000 7f9245904640 1 -- 172.21.15.44:0/3735656929 >> 172.21.15.44:0/3735656929 conn(0x7f924006c840 msgr2=0x7f924011c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:18.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.464+0000 7f9245904640 1 -- 172.21.15.44:0/3735656929 shutdown_connections 2024-08-20T03:12:18.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:18.464+0000 7f9245904640 1 -- 172.21.15.44:0/3735656929 wait complete. 2024-08-20T03:12:19.201 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T03:12:19.636 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:20.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.227+0000 7f2b4d01a640 1 -- 172.21.15.44:0/2478945025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b48075270 msgr2=0x7f2b48075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:20.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.227+0000 7f2b4d01a640 1 --2- 172.21.15.44:0/2478945025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b48075270 0x7f2b48075650 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009900 tx=0x7f2b3802a440 comp rx=0 tx=0).stop 2024-08-20T03:12:20.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.230+0000 7f2b4d01a640 1 -- 172.21.15.44:0/2478945025 shutdown_connections 2024-08-20T03:12:20.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.230+0000 7f2b4d01a640 1 --2- 172.21.15.44:0/2478945025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b48113680 0x7f2b48115a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.230+0000 7f2b4d01a640 1 --2- 172.21.15.44:0/2478945025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b48075b90 0x7f2b48113140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.230+0000 7f2b4d01a640 1 --2- 172.21.15.44:0/2478945025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b48075270 0x7f2b48075650 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.230+0000 7f2b4d01a640 1 -- 172.21.15.44:0/2478945025 >> 172.21.15.44:0/2478945025 conn(0x7f2b480fe350 msgr2=0x7f2b48100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:20.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.231+0000 7f2b4d01a640 1 -- 172.21.15.44:0/2478945025 shutdown_connections 2024-08-20T03:12:20.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.235+0000 7f2b4d01a640 1 -- 172.21.15.44:0/2478945025 wait complete. 2024-08-20T03:12:20.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.235+0000 7f2b4d01a640 1 Processor -- start 2024-08-20T03:12:20.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.236+0000 7f2b4d01a640 1 -- start start 2024-08-20T03:12:20.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.236+0000 7f2b4d01a640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b48075270 0x7f2b48112950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.236+0000 7f2b4d01a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b48075b90 0x7f2b48112e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.236+0000 7f2b4d01a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b48113680 0x7f2b4810bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.236+0000 7f2b4d01a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b48118fd0 con 0x7f2b48113680 2024-08-20T03:12:20.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.236+0000 7f2b4d01a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2b48118e50 con 0x7f2b48075b90 2024-08-20T03:12:20.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.236+0000 7f2b4d01a640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2b48119150 con 0x7f2b48075270 2024-08-20T03:12:20.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.236+0000 7f2b4c819640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b48113680 0x7f2b4810bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.236+0000 7f2b4c819640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b48113680 0x7f2b4810bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48114/0 (socket says 172.21.15.44:48114) 2024-08-20T03:12:20.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.236+0000 7f2b4c819640 1 -- 172.21.15.44:0/1729693181 learned_addr learned my addr 172.21.15.44:0/1729693181 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:20.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.236+0000 7f2b47fff640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b48075270 0x7f2b48112950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.237+0000 7f2b4c819640 1 -- 172.21.15.44:0/1729693181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b48075270 msgr2=0x7f2b48112950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:20.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.237+0000 7f2b4c819640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b48075270 0x7f2b48112950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.237+0000 7f2b4c819640 1 -- 172.21.15.44:0/1729693181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b48075b90 msgr2=0x7f2b48112e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-20T03:12:20.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.237+0000 7f2b4c819640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b48075b90 0x7f2b48112e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.237+0000 7f2b4c819640 1 -- 172.21.15.44:0/1729693181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4810c440 con 0x7f2b48113680 2024-08-20T03:12:20.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.237+0000 7f2b4c819640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b48113680 0x7f2b4810bb80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000e9d0 tx=0x7f2b4000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:20.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.238+0000 7f2b457fa640 1 -- 172.21.15.44:0/1729693181 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4000cd80 con 0x7f2b48113680 2024-08-20T03:12:20.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.238+0000 7f2b457fa640 1 -- 172.21.15.44:0/1729693181 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b40005020 con 0x7f2b48113680 2024-08-20T03:12:20.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.238+0000 7f2b457fa640 1 -- 172.21.15.44:0/1729693181 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b400116d0 con 0x7f2b48113680 2024-08-20T03:12:20.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.238+0000 7f2b4d01a640 1 -- 172.21.15.44:0/1729693181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4810c730 con 0x7f2b48113680 2024-08-20T03:12:20.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.238+0000 7f2b4d01a640 1 -- 172.21.15.44:0/1729693181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b48111790 con 0x7f2b48113680 2024-08-20T03:12:20.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.245+0000 7f2b457fa640 1 -- 172.21.15.44:0/1729693181 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2b40011870 con 0x7f2b48113680 2024-08-20T03:12:20.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.247+0000 7f2b457fa640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2b300b3f90 0x7f2b300b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.248+0000 7f2b50675740 1 Processor -- start 2024-08-20T03:12:20.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.249+0000 7f2b457fa640 1 -- 172.21.15.44:0/1729693181 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f2b400e3f20 con 0x7f2b48113680 2024-08-20T03:12:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.249+0000 7f2b50675740 1 -- start start 2024-08-20T03:12:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.249+0000 7f2b50675740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 0x55f05d5b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.249+0000 7f2b50675740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b5350 0x55f05d5c0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.249+0000 7f2b50675740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 0x55f05d5c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.249+0000 7f2b50675740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f05d52b930 con 0x55f05d5b4a30 2024-08-20T03:12:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.249+0000 7f2b50675740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f05d51ae30 con 0x55f05d5c12e0 2024-08-20T03:12:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.249+0000 7f2b50675740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f05d51ee70 con 0x55f05d5b5350 2024-08-20T03:12:20.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.250+0000 7f2b47fff640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2b300b3f90 0x7f2b300b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.250+0000 7f2b47fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 0x55f05d5b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.250+0000 7f2b47fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 0x55f05d5b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48116/0 (socket says 172.21.15.44:48116) 2024-08-20T03:12:20.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.250+0000 7f2b47fff640 1 -- 172.21.15.44:0/1743705012 learned_addr learned my addr 172.21.15.44:0/1743705012 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:20.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.250+0000 7f2b47fff640 1 -- 172.21.15.44:0/1743705012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b5350 msgr2=0x55f05d5c0da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:12:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.250+0000 7f2b47fff640 1 --2- 172.21.15.44:0/1743705012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b5350 0x55f05d5c0da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.250+0000 7f2b47fff640 1 -- 172.21.15.44:0/1743705012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 msgr2=0x55f05d5c36d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T03:12:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.250+0000 7f2b47fff640 1 --2- 172.21.15.44:0/1743705012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 0x55f05d5c36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.250+0000 7f2b47fff640 1 -- 172.21.15.44:0/1743705012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f05d27a650 con 0x55f05d5b4a30 2024-08-20T03:12:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.251+0000 7f2b47fff640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2b300b3f90 0x7f2b300b6450 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2b38004730 tx=0x7f2b380314b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:20.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.251+0000 7f2b47fff640 1 --2- 172.21.15.44:0/1743705012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 0x55f05d5b4e10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009850 tx=0x7f2b38047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83605a3148037777 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.254+0000 7f2b2e7fc640 1 -- 172.21.15.44:0/1743705012 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3804d070 con 0x55f05d5b4a30 2024-08-20T03:12:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.254+0000 7f2b2e7fc640 1 -- 172.21.15.44:0/1743705012 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b3803ddf0 con 0x55f05d5b4a30 2024-08-20T03:12:20.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.254+0000 7f2b2e7fc640 1 -- 172.21.15.44:0/1743705012 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38049db0 con 0x55f05d5b4a30 2024-08-20T03:12:20.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.255+0000 7f2b50675740 1 -- 172.21.15.44:0/1743705012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 msgr2=0x55f05d5b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:20.257 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.255+0000 7f2b50675740 1 --2- 172.21.15.44:0/1743705012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 0x55f05d5b4e10 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009850 tx=0x7f2b38047000 comp rx=0 tx=0).stop 2024-08-20T03:12:20.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.258+0000 7f2b50675740 1 -- 172.21.15.44:0/1743705012 shutdown_connections 2024-08-20T03:12:20.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.258+0000 7f2b50675740 1 --2- 172.21.15.44:0/1743705012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 0x55f05d5c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.258+0000 7f2b50675740 1 --2- 172.21.15.44:0/1743705012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b5350 0x55f05d5c0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.258+0000 7f2b50675740 1 --2- 172.21.15.44:0/1743705012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 0x55f05d5b4e10 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.258+0000 7f2b50675740 1 -- 172.21.15.44:0/1743705012 >> 172.21.15.44:0/1743705012 conn(0x55f05d5b2860 msgr2=0x55f05d5b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.264+0000 7f2b50675740 1 -- 172.21.15.44:0/1743705012 shutdown_connections 2024-08-20T03:12:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.264+0000 7f2b50675740 1 -- 172.21.15.44:0/1743705012 wait complete. 2024-08-20T03:12:20.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.264+0000 7f2b50675740 1 -- start start 2024-08-20T03:12:20.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.265+0000 7f2b50675740 1 Processor -- start 2024-08-20T03:12:20.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.265+0000 7f2b50675740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 0x55f05d5be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.266+0000 7f2b50675740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b5350 0x55f05d5be820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.266+0000 7f2b50675740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 0x55f05d5bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.266+0000 7f2b50675740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55f05d37d700 con 0x55f05d5b4a30 2024-08-20T03:12:20.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.266+0000 7f2b50675740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f05d4a1810 con 0x55f05d5c12e0 2024-08-20T03:12:20.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.266+0000 7f2b50675740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55f05d5d4f90 con 0x55f05d5b5350 2024-08-20T03:12:20.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.266+0000 7f2b50675740 10 client.0 _ll_register_callbacks cb 0x55f05d514700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:12:20.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.266+0000 7f2b4c819640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 0x55f05d5bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.266+0000 7f2b47fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 0x55f05d5be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.267+0000 7f2b47fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 0x55f05d5be2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48130/0 (socket says 172.21.15.44:48130) 2024-08-20T03:12:20.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.267+0000 7f2b47fff640 1 -- 172.21.15.44:0/805621371 learned_addr learned my addr 172.21.15.44:0/805621371 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:20.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.267+0000 7f2b477fe640 1 --2- 172.21.15.44:0/805621371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b5350 0x55f05d5be820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.267+0000 7f2b47fff640 1 -- 172.21.15.44:0/805621371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b5350 msgr2=0x55f05d5be820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:20.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.267+0000 7f2b47fff640 1 --2- 172.21.15.44:0/805621371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b5350 0x55f05d5be820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.267+0000 7f2b47fff640 1 -- 172.21.15.44:0/805621371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 msgr2=0x55f05d5bed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:20.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.267+0000 7f2b47fff640 1 --2- 172.21.15.44:0/805621371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 0x55f05d5bed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.267+0000 7f2b47fff640 1 -- 172.21.15.44:0/805621371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f05d5de410 con 0x55f05d5b4a30 2024-08-20T03:12:20.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.268+0000 7f2b47fff640 1 --2- 172.21.15.44:0/805621371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 0x55f05d5be2e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2b38031980 tx=0x7f2b3802a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68164acdfebabc8f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:20.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.268+0000 7f2b277fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:12:20.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.268+0000 7f2b277fe640 1 -- 172.21.15.44:0/805621371 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38056070 con 0x55f05d5b4a30 2024-08-20T03:12:20.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.268+0000 7f2b50675740 1 -- 172.21.15.44:0/805621371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f05d4ca150 con 0x55f05d5b4a30 2024-08-20T03:12:20.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.268+0000 7f2b277fe640 1 -- 172.21.15.44:0/805621371 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f2b3803d940 con 0x55f05d5b4a30 2024-08-20T03:12:20.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.269+0000 7f2b277fe640 1 -- 172.21.15.44:0/805621371 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2b38062050 con 0x55f05d5b4a30 2024-08-20T03:12:20.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.269+0000 7f2b50675740 10 client.17511 fetch_fsmap learned FSMap version 44 2024-08-20T03:12:20.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.269+0000 7f2b50675740 1 -- 172.21.15.44:0/805621371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f05d5e0da0 con 0x55f05d5b4a30 2024-08-20T03:12:20.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.271+0000 7f2b277fe640 1 -- 172.21.15.44:0/805621371 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b3803db80 con 0x55f05d5b4a30 2024-08-20T03:12:20.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.272+0000 7f2b277fe640 1 -- 172.21.15.44:0/805621371 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7f2b3803a040 con 0x55f05d5b4a30 2024-08-20T03:12:20.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.272+0000 7f2b50675740 10 client.17511 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T03:12:20.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.272+0000 7f2b50675740 10 client.17511 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:12:20.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.272+0000 7f2b50675740 20 client.17511 populate_metadata read hostname 'smithi044' 2024-08-20T03:12:20.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.272+0000 7f2b50675740 1 --2- 172.21.15.44:0/805621371 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55f05d5e20f0 0x55f05d5e44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.272+0000 7f2b50675740 4 client.17511 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:12:20.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.272+0000 7f2b50675740 1 -- 172.21.15.44:0/805621371 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f05d4ca150 con 0x55f05d5e20f0 2024-08-20T03:12:20.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.273+0000 7f2b477fe640 1 --2- 172.21.15.44:0/805621371 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55f05d5e20f0 0x55f05d5e44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.274+0000 7f2b477fe640 1 --2- 172.21.15.44:0/805621371 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55f05d5e20f0 0x55f05d5e44e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41d3e153f38e26c1 server_cookie=87201fca7f90262f in_seq=0 out_seq=0 2024-08-20T03:12:20.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.274+0000 7f2b277fe640 10 client.17511 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:12:20.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.275+0000 7f2b277fe640 1 -- 172.21.15.44:0/805621371 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f05d4ca150 con 0x55f05d5e20f0 2024-08-20T03:12:20.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.275+0000 7f2b277fe640 10 client.17511 handle_command_reply: tid=0 2024-08-20T03:12:20.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.275+0000 7f2b50675740 1 client.17511 shutdown 2024-08-20T03:12:20.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.278+0000 7f2b50675740 1 -- 172.21.15.44:0/805621371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 msgr2=0x55f05d5be2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:20.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.278+0000 7f2b50675740 1 --2- 172.21.15.44:0/805621371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 0x55f05d5be2e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2b38031980 tx=0x7f2b3802a440 comp rx=0 tx=0).stop 2024-08-20T03:12:20.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.280+0000 7f2b50675740 1 -- 172.21.15.44:0/805621371 shutdown_connections 2024-08-20T03:12:20.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.280+0000 7f2b50675740 1 --2- 172.21.15.44:0/805621371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 0x55f05d5bed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.280+0000 7f2b50675740 1 --2- 172.21.15.44:0/805621371 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55f05d5e20f0 0x55f05d5e44e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.280+0000 7f2b50675740 1 --2- 172.21.15.44:0/805621371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b5350 0x55f05d5be820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.280+0000 7f2b50675740 1 --2- 172.21.15.44:0/805621371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b4a30 0x55f05d5be2e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.280+0000 7f2b50675740 1 -- 172.21.15.44:0/805621371 >> 172.21.15.44:0/805621371 conn(0x55f05d5b2860 msgr2=0x55f05d5b8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:20.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.280+0000 7f2b277fe640 0 client.17511 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:12:20.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.280+0000 7f2b50675740 1 -- 172.21.15.44:0/805621371 shutdown_connections 2024-08-20T03:12:20.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.280+0000 7f2b50675740 1 -- 172.21.15.44:0/805621371 wait complete. 2024-08-20T03:12:20.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.282+0000 7f2b50675740 20 client.17511 trim_cache size 0 max 16384 2024-08-20T03:12:20.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.309+0000 7f2b50675740 1 Processor -- start 2024-08-20T03:12:20.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.309+0000 7f2b50675740 1 -- start start 2024-08-20T03:12:20.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.309+0000 7f2b50675740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b4a30 0x55f05d5be7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.309+0000 7f2b50675740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b5350 0x55f05d5bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.309+0000 7f2b50675740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 0x55f05d5e7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.309+0000 7f2b50675740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3400e870 con 0x55f05d5b5350 2024-08-20T03:12:20.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.309+0000 7f2b50675740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2b340104a0 con 0x55f05d5c12e0 2024-08-20T03:12:20.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.309+0000 7f2b50675740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3400efd0 con 0x55f05d5b4a30 2024-08-20T03:12:20.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.309+0000 7f2b47fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b4a30 0x55f05d5be7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.309+0000 7f2b47fff640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b4a30 0x55f05d5be7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46092/0 (socket says 172.21.15.44:46092) 2024-08-20T03:12:20.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.309+0000 7f2b47fff640 1 -- 172.21.15.44:0/2818405097 learned_addr learned my addr 172.21.15.44:0/2818405097 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:20.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.310+0000 7f2b477fe640 1 --2- 172.21.15.44:0/2818405097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b5350 0x55f05d5bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.310+0000 7f2b47fff640 1 -- 172.21.15.44:0/2818405097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 msgr2=0x55f05d5e7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:20.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.310+0000 7f2b47fff640 1 --2- 172.21.15.44:0/2818405097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 0x55f05d5e7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.310+0000 7f2b47fff640 1 -- 172.21.15.44:0/2818405097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b5350 msgr2=0x55f05d5bed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:20.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.310+0000 7f2b47fff640 1 --2- 172.21.15.44:0/2818405097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b5350 0x55f05d5bed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.313 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.310+0000 7f2b47fff640 1 -- 172.21.15.44:0/2818405097 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f05d5b30e0 con 0x55f05d5b4a30 2024-08-20T03:12:20.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.311+0000 7f2b47fff640 1 --2- 172.21.15.44:0/2818405097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b4a30 0x55f05d5be7e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009850 tx=0x7f2b380046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ceb49a322b3079d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:20.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.311+0000 7f2b2dffb640 1 -- 172.21.15.44:0/2818405097 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3806b070 con 0x55f05d5b4a30 2024-08-20T03:12:20.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.312+0000 7f2b2dffb640 1 -- 172.21.15.44:0/2818405097 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b3803ddf0 con 0x55f05d5b4a30 2024-08-20T03:12:20.319 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.312+0000 7f2b50675740 1 -- 172.21.15.44:0/2818405097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b4a30 msgr2=0x55f05d5be7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:20.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.312+0000 7f2b50675740 1 --2- 172.21.15.44:0/2818405097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b4a30 0x55f05d5be7e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009850 tx=0x7f2b380046c0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.313+0000 7f2b50675740 1 -- 172.21.15.44:0/2818405097 shutdown_connections 2024-08-20T03:12:20.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.313+0000 7f2b50675740 1 --2- 172.21.15.44:0/2818405097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5c12e0 0x55f05d5e7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.313+0000 7f2b50675740 1 --2- 172.21.15.44:0/2818405097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b5350 0x55f05d5bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.313+0000 7f2b50675740 1 --2- 172.21.15.44:0/2818405097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5b4a30 0x55f05d5be7e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.320 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.313+0000 7f2b50675740 1 -- 172.21.15.44:0/2818405097 >> 172.21.15.44:0/2818405097 conn(0x55f05d5b2860 msgr2=0x55f05d5c0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:20.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.313+0000 7f2b50675740 1 -- 172.21.15.44:0/2818405097 shutdown_connections 2024-08-20T03:12:20.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.313+0000 7f2b50675740 1 -- 172.21.15.44:0/2818405097 wait complete. 2024-08-20T03:12:20.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.314+0000 7f2b50675740 1 -- start start 2024-08-20T03:12:20.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.314+0000 7f2b50675740 1 Processor -- start 2024-08-20T03:12:20.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.314+0000 7f2b50675740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5b4a30 0x55f05d5e7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.314+0000 7f2b50675740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b5350 0x55f05d5e5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.321 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.314+0000 7f2b50675740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5c12e0 0x55f05d5e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.315+0000 7f2b50675740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3400fdd0 con 0x55f05d5b5350 2024-08-20T03:12:20.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.315+0000 7f2b50675740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3400f6d0 con 0x55f05d5b4a30 2024-08-20T03:12:20.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.315+0000 7f2b50675740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3400c870 con 0x55f05d5c12e0 2024-08-20T03:12:20.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.315+0000 7f2b50675740 10 client.0 _ll_register_callbacks cb 0x7f2b380317a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:12:20.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.315+0000 7f2b477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b5350 0x55f05d5e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.322 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.315+0000 7f2b477fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b5350 0x55f05d5e5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48146/0 (socket says 172.21.15.44:48146) 2024-08-20T03:12:20.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.315+0000 7f2b477fe640 1 -- 172.21.15.44:0/2899030560 learned_addr learned my addr 172.21.15.44:0/2899030560 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:20.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.315+0000 7f2b47fff640 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5b4a30 0x55f05d5e7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.315+0000 7f2b4c819640 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5c12e0 0x55f05d5e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.316+0000 7f2b477fe640 1 -- 172.21.15.44:0/2899030560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5c12e0 msgr2=0x55f05d5e57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:20.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.316+0000 7f2b477fe640 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5c12e0 0x55f05d5e57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.323 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.316+0000 7f2b477fe640 1 -- 172.21.15.44:0/2899030560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5b4a30 msgr2=0x55f05d5e7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:20.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.316+0000 7f2b477fe640 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5b4a30 0x55f05d5e7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.316+0000 7f2b477fe640 1 -- 172.21.15.44:0/2899030560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f05d5acb10 con 0x55f05d5b5350 2024-08-20T03:12:20.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.316+0000 7f2b477fe640 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b5350 0x55f05d5e5190 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c006dc0 tx=0x7f2b3c00ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30d71dd232acc54 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:20.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.316+0000 7f2b277fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:12:20.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.316+0000 7f2b277fe640 1 -- 172.21.15.44:0/2899030560 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c005c40 con 0x55f05d5b5350 2024-08-20T03:12:20.324 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.316+0000 7f2b50675740 1 -- 172.21.15.44:0/2899030560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b3803ddf0 con 0x55f05d5b5350 2024-08-20T03:12:20.325 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.316+0000 7f2b277fe640 1 -- 172.21.15.44:0/2899030560 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f2b3c00f040 con 0x55f05d5b5350 2024-08-20T03:12:20.326 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.317+0000 7f2b277fe640 1 -- 172.21.15.44:0/2899030560 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f2b3c005c40 con 0x55f05d5b5350 2024-08-20T03:12:20.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.317+0000 7f2b50675740 10 client.17514 fetch_fsmap learned FSMap version 44 2024-08-20T03:12:20.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.317+0000 7f2b50675740 1 -- 172.21.15.44:0/2899030560 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f05d5b32e0 con 0x55f05d5b5350 2024-08-20T03:12:20.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.317+0000 7f2b277fe640 1 -- 172.21.15.44:0/2899030560 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b3803ddf0 con 0x55f05d5b5350 2024-08-20T03:12:20.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.318+0000 7f2b277fe640 1 -- 172.21.15.44:0/2899030560 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7f2b3c006050 con 0x55f05d5b5350 2024-08-20T03:12:20.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.318+0000 7f2b50675740 10 client.17514 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T03:12:20.327 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.318+0000 7f2b50675740 10 client.17514 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-08-20T03:12:20.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.318+0000 7f2b50675740 20 client.17514 populate_metadata read hostname 'smithi044' 2024-08-20T03:12:20.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.318+0000 7f2b50675740 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55f05d64cfd0 0x55f05d64f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:20.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.318+0000 7f2b50675740 4 client.17514 mds_command: new command op to 24589 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:12:20.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.318+0000 7f2b50675740 1 -- 172.21.15.44:0/2899030560 --> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f05d52d880 con 0x55f05d64cfd0 2024-08-20T03:12:20.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.320+0000 7f2b47fff640 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55f05d64cfd0 0x55f05d64f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:20.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.324+0000 7f2b47fff640 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55f05d64cfd0 0x55f05d64f3f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87fe2056da03f38a server_cookie=1f55bde9eefad7a in_seq=0 out_seq=0 2024-08-20T03:12:20.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.325+0000 7f2b277fe640 10 client.17514 ms_handle_connect on v2:172.21.15.165:6836/3478633280 2024-08-20T03:12:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.326+0000 7f2b277fe640 1 -- 172.21.15.44:0/2899030560 <== mds.2 v2:172.21.15.165:6836/3478633280 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f05d52d880 con 0x55f05d64cfd0 2024-08-20T03:12:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.326+0000 7f2b277fe640 10 client.17514 handle_command_reply: tid=0 2024-08-20T03:12:20.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.326+0000 7f2b50675740 1 client.17514 shutdown 2024-08-20T03:12:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.329+0000 7f2b50675740 1 -- 172.21.15.44:0/2899030560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b5350 msgr2=0x55f05d5e5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.329+0000 7f2b50675740 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b5350 0x55f05d5e5190 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c006dc0 tx=0x7f2b3c00ba00 comp rx=0 tx=0).stop 2024-08-20T03:12:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.329+0000 7f2b50675740 1 -- 172.21.15.44:0/2899030560 shutdown_connections 2024-08-20T03:12:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.329+0000 7f2b50675740 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.165:6836/3478633280,v1:172.21.15.165:6837/3478633280] conn(0x55f05d64cfd0 0x55f05d64f3f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.332 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.329+0000 7f2b50675740 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f05d5c12e0 0x55f05d5e57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.329+0000 7f2b50675740 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f05d5b5350 0x55f05d5e5190 secure :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c006dc0 tx=0x7f2b3c00ba00 comp rx=0 tx=0).stop 2024-08-20T03:12:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.329+0000 7f2b50675740 1 --2- 172.21.15.44:0/2899030560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f05d5b4a30 0x55f05d5e7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.329+0000 7f2b50675740 1 -- 172.21.15.44:0/2899030560 >> 172.21.15.44:0/2899030560 conn(0x55f05d5b2860 msgr2=0x55f05d5b7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:20.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.330+0000 7f2b277fe640 0 client.17514 ms_handle_reset on v2:172.21.15.165:6836/3478633280 2024-08-20T03:12:20.336 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:12:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.334+0000 7f2b50675740 1 -- 172.21.15.44:0/2899030560 shutdown_connections 2024-08-20T03:12:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.334+0000 7f2b50675740 1 -- 172.21.15.44:0/2899030560 wait complete. 2024-08-20T03:12:20.337 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.334+0000 7f2b50675740 20 client.17514 trim_cache size 0 max 16384 2024-08-20T03:12:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.335+0000 7f2b4d01a640 1 -- 172.21.15.44:0/1729693181 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2b300b3f90 msgr2=0x7f2b300b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.335+0000 7f2b4d01a640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2b300b3f90 0x7f2b300b6450 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2b38004730 tx=0x7f2b380314b0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.335+0000 7f2b4d01a640 1 -- 172.21.15.44:0/1729693181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b48113680 msgr2=0x7f2b4810bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.335+0000 7f2b4d01a640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b48113680 0x7f2b4810bb80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000e9d0 tx=0x7f2b4000ee90 comp rx=0 tx=0).stop 2024-08-20T03:12:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.336+0000 7f2b4d01a640 1 -- 172.21.15.44:0/1729693181 shutdown_connections 2024-08-20T03:12:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.336+0000 7f2b4d01a640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2b300b3f90 0x7f2b300b6450 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.336+0000 7f2b4d01a640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b48113680 0x7f2b4810bb80 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.336+0000 7f2b4d01a640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b48075b90 0x7f2b48112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.336+0000 7f2b4d01a640 1 --2- 172.21.15.44:0/1729693181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b48075270 0x7f2b48112950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.336+0000 7f2b4d01a640 1 -- 172.21.15.44:0/1729693181 >> 172.21.15.44:0/1729693181 conn(0x7f2b480fe350 msgr2=0x7f2b480fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.336+0000 7f2b4d01a640 1 -- 172.21.15.44:0/1729693181 shutdown_connections 2024-08-20T03:12:20.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:20.336+0000 7f2b4d01a640 1 -- 172.21.15.44:0/1729693181 wait complete. 2024-08-20T03:12:20.341 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:20 smithi044 ceph-mon[18790]: pgmap v622: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 806 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-08-20T03:12:20.385 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:20 smithi165 ceph-mon[22357]: pgmap v622: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 806 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-08-20T03:12:20.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:20 smithi099 ceph-mon[21984]: pgmap v622: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 806 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-08-20T03:12:21.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T03:12:21.571 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:22.361 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:22 smithi044 ceph-mon[18790]: pgmap v623: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 802 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 193 B/s wr, 9 op/s 2024-08-20T03:12:22.385 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:22 smithi165 ceph-mon[22357]: pgmap v623: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 802 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 193 B/s wr, 9 op/s 2024-08-20T03:12:22.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:22 smithi099 ceph-mon[21984]: pgmap v623: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 802 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 193 B/s wr, 9 op/s 2024-08-20T03:12:22.628 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.626+0000 7fa14159a640 1 -- 172.21.15.44:0/2430070409 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c02f310 con 0x7fa144106bd0 2024-08-20T03:12:22.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.626+0000 7fa148c8f640 1 -- 172.21.15.44:0/2430070409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa144106bd0 msgr2=0x7fa144107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:22.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.626+0000 7fa148c8f640 1 --2- 172.21.15.44:0/2430070409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa144106bd0 0x7fa144107020 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009ca0 tx=0x7fa13c02a8c0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.626+0000 7fa148c8f640 1 -- 172.21.15.44:0/2430070409 shutdown_connections 2024-08-20T03:12:22.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.626+0000 7fa148c8f640 1 --2- 172.21.15.44:0/2430070409 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa144106bd0 0x7fa144107020 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.626+0000 7fa148c8f640 1 --2- 172.21.15.44:0/2430070409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1441020b0 0x7fa144106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.626+0000 7fa148c8f640 1 --2- 172.21.15.44:0/2430070409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa144101790 0x7fa144101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.629 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.626+0000 7fa148c8f640 1 -- 172.21.15.44:0/2430070409 >> 172.21.15.44:0/2430070409 conn(0x7fa1440fd400 msgr2=0x7fa1440ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:22.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.628+0000 7fa148c8f640 1 -- 172.21.15.44:0/2430070409 shutdown_connections 2024-08-20T03:12:22.630 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.628+0000 7fa148c8f640 1 -- 172.21.15.44:0/2430070409 wait complete. 2024-08-20T03:12:22.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.629+0000 7fa148c8f640 1 Processor -- start 2024-08-20T03:12:22.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.629+0000 7fa148c8f640 1 -- start start 2024-08-20T03:12:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.629+0000 7fa148c8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa144101790 0x7fa144115750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.629+0000 7fa148c8f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1441020b0 0x7fa144115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.629+0000 7fa148c8f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa144106bd0 0x7fa14410ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.629+0000 7fa148c8f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa1441190a0 con 0x7fa144101790 2024-08-20T03:12:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.629+0000 7fa148c8f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa144118f20 con 0x7fa144106bd0 2024-08-20T03:12:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.629+0000 7fa148c8f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa144119220 con 0x7fa1441020b0 2024-08-20T03:12:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.631+0000 7fa141d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1441020b0 0x7fa144115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.631+0000 7fa14259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa144101790 0x7fa144115750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.631+0000 7fa14259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa144101790 0x7fa144115750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48158/0 (socket says 172.21.15.44:48158) 2024-08-20T03:12:22.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.631+0000 7fa14259c640 1 -- 172.21.15.44:0/3850455263 learned_addr learned my addr 172.21.15.44:0/3850455263 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:22.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.632+0000 7fa142d9d640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa144106bd0 0x7fa14410ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.632+0000 7fa141d9b640 1 -- 172.21.15.44:0/3850455263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa144106bd0 msgr2=0x7fa14410ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.632+0000 7fa141d9b640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa144106bd0 0x7fa14410ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.632+0000 7fa141d9b640 1 -- 172.21.15.44:0/3850455263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa144101790 msgr2=0x7fa144115750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.632+0000 7fa141d9b640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa144101790 0x7fa144115750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.632+0000 7fa141d9b640 1 -- 172.21.15.44:0/3850455263 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14410f360 con 0x7fa1441020b0 2024-08-20T03:12:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.633+0000 7fa141d9b640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1441020b0 0x7fa144115c90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa13800d650 tx=0x7fa13800db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.633+0000 7fa1337fe640 1 -- 172.21.15.44:0/3850455263 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138017070 con 0x7fa1441020b0 2024-08-20T03:12:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.633+0000 7fa148c8f640 1 -- 172.21.15.44:0/3850455263 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14410f5f0 con 0x7fa1441020b0 2024-08-20T03:12:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.633+0000 7fa148c8f640 1 -- 172.21.15.44:0/3850455263 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1441145e0 con 0x7fa1441020b0 2024-08-20T03:12:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.633+0000 7fa1337fe640 1 -- 172.21.15.44:0/3850455263 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa138009c70 con 0x7fa1441020b0 2024-08-20T03:12:22.636 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.634+0000 7fa1337fe640 1 -- 172.21.15.44:0/3850455263 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138003b30 con 0x7fa1441020b0 2024-08-20T03:12:22.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.637+0000 7fa14b088740 1 Processor -- start 2024-08-20T03:12:22.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.638+0000 7fa14b088740 1 -- start start 2024-08-20T03:12:22.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.639+0000 7fa14b088740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb640f880 0x561cb640fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.639+0000 7fa14b088740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb641bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.639+0000 7fa14b088740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb641c130 0x561cb641e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.639+0000 7fa14b088740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561cb6386930 con 0x561cb640f880 2024-08-20T03:12:22.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.639+0000 7fa14b088740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561cb6375e30 con 0x561cb641c130 2024-08-20T03:12:22.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.639+0000 7fa14b088740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561cb6379e70 con 0x561cb64101a0 2024-08-20T03:12:22.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.639+0000 7fa14259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb641bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.639+0000 7fa14259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb641bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46130/0 (socket says 172.21.15.44:46130) 2024-08-20T03:12:22.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.639+0000 7fa14259c640 1 -- 172.21.15.44:0/813091273 learned_addr learned my addr 172.21.15.44:0/813091273 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:22.642 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.639+0000 7fa142d9d640 1 --2- 172.21.15.44:0/813091273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb640f880 0x561cb640fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.640+0000 7fa142d9d640 1 -- 172.21.15.44:0/813091273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 msgr2=0x561cb641bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:22.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.640+0000 7fa142d9d640 1 --2- 172.21.15.44:0/813091273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb641bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.640+0000 7fa142d9d640 1 -- 172.21.15.44:0/813091273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb641c130 msgr2=0x561cb641e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:22.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.640+0000 7fa142d9d640 1 --2- 172.21.15.44:0/813091273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb641c130 0x561cb641e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.640+0000 7fa142d9d640 1 -- 172.21.15.44:0/813091273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb60d5650 con 0x561cb640f880 2024-08-20T03:12:22.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.640+0000 7fa1337fe640 1 -- 172.21.15.44:0/3850455263 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fa138003cd0 con 0x7fa1441020b0 2024-08-20T03:12:22.643 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.641+0000 7fa142d9d640 1 --2- 172.21.15.44:0/813091273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb640f880 0x561cb640fc60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa13c02ade0 tx=0x7fa13c02f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b78957e255d5d16 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:22.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.641+0000 7fa130ff9640 1 -- 172.21.15.44:0/813091273 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c031db0 con 0x561cb640f880 2024-08-20T03:12:22.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.642+0000 7fa130ff9640 1 -- 172.21.15.44:0/813091273 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa13c036070 con 0x561cb640f880 2024-08-20T03:12:22.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.642+0000 7fa130ff9640 1 -- 172.21.15.44:0/813091273 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0042a0 con 0x561cb640f880 2024-08-20T03:12:22.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.642+0000 7fa1337fe640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa12c0b3e40 0x7fa12c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.643+0000 7fa1337fe640 1 -- 172.21.15.44:0/3850455263 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fa1380e3aa0 con 0x7fa1441020b0 2024-08-20T03:12:22.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.643+0000 7fa14259c640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa12c0b3e40 0x7fa12c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.644+0000 7fa14b088740 1 -- 172.21.15.44:0/813091273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb640f880 msgr2=0x561cb640fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:22.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.644+0000 7fa14b088740 1 --2- 172.21.15.44:0/813091273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb640f880 0x561cb640fc60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa13c02ade0 tx=0x7fa13c02f3c0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.644+0000 7fa14259c640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa12c0b3e40 0x7fa12c0b6300 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x561cb6412a00 tx=0x7fa134005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:22.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.645+0000 7fa14b088740 1 -- 172.21.15.44:0/813091273 shutdown_connections 2024-08-20T03:12:22.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.645+0000 7fa14b088740 1 --2- 172.21.15.44:0/813091273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb641c130 0x561cb641e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.645+0000 7fa14b088740 1 --2- 172.21.15.44:0/813091273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb641bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.645+0000 7fa14b088740 1 --2- 172.21.15.44:0/813091273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb640f880 0x561cb640fc60 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.645+0000 7fa14b088740 1 -- 172.21.15.44:0/813091273 >> 172.21.15.44:0/813091273 conn(0x561cb640d6b0 msgr2=0x561cb640e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.645+0000 7fa14b088740 1 -- 172.21.15.44:0/813091273 shutdown_connections 2024-08-20T03:12:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.645+0000 7fa14b088740 1 -- 172.21.15.44:0/813091273 wait complete. 2024-08-20T03:12:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.645+0000 7fa14b088740 1 -- start start 2024-08-20T03:12:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.646+0000 7fa14b088740 1 Processor -- start 2024-08-20T03:12:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.646+0000 7fa14b088740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb640f880 0x561cb6419120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.646+0000 7fa14b088740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb64101a0 0x561cb6419660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.646+0000 7fa14b088740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb641c130 0x561cb6419ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.646+0000 7fa14b088740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x561cb61d8700 con 0x561cb641c130 2024-08-20T03:12:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.646+0000 7fa14b088740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561cb62fc810 con 0x561cb64101a0 2024-08-20T03:12:22.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.646+0000 7fa14b088740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x561cb641ec00 con 0x561cb640f880 2024-08-20T03:12:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.646+0000 7fa14259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb640f880 0x561cb6419120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.646+0000 7fa14259c640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb640f880 0x561cb6419120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46146/0 (socket says 172.21.15.44:46146) 2024-08-20T03:12:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.646+0000 7fa14b088740 10 client.0 _ll_register_callbacks cb 0x561cb636f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:12:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.646+0000 7fa14259c640 1 -- 172.21.15.44:0/3861012564 learned_addr learned my addr 172.21.15.44:0/3861012564 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa142d9d640 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb641c130 0x561cb6419ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa141d9b640 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb64101a0 0x561cb6419660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa14259c640 1 -- 172.21.15.44:0/3861012564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb64101a0 msgr2=0x561cb6419660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa14259c640 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb64101a0 0x561cb6419660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa14259c640 1 -- 172.21.15.44:0/3861012564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb641c130 msgr2=0x561cb6419ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa14259c640 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb641c130 0x561cb6419ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa14259c640 1 -- 172.21.15.44:0/3861012564 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb6428060 con 0x561cb640f880 2024-08-20T03:12:22.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa142d9d640 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb641c130 0x561cb6419ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:12:22.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa14259c640 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb640f880 0x561cb6419120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa1340209e0 tx=0x7fa134004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7086488f3df423a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:22.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa129ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:12:22.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa129ffb640 1 -- 172.21.15.44:0/3861012564 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13402d060 con 0x561cb640f880 2024-08-20T03:12:22.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa14b088740 1 -- 172.21.15.44:0/3861012564 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561cb6325150 con 0x561cb640f880 2024-08-20T03:12:22.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.647+0000 7fa129ffb640 1 -- 172.21.15.44:0/3861012564 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fa134018030 con 0x561cb640f880 2024-08-20T03:12:22.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.648+0000 7fa129ffb640 1 -- 172.21.15.44:0/3861012564 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa134011b50 con 0x561cb640f880 2024-08-20T03:12:22.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.648+0000 7fa14b088740 10 client.26851 fetch_fsmap learned FSMap version 44 2024-08-20T03:12:22.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.648+0000 7fa14b088740 1 -- 172.21.15.44:0/3861012564 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561cb642aa70 con 0x561cb640f880 2024-08-20T03:12:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.648+0000 7fa129ffb640 1 -- 172.21.15.44:0/3861012564 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561cb6325150 con 0x561cb640f880 2024-08-20T03:12:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.648+0000 7fa129ffb640 1 -- 172.21.15.44:0/3861012564 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7fa13401d3f0 con 0x561cb640f880 2024-08-20T03:12:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.648+0000 7fa14b088740 10 client.26851 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T03:12:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.648+0000 7fa14b088740 10 client.26851 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:12:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.648+0000 7fa14b088740 20 client.26851 populate_metadata read hostname 'smithi044' 2024-08-20T03:12:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.648+0000 7fa14b088740 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561cb642c1c0 0x561cb642e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.649+0000 7fa14b088740 4 client.26851 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:12:22.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.649+0000 7fa14b088740 1 -- 172.21.15.44:0/3861012564 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cb6388880 con 0x561cb642c1c0 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.649+0000 7fa141d9b640 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561cb642c1c0 0x561cb642e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.649+0000 7fa141d9b640 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561cb642c1c0 0x561cb642e5b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=514203cc4508dd9 server_cookie=da7ee86899a059f7 in_seq=0 out_seq=0 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.649+0000 7fa129ffb640 10 client.26851 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.650+0000 7fa129ffb640 1 -- 172.21.15.44:0/3861012564 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa1380b3060 con 0x561cb642c1c0 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.650+0000 7fa129ffb640 10 client.26851 handle_command_reply: tid=0 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.650+0000 7fa14b088740 1 client.26851 shutdown 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.651+0000 7fa14b088740 1 -- 172.21.15.44:0/3861012564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb640f880 msgr2=0x561cb6419120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.651+0000 7fa14b088740 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb640f880 0x561cb6419120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa1340209e0 tx=0x7fa134004730 comp rx=0 tx=0).stop 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.651+0000 7fa14b088740 1 -- 172.21.15.44:0/3861012564 shutdown_connections 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.651+0000 7fa14b088740 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561cb642c1c0 0x561cb642e5b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.651+0000 7fa14b088740 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb641c130 0x561cb6419ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.651+0000 7fa14b088740 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb64101a0 0x561cb6419660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.651+0000 7fa14b088740 1 --2- 172.21.15.44:0/3861012564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb640f880 0x561cb6419120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.651+0000 7fa14b088740 1 -- 172.21.15.44:0/3861012564 >> 172.21.15.44:0/3861012564 conn(0x561cb640d6b0 msgr2=0x561cb6413250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:22.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.651+0000 7fa129ffb640 0 client.26851 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:12:22.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.651+0000 7fa14b088740 1 -- 172.21.15.44:0/3861012564 shutdown_connections 2024-08-20T03:12:22.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.651+0000 7fa14b088740 1 -- 172.21.15.44:0/3861012564 wait complete. 2024-08-20T03:12:22.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.651+0000 7fa14b088740 20 client.26851 trim_cache size 0 max 16384 2024-08-20T03:12:22.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.677+0000 7fa14b088740 1 Processor -- start 2024-08-20T03:12:22.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.677+0000 7fa14b088740 1 -- start start 2024-08-20T03:12:22.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.677+0000 7fa14b088740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb640f880 0x561cb640c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.677+0000 7fa14b088740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb6433c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.677+0000 7fa14b088740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb641c130 0x561cb6434150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.677+0000 7fa14b088740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa12400e750 con 0x561cb641c130 2024-08-20T03:12:22.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.677+0000 7fa14b088740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa124010380 con 0x561cb640f880 2024-08-20T03:12:22.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.677+0000 7fa14b088740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa12400eeb0 con 0x561cb64101a0 2024-08-20T03:12:22.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.677+0000 7fa142d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb641c130 0x561cb6434150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.678+0000 7fa14259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb640f880 0x561cb640c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.678+0000 7fa14259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb640f880 0x561cb640c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:60984/0 (socket says 172.21.15.44:60984) 2024-08-20T03:12:22.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.678+0000 7fa14259c640 1 -- 172.21.15.44:0/2380770820 learned_addr learned my addr 172.21.15.44:0/2380770820 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:22.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.678+0000 7fa142d9d640 1 -- 172.21.15.44:0/2380770820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 msgr2=0x561cb6433c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T03:12:22.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.678+0000 7fa142d9d640 1 --2- 172.21.15.44:0/2380770820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb6433c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.678+0000 7fa142d9d640 1 -- 172.21.15.44:0/2380770820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb640f880 msgr2=0x561cb640c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:22.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.678+0000 7fa142d9d640 1 --2- 172.21.15.44:0/2380770820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb640f880 0x561cb640c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.678+0000 7fa142d9d640 1 -- 172.21.15.44:0/2380770820 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cb640ffd0 con 0x561cb641c130 2024-08-20T03:12:22.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.679+0000 7fa142d9d640 1 --2- 172.21.15.44:0/2380770820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb641c130 0x561cb6434150 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa13c004af0 tx=0x7fa13c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809c35a39c9c7b3a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:22.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.680+0000 7fa12bfff640 1 -- 172.21.15.44:0/2380770820 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c031db0 con 0x561cb641c130 2024-08-20T03:12:22.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.680+0000 7fa12bfff640 1 -- 172.21.15.44:0/2380770820 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa13c036040 con 0x561cb641c130 2024-08-20T03:12:22.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.680+0000 7fa12bfff640 1 -- 172.21.15.44:0/2380770820 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0042a0 con 0x561cb641c130 2024-08-20T03:12:22.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.681+0000 7fa14b088740 1 -- 172.21.15.44:0/2380770820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb641c130 msgr2=0x561cb6434150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.681+0000 7fa14b088740 1 --2- 172.21.15.44:0/2380770820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb641c130 0x561cb6434150 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa13c004af0 tx=0x7fa13c009b20 comp rx=0 tx=0).stop 2024-08-20T03:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.682+0000 7fa14259c640 1 --2- 172.21.15.44:0/2380770820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb640f880 0x561cb640c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.682+0000 7fa14b088740 1 -- 172.21.15.44:0/2380770820 shutdown_connections 2024-08-20T03:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.682+0000 7fa14b088740 1 --2- 172.21.15.44:0/2380770820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb641c130 0x561cb6434150 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.682+0000 7fa14b088740 1 --2- 172.21.15.44:0/2380770820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb6433c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.682+0000 7fa14b088740 1 --2- 172.21.15.44:0/2380770820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb640f880 0x561cb640c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.684 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.682+0000 7fa14b088740 1 -- 172.21.15.44:0/2380770820 >> 172.21.15.44:0/2380770820 conn(0x561cb640d6b0 msgr2=0x561cb640bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:22.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.682+0000 7fa14b088740 1 -- 172.21.15.44:0/2380770820 shutdown_connections 2024-08-20T03:12:22.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.683+0000 7fa14b088740 1 -- 172.21.15.44:0/2380770820 wait complete. 2024-08-20T03:12:22.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.683+0000 7fa14b088740 1 -- start start 2024-08-20T03:12:22.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.683+0000 7fa14b088740 1 Processor -- start 2024-08-20T03:12:22.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.683+0000 7fa14b088740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb640f880 0x561cb6492180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.684+0000 7fa14b088740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb6423ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.684+0000 7fa14b088740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb641c130 0x561cb6424430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.684+0000 7fa14b088740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa12400fcb0 con 0x561cb640f880 2024-08-20T03:12:22.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.684+0000 7fa14b088740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa12400f5b0 con 0x561cb641c130 2024-08-20T03:12:22.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.684+0000 7fa14b088740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fa12400c750 con 0x561cb64101a0 2024-08-20T03:12:22.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.684+0000 7fa14b088740 10 client.0 _ll_register_callbacks cb 0x7fa1340208a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:12:22.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.685+0000 7fa141d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb6423ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.686+0000 7fa141d9b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb6423ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46160/0 (socket says 172.21.15.44:46160) 2024-08-20T03:12:22.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.686+0000 7fa141d9b640 1 -- 172.21.15.44:0/2163424020 learned_addr learned my addr 172.21.15.44:0/2163424020 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:22.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.686+0000 7fa142d9d640 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb641c130 0x561cb6424430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.687+0000 7fa14259c640 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb640f880 0x561cb6492180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.687+0000 7fa141d9b640 1 -- 172.21.15.44:0/2163424020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb641c130 msgr2=0x561cb6424430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:22.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.687+0000 7fa141d9b640 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb641c130 0x561cb6424430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.687+0000 7fa141d9b640 1 -- 172.21.15.44:0/2163424020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb640f880 msgr2=0x561cb6492180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:22.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.687+0000 7fa141d9b640 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb640f880 0x561cb6492180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.687+0000 7fa141d9b640 1 -- 172.21.15.44:0/2163424020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cb64249f0 con 0x561cb64101a0 2024-08-20T03:12:22.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.687+0000 7fa141d9b640 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb6423ef0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa138004610 tx=0x7fa1380a9ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89b7ce505db72d94 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:22.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.688+0000 7fa129ffb640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-08-20T03:12:22.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.689+0000 7fa129ffb640 1 -- 172.21.15.44:0/2163424020 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138020040 con 0x561cb64101a0 2024-08-20T03:12:22.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.689+0000 7fa129ffb640 1 -- 172.21.15.44:0/2163424020 <== mon.1 v2:172.21.15.165:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fa1380a6900 con 0x561cb64101a0 2024-08-20T03:12:22.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.689+0000 7fa14b088740 1 -- 172.21.15.44:0/2163424020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa13c036040 con 0x561cb64101a0 2024-08-20T03:12:22.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.689+0000 7fa129ffb640 1 -- 172.21.15.44:0/2163424020 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa1380aad70 con 0x561cb64101a0 2024-08-20T03:12:22.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.690+0000 7fa14b088740 10 client.26854 fetch_fsmap learned FSMap version 44 2024-08-20T03:12:22.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.690+0000 7fa14b088740 1 -- 172.21.15.44:0/2163424020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561cb6492bc0 con 0x561cb64101a0 2024-08-20T03:12:22.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.690+0000 7fa129ffb640 1 -- 172.21.15.44:0/2163424020 <== mon.1 v2:172.21.15.165:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa13c036040 con 0x561cb64101a0 2024-08-20T03:12:22.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.691+0000 7fa129ffb640 1 -- 172.21.15.44:0/2163424020 <== mon.1 v2:172.21.15.165:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7fa1380e1430 con 0x561cb64101a0 2024-08-20T03:12:22.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.691+0000 7fa14b088740 10 client.26854 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T03:12:22.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.691+0000 7fa14b088740 10 client.26854 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-08-20T03:12:22.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.691+0000 7fa14b088740 20 client.26854 populate_metadata read hostname 'smithi044' 2024-08-20T03:12:22.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.691+0000 7fa14b088740 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561cb64933a0 0x561cb64957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:22.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.691+0000 7fa14b088740 4 client.26854 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:12:22.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.691+0000 7fa14b088740 1 -- 172.21.15.44:0/2163424020 --> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cb6384320 con 0x561cb64933a0 2024-08-20T03:12:22.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.692+0000 7fa14259c640 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561cb64933a0 0x561cb64957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:22.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.693+0000 7fa14259c640 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561cb64933a0 0x561cb64957c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49f6436965afcb76 server_cookie=d60ef32506c1a3c1 in_seq=0 out_seq=0 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.693+0000 7fa129ffb640 10 client.26854 ms_handle_connect on v2:172.21.15.99:6838/1272675547 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.693+0000 7fa129ffb640 1 -- 172.21.15.44:0/2163424020 <== mds.1 v2:172.21.15.99:6838/1272675547 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cb6384320 con 0x561cb64933a0 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.693+0000 7fa129ffb640 10 client.26854 handle_command_reply: tid=0 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.693+0000 7fa14b088740 1 client.26854 shutdown 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.693+0000 7fa14b088740 1 -- 172.21.15.44:0/2163424020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 msgr2=0x561cb6423ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.693+0000 7fa14b088740 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb6423ef0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa138004610 tx=0x7fa1380a9ee0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.694+0000 7fa14b088740 1 -- 172.21.15.44:0/2163424020 shutdown_connections 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.694+0000 7fa14b088740 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.99:6838/1272675547,v1:172.21.15.99:6839/1272675547] conn(0x561cb64933a0 0x561cb64957c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.694+0000 7fa14b088740 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561cb641c130 0x561cb6424430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.694+0000 7fa129ffb640 0 client.26854 ms_handle_reset on v2:172.21.15.99:6838/1272675547 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.694+0000 7fa14b088740 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561cb64101a0 0x561cb6423ef0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.694+0000 7fa14b088740 1 --2- 172.21.15.44:0/2163424020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561cb640f880 0x561cb6492180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.694+0000 7fa14b088740 1 -- 172.21.15.44:0/2163424020 >> 172.21.15.44:0/2163424020 conn(0x561cb640d6b0 msgr2=0x561cb6417570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:22.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.694+0000 7fa14b088740 1 -- 172.21.15.44:0/2163424020 shutdown_connections 2024-08-20T03:12:22.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.694+0000 7fa14b088740 1 -- 172.21.15.44:0/2163424020 wait complete. 2024-08-20T03:12:22.698 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.694+0000 7fa14b088740 20 client.26854 trim_cache size 0 max 16384 2024-08-20T03:12:22.698 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:12:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.701+0000 7fa148c8f640 1 -- 172.21.15.44:0/3850455263 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa12c0b3e40 msgr2=0x7fa12c0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.701+0000 7fa148c8f640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa12c0b3e40 0x7fa12c0b6300 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x561cb6412a00 tx=0x7fa134005f90 comp rx=0 tx=0).stop 2024-08-20T03:12:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.701+0000 7fa148c8f640 1 -- 172.21.15.44:0/3850455263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1441020b0 msgr2=0x7fa144115c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.701+0000 7fa148c8f640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1441020b0 0x7fa144115c90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa13800d650 tx=0x7fa13800db10 comp rx=0 tx=0).stop 2024-08-20T03:12:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.701+0000 7fa148c8f640 1 -- 172.21.15.44:0/3850455263 shutdown_connections 2024-08-20T03:12:22.703 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.701+0000 7fa148c8f640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fa12c0b3e40 0x7fa12c0b6300 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.701+0000 7fa148c8f640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa144106bd0 0x7fa14410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.701+0000 7fa148c8f640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1441020b0 0x7fa144115c90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.701+0000 7fa148c8f640 1 --2- 172.21.15.44:0/3850455263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa144101790 0x7fa144115750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:22.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.701+0000 7fa148c8f640 1 -- 172.21.15.44:0/3850455263 >> 172.21.15.44:0/3850455263 conn(0x7fa1440fd400 msgr2=0x7fa144104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:22.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.703+0000 7fa148c8f640 1 -- 172.21.15.44:0/3850455263 shutdown_connections 2024-08-20T03:12:22.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:22.705+0000 7fa148c8f640 1 -- 172.21.15.44:0/3850455263 wait complete. 2024-08-20T03:12:22.939 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-20T03:12:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.115+0000 7fce1b94b640 1 -- 172.21.15.44:0/1670672706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce14104c10 msgr2=0x7fce14104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:23.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.115+0000 7fce1b94b640 1 --2- 172.21.15.44:0/1670672706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce14104c10 0x7fce14104ff0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fce04009860 tx=0x7fce0402a340 comp rx=0 tx=0).stop 2024-08-20T03:12:23.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.117+0000 7fce1b94b640 1 -- 172.21.15.44:0/1670672706 shutdown_connections 2024-08-20T03:12:23.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.118+0000 7fce1b94b640 1 --2- 172.21.15.44:0/1670672706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1410af90 0x7fce14111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.118+0000 7fce1b94b640 1 --2- 172.21.15.44:0/1670672706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce1410a610 0x7fce1410aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.118+0000 7fce1b94b640 1 --2- 172.21.15.44:0/1670672706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce14104c10 0x7fce14104ff0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.118+0000 7fce1b94b640 1 -- 172.21.15.44:0/1670672706 >> 172.21.15.44:0/1670672706 conn(0x7fce14100670 msgr2=0x7fce14102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:23.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.119+0000 7fce1b94b640 1 -- 172.21.15.44:0/1670672706 shutdown_connections 2024-08-20T03:12:23.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.119+0000 7fce1b94b640 1 -- 172.21.15.44:0/1670672706 wait complete. 2024-08-20T03:12:23.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.120+0000 7fce1b94b640 1 Processor -- start 2024-08-20T03:12:23.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.120+0000 7fce1b94b640 1 -- start start 2024-08-20T03:12:23.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.120+0000 7fce1b94b640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce14104c10 0x7fce14111530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:23.123 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.120+0000 7fce1b94b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1410a610 0x7fce1410e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:23.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.121+0000 7fce1b94b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce1410af90 0x7fce1410ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:23.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.121+0000 7fce1b94b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fce14114c50 con 0x7fce1410a610 2024-08-20T03:12:23.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.121+0000 7fce1b94b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce14114ad0 con 0x7fce1410af90 2024-08-20T03:12:23.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.121+0000 7fce1b94b640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fce14114dd0 con 0x7fce14104c10 2024-08-20T03:12:23.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.122+0000 7fce19ee8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce1410af90 0x7fce1410ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:23.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.122+0000 7fce19ee8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce1410af90 0x7fce1410ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32790/0 (socket says 172.21.15.44:32790) 2024-08-20T03:12:23.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.122+0000 7fce19ee8640 1 -- 172.21.15.44:0/4292668992 learned_addr learned my addr 172.21.15.44:0/4292668992 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:23.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.122+0000 7fce196e7640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce14104c10 0x7fce14111530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:23.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.123+0000 7fce196e7640 1 -- 172.21.15.44:0/4292668992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce1410af90 msgr2=0x7fce1410ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:23.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.123+0000 7fce196e7640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce1410af90 0x7fce1410ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.123+0000 7fce196e7640 1 -- 172.21.15.44:0/4292668992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1410a610 msgr2=0x7fce1410e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:23.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.123+0000 7fce196e7640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1410a610 0x7fce1410e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.123+0000 7fce196e7640 1 -- 172.21.15.44:0/4292668992 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1410f590 con 0x7fce14104c10 2024-08-20T03:12:23.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.123+0000 7fce196e7640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce14104c10 0x7fce14111530 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fce04035040 tx=0x7fce04030e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:23.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.124+0000 7fce0a7fc640 1 -- 172.21.15.44:0/4292668992 <== mon.1 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04037040 con 0x7fce14104c10 2024-08-20T03:12:23.127 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.124+0000 7fce0a7fc640 1 -- 172.21.15.44:0/4292668992 <== mon.1 v2:172.21.15.165:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce0402adc0 con 0x7fce14104c10 2024-08-20T03:12:23.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.124+0000 7fce0a7fc640 1 -- 172.21.15.44:0/4292668992 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04003850 con 0x7fce14104c10 2024-08-20T03:12:23.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.125+0000 7fce1b94b640 1 -- 172.21.15.44:0/4292668992 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce141a1210 con 0x7fce14104c10 2024-08-20T03:12:23.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.125+0000 7fce1b94b640 1 -- 172.21.15.44:0/4292668992 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce141a1650 con 0x7fce14104c10 2024-08-20T03:12:23.129 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.126+0000 7fce1b94b640 1 -- 172.21.15.44:0/4292668992 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce14104ff0 con 0x7fce14104c10 2024-08-20T03:12:23.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.129+0000 7fce0a7fc640 1 -- 172.21.15.44:0/4292668992 <== mon.1 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fce04043060 con 0x7fce14104c10 2024-08-20T03:12:23.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.130+0000 7fce0a7fc640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fce000b3d40 0x7fce000b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:23.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.131+0000 7fce0a7fc640 1 -- 172.21.15.44:0/4292668992 <== mon.1 v2:172.21.15.165:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fce04103450 con 0x7fce14104c10 2024-08-20T03:12:23.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.131+0000 7fce18ee6640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fce000b3d40 0x7fce000b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:23.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.131+0000 7fce18ee6640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fce000b3d40 0x7fce000b6200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fce14111270 tx=0x7fce10006c90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:23.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.136+0000 7fce0a7fc640 1 -- 172.21.15.44:0/4292668992 <== mon.1 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fce040ca890 con 0x7fce14104c10 2024-08-20T03:12:23.512 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.510+0000 7fce1b94b640 1 -- 172.21.15.44:0/4292668992 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fce14078510 con 0x7fce000b3d40 2024-08-20T03:12:23.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.536+0000 7fce0a7fc640 1 -- 172.21.15.44:0/4292668992 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fce14078510 con 0x7fce000b3d40 2024-08-20T03:12:23.539 INFO:teuthology.orchestra.run.smithi044.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-20T00:00:00", "created": "2024-08-20T03:02:21", "first": "2024-08-20T03:03:00", "last": "2024-08-20T03:12:00", "last_pruned": "2024-08-20T03:12:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-08-20T03:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.539+0000 7fce1b94b640 1 -- 172.21.15.44:0/4292668992 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fce000b3d40 msgr2=0x7fce000b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.540+0000 7fce1b94b640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fce000b3d40 0x7fce000b6200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fce14111270 tx=0x7fce10006c90 comp rx=0 tx=0).stop 2024-08-20T03:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.540+0000 7fce1b94b640 1 -- 172.21.15.44:0/4292668992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce14104c10 msgr2=0x7fce14111530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.540+0000 7fce1b94b640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce14104c10 0x7fce14111530 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fce04035040 tx=0x7fce04030e00 comp rx=0 tx=0).stop 2024-08-20T03:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.540+0000 7fce1b94b640 1 -- 172.21.15.44:0/4292668992 shutdown_connections 2024-08-20T03:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.540+0000 7fce1b94b640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fce000b3d40 0x7fce000b6200 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.540+0000 7fce1b94b640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce1410af90 0x7fce1410ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.540+0000 7fce1b94b640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce1410a610 0x7fce1410e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.542 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.540+0000 7fce1b94b640 1 --2- 172.21.15.44:0/4292668992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce14104c10 0x7fce14111530 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.540+0000 7fce1b94b640 1 -- 172.21.15.44:0/4292668992 >> 172.21.15.44:0/4292668992 conn(0x7fce14100670 msgr2=0x7fce14102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.540+0000 7fce1b94b640 1 -- 172.21.15.44:0/4292668992 shutdown_connections 2024-08-20T03:12:23.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.540+0000 7fce1b94b640 1 -- 172.21.15.44:0/4292668992 wait complete. 2024-08-20T03:12:23.560 DEBUG:teuthology.orchestra.run.smithi044:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-20T03:12:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.718+0000 7fb27e6e7640 1 -- 172.21.15.44:0/2582804494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2781038e0 msgr2=0x7fb278103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.718+0000 7fb27e6e7640 1 --2- 172.21.15.44:0/2582804494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2781038e0 0x7fb278103cc0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb268009900 tx=0x7fb26802a410 comp rx=0 tx=0).stop 2024-08-20T03:12:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.718+0000 7fb27e6e7640 1 -- 172.21.15.44:0/2582804494 shutdown_connections 2024-08-20T03:12:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.718+0000 7fb27e6e7640 1 --2- 172.21.15.44:0/2582804494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27810cd30 0x7fb27810f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.718+0000 7fb27e6e7640 1 --2- 172.21.15.44:0/2582804494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb278104200 0x7fb27810c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.718+0000 7fb27e6e7640 1 --2- 172.21.15.44:0/2582804494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2781038e0 0x7fb278103cc0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.718+0000 7fb27e6e7640 1 -- 172.21.15.44:0/2582804494 >> 172.21.15.44:0/2582804494 conn(0x7fb2780fd350 msgr2=0x7fb2780ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.718+0000 7fb27e6e7640 1 -- 172.21.15.44:0/2582804494 shutdown_connections 2024-08-20T03:12:23.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.718+0000 7fb27e6e7640 1 -- 172.21.15.44:0/2582804494 wait complete. 2024-08-20T03:12:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.719+0000 7fb27e6e7640 1 Processor -- start 2024-08-20T03:12:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.719+0000 7fb27e6e7640 1 -- start start 2024-08-20T03:12:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.719+0000 7fb27e6e7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2781038e0 0x7fb27810bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.719+0000 7fb27e6e7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb278104200 0x7fb27810c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb277fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2781038e0 0x7fb27810bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb277fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2781038e0 0x7fb27810bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32804/0 (socket says 172.21.15.44:32804) 2024-08-20T03:12:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb277fff640 1 -- 172.21.15.44:0/1264510951 learned_addr learned my addr 172.21.15.44:0/1264510951 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb27e6e7640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27810cd30 0x7fb27819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:23.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb278112a10 con 0x7fb278104200 2024-08-20T03:12:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb278112890 con 0x7fb2781038e0 2024-08-20T03:12:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb278112b90 con 0x7fb27810cd30 2024-08-20T03:12:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb2777fe640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb278104200 0x7fb27810c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:23.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb27cc84640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27810cd30 0x7fb27819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb2777fe640 1 -- 172.21.15.44:0/1264510951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27810cd30 msgr2=0x7fb27819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb2777fe640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27810cd30 0x7fb27819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb2777fe640 1 -- 172.21.15.44:0/1264510951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2781038e0 msgr2=0x7fb27810bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb2777fe640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2781038e0 0x7fb27810bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.720+0000 7fb2777fe640 1 -- 172.21.15.44:0/1264510951 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27819ffe0 con 0x7fb278104200 2024-08-20T03:12:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.721+0000 7fb2777fe640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb278104200 0x7fb27810c080 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00e9c0 tx=0x7fb26c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.721+0000 7fb2757fa640 1 -- 172.21.15.44:0/1264510951 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c00ccc0 con 0x7fb278104200 2024-08-20T03:12:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.721+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2781a02d0 con 0x7fb278104200 2024-08-20T03:12:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.721+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27819c870 con 0x7fb278104200 2024-08-20T03:12:23.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.721+0000 7fb2757fa640 1 -- 172.21.15.44:0/1264510951 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb26c0108e0 con 0x7fb278104200 2024-08-20T03:12:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.721+0000 7fb2757fa640 1 -- 172.21.15.44:0/1264510951 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c018780 con 0x7fb278104200 2024-08-20T03:12:23.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.722+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb278103cc0 con 0x7fb278104200 2024-08-20T03:12:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.727+0000 7fb2757fa640 1 -- 172.21.15.44:0/1264510951 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb26c01f020 con 0x7fb278104200 2024-08-20T03:12:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.728+0000 7fb2757fa640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb2600b3e40 0x7fb2600b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:23.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.729+0000 7fb2757fa640 1 -- 172.21.15.44:0/1264510951 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fb26c0e44d0 con 0x7fb278104200 2024-08-20T03:12:23.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.732+0000 7fb2757fa640 1 -- 172.21.15.44:0/1264510951 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb26c0ab7a0 con 0x7fb278104200 2024-08-20T03:12:23.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.733+0000 7fb277fff640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb2600b3e40 0x7fb2600b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:23.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:23.734+0000 7fb277fff640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb2600b3e40 0x7fb2600b6300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb2680096c0 tx=0x7fb268005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:23.903 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T03:12:24.150 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.148+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 --> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb278102c90 con 0x7fb2600b3e40 2024-08-20T03:12:24.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.180+0000 7fb2757fa640 1 -- 172.21.15.44:0/1264510951 <== mgr.14220 v2:172.21.15.44:6800/587429394 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb278102c90 con 0x7fb2600b3e40 2024-08-20T03:12:24.182 INFO:teuthology.orchestra.run.smithi044.stdout:/ 1m 6m3h 2024-08-20T03:12:24.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.184+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb2600b3e40 msgr2=0x7fb2600b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:24.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.184+0000 7fb27e6e7640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb2600b3e40 0x7fb2600b6300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb2680096c0 tx=0x7fb268005a20 comp rx=0 tx=0).stop 2024-08-20T03:12:24.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.184+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb278104200 msgr2=0x7fb27810c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:24.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.184+0000 7fb27e6e7640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb278104200 0x7fb27810c080 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00e9c0 tx=0x7fb26c00ee80 comp rx=0 tx=0).stop 2024-08-20T03:12:24.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.184+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 shutdown_connections 2024-08-20T03:12:24.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.184+0000 7fb27e6e7640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb2600b3e40 0x7fb2600b6300 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:24.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.184+0000 7fb27e6e7640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb27810cd30 0x7fb27819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:24.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.184+0000 7fb27e6e7640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb278104200 0x7fb27810c080 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:24.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.184+0000 7fb27e6e7640 1 --2- 172.21.15.44:0/1264510951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2781038e0 0x7fb27810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:24.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.184+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 >> 172.21.15.44:0/1264510951 conn(0x7fb2780fd350 msgr2=0x7fb27810ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:24.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.184+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 shutdown_connections 2024-08-20T03:12:24.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.184+0000 7fb27e6e7640 1 -- 172.21.15.44:0/1264510951 wait complete. 2024-08-20T03:12:24.205 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-08-20T03:12:24.267 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-08-20T03:12:24.267 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:24 smithi044 ceph-mon[18790]: pgmap v624: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 800 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 10 op/s 2024-08-20T03:12:24.330 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:24.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:24 smithi165 ceph-mon[22357]: pgmap v624: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 800 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 10 op/s 2024-08-20T03:12:24.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:24 smithi099 ceph-mon[21984]: pgmap v624: 97 pgs: 21 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 36 active+clean; 800 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 10 op/s 2024-08-20T03:12:25.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.922+0000 7f0922155640 1 -- 172.21.15.44:0/631237892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c104b20 msgr2=0x7f091c104f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:25.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.922+0000 7f0922155640 1 --2- 172.21.15.44:0/631237892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c104b20 0x7f091c104f00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f090c009900 tx=0x7f090c02a400 comp rx=0 tx=0).stop 2024-08-20T03:12:25.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.922+0000 7f0922155640 1 -- 172.21.15.44:0/631237892 shutdown_connections 2024-08-20T03:12:25.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.922+0000 7f0922155640 1 --2- 172.21.15.44:0/631237892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c10aeb0 0x7f091c1116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.922+0000 7f0922155640 1 --2- 172.21.15.44:0/631237892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f091c10a530 0x7f091c10a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.922+0000 7f0922155640 1 --2- 172.21.15.44:0/631237892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c104b20 0x7f091c104f00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.922+0000 7f0922155640 1 -- 172.21.15.44:0/631237892 >> 172.21.15.44:0/631237892 conn(0x7f091c100570 msgr2=0x7f091c102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:25.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.922+0000 7f0922155640 1 -- 172.21.15.44:0/631237892 shutdown_connections 2024-08-20T03:12:25.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.923+0000 7f0922155640 1 -- 172.21.15.44:0/631237892 wait complete. 2024-08-20T03:12:25.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.923+0000 7f0922155640 1 Processor -- start 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.923+0000 7f0922155640 1 -- start start 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.924+0000 7f0922155640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c104b20 0x7f091c19af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.924+0000 7f0922155640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f091c10a530 0x7f091c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.924+0000 7f0922155640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c10aeb0 0x7f091c1a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.924+0000 7f0922155640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f091c114bc0 con 0x7f091c10aeb0 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.924+0000 7f0922155640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f091c114a40 con 0x7f091c104b20 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.924+0000 7f0922155640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f091c114d40 con 0x7f091c10a530 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.924+0000 7f091bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c10aeb0 0x7f091c1a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.924+0000 7f091bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c10aeb0 0x7f091c1a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48248/0 (socket says 172.21.15.44:48248) 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.924+0000 7f091bfff640 1 -- 172.21.15.44:0/2074309620 learned_addr learned my addr 172.21.15.44:0/2074309620 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.925+0000 7f091bfff640 1 -- 172.21.15.44:0/2074309620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f091c10a530 msgr2=0x7f091c19b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.925+0000 7f091bfff640 1 --2- 172.21.15.44:0/2074309620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f091c10a530 0x7f091c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.925+0000 7f091bfff640 1 -- 172.21.15.44:0/2074309620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c104b20 msgr2=0x7f091c19af50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.925+0000 7f091bfff640 1 --2- 172.21.15.44:0/2074309620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c104b20 0x7f091c19af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.925+0000 7f091bfff640 1 -- 172.21.15.44:0/2074309620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091c1a1000 con 0x7f091c10aeb0 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.925+0000 7f091bfff640 1 --2- 172.21.15.44:0/2074309620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c10aeb0 0x7f091c1a0830 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f091400eb10 tx=0x7f091400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.926+0000 7f09197fa640 1 -- 172.21.15.44:0/2074309620 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914004b20 con 0x7f091c10aeb0 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.926+0000 7f09197fa640 1 -- 172.21.15.44:0/2074309620 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0914004cc0 con 0x7f091c10aeb0 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.926+0000 7f09197fa640 1 -- 172.21.15.44:0/2074309620 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914010670 con 0x7f091c10aeb0 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.926+0000 7f0922155640 1 -- 172.21.15.44:0/2074309620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091c1a12f0 con 0x7f091c10aeb0 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.926+0000 7f0922155640 1 -- 172.21.15.44:0/2074309620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091c0772c0 con 0x7f091c10aeb0 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.931+0000 7f09197fa640 1 -- 172.21.15.44:0/2074309620 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f0914005020 con 0x7f091c10aeb0 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.931+0000 7f092454e740 1 Processor -- start 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.931+0000 7f092454e740 1 -- start start 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.932+0000 7f092454e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f57910 0x55d7f9f57cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.932+0000 7f092454e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f58230 0x55d7f9f63c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.932+0000 7f092454e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f641c0 0x55d7f9f665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.932+0000 7f092454e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d7f9ece930 con 0x55d7f9f57910 2024-08-20T03:12:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.932+0000 7f092454e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d7f9ebde30 con 0x55d7f9f58230 2024-08-20T03:12:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.932+0000 7f092454e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d7f9ec1e70 con 0x55d7f9f641c0 2024-08-20T03:12:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.932+0000 7f09197fa640 1 --2- 172.21.15.44:0/2074309620 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f09080b3fc0 0x7f09080b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.932+0000 7f091b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f58230 0x55d7f9f63c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.932+0000 7f091b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f58230 0x55d7f9f63c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32822/0 (socket says 172.21.15.44:32822) 2024-08-20T03:12:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.932+0000 7f091b7fe640 1 -- 172.21.15.44:0/1066658213 learned_addr learned my addr 172.21.15.44:0/1066658213 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.933+0000 7f091bfff640 1 --2- 172.21.15.44:0/2074309620 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f09080b3fc0 0x7f09080b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.933+0000 7f09197fa640 1 -- 172.21.15.44:0/2074309620 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f09140e44a0 con 0x7f091c10aeb0 2024-08-20T03:12:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.933+0000 7f0912ffd640 1 --2- 172.21.15.44:0/1066658213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f57910 0x55d7f9f57cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.933+0000 7f0912ffd640 1 --2- 172.21.15.44:0/1066658213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f641c0 0x55d7f9f665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.933+0000 7f091b7fe640 1 -- 172.21.15.44:0/1066658213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f641c0 msgr2=0x55d7f9f665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.933+0000 7f091b7fe640 1 --2- 172.21.15.44:0/1066658213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f641c0 0x55d7f9f665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.933+0000 7f091b7fe640 1 -- 172.21.15.44:0/1066658213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f57910 msgr2=0x55d7f9f57cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.933+0000 7f091b7fe640 1 --2- 172.21.15.44:0/1066658213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f57910 0x55d7f9f57cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.933+0000 7f091b7fe640 1 -- 172.21.15.44:0/1066658213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f9c1d650 con 0x55d7f9f58230 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.933+0000 7f0912ffd640 1 --2- 172.21.15.44:0/1066658213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f641c0 0x55d7f9f665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.933+0000 7f091bfff640 1 --2- 172.21.15.44:0/2074309620 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f09080b3fc0 0x7f09080b6480 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f09140ab000 tx=0x7f091401ffa0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.934+0000 7f091b7fe640 1 --2- 172.21.15.44:0/1066658213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f58230 0x55d7f9f63c80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f090c009a20 tx=0x7f090c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a07f65d315f119d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.935+0000 7f0911ffb640 1 -- 172.21.15.44:0/1066658213 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c03e030 con 0x55d7f9f58230 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.935+0000 7f0911ffb640 1 -- 172.21.15.44:0/1066658213 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f090c004270 con 0x55d7f9f58230 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.935+0000 7f0911ffb640 1 -- 172.21.15.44:0/1066658213 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c035040 con 0x55d7f9f58230 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.936+0000 7f092454e740 1 -- 172.21.15.44:0/1066658213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f58230 msgr2=0x55d7f9f63c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.936+0000 7f092454e740 1 --2- 172.21.15.44:0/1066658213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f58230 0x55d7f9f63c80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f090c009a20 tx=0x7f090c004040 comp rx=0 tx=0).stop 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.936+0000 7f092454e740 1 -- 172.21.15.44:0/1066658213 shutdown_connections 2024-08-20T03:12:25.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.936+0000 7f092454e740 1 --2- 172.21.15.44:0/1066658213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f641c0 0x55d7f9f665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.936+0000 7f092454e740 1 --2- 172.21.15.44:0/1066658213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f58230 0x55d7f9f63c80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.936+0000 7f092454e740 1 --2- 172.21.15.44:0/1066658213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f57910 0x55d7f9f57cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.936+0000 7f092454e740 1 -- 172.21.15.44:0/1066658213 >> 172.21.15.44:0/1066658213 conn(0x55d7f9f55740 msgr2=0x55d7f9f568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.936+0000 7f092454e740 1 -- 172.21.15.44:0/1066658213 shutdown_connections 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.936+0000 7f092454e740 1 -- 172.21.15.44:0/1066658213 wait complete. 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.937+0000 7f092454e740 1 -- start start 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.937+0000 7f092454e740 1 Processor -- start 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.937+0000 7f092454e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f57910 0x55d7f9f610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.937+0000 7f092454e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 0x55d7f9f61620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f092454e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f641c0 0x55d7f9f61b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f092454e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55d7f9d20700 con 0x55d7f9f58230 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f091b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 0x55d7f9f61620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f091b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 0x55d7f9f61620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48258/0 (socket says 172.21.15.44:48258) 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f091b7fe640 1 -- 172.21.15.44:0/273522577 learned_addr learned my addr 172.21.15.44:0/273522577 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f092454e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d7f9e44810 con 0x55d7f9f641c0 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f092454e740 1 -- 172.21.15.44:0/273522577 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x55d7f9f66e10 con 0x55d7f9f57910 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f092454e740 10 client.0 _ll_register_callbacks cb 0x55d7f9eb7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f0912ffd640 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f641c0 0x55d7f9f61b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f091b7fe640 1 -- 172.21.15.44:0/273522577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f57910 msgr2=0x55d7f9f610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f0912ffd640 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f57910 0x55d7f9f610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f091b7fe640 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f57910 0x55d7f9f610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f091b7fe640 1 -- 172.21.15.44:0/273522577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f641c0 msgr2=0x55d7f9f61b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f091b7fe640 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f641c0 0x55d7f9f61b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f091b7fe640 1 -- 172.21.15.44:0/273522577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f9f701d0 con 0x55d7f9f58230 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f0912ffd640 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f57910 0x55d7f9f610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:12:25.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f091b7fe640 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 0x55d7f9f61620 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f090c0038d0 tx=0x7f090c003900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6f1c8a9fb9ce5c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.938+0000 7f08faffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.940+0000 7f08faffd640 1 -- 172.21.15.44:0/273522577 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c03e040 con 0x55d7f9f58230 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.940+0000 7f08faffd640 1 -- 172.21.15.44:0/273522577 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f090c003d20 con 0x55d7f9f58230 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.940+0000 7f092454e740 1 -- 172.21.15.44:0/273522577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7f9e6d150 con 0x55d7f9f58230 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.942+0000 7f08faffd640 1 -- 172.21.15.44:0/273522577 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f090c035040 con 0x55d7f9f58230 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.942+0000 7f092454e740 10 client.17553 fetch_fsmap learned FSMap version 44 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.942+0000 7f092454e740 1 -- 172.21.15.44:0/273522577 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d7f9f72ad0 con 0x55d7f9f58230 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.942+0000 7f08faffd640 1 -- 172.21.15.44:0/273522577 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d7f9e6d150 con 0x55d7f9f58230 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.943+0000 7f08faffd640 1 -- 172.21.15.44:0/273522577 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7f090c04c4a0 con 0x55d7f9f58230 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.943+0000 7f092454e740 10 client.17553 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.943+0000 7f092454e740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.943+0000 7f092454e740 20 client.17553 populate_metadata read hostname 'smithi044' 2024-08-20T03:12:25.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.943+0000 7f092454e740 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f9f73e20 0x55d7f9f76210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.943+0000 7f092454e740 4 client.17553 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.943+0000 7f092454e740 1 -- 172.21.15.44:0/273522577 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7f9ed0880 con 0x55d7f9f73e20 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.943+0000 7f0912ffd640 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f9f73e20 0x55d7f9f76210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.944+0000 7f08faffd640 10 client.17553 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.944+0000 7f0912ffd640 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f9f73e20 0x55d7f9f76210 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b44e7d9c490010d server_cookie=a95ff97c926b8e6a in_seq=0 out_seq=0 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.945+0000 7f08faffd640 1 -- 172.21.15.44:0/273522577 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7f9ed0880 con 0x55d7f9f73e20 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.945+0000 7f08faffd640 10 client.17553 handle_command_reply: tid=0 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.945+0000 7f092454e740 1 client.17553 shutdown 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.947+0000 7f092454e740 1 -- 172.21.15.44:0/273522577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 msgr2=0x55d7f9f61620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.947+0000 7f092454e740 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 0x55d7f9f61620 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f090c0038d0 tx=0x7f090c003900 comp rx=0 tx=0).stop 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.947+0000 7f092454e740 1 -- 172.21.15.44:0/273522577 shutdown_connections 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.947+0000 7f092454e740 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f9f73e20 0x55d7f9f76210 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.947+0000 7f092454e740 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f641c0 0x55d7f9f61b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.947+0000 7f092454e740 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 0x55d7f9f61620 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.947+0000 7f092454e740 1 --2- 172.21.15.44:0/273522577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f57910 0x55d7f9f610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.947+0000 7f092454e740 1 -- 172.21.15.44:0/273522577 >> 172.21.15.44:0/273522577 conn(0x55d7f9f55740 msgr2=0x55d7f9f5b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.947+0000 7f08faffd640 0 client.17553 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.947+0000 7f092454e740 1 -- 172.21.15.44:0/273522577 shutdown_connections 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.947+0000 7f092454e740 1 -- 172.21.15.44:0/273522577 wait complete. 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.948+0000 7f092454e740 20 client.17553 trim_cache size 0 max 16384 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f092454e740 1 Processor -- start 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f092454e740 1 -- start start 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f092454e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f57910 0x55d7f9f61970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f092454e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f58230 0x55d7f9f7b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f092454e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f641c0 0x55d7f9f7bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f092454e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc00e750 con 0x55d7f9f641c0 2024-08-20T03:12:25.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f0912ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f57910 0x55d7f9f61970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f092454e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc010380 con 0x55d7f9f57910 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f092454e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc00eeb0 con 0x55d7f9f58230 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f0912ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f57910 0x55d7f9f61970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32840/0 (socket says 172.21.15.44:32840) 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f0912ffd640 1 -- 172.21.15.44:0/2884556473 learned_addr learned my addr 172.21.15.44:0/2884556473 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f091b7fe640 1 --2- 172.21.15.44:0/2884556473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f58230 0x55d7f9f7b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.971+0000 7f0912ffd640 1 --2- 172.21.15.44:0/2884556473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f641c0 0x55d7f9f7bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.972+0000 7f0912ffd640 1 -- 172.21.15.44:0/2884556473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f58230 msgr2=0x55d7f9f7b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.972+0000 7f0912ffd640 1 --2- 172.21.15.44:0/2884556473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f58230 0x55d7f9f7b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.972+0000 7f0912ffd640 1 -- 172.21.15.44:0/2884556473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f641c0 msgr2=0x55d7f9f7bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.972+0000 7f0912ffd640 1 --2- 172.21.15.44:0/2884556473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f641c0 0x55d7f9f7bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.972+0000 7f0912ffd640 1 -- 172.21.15.44:0/2884556473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f9f576c0 con 0x55d7f9f57910 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.972+0000 7f0912ffd640 1 --2- 172.21.15.44:0/2884556473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f57910 0x55d7f9f61970 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55d7f9f578c0 tx=0x7f0904005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7269e7d145c8d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:25.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.972+0000 7f09117fa640 1 -- 172.21.15.44:0/2884556473 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904004030 con 0x55d7f9f57910 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.972+0000 7f09117fa640 1 -- 172.21.15.44:0/2884556473 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0904010980 con 0x55d7f9f57910 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.972+0000 7f09117fa640 1 -- 172.21.15.44:0/2884556473 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904005440 con 0x55d7f9f57910 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.973+0000 7f092454e740 1 -- 172.21.15.44:0/2884556473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f57910 msgr2=0x55d7f9f61970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.974+0000 7f092454e740 1 --2- 172.21.15.44:0/2884556473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f57910 0x55d7f9f61970 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55d7f9f578c0 tx=0x7f0904005850 comp rx=0 tx=0).stop 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.974+0000 7f092454e740 1 -- 172.21.15.44:0/2884556473 shutdown_connections 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.974+0000 7f092454e740 1 --2- 172.21.15.44:0/2884556473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f641c0 0x55d7f9f7bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.974+0000 7f092454e740 1 --2- 172.21.15.44:0/2884556473 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f58230 0x55d7f9f7b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.974+0000 7f092454e740 1 --2- 172.21.15.44:0/2884556473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f57910 0x55d7f9f61970 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.974+0000 7f092454e740 1 -- 172.21.15.44:0/2884556473 >> 172.21.15.44:0/2884556473 conn(0x55d7f9f55740 msgr2=0x55d7f9f62f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.974+0000 7f092454e740 1 -- 172.21.15.44:0/2884556473 shutdown_connections 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.974+0000 7f092454e740 1 -- 172.21.15.44:0/2884556473 wait complete. 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.975+0000 7f092454e740 1 -- start start 2024-08-20T03:12:25.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.976+0000 7f092454e740 1 Processor -- start 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.976+0000 7f092454e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f57910 0x55d7f9f69920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.976+0000 7f092454e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 0x55d7f9f69e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f092454e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f641c0 0x55d7f9fdc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f092454e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc00fcb0 con 0x55d7f9f58230 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f092454e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc00f5b0 con 0x55d7f9f641c0 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f092454e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc00c750 con 0x55d7f9f57910 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f092454e740 10 client.0 _ll_register_callbacks cb 0x7f0904005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f091b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 0x55d7f9f69e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f091b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 0x55d7f9f69e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:48278/0 (socket says 172.21.15.44:48278) 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f091b7fe640 1 -- 172.21.15.44:0/3944613568 learned_addr learned my addr 172.21.15.44:0/3944613568 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f0912ffd640 1 --2- 172.21.15.44:0/3944613568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f57910 0x55d7f9f69920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f091b7fe640 1 -- 172.21.15.44:0/3944613568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f57910 msgr2=0x55d7f9f69920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:25.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f091b7fe640 1 --2- 172.21.15.44:0/3944613568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f57910 0x55d7f9f69920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f091b7fe640 1 -- 172.21.15.44:0/3944613568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f641c0 msgr2=0x55d7f9fdc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f091b7fe640 1 --2- 172.21.15.44:0/3944613568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f641c0 0x55d7f9fdc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.977+0000 7f091b7fe640 1 -- 172.21.15.44:0/3944613568 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f9fdced0 con 0x55d7f9f58230 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.978+0000 7f091b7fe640 1 --2- 172.21.15.44:0/3944613568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 0x55d7f9f69e60 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f090c009900 tx=0x7f090c04c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22aecd6f79a8179b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.978+0000 7f08faffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.978+0000 7f08faffd640 1 -- 172.21.15.44:0/3944613568 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c03e040 con 0x55d7f9f58230 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.978+0000 7f08faffd640 1 -- 172.21.15.44:0/3944613568 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f090c005240 con 0x55d7f9f58230 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.978+0000 7f092454e740 1 -- 172.21.15.44:0/3944613568 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0904010980 con 0x55d7f9f58230 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.979+0000 7f08faffd640 1 -- 172.21.15.44:0/3944613568 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f090c064050 con 0x55d7f9f58230 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.979+0000 7f092454e740 10 client.17559 fetch_fsmap learned FSMap version 44 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.979+0000 7f092454e740 1 -- 172.21.15.44:0/3944613568 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d7f9fde1b0 con 0x55d7f9f58230 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.979+0000 7f08faffd640 1 -- 172.21.15.44:0/3944613568 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0904010980 con 0x55d7f9f58230 2024-08-20T03:12:25.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.979+0000 7f08faffd640 1 -- 172.21.15.44:0/3944613568 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 44) ==== 8386+0+0 (secure 0 0 0) 0x7f090c0046c0 con 0x55d7f9f58230 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.979+0000 7f092454e740 10 client.17559 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.979+0000 7f092454e740 10 client.17559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.979+0000 7f092454e740 20 client.17559 populate_metadata read hostname 'smithi044' 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.979+0000 7f092454e740 1 --2- 172.21.15.44:0/3944613568 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f9fde9b0 0x55d7f9fe0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.979+0000 7f092454e740 4 client.17559 mds_command: new command op to 24604 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.979+0000 7f092454e740 1 -- 172.21.15.44:0/3944613568 --> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7f9f62050 con 0x55d7f9fde9b0 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.979+0000 7f0912ffd640 1 --2- 172.21.15.44:0/3944613568 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f9fde9b0 0x55d7f9fe0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.980+0000 7f08faffd640 10 client.17559 ms_handle_connect on v2:172.21.15.165:6838/3029973590 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.980+0000 7f0912ffd640 1 --2- 172.21.15.44:0/3944613568 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f9fde9b0 0x55d7f9fe0dd0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85d3a851fe52f285 server_cookie=98d35af7d4a79bfe in_seq=0 out_seq=0 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.980+0000 7f08faffd640 1 -- 172.21.15.44:0/3944613568 <== mds.0 v2:172.21.15.165:6838/3029973590 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7f9f62050 con 0x55d7f9fde9b0 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.980+0000 7f08faffd640 10 client.17559 handle_command_reply: tid=0 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.981+0000 7f092454e740 1 client.17559 shutdown 2024-08-20T03:12:25.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.981+0000 7f092454e740 1 -- 172.21.15.44:0/3944613568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 msgr2=0x55d7f9f69e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:25.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.981+0000 7f092454e740 1 --2- 172.21.15.44:0/3944613568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 0x55d7f9f69e60 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f090c009900 tx=0x7f090c04c7b0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.981+0000 7f092454e740 1 -- 172.21.15.44:0/3944613568 shutdown_connections 2024-08-20T03:12:25.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.981+0000 7f092454e740 1 --2- 172.21.15.44:0/3944613568 >> [v2:172.21.15.165:6838/3029973590,v1:172.21.15.165:6839/3029973590] conn(0x55d7f9fde9b0 0x55d7f9fe0dd0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.981+0000 7f092454e740 1 --2- 172.21.15.44:0/3944613568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d7f9f641c0 0x55d7f9fdc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.981+0000 7f08faffd640 0 client.17559 ms_handle_reset on v2:172.21.15.165:6838/3029973590 2024-08-20T03:12:25.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.981+0000 7f092454e740 1 --2- 172.21.15.44:0/3944613568 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d7f9f58230 0x55d7f9f69e60 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.310 INFO:teuthology.orchestra.run.smithi044.stdout:[] 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.981+0000 7f092454e740 1 --2- 172.21.15.44:0/3944613568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d7f9f57910 0x55d7f9f69920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.981+0000 7f092454e740 1 -- 172.21.15.44:0/3944613568 >> 172.21.15.44:0/3944613568 conn(0x55d7f9f55740 msgr2=0x55d7f9f53e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.981+0000 7f092454e740 1 -- 172.21.15.44:0/3944613568 shutdown_connections 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.981+0000 7f092454e740 1 -- 172.21.15.44:0/3944613568 wait complete. 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.982+0000 7f092454e740 20 client.17559 trim_cache size 0 max 16384 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.982+0000 7f0922155640 1 -- 172.21.15.44:0/2074309620 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f09080b3fc0 msgr2=0x7f09080b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.982+0000 7f0922155640 1 --2- 172.21.15.44:0/2074309620 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f09080b3fc0 0x7f09080b6480 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f09140ab000 tx=0x7f091401ffa0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.982+0000 7f0922155640 1 -- 172.21.15.44:0/2074309620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c10aeb0 msgr2=0x7f091c1a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.982+0000 7f0922155640 1 --2- 172.21.15.44:0/2074309620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c10aeb0 0x7f091c1a0830 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f091400eb10 tx=0x7f091400c2d0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.983+0000 7f0922155640 1 -- 172.21.15.44:0/2074309620 shutdown_connections 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.983+0000 7f0922155640 1 --2- 172.21.15.44:0/2074309620 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f09080b3fc0 0x7f09080b6480 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.983+0000 7f0922155640 1 --2- 172.21.15.44:0/2074309620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f091c10aeb0 0x7f091c1a0830 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.311 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.983+0000 7f0922155640 1 --2- 172.21.15.44:0/2074309620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f091c10a530 0x7f091c19b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.983+0000 7f0922155640 1 --2- 172.21.15.44:0/2074309620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f091c104b20 0x7f091c19af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:25.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.983+0000 7f0922155640 1 -- 172.21.15.44:0/2074309620 >> 172.21.15.44:0/2074309620 conn(0x7f091c100570 msgr2=0x7f091c101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:25.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.983+0000 7f0922155640 1 -- 172.21.15.44:0/2074309620 shutdown_connections 2024-08-20T03:12:25.312 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:24.983+0000 7f0922155640 1 -- 172.21.15.44:0/2074309620 wait complete. 2024-08-20T03:12:25.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:25 smithi165 ceph-mon[22357]: from='client.26860 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:12:25.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:25 smithi165 ceph-mon[22357]: from='client.17541 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:12:25.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:25 smithi099 ceph-mon[21984]: from='client.26860 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:12:25.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:25 smithi099 ceph-mon[21984]: from='client.17541 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:12:25.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:25 smithi044 ceph-mon[18790]: from='client.26860 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:12:25.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:25 smithi044 ceph-mon[18790]: from='client.17541 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T03:12:26.151 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-08-20T03:12:26.152 INFO:tasks.fwd_scrub:done joining 2024-08-20T03:12:26.152 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-08-20T03:12:26.168 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-20T03:12:26.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.336+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3715190717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3d811bdd0 msgr2=0x7fb3d811e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:26.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.336+0000 7fb3e03a1640 1 --2- 172.21.15.44:0/3715190717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3d811bdd0 0x7fb3d811e1c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0009ca0 tx=0x7fb3d002a960 comp rx=0 tx=0).stop 2024-08-20T03:12:26.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.336+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3715190717 shutdown_connections 2024-08-20T03:12:26.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.336+0000 7fb3e03a1640 1 --2- 172.21.15.44:0/3715190717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3d811bdd0 0x7fb3d811e1c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.336+0000 7fb3e03a1640 1 --2- 172.21.15.44:0/3715190717 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d810a260 0x7fb3d810a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.336+0000 7fb3e03a1640 1 --2- 172.21.15.44:0/3715190717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3d8109e80 0x7fb3d8110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.336+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3715190717 >> 172.21.15.44:0/3715190717 conn(0x7fb3d806ea20 msgr2=0x7fb3d80701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:26.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.337+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3715190717 shutdown_connections 2024-08-20T03:12:26.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.339+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3715190717 wait complete. 2024-08-20T03:12:26.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.340+0000 7fb3e03a1640 1 Processor -- start 2024-08-20T03:12:26.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.340+0000 7fb3e03a1640 1 -- start start 2024-08-20T03:12:26.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.341+0000 7fb3e03a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d8109e80 0x7fb3d81a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.341+0000 7fb3e03a1640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3d810a260 0x7fb3d81a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.341+0000 7fb3e03a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3d811bdd0 0x7fb3d81ad730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.341+0000 7fb3e03a1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d81218e0 con 0x7fb3d8109e80 2024-08-20T03:12:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.341+0000 7fb3e03a1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8121760 con 0x7fb3d811bdd0 2024-08-20T03:12:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.341+0000 7fb3e03a1640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d8121a60 con 0x7fb3d810a260 2024-08-20T03:12:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.341+0000 7fb3de93e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3d811bdd0 0x7fb3d81ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:26.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.341+0000 7fb3de93e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3d811bdd0 0x7fb3d81ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:32858/0 (socket says 172.21.15.44:32858) 2024-08-20T03:12:26.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.341+0000 7fb3de93e640 1 -- 172.21.15.44:0/3677265586 learned_addr learned my addr 172.21.15.44:0/3677265586 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:26.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.342+0000 7fb3de93e640 1 -- 172.21.15.44:0/3677265586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3d810a260 msgr2=0x7fb3d81a83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:26.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.342+0000 7fb3de93e640 1 --2- 172.21.15.44:0/3677265586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3d810a260 0x7fb3d81a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.342+0000 7fb3de93e640 1 -- 172.21.15.44:0/3677265586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d8109e80 msgr2=0x7fb3d81a7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:26.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.342+0000 7fb3de93e640 1 --2- 172.21.15.44:0/3677265586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d8109e80 0x7fb3d81a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.342+0000 7fb3de93e640 1 -- 172.21.15.44:0/3677265586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d81adf00 con 0x7fb3d811bdd0 2024-08-20T03:12:26.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.342+0000 7fb3de93e640 1 --2- 172.21.15.44:0/3677265586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3d811bdd0 0x7fb3d81ad730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00098d0 tx=0x7fb3d002fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:26.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.342+0000 7fb3cf7fe640 1 -- 172.21.15.44:0/3677265586 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0034030 con 0x7fb3d811bdd0 2024-08-20T03:12:26.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.342+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3677265586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d81ae190 con 0x7fb3d811bdd0 2024-08-20T03:12:26.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.343+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3677265586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d8111630 con 0x7fb3d811bdd0 2024-08-20T03:12:26.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.343+0000 7fb3cf7fe640 1 -- 172.21.15.44:0/3677265586 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3d0030d10 con 0x7fb3d811bdd0 2024-08-20T03:12:26.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.343+0000 7fb3cf7fe640 1 -- 172.21.15.44:0/3677265586 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0039420 con 0x7fb3d811bdd0 2024-08-20T03:12:26.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.346+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3677265586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3bc0052c0 con 0x7fb3d811bdd0 2024-08-20T03:12:26.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.347+0000 7fb3cf7fe640 1 -- 172.21.15.44:0/3677265586 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb3d0037020 con 0x7fb3d811bdd0 2024-08-20T03:12:26.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.348+0000 7fb3cf7fe640 1 --2- 172.21.15.44:0/3677265586 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb3c40b3e40 0x7fb3c40b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:26.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.348+0000 7fb3cf7fe640 1 -- 172.21.15.44:0/3677265586 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fb3d0104d80 con 0x7fb3d811bdd0 2024-08-20T03:12:26.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.348+0000 7fb3de13d640 1 --2- 172.21.15.44:0/3677265586 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb3c40b3e40 0x7fb3c40b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:26.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.349+0000 7fb3de13d640 1 --2- 172.21.15.44:0/3677265586 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb3c40b3e40 0x7fb3c40b6300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40096f0 tx=0x7fb3d4006c40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:26.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.355+0000 7fb3cf7fe640 1 -- 172.21.15.44:0/3677265586 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb3d00cc050 con 0x7fb3d811bdd0 2024-08-20T03:12:26.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:26 smithi165 ceph-mon[22357]: pgmap v625: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 796 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-08-20T03:12:26.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:26 smithi099 ceph-mon[21984]: pgmap v625: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 796 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-08-20T03:12:26.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:26 smithi044 ceph-mon[18790]: pgmap v625: 97 pgs: 19 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 40 active+clean; 796 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-08-20T03:12:26.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.705+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3677265586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3bc005880 con 0x7fb3d811bdd0 2024-08-20T03:12:26.709 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.707+0000 7fb3cf7fe640 1 -- 172.21.15.44:0/3677265586 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+23111 (secure 0 0 0) 0x7fb3d00cb950 con 0x7fb3d811bdd0 2024-08-20T03:12:26.713 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:12:26.714 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":84,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","created":"2024-08-20T02:52:10.979264+0000","modified":"2024-08-20T03:12:10.035989+0000","last_up_change":"2024-08-20T02:59:26.404917+0000","last_in_change":"2024-08-20T02:59:03.707380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T02:55:46.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T03:00:14.354568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T03:00:16.869335+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7708f8c-a799-4b8f-ab1e-78482d73ca1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6803","nonce":858898316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6805","nonce":858898316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6809","nonce":858898316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6807","nonce":858898316}]},"public_addr":"172.21.15.44:6803/858898316","cluster_addr":"172.21.15.44:6805/858898316","heartbeat_back_addr":"172.21.15.44:6809/858898316","heartbeat_front_addr":"172.21.15.44:6807/858898316","state":["exists","up"]},{"osd":1,"uuid":"12f07986-60bf-498f-a012-cd7ea607b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6801","nonce":4252911663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6803","nonce":4252911663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6807","nonce":4252911663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6805","nonce":4252911663}]},"public_addr":"172.21.15.99:6801/4252911663","cluster_addr":"172.21.15.99:6803/4252911663","heartbeat_back_addr":"172.21.15.99:6807/4252911663","heartbeat_front_addr":"172.21.15.99:6805/4252911663","state":["exists","up"]},{"osd":2,"uuid":"0f189000-ba7f-4412-b0d3-75e8265f6685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6801","nonce":1492740465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6803","nonce":1492740465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6807","nonce":1492740465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6805","nonce":1492740465}]},"public_addr":"172.21.15.165:6801/1492740465","cluster_addr":"172.21.15.165:6803/1492740465","heartbeat_back_addr":"172.21.15.165:6807/1492740465","heartbeat_front_addr":"172.21.15.165:6805/1492740465","state":["exists","up"]},{"osd":3,"uuid":"8f9b042c-d6b5-4eee-aa29-ffc41a48c20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6811","nonce":3423983702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6813","nonce":3423983702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6817","nonce":3423983702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6815","nonce":3423983702}]},"public_addr":"172.21.15.44:6811/3423983702","cluster_addr":"172.21.15.44:6813/3423983702","heartbeat_back_addr":"172.21.15.44:6817/3423983702","heartbeat_front_addr":"172.21.15.44:6815/3423983702","state":["exists","up"]},{"osd":4,"uuid":"4edebfdb-af10-49ee-b724-2a63b9219221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6809","nonce":3099617325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6811","nonce":3099617325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6815","nonce":3099617325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6813","nonce":3099617325}]},"public_addr":"172.21.15.99:6809/3099617325","cluster_addr":"172.21.15.99:6811/3099617325","heartbeat_back_addr":"172.21.15.99:6815/3099617325","heartbeat_front_addr":"172.21.15.99:6813/3099617325","state":["exists","up"]},{"osd":5,"uuid":"6d22c21f-cc43-4ef5-a75c-d4fc1a361c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6809","nonce":523245255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6811","nonce":523245255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6815","nonce":523245255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6813","nonce":523245255}]},"public_addr":"172.21.15.165:6809/523245255","cluster_addr":"172.21.15.165:6811/523245255","heartbeat_back_addr":"172.21.15.165:6815/523245255","heartbeat_front_addr":"172.21.15.165:6813/523245255","state":["exists","up"]},{"osd":6,"uuid":"03df0c2f-1b7c-4242-888c-e047322fd8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6819","nonce":1039098986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6821","nonce":1039098986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6825","nonce":1039098986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6823","nonce":1039098986}]},"public_addr":"172.21.15.44:6819/1039098986","cluster_addr":"172.21.15.44:6821/1039098986","heartbeat_back_addr":"172.21.15.44:6825/1039098986","heartbeat_front_addr":"172.21.15.44:6823/1039098986","state":["exists","up"]},{"osd":7,"uuid":"84bbc683-9b13-448e-897c-114458dae33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6817","nonce":1438613564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6819","nonce":1438613564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6823","nonce":1438613564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6821","nonce":1438613564}]},"public_addr":"172.21.15.99:6817/1438613564","cluster_addr":"172.21.15.99:6819/1438613564","heartbeat_back_addr":"172.21.15.99:6823/1438613564","heartbeat_front_addr":"172.21.15.99:6821/1438613564","state":["exists","up"]},{"osd":8,"uuid":"14d7ffd3-db05-4228-a12e-4c69ef65ef8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6817","nonce":632957688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6819","nonce":632957688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6823","nonce":632957688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6821","nonce":632957688}]},"public_addr":"172.21.15.165:6817/632957688","cluster_addr":"172.21.15.165:6819/632957688","heartbeat_back_addr":"172.21.15.165:6823/632957688","heartbeat_front_addr":"172.21.15.165:6821/632957688","state":["exists","up"]},{"osd":9,"uuid":"1ae468a5-3d72-4eb0-a1a6-cd87eae740b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6827","nonce":2426940093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6829","nonce":2426940093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6833","nonce":2426940093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6831","nonce":2426940093}]},"public_addr":"172.21.15.44:6827/2426940093","cluster_addr":"172.21.15.44:6829/2426940093","heartbeat_back_addr":"172.21.15.44:6833/2426940093","heartbeat_front_addr":"172.21.15.44:6831/2426940093","state":["exists","up"]},{"osd":10,"uuid":"fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6825","nonce":3275793038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6827","nonce":3275793038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6831","nonce":3275793038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6829","nonce":3275793038}]},"public_addr":"172.21.15.99:6825/3275793038","cluster_addr":"172.21.15.99:6827/3275793038","heartbeat_back_addr":"172.21.15.99:6831/3275793038","heartbeat_front_addr":"172.21.15.99:6829/3275793038","state":["exists","up"]},{"osd":11,"uuid":"50715410-1fc2-4818-b600-604a4b7cce9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6825","nonce":107688415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6827","nonce":107688415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6831","nonce":107688415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6829","nonce":107688415}]},"public_addr":"172.21.15.165:6825/107688415","cluster_addr":"172.21.15.165:6827/107688415","heartbeat_back_addr":"172.21.15.165:6831/107688415","heartbeat_front_addr":"172.21.15.165:6829/107688415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:54:56.697075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:20.486846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:43.353376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:08.333879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:32.186492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:55.891366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:19.681421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:45.475926+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:10.109795+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:34.444169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:59.147646+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:59:25.403071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2597827045":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/3582612619":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/1916809746":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2919033419":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:6800/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2331034359":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2572185146":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/4176802819":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/3654439062":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6801/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/2376733275":"2024-08-21T02:53:58.312659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T03:12:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.710+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3677265586 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb3c40b3e40 msgr2=0x7fb3c40b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.710+0000 7fb3e03a1640 1 --2- 172.21.15.44:0/3677265586 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb3c40b3e40 0x7fb3c40b6300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40096f0 tx=0x7fb3d4006c40 comp rx=0 tx=0).stop 2024-08-20T03:12:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.711+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3677265586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3d811bdd0 msgr2=0x7fb3d81ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.711+0000 7fb3e03a1640 1 --2- 172.21.15.44:0/3677265586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3d811bdd0 0x7fb3d81ad730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00098d0 tx=0x7fb3d002fb20 comp rx=0 tx=0).stop 2024-08-20T03:12:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.711+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3677265586 shutdown_connections 2024-08-20T03:12:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.711+0000 7fb3e03a1640 1 --2- 172.21.15.44:0/3677265586 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb3c40b3e40 0x7fb3c40b6300 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.711+0000 7fb3e03a1640 1 --2- 172.21.15.44:0/3677265586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3d811bdd0 0x7fb3d81ad730 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.711+0000 7fb3e03a1640 1 --2- 172.21.15.44:0/3677265586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3d810a260 0x7fb3d81a83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.711+0000 7fb3e03a1640 1 --2- 172.21.15.44:0/3677265586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb3d8109e80 0x7fb3d81a7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.711+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3677265586 >> 172.21.15.44:0/3677265586 conn(0x7fb3d806ea20 msgr2=0x7fb3d811c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:26.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.715+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3677265586 shutdown_connections 2024-08-20T03:12:26.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.715+0000 7fb3e03a1640 1 -- 172.21.15.44:0/3677265586 wait complete. 2024-08-20T03:12:26.735 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-20T02:55:46.770770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-20T03:00:14.354568+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-20T03:00:16.869335+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 84, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-20T03:12:26.736 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-20T03:12:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.909+0000 7f32eeb1d640 1 -- 172.21.15.44:0/1516367018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e810a260 msgr2=0x7f32e810a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.909+0000 7f32eeb1d640 1 --2- 172.21.15.44:0/1516367018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e810a260 0x7f32e810a680 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009900 tx=0x7f32d802a510 comp rx=0 tx=0).stop 2024-08-20T03:12:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.909+0000 7f32eeb1d640 1 -- 172.21.15.44:0/1516367018 shutdown_connections 2024-08-20T03:12:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.909+0000 7f32eeb1d640 1 --2- 172.21.15.44:0/1516367018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32e811bdd0 0x7f32e811e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.909+0000 7f32eeb1d640 1 --2- 172.21.15.44:0/1516367018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e810a260 0x7f32e810a680 secure :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009900 tx=0x7f32d802a510 comp rx=0 tx=0).stop 2024-08-20T03:12:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.909+0000 7f32eeb1d640 1 --2- 172.21.15.44:0/1516367018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32e8109e80 0x7f32e8110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.909+0000 7f32eeb1d640 1 -- 172.21.15.44:0/1516367018 >> 172.21.15.44:0/1516367018 conn(0x7f32e806ea00 msgr2=0x7f32e80701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:26.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.912+0000 7f32eeb1d640 1 -- 172.21.15.44:0/1516367018 shutdown_connections 2024-08-20T03:12:26.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.913+0000 7f32eeb1d640 1 -- 172.21.15.44:0/1516367018 wait complete. 2024-08-20T03:12:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.914+0000 7f32eeb1d640 1 Processor -- start 2024-08-20T03:12:26.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.914+0000 7f32eeb1d640 1 -- start start 2024-08-20T03:12:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.914+0000 7f32eeb1d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e8109e80 0x7f32e81118d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.914+0000 7f32eeb1d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32e811bdd0 0x7f32e8111e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.914+0000 7f32eeb1d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32e81171a0 0x7f32e8117650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.914+0000 7f32eeb1d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f32e8121840 con 0x7f32e8109e80 2024-08-20T03:12:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.914+0000 7f32eeb1d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f32e81216c0 con 0x7f32e811bdd0 2024-08-20T03:12:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.914+0000 7f32eeb1d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f32e81219c0 con 0x7f32e81171a0 2024-08-20T03:12:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.915+0000 7f32ed0ba640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32e81171a0 0x7f32e8117650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.915+0000 7f32ed0ba640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32e81171a0 0x7f32e8117650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:46300/0 (socket says 172.21.15.44:46300) 2024-08-20T03:12:26.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.915+0000 7f32ed0ba640 1 -- 172.21.15.44:0/995610665 learned_addr learned my addr 172.21.15.44:0/995610665 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:26.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.916+0000 7f32ec8b9640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e8109e80 0x7f32e81118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:26.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.916+0000 7f32e7fff640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32e811bdd0 0x7f32e8111e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:26.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.917+0000 7f32ec8b9640 1 -- 172.21.15.44:0/995610665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32e81171a0 msgr2=0x7f32e8117650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:26.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.917+0000 7f32ec8b9640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32e81171a0 0x7f32e8117650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.917+0000 7f32ec8b9640 1 -- 172.21.15.44:0/995610665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32e811bdd0 msgr2=0x7f32e8111e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:26.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.917+0000 7f32ec8b9640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32e811bdd0 0x7f32e8111e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.917+0000 7f32ec8b9640 1 -- 172.21.15.44:0/995610665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e8117d80 con 0x7f32e8109e80 2024-08-20T03:12:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.917+0000 7f32ec8b9640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e8109e80 0x7f32e81118d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9f0 tx=0x7f32e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.918+0000 7f32e5ffb640 1 -- 172.21.15.44:0/995610665 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e000cd80 con 0x7f32e8109e80 2024-08-20T03:12:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.918+0000 7f32e5ffb640 1 -- 172.21.15.44:0/995610665 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e0011920 con 0x7f32e8109e80 2024-08-20T03:12:26.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.918+0000 7f32e5ffb640 1 -- 172.21.15.44:0/995610665 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e000cd80 con 0x7f32e8109e80 2024-08-20T03:12:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.921+0000 7f32eeb1d640 1 -- 172.21.15.44:0/995610665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e8118310 con 0x7f32e8109e80 2024-08-20T03:12:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.921+0000 7f32eeb1d640 1 -- 172.21.15.44:0/995610665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e8118890 con 0x7f32e8109e80 2024-08-20T03:12:26.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.922+0000 7f32eeb1d640 1 -- 172.21.15.44:0/995610665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e8074540 con 0x7f32e8109e80 2024-08-20T03:12:26.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.926+0000 7f32e5ffb640 1 -- 172.21.15.44:0/995610665 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f32e0011470 con 0x7f32e8109e80 2024-08-20T03:12:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.927+0000 7f32e5ffb640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f32d40b4140 0x7f32d40b6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.927+0000 7f32e5ffb640 1 -- 172.21.15.44:0/995610665 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f32e000cb80 con 0x7f32e8109e80 2024-08-20T03:12:26.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.927+0000 7f32e7fff640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f32d40b4140 0x7f32d40b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:26.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.928+0000 7f32e7fff640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f32d40b4140 0x7f32d40b6600 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f32d8006010 tx=0x7f32d8000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:26.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:26.936+0000 7f32e5ffb640 1 -- 172.21.15.44:0/995610665 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f32e0016030 con 0x7f32e8109e80 2024-08-20T03:12:27.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.290+0000 7f32eeb1d640 1 -- 172.21.15.44:0/995610665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32e8118bb0 con 0x7f32e8109e80 2024-08-20T03:12:27.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.291+0000 7f32e5ffb640 1 -- 172.21.15.44:0/995610665 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f32e0016030 con 0x7f32e8109e80 2024-08-20T03:12:27.293 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-08-20T03:12:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.294+0000 7f32eeb1d640 1 -- 172.21.15.44:0/995610665 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f32d40b4140 msgr2=0x7f32d40b6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:27.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.295+0000 7f32eeb1d640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f32d40b4140 0x7f32d40b6600 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f32d8006010 tx=0x7f32d8000bc0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.295+0000 7f32eeb1d640 1 -- 172.21.15.44:0/995610665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e8109e80 msgr2=0x7f32e81118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.295+0000 7f32eeb1d640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e8109e80 0x7f32e81118d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9f0 tx=0x7f32e000eeb0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.295+0000 7f32eeb1d640 1 -- 172.21.15.44:0/995610665 shutdown_connections 2024-08-20T03:12:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.295+0000 7f32eeb1d640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f32d40b4140 0x7f32d40b6600 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.295+0000 7f32eeb1d640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32e81171a0 0x7f32e8117650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.295+0000 7f32eeb1d640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32e811bdd0 0x7f32e8111e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.295+0000 7f32eeb1d640 1 --2- 172.21.15.44:0/995610665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32e8109e80 0x7f32e81118d0 secure :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9f0 tx=0x7f32e000eeb0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.295+0000 7f32eeb1d640 1 -- 172.21.15.44:0/995610665 >> 172.21.15.44:0/995610665 conn(0x7f32e806ea00 msgr2=0x7f32e811c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.295+0000 7f32eeb1d640 1 -- 172.21.15.44:0/995610665 shutdown_connections 2024-08-20T03:12:27.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.296+0000 7f32eeb1d640 1 -- 172.21.15.44:0/995610665 wait complete. 2024-08-20T03:12:27.315 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-20T03:12:27.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:27 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3677265586' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:12:27.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:27 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3677265586' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:12:27.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.478+0000 7f8255a29640 1 -- 172.21.15.44:0/3399128820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82501037c0 msgr2=0x7f8250103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:27.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.478+0000 7f8255a29640 1 --2- 172.21.15.44:0/3399128820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82501037c0 0x7f8250103ba0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8240009900 tx=0x7f824002a410 comp rx=0 tx=0).stop 2024-08-20T03:12:27.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.478+0000 7f8255a29640 1 -- 172.21.15.44:0/3399128820 shutdown_connections 2024-08-20T03:12:27.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.478+0000 7f8255a29640 1 --2- 172.21.15.44:0/3399128820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f825010cce0 0x7f825010f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.478+0000 7f8255a29640 1 --2- 172.21.15.44:0/3399128820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82501040e0 0x7f825010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.478+0000 7f8255a29640 1 --2- 172.21.15.44:0/3399128820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82501037c0 0x7f8250103ba0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.478+0000 7f8255a29640 1 -- 172.21.15.44:0/3399128820 >> 172.21.15.44:0/3399128820 conn(0x7f82500fd310 msgr2=0x7f82500ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:27.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.478+0000 7f8255a29640 1 -- 172.21.15.44:0/3399128820 shutdown_connections 2024-08-20T03:12:27.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.478+0000 7f8255a29640 1 -- 172.21.15.44:0/3399128820 wait complete. 2024-08-20T03:12:27.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.479+0000 7f8255a29640 1 Processor -- start 2024-08-20T03:12:27.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.479+0000 7f8255a29640 1 -- start start 2024-08-20T03:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.479+0000 7f8255a29640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82501037c0 0x7f8250198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f8255a29640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82501040e0 0x7f8250199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f824effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82501037c0 0x7f8250198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f824effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82501037c0 0x7f8250198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34776/0 (socket says 172.21.15.44:34776) 2024-08-20T03:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f824effd640 1 -- 172.21.15.44:0/2520459746 learned_addr learned my addr 172.21.15.44:0/2520459746 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f8255a29640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f825010cce0 0x7f825019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f824e7fc640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82501040e0 0x7f8250199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:27.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f8255a29640 1 -- 172.21.15.44:0/2520459746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8250112a00 con 0x7f82501037c0 2024-08-20T03:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f8255a29640 1 -- 172.21.15.44:0/2520459746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8250112880 con 0x7f825010cce0 2024-08-20T03:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f8255a29640 1 -- 172.21.15.44:0/2520459746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8250112b80 con 0x7f82501040e0 2024-08-20T03:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f824effd640 1 -- 172.21.15.44:0/2520459746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82501040e0 msgr2=0x7f8250199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f824effd640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82501040e0 0x7f8250199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f824effd640 1 -- 172.21.15.44:0/2520459746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f825010cce0 msgr2=0x7f825019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f824effd640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f825010cce0 0x7f825019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f824effd640 1 -- 172.21.15.44:0/2520459746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f825019ef80 con 0x7f82501037c0 2024-08-20T03:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.480+0000 7f824e7fc640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82501040e0 0x7f8250199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.481+0000 7f824effd640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82501037c0 0x7f8250198ee0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8240005ab0 tx=0x7f8240003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.481+0000 7f8254a27640 1 -- 172.21.15.44:0/2520459746 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824003f070 con 0x7f82501037c0 2024-08-20T03:12:27.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.481+0000 7f8254a27640 1 -- 172.21.15.44:0/2520459746 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8240031d60 con 0x7f82501037c0 2024-08-20T03:12:27.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.482+0000 7f8254a27640 1 -- 172.21.15.44:0/2520459746 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240035030 con 0x7f82501037c0 2024-08-20T03:12:27.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.482+0000 7f8255a29640 1 -- 172.21.15.44:0/2520459746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825019f210 con 0x7f82501037c0 2024-08-20T03:12:27.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.482+0000 7f8255a29640 1 -- 172.21.15.44:0/2520459746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82501a5930 con 0x7f82501037c0 2024-08-20T03:12:27.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.484+0000 7f82367fc640 1 -- 172.21.15.44:0/2520459746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8250103ba0 con 0x7f82501037c0 2024-08-20T03:12:27.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.490+0000 7f8254a27640 1 -- 172.21.15.44:0/2520459746 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8240042070 con 0x7f82501037c0 2024-08-20T03:12:27.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.491+0000 7f8254a27640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f82380b3dc0 0x7f82380b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:27.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.491+0000 7f8254a27640 1 -- 172.21.15.44:0/2520459746 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f8240104500 con 0x7f82501037c0 2024-08-20T03:12:27.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.493+0000 7f824e7fc640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f82380b3dc0 0x7f82380b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:27.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.494+0000 7f824e7fc640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f82380b3dc0 0x7f82380b6280 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f82500ffb60 tx=0x7f8244005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:27.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.495+0000 7f8254a27640 1 -- 172.21.15.44:0/2520459746 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f82400cb7d0 con 0x7f82501037c0 2024-08-20T03:12:27.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:27 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3677265586' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:12:27.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.850+0000 7f82367fc640 1 -- 172.21.15.44:0/2520459746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8250063a80 con 0x7f82501037c0 2024-08-20T03:12:27.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.851+0000 7f8254a27640 1 -- 172.21.15.44:0/2520459746 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7f82400cb0d0 con 0x7f82501037c0 2024-08-20T03:12:27.853 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-08-20T03:12:27.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.855+0000 7f82367fc640 1 -- 172.21.15.44:0/2520459746 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f82380b3dc0 msgr2=0x7f82380b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.855+0000 7f82367fc640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f82380b3dc0 0x7f82380b6280 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f82500ffb60 tx=0x7f8244005f90 comp rx=0 tx=0).stop 2024-08-20T03:12:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.855+0000 7f82367fc640 1 -- 172.21.15.44:0/2520459746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82501037c0 msgr2=0x7f8250198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.855+0000 7f82367fc640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82501037c0 0x7f8250198ee0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8240005ab0 tx=0x7f8240003c80 comp rx=0 tx=0).stop 2024-08-20T03:12:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.855+0000 7f82367fc640 1 -- 172.21.15.44:0/2520459746 shutdown_connections 2024-08-20T03:12:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.855+0000 7f82367fc640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f82380b3dc0 0x7f82380b6280 secure :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f82500ffb60 tx=0x7f8244005f90 comp rx=0 tx=0).stop 2024-08-20T03:12:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.855+0000 7f82367fc640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f825010cce0 0x7f825019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.855+0000 7f82367fc640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82501040e0 0x7f8250199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.855+0000 7f82367fc640 1 --2- 172.21.15.44:0/2520459746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82501037c0 0x7f8250198ee0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:27.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.855+0000 7f82367fc640 1 -- 172.21.15.44:0/2520459746 >> 172.21.15.44:0/2520459746 conn(0x7f82500fd310 msgr2=0x7f825010ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.856+0000 7f82367fc640 1 -- 172.21.15.44:0/2520459746 shutdown_connections 2024-08-20T03:12:27.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:27.856+0000 7f82367fc640 1 -- 172.21.15.44:0/2520459746 wait complete. 2024-08-20T03:12:27.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-20T03:12:28.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.051+0000 7fb94b7fe640 1 -- 172.21.15.44:0/1414250239 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938031810 con 0x7fb94c10a260 2024-08-20T03:12:28.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.052+0000 7fb953193640 1 -- 172.21.15.44:0/1414250239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb94c10a260 msgr2=0x7fb94c10a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:28.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.052+0000 7fb953193640 1 --2- 172.21.15.44:0/1414250239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb94c10a260 0x7fb94c10a680 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb938009900 tx=0x7fb93802a560 comp rx=0 tx=0).stop 2024-08-20T03:12:28.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.052+0000 7fb953193640 1 -- 172.21.15.44:0/1414250239 shutdown_connections 2024-08-20T03:12:28.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.052+0000 7fb953193640 1 --2- 172.21.15.44:0/1414250239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb94c11bdd0 0x7fb94c11e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.052+0000 7fb953193640 1 --2- 172.21.15.44:0/1414250239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb94c10a260 0x7fb94c10a680 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.052+0000 7fb953193640 1 --2- 172.21.15.44:0/1414250239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb94c109e80 0x7fb94c110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.052+0000 7fb953193640 1 -- 172.21.15.44:0/1414250239 >> 172.21.15.44:0/1414250239 conn(0x7fb94c06ea00 msgr2=0x7fb94c0701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:28.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.053+0000 7fb953193640 1 -- 172.21.15.44:0/1414250239 shutdown_connections 2024-08-20T03:12:28.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.055+0000 7fb953193640 1 -- 172.21.15.44:0/1414250239 wait complete. 2024-08-20T03:12:28.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.055+0000 7fb953193640 1 Processor -- start 2024-08-20T03:12:28.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.055+0000 7fb953193640 1 -- start start 2024-08-20T03:12:28.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb953193640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb94c109e80 0x7fb94c119270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:28.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb953193640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb94c10a260 0x7fb94c1197b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:28.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb953193640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb94c11bdd0 0x7fb94c112680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:28.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb953193640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c121890 con 0x7fb94c10a260 2024-08-20T03:12:28.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb953193640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c121710 con 0x7fb94c109e80 2024-08-20T03:12:28.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb953193640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c121a10 con 0x7fb94c11bdd0 2024-08-20T03:12:28.060 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb951730640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb94c11bdd0 0x7fb94c112680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:28.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb950f2f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb94c109e80 0x7fb94c119270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:28.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb951730640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb94c11bdd0 0x7fb94c112680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.44:33918/0 (socket says 172.21.15.44:33918) 2024-08-20T03:12:28.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb951730640 1 -- 172.21.15.44:0/509344002 learned_addr learned my addr 172.21.15.44:0/509344002 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:28.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb950f2f640 1 -- 172.21.15.44:0/509344002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb94c11bdd0 msgr2=0x7fb94c112680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:28.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb950f2f640 1 --2- 172.21.15.44:0/509344002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb94c11bdd0 0x7fb94c112680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb950f2f640 1 -- 172.21.15.44:0/509344002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb94c10a260 msgr2=0x7fb94c1197b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:28.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb950f2f640 1 --2- 172.21.15.44:0/509344002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb94c10a260 0x7fb94c1197b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.056+0000 7fb950f2f640 1 -- 172.21.15.44:0/509344002 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c112f40 con 0x7fb94c109e80 2024-08-20T03:12:28.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.057+0000 7fb950f2f640 1 --2- 172.21.15.44:0/509344002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb94c109e80 0x7fb94c119270 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb940009fd0 tx=0x7fb94000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:28.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.058+0000 7fb949ffb640 1 -- 172.21.15.44:0/509344002 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940015020 con 0x7fb94c109e80 2024-08-20T03:12:28.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.058+0000 7fb953193640 1 -- 172.21.15.44:0/509344002 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94c113230 con 0x7fb94c109e80 2024-08-20T03:12:28.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.058+0000 7fb949ffb640 1 -- 172.21.15.44:0/509344002 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb940011ce0 con 0x7fb94c109e80 2024-08-20T03:12:28.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.058+0000 7fb949ffb640 1 -- 172.21.15.44:0/509344002 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940015020 con 0x7fb94c109e80 2024-08-20T03:12:28.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.058+0000 7fb953193640 1 -- 172.21.15.44:0/509344002 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94c1181f0 con 0x7fb94c109e80 2024-08-20T03:12:28.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.061+0000 7fb953193640 1 -- 172.21.15.44:0/509344002 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94c10a680 con 0x7fb94c109e80 2024-08-20T03:12:28.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.062+0000 7fb949ffb640 1 -- 172.21.15.44:0/509344002 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fb940018020 con 0x7fb94c109e80 2024-08-20T03:12:28.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.063+0000 7fb949ffb640 1 --2- 172.21.15.44:0/509344002 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb93c0b4140 0x7fb93c0b6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:28.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.063+0000 7fb949ffb640 1 -- 172.21.15.44:0/509344002 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7fb940011630 con 0x7fb94c109e80 2024-08-20T03:12:28.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.064+0000 7fb94bfff640 1 --2- 172.21.15.44:0/509344002 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb93c0b4140 0x7fb93c0b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:28.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.065+0000 7fb94bfff640 1 --2- 172.21.15.44:0/509344002 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb93c0b4140 0x7fb93c0b6600 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb93802aa80 tx=0x7fb938000bc0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:28.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.071+0000 7fb949ffb640 1 -- 172.21.15.44:0/509344002 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb94001f020 con 0x7fb94c109e80 2024-08-20T03:12:28.269 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:28 smithi044 ceph-mon[18790]: pgmap v626: 97 pgs: 17 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 44 active+clean; 795 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-08-20T03:12:28.269 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:28 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/995610665' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T03:12:28.269 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:28 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2520459746' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T03:12:28.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:28 smithi165 ceph-mon[22357]: pgmap v626: 97 pgs: 17 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 44 active+clean; 795 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-08-20T03:12:28.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:28 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/995610665' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T03:12:28.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:28 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2520459746' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T03:12:28.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:28 smithi099 ceph-mon[21984]: pgmap v626: 97 pgs: 17 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 44 active+clean; 795 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-08-20T03:12:28.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:28 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/995610665' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T03:12:28.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:28 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2520459746' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T03:12:28.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.430+0000 7fb953193640 1 -- 172.21.15.44:0/509344002 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb94c1186b0 con 0x7fb94c109e80 2024-08-20T03:12:28.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.432+0000 7fb949ffb640 1 -- 172.21.15.44:0/509344002 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7fb9400aa620 con 0x7fb94c109e80 2024-08-20T03:12:28.434 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-08-20T03:12:28.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.438+0000 7fb953193640 1 -- 172.21.15.44:0/509344002 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb93c0b4140 msgr2=0x7fb93c0b6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:28.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.438+0000 7fb953193640 1 --2- 172.21.15.44:0/509344002 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb93c0b4140 0x7fb93c0b6600 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb93802aa80 tx=0x7fb938000bc0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.438+0000 7fb953193640 1 -- 172.21.15.44:0/509344002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb94c109e80 msgr2=0x7fb94c119270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:28.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.438+0000 7fb953193640 1 --2- 172.21.15.44:0/509344002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb94c109e80 0x7fb94c119270 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb940009fd0 tx=0x7fb94000b870 comp rx=0 tx=0).stop 2024-08-20T03:12:28.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.438+0000 7fb953193640 1 -- 172.21.15.44:0/509344002 shutdown_connections 2024-08-20T03:12:28.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.438+0000 7fb953193640 1 --2- 172.21.15.44:0/509344002 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fb93c0b4140 0x7fb93c0b6600 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.438+0000 7fb953193640 1 --2- 172.21.15.44:0/509344002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb94c11bdd0 0x7fb94c112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.438+0000 7fb953193640 1 --2- 172.21.15.44:0/509344002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb94c10a260 0x7fb94c1197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.438+0000 7fb953193640 1 --2- 172.21.15.44:0/509344002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb94c109e80 0x7fb94c119270 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.438+0000 7fb953193640 1 -- 172.21.15.44:0/509344002 >> 172.21.15.44:0/509344002 conn(0x7fb94c06ea00 msgr2=0x7fb94c11c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:28.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.438+0000 7fb953193640 1 -- 172.21.15.44:0/509344002 shutdown_connections 2024-08-20T03:12:28.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.438+0000 7fb953193640 1 -- 172.21.15.44:0/509344002 wait complete. 2024-08-20T03:12:28.459 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-08-20T03:12:28.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.628+0000 7f8b6eb31640 1 -- 172.21.15.44:0/2994166644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b681040d0 msgr2=0x7f8b68104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:28.631 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.628+0000 7f8b6eb31640 1 --2- 172.21.15.44:0/2994166644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b681040d0 0x7f8b68104530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8b580108e0 tx=0x7f8b5802e2e0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.629+0000 7f8b6eb31640 1 -- 172.21.15.44:0/2994166644 shutdown_connections 2024-08-20T03:12:28.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.629+0000 7f8b6eb31640 1 --2- 172.21.15.44:0/2994166644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b68104a70 0x7f8b6810e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.629+0000 7f8b6eb31640 1 --2- 172.21.15.44:0/2994166644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b681040d0 0x7f8b68104530 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.629+0000 7f8b6eb31640 1 --2- 172.21.15.44:0/2994166644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b68101cb0 0x7f8b68102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.634 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.629+0000 7f8b6eb31640 1 -- 172.21.15.44:0/2994166644 >> 172.21.15.44:0/2994166644 conn(0x7f8b680773c0 msgr2=0x7f8b680777d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:28.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.633+0000 7f8b6eb31640 1 -- 172.21.15.44:0/2994166644 shutdown_connections 2024-08-20T03:12:28.635 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.633+0000 7f8b6eb31640 1 -- 172.21.15.44:0/2994166644 wait complete. 2024-08-20T03:12:28.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.634+0000 7f8b6eb31640 1 Processor -- start 2024-08-20T03:12:28.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.634+0000 7f8b6eb31640 1 -- start start 2024-08-20T03:12:28.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.634+0000 7f8b6eb31640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b68101cb0 0x7f8b6806cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:28.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.634+0000 7f8b6eb31640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b681040d0 0x7f8b6806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:28.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6eb31640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b68104a70 0x7f8b68072470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:28.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6eb31640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b681190a0 con 0x7f8b68104a70 2024-08-20T03:12:28.638 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6eb31640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68118f20 con 0x7f8b68101cb0 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6eb31640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68119220 con 0x7f8b681040d0 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6c8cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b68101cb0 0x7f8b6806cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6c8cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b68101cb0 0x7f8b6806cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.44:54262/0 (socket says 172.21.15.44:54262) 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6c8cd640 1 -- 172.21.15.44:0/979261349 learned_addr learned my addr 172.21.15.44:0/979261349 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6d0ce640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b68104a70 0x7f8b68072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b5ffff640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b681040d0 0x7f8b6806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6c8cd640 1 -- 172.21.15.44:0/979261349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b681040d0 msgr2=0x7f8b6806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6c8cd640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b681040d0 0x7f8b6806d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6c8cd640 1 -- 172.21.15.44:0/979261349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b68104a70 msgr2=0x7f8b68072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6c8cd640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b68104a70 0x7f8b68072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.635+0000 7f8b6c8cd640 1 -- 172.21.15.44:0/979261349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b68072c40 con 0x7f8b68101cb0 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.636+0000 7f8b6d0ce640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b68104a70 0x7f8b68072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:12:28.639 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.636+0000 7f8b6c8cd640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b68101cb0 0x7f8b6806cba0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400d770 tx=0x7f8b6400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:28.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.637+0000 7f8b5dffb640 1 -- 172.21.15.44:0/979261349 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64017070 con 0x7f8b68101cb0 2024-08-20T03:12:28.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.637+0000 7f8b5dffb640 1 -- 172.21.15.44:0/979261349 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b640054d0 con 0x7f8b68101cb0 2024-08-20T03:12:28.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.637+0000 7f8b6eb31640 1 -- 172.21.15.44:0/979261349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b68072f30 con 0x7f8b68101cb0 2024-08-20T03:12:28.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.637+0000 7f8b5dffb640 1 -- 172.21.15.44:0/979261349 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64004d50 con 0x7f8b68101cb0 2024-08-20T03:12:28.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.637+0000 7f8b6eb31640 1 -- 172.21.15.44:0/979261349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b680735e0 con 0x7f8b68101cb0 2024-08-20T03:12:28.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.638+0000 7f8b6eb31640 1 -- 172.21.15.44:0/979261349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b68102c60 con 0x7f8b68101cb0 2024-08-20T03:12:28.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.646+0000 7f8b5dffb640 1 -- 172.21.15.44:0/979261349 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8b64009c30 con 0x7f8b68101cb0 2024-08-20T03:12:28.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.647+0000 7f8b5dffb640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8b500b3f50 0x7f8b500b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:28.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.647+0000 7f8b5dffb640 1 -- 172.21.15.44:0/979261349 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8493+0+0 (secure 0 0 0) 0x7f8b640e3df0 con 0x7f8b68101cb0 2024-08-20T03:12:28.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.648+0000 7f8b5dffb640 1 -- 172.21.15.44:0/979261349 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8b640ab1c0 con 0x7f8b68101cb0 2024-08-20T03:12:28.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.648+0000 7f8b5ffff640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8b500b3f50 0x7f8b500b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:28.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:28.662+0000 7f8b5ffff640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8b500b3f50 0x7f8b500b6410 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8b58000c00 tx=0x7f8b580105a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:29.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.112+0000 7f8b6eb31640 1 -- 172.21.15.44:0/979261349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f8b68102090 con 0x7f8b68101cb0 2024-08-20T03:12:29.128 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.126+0000 7f8b5dffb640 1 -- 172.21.15.44:0/979261349 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+821907 (secure 0 0 0) 0x7f8b640aaac0 con 0x7f8b68101cb0 2024-08-20T03:12:29.139 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:12:29.140 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:12:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 27, 2024-08-20T03:12:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: "active_gid": 14220, 2024-08-20T03:12:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: "active_name": "x", 2024-08-20T03:12:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: "active_addrs": { 2024-08-20T03:12:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-08-20T03:12:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-08-20T03:12:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6800", 2024-08-20T03:12:29.140 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 587429394 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:6801", 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 587429394 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: "active_addr": "172.21.15.44:6801/587429394", 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: "active_change": "2024-08-20T02:53:58.312813+0000", 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: "active_mgr_features": 4540701547738038271, 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: "available": true, 2024-08-20T03:12:29.141 INFO:teuthology.orchestra.run.smithi044.stdout: "standbys": [ 2024-08-20T03:12:29.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.142 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 24142, 2024-08-20T03:12:29.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "z", 2024-08-20T03:12:29.142 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_features": 4540701547738038271, 2024-08-20T03:12:29.142 INFO:teuthology.orchestra.run.smithi044.stdout: "available_modules": [ 2024-08-20T03:12:29.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "alerts", 2024-08-20T03:12:29.142 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.142 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.142 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.142 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:29.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How frequently to reexamine health status", 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.143 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.144 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.145 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.146 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.147 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.148 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_destination": { 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_destination", 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.149 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email address to send alerts to", 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_from_name": { 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_from_name", 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "Ceph", 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.150 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email From: name", 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_host": { 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_host", 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.151 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP server", 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_password": { 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_password", 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.152 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Password to authenticate with", 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_port": { 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_port", 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "465", 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.153 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.154 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.154 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP port", 2024-08-20T03:12:29.154 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.154 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.154 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.154 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_sender": { 2024-08-20T03:12:29.154 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_sender", 2024-08-20T03:12:29.154 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.155 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.155 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.155 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.155 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.155 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.155 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.155 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP envelope sender", 2024-08-20T03:12:29.155 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.155 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.156 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.156 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_ssl": { 2024-08-20T03:12:29.156 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_ssl", 2024-08-20T03:12:29.156 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.156 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.156 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.156 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.156 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.156 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.156 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_user": { 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_user", 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.157 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.158 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "User to authenticate as", 2024-08-20T03:12:29.158 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.158 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.158 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.158 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.158 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.158 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.158 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.158 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.158 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "balancer", 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.159 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "active": { 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "active", 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically balance PGs across cluster", 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.160 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_time": { 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_time", 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0000", 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "beginning time of day to automatically balance", 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.161 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_weekday": { 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_weekday", 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.162 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_max_iterations": { 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_max_iterations", 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "25", 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "250", 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.163 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_metrics": { 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_metrics", 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "pgs,objects,bytes", 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-20T03:12:29.164 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.165 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.165 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_step": { 2024-08-20T03:12:29.165 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_step", 2024-08-20T03:12:29.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:29.165 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.5", 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0.001", 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "0.999", 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "aggressiveness of optimization", 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: "end_time": { 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_time", 2024-08-20T03:12:29.166 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2359", 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ending time of day to automatically balance", 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "end_weekday": { 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_weekday", 2024-08-20T03:12:29.167 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.168 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.169 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.170 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.170 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.170 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.170 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.170 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.170 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.170 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.170 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.170 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.170 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.170 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.171 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.173 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: "min_score": { 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_score", 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.174 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "upmap", 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-compat", 2024-08-20T03:12:29.175 INFO:teuthology.orchestra.run.smithi044.stdout: "none", 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: "read", 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap", 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap-read" 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Balancer mode", 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_ids": { 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_ids", 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.176 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.177 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.177 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.177 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.177 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.177 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-20T03:12:29.177 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.177 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.177 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.177 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:29.177 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:29.177 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.178 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_deviation": { 2024-08-20T03:12:29.179 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_deviation", 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_optimizations": { 2024-08-20T03:12:29.180 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_optimizations", 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.181 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm", 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_down_multiplier": { 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_down_multiplier", 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-08-20T03:12:29.182 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.183 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.183 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.183 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-20T03:12:29.183 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.183 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.183 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.183 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.183 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_refresh_rate": { 2024-08-20T03:12:29.183 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_refresh_rate", 2024-08-20T03:12:29.183 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.183 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "20", 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_starting_port": { 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_starting_port", 2024-08-20T03:12:29.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4721", 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_ptrace": { 2024-08-20T03:12:29.185 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_ptrace", 2024-08-20T03:12:29.186 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.186 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.186 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.186 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.186 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.186 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.186 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.186 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-20T03:12:29.186 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-20T03:12:29.186 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.186 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_interval": { 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_interval", 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.187 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.188 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.188 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_memory_target_ratio": { 2024-08-20T03:12:29.188 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_memory_target_ratio", 2024-08-20T03:12:29.188 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:29.188 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.188 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.188 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.7", 2024-08-20T03:12:29.188 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.188 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.188 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.189 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-20T03:12:29.189 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.189 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.189 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.189 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.189 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm_log_destination": { 2024-08-20T03:12:29.189 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm_log_destination", 2024-08-20T03:12:29.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.189 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.189 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.189 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: "file", 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: "file,syslog", 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: "syslog" 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.190 INFO:teuthology.orchestra.run.smithi044.stdout: "cgroups_split": { 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cgroups_split", 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "config_checks_enabled": { 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_checks_enabled", 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.192 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "config_dashboard": { 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_dashboard", 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.193 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.194 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_alertmanager": { 2024-08-20T03:12:29.194 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_alertmanager", 2024-08-20T03:12:29.194 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.194 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.194 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.194 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-20T03:12:29.194 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.194 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.194 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_base": { 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_base", 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.195 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Container image name, without the tag", 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_elasticsearch": { 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_elasticsearch", 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-20T03:12:29.196 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.197 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.197 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.197 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "elasticsearch container image", 2024-08-20T03:12:29.197 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.197 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.197 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.197 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_grafana": { 2024-08-20T03:12:29.197 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_grafana", 2024-08-20T03:12:29.197 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.197 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.197 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_haproxy": { 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_haproxy", 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.198 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.199 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.199 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-20T03:12:29.199 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.199 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.199 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.199 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "HAproxy container image", 2024-08-20T03:12:29.199 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.199 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.199 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.199 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_agent": { 2024-08-20T03:12:29.199 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_agent", 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger agent container image", 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_collector": { 2024-08-20T03:12:29.200 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_collector", 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger collector container image", 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.201 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_query": { 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_query", 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger query container image", 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_keepalived": { 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_keepalived", 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Keepalived container image", 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_loki": { 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_loki", 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Loki container image", 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.204 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nginx": { 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nginx", 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nginx container image", 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.205 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_node_exporter": { 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_node_exporter", 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-08-20T03:12:29.206 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nvmeof": { 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nvmeof", 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nvme-of container image", 2024-08-20T03:12:29.207 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_prometheus": { 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_prometheus", 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-08-20T03:12:29.208 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.209 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.209 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.209 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.209 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_promtail": { 2024-08-20T03:12:29.209 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_promtail", 2024-08-20T03:12:29.209 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.209 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.209 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.209 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-20T03:12:29.209 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.209 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.209 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Promtail container image", 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_samba": { 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_samba", 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.210 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.211 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.211 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Samba/SMB container image", 2024-08-20T03:12:29.211 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.211 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.211 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.211 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.211 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_snmp_gateway": { 2024-08-20T03:12:29.211 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_snmp_gateway", 2024-08-20T03:12:29.211 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.211 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.211 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.211 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-20T03:12:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SNMP Gateway container image", 2024-08-20T03:12:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: "container_init": { 2024-08-20T03:12:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_init", 2024-08-20T03:12:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.212 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Run podman/docker with `--init`", 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "daemon_cache_timeout": { 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "daemon_cache_timeout", 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.213 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.214 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.214 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:29.214 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.214 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.214 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.214 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-20T03:12:29.214 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.214 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.214 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "default_cephadm_command_timeout": { 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_cephadm_command_timeout", 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "900", 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.215 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.216 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.216 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.216 INFO:teuthology.orchestra.run.smithi044.stdout: "default_registry": { 2024-08-20T03:12:29.216 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_registry", 2024-08-20T03:12:29.216 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.216 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.216 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.216 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io", 2024-08-20T03:12:29.216 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.216 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.216 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.216 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: "device_cache_timeout": { 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_cache_timeout", 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1800", 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.217 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache device inventory", 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: "device_enhanced_scan": { 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_enhanced_scan", 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.218 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: "facts_cache_timeout": { 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "facts_cache_timeout", 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:29.219 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache host facts data", 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: "grafana_dashboards_path": { 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "grafana_dashboards_path", 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.220 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "host_check_interval": { 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "host_check_interval", 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.221 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to perform a host check", 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: "hw_monitoring": { 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hw_monitoring", 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.222 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "inventory_list_all": { 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "inventory_list_all", 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.223 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.224 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.225 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.225 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.225 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.225 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.225 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.225 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.225 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.225 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.225 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.225 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.225 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.225 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: "log_refresh_metadata": { 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_refresh_metadata", 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.226 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.227 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.228 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.229 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.230 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.231 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "*", 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.232 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: "max_count_per_host": { 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_count_per_host", 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of daemons per service per host", 2024-08-20T03:12:29.233 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd_draining_count": { 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_osd_draining_count", 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.234 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: "migration_current": { 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "migration_current", 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.235 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "internal - do not modify", 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "root", 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.236 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm-package", 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: "root" 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "mode for remote execution of cephadm", 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: "oob_default_addr": { 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "oob_default_addr", 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.237 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "169.254.1.1", 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: "prometheus_alerts_path": { 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus_alerts_path", 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.238 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_insecure": { 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_insecure", 2024-08-20T03:12:29.239 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_password": { 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_password", 2024-08-20T03:12:29.240 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.241 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.241 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.241 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.241 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.241 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.241 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.241 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-20T03:12:29.241 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.241 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.241 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.241 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_url": { 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_url", 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_username": { 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_username", 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.243 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "secure_monitoring_stack": { 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secure_monitoring_stack", 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.244 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "service_discovery_port": { 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "service_discovery_port", 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8765", 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "cephadm service discovery port", 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.245 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.246 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: "ssh_config_file": { 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssh_config_file", 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.247 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "use_agent": { 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_agent", 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.248 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: "use_repo_digest": { 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_repo_digest", 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-20T03:12:29.249 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_failed_host_check": { 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_failed_host_check", 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.250 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if the host check fails", 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_daemons": { 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_daemons", 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.251 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_hosts": { 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_hosts", 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.252 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crash", 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.253 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.254 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.255 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.256 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.257 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.258 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: "retain_interval": { 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retain_interval", 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "31536000", 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.259 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.260 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_recent_interval": { 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_recent_interval", 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1209600", 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.261 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dashboard", 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-20T03:12:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-20T03:12:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.263 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_HOST": { 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.264 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.265 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_ENABLED": { 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_ENABLED", 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.266 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.267 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: "ENABLE_BROWSABLE_API": { 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.268 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.269 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.270 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.270 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.270 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.270 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.270 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.270 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-20T03:12:29.270 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-20T03:12:29.270 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.270 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.270 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.270 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.271 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.272 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.273 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.274 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.275 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.276 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_PASSWORD": { 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.277 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.278 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.278 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-08-20T03:12:29.278 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.278 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.278 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.278 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.278 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.278 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.278 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.278 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-20T03:12:29.278 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.279 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_URL": { 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_URL", 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.280 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_USERNAME": { 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.283 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.284 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.285 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "[]", 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.286 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "MULTICLUSTER_CONFIG": { 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "{}", 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.287 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_HOST": { 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.288 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.289 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.290 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-20T03:12:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-20T03:12:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.291 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.292 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.293 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-20T03:12:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-20T03:12:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.294 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-20T03:12:29.295 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-20T03:12:29.296 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_ENABLED": { 2024-08-20T03:12:29.297 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.298 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.300 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8", 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.301 INFO:teuthology.orchestra.run.smithi044.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "45", 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ACCESS_KEY": { 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.303 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.304 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SECRET_KEY": { 2024-08-20T03:12:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-20T03:12:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.305 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SSL_VERIFY": { 2024-08-20T03:12:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-20T03:12:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.306 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "UNSAFE_TLS_v1_2": { 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.307 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.308 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.309 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-20T03:12:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-20T03:12:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.310 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "cross_origin_url": { 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cross_origin_url", 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.311 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "crt_file": { 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crt_file", 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.312 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "debug": { 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "debug", 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.313 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable/disable debug options", 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "jwt_token_ttl": { 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "jwt_token_ttl", 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.314 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.315 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.315 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "28800", 2024-08-20T03:12:29.315 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.315 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.315 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.315 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.315 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.315 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.315 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.316 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.317 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.318 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.319 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.319 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.319 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.319 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.319 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.319 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.319 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.319 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.319 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.319 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.319 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.320 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.321 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.321 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.321 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.321 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.321 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.321 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.321 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.321 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.321 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.321 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.321 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: "motd": { 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "motd", 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.322 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "The message of the day", 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect_resolve_ip_addr": { 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "redirect_resolve_ip_addr", 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.323 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.324 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-08-20T03:12:29.325 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8080", 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.326 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.327 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.328 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl_server_port": { 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl_server_port", 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8443", 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.329 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "redirect", 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect" 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.330 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.331 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "url_prefix": { 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url_prefix", 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.332 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_monitoring": { 2024-08-20T03:12:29.333 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_monitoring", 2024-08-20T03:12:29.334 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.334 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.334 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.334 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.334 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.334 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.334 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.334 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "monitor device health metrics", 2024-08-20T03:12:29.334 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.334 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.334 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.335 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.336 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.337 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.338 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.339 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: "mark_out_threshold": { 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mark_out_threshold", 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2419200", 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.340 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": { 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_name", 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "device_health_metrics", 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.341 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.342 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-20T03:12:29.342 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.342 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.342 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.342 INFO:teuthology.orchestra.run.smithi044.stdout: "retention_period": { 2024-08-20T03:12:29.342 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retention_period", 2024-08-20T03:12:29.342 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.342 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.342 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.342 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15552000", 2024-08-20T03:12:29.342 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain device health metrics", 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_frequency": { 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_frequency", 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-08-20T03:12:29.343 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "self_heal": { 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "self_heal", 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:29.345 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and check device health", 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.346 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_threshold": { 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_threshold", 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.347 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7257600", 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.348 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "diskprediction_local", 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.349 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.350 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.351 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.352 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.353 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "predict_interval": { 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predict_interval", 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.354 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "predictor_model": { 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predictor_model", 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.355 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "prophetstor", 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.356 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.357 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.358 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "influx", 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": false, 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "influxdb python module not found", 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "batch_size": { 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "batch_size", 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5000", 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-20T03:12:29.359 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "database": { 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "database", 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "ceph", 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-20T03:12:29.360 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": { 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hostname", 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server hostname", 2024-08-20T03:12:29.361 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "30", 2024-08-20T03:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "5", 2024-08-20T03:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.362 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.363 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.365 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.366 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.369 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.370 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.371 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.371 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.371 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.371 INFO:teuthology.orchestra.run.smithi044.stdout: "password": { 2024-08-20T03:12:29.371 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "password", 2024-08-20T03:12:29.371 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.371 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.371 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.371 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.371 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.371 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "password of InfluxDB server user", 2024-08-20T03:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: "port": { 2024-08-20T03:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "port", 2024-08-20T03:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.372 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8086", 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server port", 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.373 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.374 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-08-20T03:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-08-20T03:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "false", 2024-08-20T03:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T03:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.375 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "threads": { 2024-08-20T03:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threads", 2024-08-20T03:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-08-20T03:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "32", 2024-08-20T03:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-20T03:12:29.376 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "username": { 2024-08-20T03:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "username", 2024-08-20T03:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.377 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "username of InfluxDB server user", 2024-08-20T03:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "verify_ssl": { 2024-08-20T03:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "verify_ssl", 2024-08-20T03:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.378 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "true", 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "insights", 2024-08-20T03:12:29.379 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.380 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.381 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.382 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.384 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.385 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.386 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "iostat", 2024-08-20T03:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.387 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.388 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.389 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.390 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.391 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.392 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.393 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.394 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "k8sevents", 2024-08-20T03:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_event_retention_days": { 2024-08-20T03:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ceph_event_retention_days", 2024-08-20T03:12:29.395 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7", 2024-08-20T03:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-20T03:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "config_check_secs": { 2024-08-20T03:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_check_secs", 2024-08-20T03:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "10", 2024-08-20T03:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-20T03:12:29.397 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.398 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.399 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.400 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.401 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.402 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.403 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.404 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.405 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "localpool", 2024-08-20T03:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "failure_domain": { 2024-08-20T03:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "failure_domain", 2024-08-20T03:12:29.406 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "host", 2024-08-20T03:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "failure domain for any created local pool", 2024-08-20T03:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-20T03:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.407 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.408 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.409 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.411 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.412 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.413 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": { 2024-08-20T03:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_size", 2024-08-20T03:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.414 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default min_size for any created local pool", 2024-08-20T03:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-20T03:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "num_rep": { 2024-08-20T03:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "num_rep", 2024-08-20T03:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.415 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3", 2024-08-20T03:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default replica count for any created local pool", 2024-08-20T03:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": { 2024-08-20T03:12:29.416 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_num", 2024-08-20T03:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "128", 2024-08-20T03:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default pg_num for any created local pool", 2024-08-20T03:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.417 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "prefix": { 2024-08-20T03:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prefix", 2024-08-20T03:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name prefix for any created local pool", 2024-08-20T03:12:29.418 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.419 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "subtree": { 2024-08-20T03:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subtree", 2024-08-20T03:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "rack", 2024-08-20T03:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.420 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mds_autoscaler", 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.421 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.422 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.423 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.424 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.425 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.426 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.427 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.428 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirroring", 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.429 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.430 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.431 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.432 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.433 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.434 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.435 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.436 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "nfs", 2024-08-20T03:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.437 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.439 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.439 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.439 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.439 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.439 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.439 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.439 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.439 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.439 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.439 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.439 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.439 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.440 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.440 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.440 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.440 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.440 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.440 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.440 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.440 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.440 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.440 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.441 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.441 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.441 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.441 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.441 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.441 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.441 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.441 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.441 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.441 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.441 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.442 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.442 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.442 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.442 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.442 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.442 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.442 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.442 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.442 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.442 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.443 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.444 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.444 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.444 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.444 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.444 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.444 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.444 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.444 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-08-20T03:12:29.444 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.444 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.444 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "fail_fs": { 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "fail_fs", 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.445 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.446 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.446 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.446 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.446 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.446 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.446 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.446 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.446 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.446 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.446 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.446 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.447 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.447 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.447 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.447 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.447 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.447 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.447 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.447 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.447 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.447 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.447 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.448 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.448 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.448 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.448 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.448 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.448 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.448 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.448 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.448 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.448 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.448 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.449 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.449 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.449 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.449 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.449 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.449 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.449 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.449 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.449 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.449 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.449 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.450 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.450 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.450 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.450 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.450 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.450 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.450 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.450 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.450 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.450 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.452 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator": { 2024-08-20T03:12:29.452 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-08-20T03:12:29.452 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.452 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.452 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.452 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.452 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.452 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.452 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.452 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2024-08-20T03:12:29.452 INFO:teuthology.orchestra.run.smithi044.stdout: "rook", 2024-08-20T03:12:29.452 INFO:teuthology.orchestra.run.smithi044.stdout: "test_orchestrator" 2024-08-20T03:12:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Orchestrator backend", 2024-08-20T03:12:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.453 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.454 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.454 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.454 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.454 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.454 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.454 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.454 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.454 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.454 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.454 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.454 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_perf_query", 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.455 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.456 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.456 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.456 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.456 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.456 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.456 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.456 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.456 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.456 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.456 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.457 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.458 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.458 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.458 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.458 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.458 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.458 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.458 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.458 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.458 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.459 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.459 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.459 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.459 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.459 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.459 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.459 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.459 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.459 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.459 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.459 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.459 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.460 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.460 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.460 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.460 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.460 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.460 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.460 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.460 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.460 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.460 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.460 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.460 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.461 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.462 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.462 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.462 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.462 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.462 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.462 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.462 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_support", 2024-08-20T03:12:29.462 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.462 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.462 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.462 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.463 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.463 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.463 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.463 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.463 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.463 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.463 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.463 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.463 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.463 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.463 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.464 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.464 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.464 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.464 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.464 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.464 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.464 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.464 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.464 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.464 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.465 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.465 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.465 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.465 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.465 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.465 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.465 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.465 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.465 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.465 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.465 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.466 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.466 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.466 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.466 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.466 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.466 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.466 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.466 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.466 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.466 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.466 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.467 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.467 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.467 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.467 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.467 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.467 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.467 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.467 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.467 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.467 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.467 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.468 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.468 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.468 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.468 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.468 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.468 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.468 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.468 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.468 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.468 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.468 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.469 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.469 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.469 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.469 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.469 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.469 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.469 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.469 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.469 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.469 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.469 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.469 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.470 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.470 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.470 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.470 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.470 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_autoscaler", 2024-08-20T03:12:29.470 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.470 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.470 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.470 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.470 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.470 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.471 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.471 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.471 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.471 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.471 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.471 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.471 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.471 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.471 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.471 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.471 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.471 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.472 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.472 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.472 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.472 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.472 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.472 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.472 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.472 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.472 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.472 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.472 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.473 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.473 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.473 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.473 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.473 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.473 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.473 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.473 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.473 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.473 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.474 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.474 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.474 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.474 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.474 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.474 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.474 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.474 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.474 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.474 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.474 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.475 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.475 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.475 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.475 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.475 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.475 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.475 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.475 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.475 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.475 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.475 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.475 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.476 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.476 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.476 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.476 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.476 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.476 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.476 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.476 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.476 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:29.476 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:29.476 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.477 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.477 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.477 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:29.477 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.477 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.477 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.477 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.477 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.477 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.477 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.478 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.478 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.478 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.478 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.478 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.478 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.478 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.478 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.478 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.478 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.478 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.479 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.479 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.479 INFO:teuthology.orchestra.run.smithi044.stdout: "threshold": { 2024-08-20T03:12:29.479 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threshold", 2024-08-20T03:12:29.479 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:29.479 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.479 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.479 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-08-20T03:12:29.479 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1.0", 2024-08-20T03:12:29.479 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.480 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.480 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "scaling threshold", 2024-08-20T03:12:29.480 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-20T03:12:29.480 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.480 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.480 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.480 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.480 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.480 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "progress", 2024-08-20T03:12:29.480 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_pg_recovery_event": { 2024-08-20T03:12:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_pg_recovery_event", 2024-08-20T03:12:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.481 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-20T03:12:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-08-20T03:12:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-08-20T03:12:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.482 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.483 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.484 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.485 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.486 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.487 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.488 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.489 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "max_completed_events": { 2024-08-20T03:12:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_completed_events", 2024-08-20T03:12:29.490 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "50", 2024-08-20T03:12:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "number of past completed events to remember", 2024-08-20T03:12:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long the module is going to sleep", 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.492 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.493 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus", 2024-08-20T03:12:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: "cache": { 2024-08-20T03:12:29.494 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cache", 2024-08-20T03:12:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.495 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "exclude_perf_counters": { 2024-08-20T03:12:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "exclude_perf_counters", 2024-08-20T03:12:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.496 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-20T03:12:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-20T03:12:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.497 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.498 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.499 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.500 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.501 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.502 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools": { 2024-08-20T03:12:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools", 2024-08-20T03:12:29.503 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.504 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-20T03:12:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-20T03:12:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300", 2024-08-20T03:12:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.505 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_interval": { 2024-08-20T03:12:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_interval", 2024-08-20T03:12:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15.0", 2024-08-20T03:12:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.506 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-08-20T03:12:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-08-20T03:12:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-08-20T03:12:29.507 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-20T03:12:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-08-20T03:12:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-08-20T03:12:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.508 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "9283", 2024-08-20T03:12:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-20T03:12:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.509 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "stale_cache_strategy": { 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stale_cache_strategy", 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "log", 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.511 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-08-20T03:12:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-08-20T03:12:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "default", 2024-08-20T03:12:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "default", 2024-08-20T03:12:29.512 INFO:teuthology.orchestra.run.smithi044.stdout: "error" 2024-08-20T03:12:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-08-20T03:12:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-08-20T03:12:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.513 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-08-20T03:12:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-08-20T03:12:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-08-20T03:12:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.514 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2024-08-20T03:12:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.515 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.516 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.516 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.516 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.516 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.516 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.516 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.516 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.516 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.516 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.516 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.516 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.517 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.517 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.517 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.517 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.517 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.517 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.517 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.517 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.517 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.517 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.518 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.518 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.518 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.518 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.518 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.518 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.518 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.518 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.518 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.518 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.519 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.519 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.519 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.519 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.519 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.519 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.519 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.519 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.519 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.519 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.519 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.519 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.520 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_snap_create": { 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_snap_create", 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.521 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.522 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.522 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.522 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.522 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.522 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.522 INFO:teuthology.orchestra.run.smithi044.stdout: "mirror_snapshot_schedule": { 2024-08-20T03:12:29.522 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirror_snapshot_schedule", 2024-08-20T03:12:29.522 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.522 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.522 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.522 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.523 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: "trash_purge_schedule": { 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "trash_purge_schedule", 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.524 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.525 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.525 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.525 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.525 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.525 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.525 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.525 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.525 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.525 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.525 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.525 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.526 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.526 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "restful", 2024-08-20T03:12:29.526 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.526 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.526 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.526 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_auth": { 2024-08-20T03:12:29.526 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_auth", 2024-08-20T03:12:29.526 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.526 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.526 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.526 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.526 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.527 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.528 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.528 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.528 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.528 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.528 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.528 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.528 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.528 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.528 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.528 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.528 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.529 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.530 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.530 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.530 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.530 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.530 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.530 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.530 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.530 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.530 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.530 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.530 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.531 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.531 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.531 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.531 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.531 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.531 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.531 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.531 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.531 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.531 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.531 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.532 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.533 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.533 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.533 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.533 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.533 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.533 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.533 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.533 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.533 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.533 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.533 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.534 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.534 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.534 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.534 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.534 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-08-20T03:12:29.534 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-08-20T03:12:29.534 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.534 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.534 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.534 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.534 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.534 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.535 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.535 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.535 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.535 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.535 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.535 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-08-20T03:12:29.535 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-08-20T03:12:29.535 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.535 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.535 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.536 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.536 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.536 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.536 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.536 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.536 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.536 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.536 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.536 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.536 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.537 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.537 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.537 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.537 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.537 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.537 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.537 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.537 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.537 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.537 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.537 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.537 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.538 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.538 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.538 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.538 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rgw", 2024-08-20T03:12:29.538 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.538 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.538 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.538 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.538 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.538 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.539 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.539 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.539 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.539 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.539 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.539 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.539 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.539 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.539 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.539 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.539 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.539 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.540 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.540 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.540 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.540 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.540 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.540 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.540 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.540 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.540 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.540 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.540 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.541 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.541 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.541 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.541 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.541 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.541 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.541 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.541 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.541 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.541 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.541 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.542 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.542 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.542 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.542 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.542 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.542 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.542 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.542 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.542 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.542 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.542 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.543 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.543 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.543 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.543 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.543 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.543 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.543 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.543 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.543 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.543 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.543 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.544 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.544 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.544 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.544 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.544 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.544 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.544 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.544 INFO:teuthology.orchestra.run.smithi044.stdout: "secondary_zone_period_retry_limit": { 2024-08-20T03:12:29.544 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-20T03:12:29.544 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.545 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.546 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.546 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.546 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.546 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.546 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.546 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.546 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.546 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.546 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.546 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.546 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rook", 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.547 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.548 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.549 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.550 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.551 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.551 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.551 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.551 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.551 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.551 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.551 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.551 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.551 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.551 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.551 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.551 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.552 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.552 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.552 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.552 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.552 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.552 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.552 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.552 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.552 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.552 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.552 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.552 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.553 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: "storage_class": { 2024-08-20T03:12:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "storage_class", 2024-08-20T03:12:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "local", 2024-08-20T03:12:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.554 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "selftest", 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.555 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.556 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.556 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.556 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.556 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.556 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.556 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.556 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.556 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.556 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.556 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.556 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.556 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.557 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.558 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.558 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.558 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.558 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.558 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.558 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.558 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.558 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.558 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.558 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.558 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.559 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.560 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.561 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.561 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.561 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.561 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.561 INFO:teuthology.orchestra.run.smithi044.stdout: "roption1": { 2024-08-20T03:12:29.561 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption1", 2024-08-20T03:12:29.561 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.561 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.561 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.561 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.561 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.562 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.562 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.562 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.562 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.562 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.562 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.562 INFO:teuthology.orchestra.run.smithi044.stdout: "roption2": { 2024-08-20T03:12:29.562 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption2", 2024-08-20T03:12:29.562 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.562 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.562 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "xyz", 2024-08-20T03:12:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption1": { 2024-08-20T03:12:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption1", 2024-08-20T03:12:29.563 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.564 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.564 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.564 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.564 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.564 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.564 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.564 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.564 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.564 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.564 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.564 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.564 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption2": { 2024-08-20T03:12:29.565 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption2", 2024-08-20T03:12:29.565 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.565 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.565 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.565 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.565 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.565 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.565 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.565 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.565 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.565 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.565 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.566 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption3": { 2024-08-20T03:12:29.566 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption3", 2024-08-20T03:12:29.566 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:29.566 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.566 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.566 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.566 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.566 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.566 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.566 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.567 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.567 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.567 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.567 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption4": { 2024-08-20T03:12:29.567 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption4", 2024-08-20T03:12:29.567 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.567 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.567 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.567 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.567 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.567 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption5": { 2024-08-20T03:12:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption5", 2024-08-20T03:12:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.568 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption6": { 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption6", 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.569 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.570 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.570 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.570 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.570 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.570 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.570 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.570 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.570 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.570 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.570 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption7": { 2024-08-20T03:12:29.570 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption7", 2024-08-20T03:12:29.571 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.571 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.571 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.571 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.571 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-08-20T03:12:29.571 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "42", 2024-08-20T03:12:29.571 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.571 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.571 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.571 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.571 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.572 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.573 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.573 INFO:teuthology.orchestra.run.smithi044.stdout: "testkey": { 2024-08-20T03:12:29.573 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testkey", 2024-08-20T03:12:29.573 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.573 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.573 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.573 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.573 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.573 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.573 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.573 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.574 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.574 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.574 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.574 INFO:teuthology.orchestra.run.smithi044.stdout: "testlkey": { 2024-08-20T03:12:29.574 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testlkey", 2024-08-20T03:12:29.574 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.574 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.574 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.574 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.574 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.575 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.575 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.575 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.575 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.575 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.575 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.575 INFO:teuthology.orchestra.run.smithi044.stdout: "testnewline": { 2024-08-20T03:12:29.575 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testnewline", 2024-08-20T03:12:29.575 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.575 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.576 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.576 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.576 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.576 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.576 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.576 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.576 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.576 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.576 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.577 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.577 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smb", 2024-08-20T03:12:29.577 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.577 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.577 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.577 INFO:teuthology.orchestra.run.smithi044.stdout: "internal_store_backend": { 2024-08-20T03:12:29.577 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "internal_store_backend", 2024-08-20T03:12:29.577 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.577 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.577 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.577 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.577 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-20T03:12:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.578 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.579 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.579 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.579 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.579 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.579 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.579 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.579 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.579 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.579 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.579 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.579 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.579 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.580 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.580 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.580 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.580 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.580 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.580 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.580 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.580 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.580 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.580 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.580 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.581 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.581 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.581 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.581 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.581 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.581 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.581 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.581 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.581 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.581 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.581 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.582 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.582 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.582 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.582 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.582 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.582 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.582 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.582 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.582 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.582 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.582 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.582 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.583 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.583 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.583 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.583 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.583 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.583 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.583 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.583 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.583 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.583 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.584 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.584 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.584 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.584 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.584 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.584 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.584 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.584 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.584 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.584 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.584 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.584 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.585 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.585 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.585 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.585 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.585 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.585 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.585 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.585 INFO:teuthology.orchestra.run.smithi044.stdout: "update_orchestration": { 2024-08-20T03:12:29.585 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "update_orchestration", 2024-08-20T03:12:29.585 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.585 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.586 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.586 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.586 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.586 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.586 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.586 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-20T03:12:29.586 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.586 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.586 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.586 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.586 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.587 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.587 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-08-20T03:12:29.587 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.587 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.587 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.587 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_m_granularity": { 2024-08-20T03:12:29.587 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_m_granularity", 2024-08-20T03:12:29.587 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.587 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.587 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.588 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.588 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.588 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.588 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.588 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow minute scheduled snapshots", 2024-08-20T03:12:29.588 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.588 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.588 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.588 INFO:teuthology.orchestra.run.smithi044.stdout: "dump_on_update": { 2024-08-20T03:12:29.588 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dump_on_update", 2024-08-20T03:12:29.588 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.589 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.589 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.589 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.589 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.589 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.589 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.589 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "dump database to debug log on update", 2024-08-20T03:12:29.589 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.589 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.589 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.589 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.590 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.590 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.590 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.590 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.590 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.590 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.590 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.590 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.590 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.590 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.590 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.591 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.591 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.591 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.591 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.591 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.591 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.591 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.591 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.591 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.591 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.591 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.592 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.592 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.592 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.592 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.592 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.592 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.592 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.592 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.592 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.592 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.592 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.593 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.594 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.594 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.594 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.594 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.594 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.594 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.594 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.594 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.594 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.594 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.595 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.595 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.595 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.595 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.595 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.595 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.595 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.595 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.595 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.595 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.595 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.595 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.596 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.596 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.596 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.596 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.596 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.596 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.596 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.596 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.596 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.596 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.597 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.597 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.597 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.597 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.597 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.597 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stats", 2024-08-20T03:12:29.597 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.597 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.597 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.597 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.597 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.597 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.598 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.598 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.598 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.598 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.598 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.598 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.598 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.598 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.598 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.598 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.599 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.599 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.599 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.599 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.599 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.599 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.599 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.599 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.599 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.599 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.599 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.600 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.600 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.600 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.600 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.600 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.600 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.600 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.600 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.600 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.600 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.601 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.601 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.601 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.601 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.601 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.601 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.601 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.601 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.601 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.601 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.601 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.602 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.602 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.602 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.602 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.602 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.602 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.602 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.602 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.602 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.602 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.602 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.603 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.603 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.603 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.603 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.603 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.603 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.603 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.603 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.603 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.603 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.603 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.604 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.604 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.604 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.604 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.604 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.604 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.604 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.604 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.604 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.604 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.604 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.604 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.605 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.605 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.605 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.605 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.605 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "status", 2024-08-20T03:12:29.605 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.605 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.605 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.605 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.605 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.606 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.606 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.606 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.606 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.606 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.606 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.606 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.606 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.606 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.606 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.606 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.607 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.607 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.607 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.607 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.607 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.607 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.607 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.607 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.607 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.607 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.608 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.608 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.608 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.608 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.608 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.608 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.608 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.608 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.608 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.608 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.608 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.609 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.609 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.609 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.609 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.609 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.609 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.609 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.609 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.609 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.609 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.609 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.609 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.610 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.610 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.610 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.610 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.610 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.610 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.610 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.610 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.610 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.610 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.611 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.611 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.611 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.611 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.611 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.611 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.611 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.611 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.611 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.611 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.612 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.612 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.612 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.612 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.612 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.612 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.612 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.612 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.612 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.612 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.612 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.612 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.613 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.613 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.613 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.613 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.613 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telegraf", 2024-08-20T03:12:29.613 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.613 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.613 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.613 INFO:teuthology.orchestra.run.smithi044.stdout: "address": { 2024-08-20T03:12:29.613 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "address", 2024-08-20T03:12:29.613 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.614 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.614 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.614 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-20T03:12:29.614 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.614 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.614 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.614 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.614 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.614 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.614 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.615 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:29.615 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:29.615 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.615 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.615 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.615 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15", 2024-08-20T03:12:29.615 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.615 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.615 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.615 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.615 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.615 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.616 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.616 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.616 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.616 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.616 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.616 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.616 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.616 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.616 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.616 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.617 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.617 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.617 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.617 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.617 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.617 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.617 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.617 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.617 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.617 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.617 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.618 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.618 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.618 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.618 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.618 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.618 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.618 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.618 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.618 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.618 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.619 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.619 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.619 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.619 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.619 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.619 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.619 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.619 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.619 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.619 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.620 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.620 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.620 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.620 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.620 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.620 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.620 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.620 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.620 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.620 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.620 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.621 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.621 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.621 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.621 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.621 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.621 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.621 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.621 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.621 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.621 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.622 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.622 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.622 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.622 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.622 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.622 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.622 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.622 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.622 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.622 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.622 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.623 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.623 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.623 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.623 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.623 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.623 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.623 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.623 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.624 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telemetry", 2024-08-20T03:12:29.624 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.624 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.624 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.624 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_basic": { 2024-08-20T03:12:29.624 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_basic", 2024-08-20T03:12:29.624 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.624 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.624 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.624 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.624 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.624 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.625 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.625 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-20T03:12:29.625 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.625 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.625 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.625 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_crash": { 2024-08-20T03:12:29.625 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_crash", 2024-08-20T03:12:29.625 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.625 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.625 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.625 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.626 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.626 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.626 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.626 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-20T03:12:29.626 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.626 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.626 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.626 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_device": { 2024-08-20T03:12:29.626 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_device", 2024-08-20T03:12:29.626 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.626 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_ident": { 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_ident", 2024-08-20T03:12:29.627 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_perf": { 2024-08-20T03:12:29.628 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_perf", 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.629 INFO:teuthology.orchestra.run.smithi044.stdout: "contact": { 2024-08-20T03:12:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "contact", 2024-08-20T03:12:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.630 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: "description": { 2024-08-20T03:12:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "description", 2024-08-20T03:12:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.631 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: "device_url": { 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_url", 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.632 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-08-20T03:12:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-08-20T03:12:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.633 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.634 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.634 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.634 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.634 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.634 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.634 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.634 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:29.634 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:29.634 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.634 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.635 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.635 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "24", 2024-08-20T03:12:29.635 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "8", 2024-08-20T03:12:29.635 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.635 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.635 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.635 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.635 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.635 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.635 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.635 INFO:teuthology.orchestra.run.smithi044.stdout: "last_opt_revision": { 2024-08-20T03:12:29.635 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "last_opt_revision", 2024-08-20T03:12:29.636 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.636 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.636 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.636 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1", 2024-08-20T03:12:29.636 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.636 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.636 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.636 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.636 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.636 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.636 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard": { 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard", 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.637 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.638 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard_description": { 2024-08-20T03:12:29.638 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard_description", 2024-08-20T03:12:29.638 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.638 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.638 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.638 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.638 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.638 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.638 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.638 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.639 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.639 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.639 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.639 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.639 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.639 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.639 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.639 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.639 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.639 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.639 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.640 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.640 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.640 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.640 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.640 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.640 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.640 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.640 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.640 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.640 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.640 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.640 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.641 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.641 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.641 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.641 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.641 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.641 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.641 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.641 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.641 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.641 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.642 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.642 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.642 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.642 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.642 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.642 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.642 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.642 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.642 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.642 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.643 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.643 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.643 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.643 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.643 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.643 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.643 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.643 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.643 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.643 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.643 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.644 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.644 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.644 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.644 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.644 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.644 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.644 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.644 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.645 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.645 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.645 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.645 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.645 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.645 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.645 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.645 INFO:teuthology.orchestra.run.smithi044.stdout: "organization": { 2024-08-20T03:12:29.645 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "organization", 2024-08-20T03:12:29.645 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.646 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.646 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.646 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.646 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.646 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.646 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.646 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.646 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.646 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.646 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.647 INFO:teuthology.orchestra.run.smithi044.stdout: "proxy": { 2024-08-20T03:12:29.647 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "proxy", 2024-08-20T03:12:29.647 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.647 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.647 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.647 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.647 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.647 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.647 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.647 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.647 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.647 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.648 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.648 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.648 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.648 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.648 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.648 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.648 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.648 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.648 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.648 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.649 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.649 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.649 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.649 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.649 INFO:teuthology.orchestra.run.smithi044.stdout: "url": { 2024-08-20T03:12:29.649 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url", 2024-08-20T03:12:29.649 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.649 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.649 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.649 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-20T03:12:29.649 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "test_orchestrator", 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.650 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.651 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.651 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.651 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.651 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.651 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.651 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.651 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.651 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.651 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.651 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.651 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.651 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.652 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.653 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.654 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.654 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.654 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.654 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.654 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.654 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.654 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.654 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.654 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.654 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.654 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.654 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.655 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.655 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.655 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.655 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.655 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.655 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.655 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.655 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.655 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.655 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.656 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.656 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.656 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.656 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.656 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.656 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.656 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.656 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.656 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.656 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.656 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.657 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.658 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2024-08-20T03:12:29.658 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.658 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.658 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.658 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.658 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.658 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.658 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.658 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.658 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.658 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.658 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.659 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.660 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.660 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.660 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.660 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.660 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.660 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.660 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.660 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.660 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.660 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.660 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.660 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.661 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.662 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.663 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.663 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.663 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.663 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.663 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.663 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.663 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.663 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.663 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.663 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_clones": { 2024-08-20T03:12:29.663 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_clones", 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4", 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.664 INFO:teuthology.orchestra.run.smithi044.stdout: "periodic_async_work": { 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "periodic_async_work", 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Periodically check for async work", 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.666 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_delay": { 2024-08-20T03:12:29.666 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_delay", 2024-08-20T03:12:29.666 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.666 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.666 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.666 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.666 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.666 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.666 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_no_wait": { 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_no_wait", 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.667 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.668 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.669 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.669 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.669 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.669 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.669 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.669 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.669 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.669 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.669 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.669 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.669 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix", 2024-08-20T03:12:29.669 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.670 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.670 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.670 INFO:teuthology.orchestra.run.smithi044.stdout: "discovery_interval": { 2024-08-20T03:12:29.670 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "discovery_interval", 2024-08-20T03:12:29.670 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:29.670 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.670 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.670 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "100", 2024-08-20T03:12:29.670 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.670 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.670 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.670 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.671 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.671 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.671 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.671 INFO:teuthology.orchestra.run.smithi044.stdout: "identifier": { 2024-08-20T03:12:29.671 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "identifier", 2024-08-20T03:12:29.671 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.671 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.671 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.671 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.671 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.671 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:29.672 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.673 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.673 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.673 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.673 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.673 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.673 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.673 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.673 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.673 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.673 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.673 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.673 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.674 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.674 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.674 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.674 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.674 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.674 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.674 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.674 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.674 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.674 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.675 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.675 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.675 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.675 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.675 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.675 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.675 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.675 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.675 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.676 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.676 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.676 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.676 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.676 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.676 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.676 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.676 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.676 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.677 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.677 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.677 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.677 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.677 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.677 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.677 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.677 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.677 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.677 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.678 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.678 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.678 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.678 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.678 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.678 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.678 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.678 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.678 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.678 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.679 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.680 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.681 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.681 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_host": { 2024-08-20T03:12:29.681 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_host", 2024-08-20T03:12:29.681 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.681 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.681 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.681 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.681 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.681 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.681 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.682 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.682 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.682 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.682 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.682 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_port": { 2024-08-20T03:12:29.682 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_port", 2024-08-20T03:12:29.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.682 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.682 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.683 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10051", 2024-08-20T03:12:29.683 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.683 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.683 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.683 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.683 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.683 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.683 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.683 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_sender": { 2024-08-20T03:12:29.683 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_sender", 2024-08-20T03:12:29.683 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.684 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.684 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.684 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-20T03:12:29.684 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.684 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.684 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.684 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.684 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.684 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.684 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.684 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.685 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-08-20T03:12:29.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.685 INFO:teuthology.orchestra.run.smithi044.stdout: "gid": 24146, 2024-08-20T03:12:29.685 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "y", 2024-08-20T03:12:29.685 INFO:teuthology.orchestra.run.smithi044.stdout: "mgr_features": 4540701547738038271, 2024-08-20T03:12:29.685 INFO:teuthology.orchestra.run.smithi044.stdout: "available_modules": [ 2024-08-20T03:12:29.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.685 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "alerts", 2024-08-20T03:12:29.685 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.685 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.686 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.686 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:29.686 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:29.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.686 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.686 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.686 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:29.686 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.686 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.686 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.686 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How frequently to reexamine health status", 2024-08-20T03:12:29.686 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.687 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.687 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.687 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.687 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.687 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.687 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.687 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.687 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.687 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.687 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.688 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.688 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.688 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.688 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.688 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.688 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.688 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.688 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.688 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.688 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.688 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.689 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.690 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.690 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.690 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.690 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.690 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.690 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.690 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.690 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.690 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.690 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.691 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.691 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.691 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.691 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.691 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.691 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.691 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.691 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.691 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.691 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.691 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.692 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_destination": { 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_destination", 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email address to send alerts to", 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.693 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.694 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.694 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_from_name": { 2024-08-20T03:12:29.694 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_from_name", 2024-08-20T03:12:29.694 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.694 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.694 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.694 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "Ceph", 2024-08-20T03:12:29.694 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.694 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.694 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.695 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email From: name", 2024-08-20T03:12:29.695 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.695 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.695 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.695 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_host": { 2024-08-20T03:12:29.695 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_host", 2024-08-20T03:12:29.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.695 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.695 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.695 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.695 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.696 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.696 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.696 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP server", 2024-08-20T03:12:29.696 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.696 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.696 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.696 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_password": { 2024-08-20T03:12:29.696 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_password", 2024-08-20T03:12:29.696 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.696 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.696 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Password to authenticate with", 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_port": { 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_port", 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.697 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.698 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "465", 2024-08-20T03:12:29.698 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.698 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.698 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.698 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP port", 2024-08-20T03:12:29.698 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.698 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.698 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.698 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_sender": { 2024-08-20T03:12:29.698 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_sender", 2024-08-20T03:12:29.698 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP envelope sender", 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_ssl": { 2024-08-20T03:12:29.699 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_ssl", 2024-08-20T03:12:29.700 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.700 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.700 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.700 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.700 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.700 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.700 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.700 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-20T03:12:29.700 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.700 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.700 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_user": { 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_user", 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "User to authenticate as", 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.701 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.702 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.702 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.702 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.702 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.702 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.702 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.702 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.702 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.702 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.702 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.703 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.703 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.703 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.703 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.703 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.703 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "balancer", 2024-08-20T03:12:29.703 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.703 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.703 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.703 INFO:teuthology.orchestra.run.smithi044.stdout: "active": { 2024-08-20T03:12:29.704 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "active", 2024-08-20T03:12:29.704 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.704 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.704 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.704 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.704 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.704 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.704 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.704 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically balance PGs across cluster", 2024-08-20T03:12:29.704 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.704 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.704 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.705 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_time": { 2024-08-20T03:12:29.705 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_time", 2024-08-20T03:12:29.705 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.705 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.705 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.705 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0000", 2024-08-20T03:12:29.705 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.705 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.705 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.705 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "beginning time of day to automatically balance", 2024-08-20T03:12:29.705 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T03:12:29.706 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.706 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.706 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.706 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_weekday": { 2024-08-20T03:12:29.706 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_weekday", 2024-08-20T03:12:29.706 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:29.706 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.706 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.706 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.706 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-08-20T03:12:29.706 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-08-20T03:12:29.707 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.707 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-20T03:12:29.707 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T03:12:29.707 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.707 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.707 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_max_iterations": { 2024-08-20T03:12:29.707 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_max_iterations", 2024-08-20T03:12:29.707 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:29.707 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.707 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.708 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "25", 2024-08-20T03:12:29.708 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-08-20T03:12:29.708 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "250", 2024-08-20T03:12:29.708 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.708 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-20T03:12:29.708 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.708 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.708 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.708 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_metrics": { 2024-08-20T03:12:29.709 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_metrics", 2024-08-20T03:12:29.709 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.709 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.709 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.709 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "pgs,objects,bytes", 2024-08-20T03:12:29.709 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.709 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.709 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.709 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-20T03:12:29.709 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-20T03:12:29.710 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.710 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.710 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_step": { 2024-08-20T03:12:29.710 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_step", 2024-08-20T03:12:29.710 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:29.710 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.710 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.710 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.5", 2024-08-20T03:12:29.710 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0.001", 2024-08-20T03:12:29.711 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "0.999", 2024-08-20T03:12:29.711 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.711 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "aggressiveness of optimization", 2024-08-20T03:12:29.711 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-20T03:12:29.711 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.711 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.711 INFO:teuthology.orchestra.run.smithi044.stdout: "end_time": { 2024-08-20T03:12:29.711 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_time", 2024-08-20T03:12:29.711 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.711 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.712 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.712 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2359", 2024-08-20T03:12:29.712 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.712 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.712 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.712 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ending time of day to automatically balance", 2024-08-20T03:12:29.712 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T03:12:29.712 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.712 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.712 INFO:teuthology.orchestra.run.smithi044.stdout: "end_weekday": { 2024-08-20T03:12:29.712 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_weekday", 2024-08-20T03:12:29.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:29.713 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.713 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.713 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.713 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-08-20T03:12:29.713 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-08-20T03:12:29.713 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.713 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-20T03:12:29.713 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T03:12:29.713 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.713 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.714 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.714 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.714 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.714 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.714 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.714 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.714 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.714 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.714 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.714 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.715 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.716 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.717 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.717 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.717 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.717 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.717 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.717 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.717 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.717 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.717 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.717 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.718 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.718 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.718 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.718 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.718 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.718 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.718 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.718 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.718 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.718 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.718 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.719 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.719 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.719 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.719 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.719 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.719 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.719 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.719 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.719 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.719 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.719 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "min_score": { 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_score", 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.720 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "upmap", 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.721 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-compat", 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "none", 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "read", 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap", 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap-read" 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Balancer mode", 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_ids": { 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_ids", 2024-08-20T03:12:29.722 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-20T03:12:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.723 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-20T03:12:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.724 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.725 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_deviation": { 2024-08-20T03:12:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_deviation", 2024-08-20T03:12:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-08-20T03:12:29.726 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_optimizations": { 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_optimizations", 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.727 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm", 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.728 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_down_multiplier": { 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_down_multiplier", 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.729 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_refresh_rate": { 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_refresh_rate", 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "20", 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-20T03:12:29.730 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_starting_port": { 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_starting_port", 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4721", 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.731 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_ptrace": { 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_ptrace", 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.732 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_interval": { 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_interval", 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.733 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-20T03:12:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_memory_target_ratio": { 2024-08-20T03:12:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_memory_target_ratio", 2024-08-20T03:12:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.734 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.7", 2024-08-20T03:12:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-20T03:12:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm_log_destination": { 2024-08-20T03:12:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm_log_destination", 2024-08-20T03:12:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.735 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "file", 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "file,syslog", 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "syslog" 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.736 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "cgroups_split": { 2024-08-20T03:12:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cgroups_split", 2024-08-20T03:12:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-20T03:12:29.737 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "config_checks_enabled": { 2024-08-20T03:12:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_checks_enabled", 2024-08-20T03:12:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.738 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-20T03:12:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "config_dashboard": { 2024-08-20T03:12:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_dashboard", 2024-08-20T03:12:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.739 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-20T03:12:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_alertmanager": { 2024-08-20T03:12:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_alertmanager", 2024-08-20T03:12:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.740 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-20T03:12:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-08-20T03:12:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_base": { 2024-08-20T03:12:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_base", 2024-08-20T03:12:29.741 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-20T03:12:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Container image name, without the tag", 2024-08-20T03:12:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_elasticsearch": { 2024-08-20T03:12:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_elasticsearch", 2024-08-20T03:12:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-20T03:12:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "elasticsearch container image", 2024-08-20T03:12:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.743 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_grafana": { 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_grafana", 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-08-20T03:12:29.744 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_haproxy": { 2024-08-20T03:12:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_haproxy", 2024-08-20T03:12:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-20T03:12:29.745 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "HAproxy container image", 2024-08-20T03:12:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_agent": { 2024-08-20T03:12:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_agent", 2024-08-20T03:12:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.746 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-20T03:12:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger agent container image", 2024-08-20T03:12:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_collector": { 2024-08-20T03:12:29.747 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_collector", 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger collector container image", 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.748 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_query": { 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_query", 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger query container image", 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_keepalived": { 2024-08-20T03:12:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_keepalived", 2024-08-20T03:12:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-20T03:12:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Keepalived container image", 2024-08-20T03:12:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.750 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_loki": { 2024-08-20T03:12:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_loki", 2024-08-20T03:12:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-20T03:12:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.751 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Loki container image", 2024-08-20T03:12:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nginx": { 2024-08-20T03:12:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nginx", 2024-08-20T03:12:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-20T03:12:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.752 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nginx container image", 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_node_exporter": { 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_node_exporter", 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-20T03:12:29.753 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-08-20T03:12:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nvmeof": { 2024-08-20T03:12:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nvmeof", 2024-08-20T03:12:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.754 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-20T03:12:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nvme-of container image", 2024-08-20T03:12:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_prometheus": { 2024-08-20T03:12:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_prometheus", 2024-08-20T03:12:29.755 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_promtail": { 2024-08-20T03:12:29.756 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_promtail", 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Promtail container image", 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.757 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_samba": { 2024-08-20T03:12:29.758 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_samba", 2024-08-20T03:12:29.758 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.758 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.758 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.758 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-20T03:12:29.758 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.758 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.758 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.758 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Samba/SMB container image", 2024-08-20T03:12:29.758 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.758 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.758 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.759 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.759 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_snmp_gateway": { 2024-08-20T03:12:29.759 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_snmp_gateway", 2024-08-20T03:12:29.759 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.759 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.759 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.759 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-20T03:12:29.759 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.759 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.759 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.759 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SNMP Gateway container image", 2024-08-20T03:12:29.759 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: "container_init": { 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_init", 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.760 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Run podman/docker with `--init`", 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: "daemon_cache_timeout": { 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "daemon_cache_timeout", 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.761 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.762 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-20T03:12:29.762 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.762 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.762 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.762 INFO:teuthology.orchestra.run.smithi044.stdout: "default_cephadm_command_timeout": { 2024-08-20T03:12:29.762 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_cephadm_command_timeout", 2024-08-20T03:12:29.762 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.762 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.762 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.762 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "900", 2024-08-20T03:12:29.762 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: "default_registry": { 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_registry", 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.763 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io", 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: "device_cache_timeout": { 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_cache_timeout", 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.764 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.765 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1800", 2024-08-20T03:12:29.765 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.765 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.765 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.765 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache device inventory", 2024-08-20T03:12:29.765 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.765 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.765 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.765 INFO:teuthology.orchestra.run.smithi044.stdout: "device_enhanced_scan": { 2024-08-20T03:12:29.766 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_enhanced_scan", 2024-08-20T03:12:29.766 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.766 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.766 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.766 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.766 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.766 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.766 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.766 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-20T03:12:29.766 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.766 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.766 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.767 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.767 INFO:teuthology.orchestra.run.smithi044.stdout: "facts_cache_timeout": { 2024-08-20T03:12:29.767 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "facts_cache_timeout", 2024-08-20T03:12:29.767 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.767 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.767 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.767 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:29.767 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.767 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.767 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.767 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache host facts data", 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: "grafana_dashboards_path": { 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "grafana_dashboards_path", 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.768 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: "host_check_interval": { 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "host_check_interval", 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.769 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.770 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.770 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to perform a host check", 2024-08-20T03:12:29.770 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.770 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.770 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.770 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.770 INFO:teuthology.orchestra.run.smithi044.stdout: "hw_monitoring": { 2024-08-20T03:12:29.770 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hw_monitoring", 2024-08-20T03:12:29.770 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.770 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.770 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.770 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-20T03:12:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "inventory_list_all": { 2024-08-20T03:12:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "inventory_list_all", 2024-08-20T03:12:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.771 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.772 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.773 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.773 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.773 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.773 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.773 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.773 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.773 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.773 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.773 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.773 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.773 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.773 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.774 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.774 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.774 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.774 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.774 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.774 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.774 INFO:teuthology.orchestra.run.smithi044.stdout: "log_refresh_metadata": { 2024-08-20T03:12:29.774 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_refresh_metadata", 2024-08-20T03:12:29.774 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.774 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.774 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.774 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.775 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.775 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.775 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.775 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-20T03:12:29.775 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.775 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.775 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.775 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.775 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.777 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.777 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.777 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.778 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.778 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.778 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.778 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.778 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.778 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-20T03:12:29.778 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.778 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.778 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.778 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.778 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.778 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.779 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.779 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.779 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.779 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.779 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.779 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.779 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.779 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.779 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.779 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.779 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.779 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.780 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.781 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.781 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.781 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.781 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.781 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.781 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.781 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.781 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.781 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-20T03:12:29.781 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-20T03:12:29.781 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.782 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.782 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.782 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.782 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.782 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.782 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.782 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-20T03:12:29.782 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.782 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.782 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.782 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.782 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-20T03:12:29.783 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-20T03:12:29.783 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.783 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.783 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.783 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "*", 2024-08-20T03:12:29.783 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.783 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.783 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.783 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-20T03:12:29.783 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.783 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.783 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.784 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.784 INFO:teuthology.orchestra.run.smithi044.stdout: "max_count_per_host": { 2024-08-20T03:12:29.784 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_count_per_host", 2024-08-20T03:12:29.784 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.784 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.784 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.784 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.784 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.784 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.784 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.784 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of daemons per service per host", 2024-08-20T03:12:29.784 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.785 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.785 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.785 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd_draining_count": { 2024-08-20T03:12:29.785 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_osd_draining_count", 2024-08-20T03:12:29.785 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.785 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.785 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.785 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.785 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.785 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.785 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.786 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-20T03:12:29.786 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.786 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.786 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.786 INFO:teuthology.orchestra.run.smithi044.stdout: "migration_current": { 2024-08-20T03:12:29.786 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "migration_current", 2024-08-20T03:12:29.786 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.786 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.786 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.786 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "internal - do not modify", 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.787 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.788 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "root", 2024-08-20T03:12:29.788 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.788 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.788 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.788 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm-package", 2024-08-20T03:12:29.788 INFO:teuthology.orchestra.run.smithi044.stdout: "root" 2024-08-20T03:12:29.788 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.788 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "mode for remote execution of cephadm", 2024-08-20T03:12:29.788 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.788 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.788 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: "oob_default_addr": { 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "oob_default_addr", 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "169.254.1.1", 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.789 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.790 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.790 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.790 INFO:teuthology.orchestra.run.smithi044.stdout: "prometheus_alerts_path": { 2024-08-20T03:12:29.790 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus_alerts_path", 2024-08-20T03:12:29.790 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.790 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.790 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.790 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-20T03:12:29.790 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.790 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.791 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.791 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-20T03:12:29.791 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.791 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.791 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.791 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.791 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_insecure": { 2024-08-20T03:12:29.791 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_insecure", 2024-08-20T03:12:29.791 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.791 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.791 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.791 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.792 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.792 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.792 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.792 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-20T03:12:29.792 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.792 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.792 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.792 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.792 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_password": { 2024-08-20T03:12:29.792 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_password", 2024-08-20T03:12:29.792 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.793 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.793 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.793 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.793 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.793 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.793 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.793 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-20T03:12:29.793 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.793 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.793 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.793 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.793 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_url": { 2024-08-20T03:12:29.794 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_url", 2024-08-20T03:12:29.794 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.794 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.794 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.794 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.794 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.794 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.794 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.794 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-20T03:12:29.794 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.795 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.795 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.795 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_username": { 2024-08-20T03:12:29.795 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_username", 2024-08-20T03:12:29.795 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.795 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.795 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.795 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.795 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.795 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.795 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.796 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-20T03:12:29.796 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.796 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.796 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.796 INFO:teuthology.orchestra.run.smithi044.stdout: "secure_monitoring_stack": { 2024-08-20T03:12:29.796 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secure_monitoring_stack", 2024-08-20T03:12:29.796 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.796 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.796 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.796 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-20T03:12:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "service_discovery_port": { 2024-08-20T03:12:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "service_discovery_port", 2024-08-20T03:12:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.797 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.798 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.798 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8765", 2024-08-20T03:12:29.798 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.798 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.798 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.798 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "cephadm service discovery port", 2024-08-20T03:12:29.798 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.798 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.798 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.798 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.798 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.799 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.799 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.799 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.799 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.799 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.799 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.799 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.799 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.799 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.799 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.799 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.800 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.800 INFO:teuthology.orchestra.run.smithi044.stdout: "ssh_config_file": { 2024-08-20T03:12:29.800 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssh_config_file", 2024-08-20T03:12:29.800 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.800 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.800 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.800 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.800 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.800 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.800 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.800 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-20T03:12:29.800 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.801 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.801 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.801 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.801 INFO:teuthology.orchestra.run.smithi044.stdout: "use_agent": { 2024-08-20T03:12:29.801 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_agent", 2024-08-20T03:12:29.801 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.801 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.801 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.801 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.801 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.801 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.802 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.802 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-20T03:12:29.802 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.802 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.802 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.802 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.802 INFO:teuthology.orchestra.run.smithi044.stdout: "use_repo_digest": { 2024-08-20T03:12:29.802 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_repo_digest", 2024-08-20T03:12:29.802 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.802 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.802 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.802 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.803 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.803 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.803 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.803 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-20T03:12:29.803 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.803 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.803 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.803 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_failed_host_check": { 2024-08-20T03:12:29.803 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_failed_host_check", 2024-08-20T03:12:29.803 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.803 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if the host check fails", 2024-08-20T03:12:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_daemons": { 2024-08-20T03:12:29.804 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_daemons", 2024-08-20T03:12:29.805 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.805 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.805 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.805 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.805 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.805 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.805 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.805 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-20T03:12:29.805 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.805 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.805 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.805 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.806 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_hosts": { 2024-08-20T03:12:29.806 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_hosts", 2024-08-20T03:12:29.806 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.806 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.806 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.806 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.806 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.806 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.806 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.806 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-20T03:12:29.807 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.807 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.807 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.807 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.807 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.807 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.807 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.807 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crash", 2024-08-20T03:12:29.807 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.807 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.807 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.807 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.808 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.808 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.808 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.808 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.808 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.808 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.808 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.808 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.808 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.808 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.808 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.809 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.809 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.809 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.809 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.809 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.809 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.809 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.809 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.809 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.809 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.809 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.809 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.810 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.810 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.810 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.810 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.810 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.810 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.810 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.810 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.810 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.810 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.810 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.811 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.811 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.811 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.811 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.811 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.811 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.811 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.811 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.811 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.811 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.811 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.812 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.812 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.812 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.812 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.812 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.812 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.812 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.812 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.812 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.812 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.813 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.813 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.813 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.813 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.813 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.813 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.813 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.813 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.813 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.813 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.813 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.813 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.814 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.814 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.814 INFO:teuthology.orchestra.run.smithi044.stdout: "retain_interval": { 2024-08-20T03:12:29.814 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retain_interval", 2024-08-20T03:12:29.814 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.814 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.814 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.814 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "31536000", 2024-08-20T03:12:29.814 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.814 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.814 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.815 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-20T03:12:29.815 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.815 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.815 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.815 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.815 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.815 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.815 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.815 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.815 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.815 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.816 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.816 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.816 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.816 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.816 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.816 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.816 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.816 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.816 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_recent_interval": { 2024-08-20T03:12:29.816 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_recent_interval", 2024-08-20T03:12:29.816 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.816 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.817 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.817 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1209600", 2024-08-20T03:12:29.817 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.817 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.817 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.817 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-20T03:12:29.817 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.817 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.817 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.817 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.817 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.818 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.818 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.818 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dashboard", 2024-08-20T03:12:29.818 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.818 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.818 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.818 INFO:teuthology.orchestra.run.smithi044.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-20T03:12:29.818 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-20T03:12:29.818 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.818 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.818 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.819 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.819 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.819 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.819 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.819 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.819 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.819 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.819 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.819 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.819 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_HOST": { 2024-08-20T03:12:29.819 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-20T03:12:29.819 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-20T03:12:29.820 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.821 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_ENABLED": { 2024-08-20T03:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_ENABLED", 2024-08-20T03:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.822 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.823 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.823 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-20T03:12:29.823 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-20T03:12:29.823 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.823 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.823 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.823 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.823 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.823 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.823 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.823 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.823 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: "ENABLE_BROWSABLE_API": { 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.824 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.825 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.826 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.826 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.826 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.826 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.826 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.826 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-20T03:12:29.826 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-20T03:12:29.826 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.826 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.826 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.826 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.827 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.827 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.827 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.827 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.827 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.827 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.827 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.827 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.827 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-20T03:12:29.827 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-20T03:12:29.827 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.828 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.828 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.828 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.828 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.828 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.828 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.828 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.828 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.828 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.828 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.828 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.829 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.830 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.830 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-20T03:12:29.830 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-20T03:12:29.830 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.830 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.830 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.830 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.830 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.830 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.830 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.830 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.831 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.831 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.831 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.831 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.831 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-20T03:12:29.831 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-20T03:12:29.831 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.831 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.831 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.831 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.831 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.832 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.832 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.832 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.832 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.832 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.832 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.832 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.832 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-20T03:12:29.832 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-20T03:12:29.832 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.832 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.832 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.833 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.833 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.833 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.833 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.833 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.833 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.833 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.833 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.833 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.834 INFO:teuthology.orchestra.run.smithi044.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-20T03:12:29.834 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-20T03:12:29.834 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.834 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.834 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.834 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.834 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.834 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.834 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.835 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.835 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.835 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.835 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.835 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.835 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_PASSWORD": { 2024-08-20T03:12:29.835 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-20T03:12:29.835 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.835 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.835 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.836 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-08-20T03:12:29.836 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.836 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.836 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.836 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.836 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.836 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.836 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.836 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.836 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-20T03:12:29.836 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-20T03:12:29.837 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.837 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.837 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.837 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.837 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.837 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.837 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.837 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.837 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.837 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.838 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.838 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.838 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_URL": { 2024-08-20T03:12:29.838 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_URL", 2024-08-20T03:12:29.838 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.838 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.838 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.838 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.838 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.839 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.839 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.839 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.839 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.839 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.839 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.839 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.839 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_USERNAME": { 2024-08-20T03:12:29.839 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-20T03:12:29.839 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.840 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.840 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.840 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-08-20T03:12:29.840 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.840 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.840 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.840 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.840 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.840 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.840 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.841 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.841 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-20T03:12:29.841 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-20T03:12:29.841 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.841 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.841 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.841 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.841 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.841 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.841 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.841 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.841 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.842 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.842 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.842 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.842 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-20T03:12:29.842 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-20T03:12:29.842 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.842 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.842 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.842 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.842 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.842 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.842 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.843 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.843 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.843 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.843 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.843 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.843 INFO:teuthology.orchestra.run.smithi044.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-20T03:12:29.843 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-20T03:12:29.843 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.843 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.843 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.844 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.844 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.844 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.844 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.844 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.844 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.844 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.844 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.844 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.844 INFO:teuthology.orchestra.run.smithi044.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-20T03:12:29.844 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-20T03:12:29.844 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.845 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.845 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.845 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.845 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.845 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.845 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.845 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.845 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.845 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.845 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.846 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.846 INFO:teuthology.orchestra.run.smithi044.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-20T03:12:29.846 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-20T03:12:29.846 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.846 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.846 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.846 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "[]", 2024-08-20T03:12:29.846 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.846 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.846 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.846 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.847 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.847 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.847 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.847 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.847 INFO:teuthology.orchestra.run.smithi044.stdout: "MULTICLUSTER_CONFIG": { 2024-08-20T03:12:29.847 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-20T03:12:29.847 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.847 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.847 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.847 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "{}", 2024-08-20T03:12:29.847 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.847 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.848 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.848 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.848 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.848 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.848 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.848 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.848 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_HOST": { 2024-08-20T03:12:29.848 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-20T03:12:29.848 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.848 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.848 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.848 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.849 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.849 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.849 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.849 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.849 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.849 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.849 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.849 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.849 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-20T03:12:29.849 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-20T03:12:29.849 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.850 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.850 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.850 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.850 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.850 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.850 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.850 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.850 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.850 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.850 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.850 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.850 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-20T03:12:29.851 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-20T03:12:29.851 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.851 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.851 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.851 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.851 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.851 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.851 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.851 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.851 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.851 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.852 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.852 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.852 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-20T03:12:29.852 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-20T03:12:29.852 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.852 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.852 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.852 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.852 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.852 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.852 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.852 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.853 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.853 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.853 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.853 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.853 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-20T03:12:29.853 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-20T03:12:29.853 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.853 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.853 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.853 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.853 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.854 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.854 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.854 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.854 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.854 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.854 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.854 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.854 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-20T03:12:29.854 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-20T03:12:29.855 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.855 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.855 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.855 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.855 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.855 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.855 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.855 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.855 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.855 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.855 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.856 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.856 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-20T03:12:29.856 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-20T03:12:29.856 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.856 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.856 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.856 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.856 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.856 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.856 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.856 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.857 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.857 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.857 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.857 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.857 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-20T03:12:29.857 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-20T03:12:29.857 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.857 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.857 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.857 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.857 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.857 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.858 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.858 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.858 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.858 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.858 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.858 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.858 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-20T03:12:29.858 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-20T03:12:29.858 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.858 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.858 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.859 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.859 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.859 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.859 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.859 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.859 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.859 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.859 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.859 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.859 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_ENABLED": { 2024-08-20T03:12:29.859 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-20T03:12:29.859 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-20T03:12:29.860 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-20T03:12:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-20T03:12:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.861 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-20T03:12:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-20T03:12:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.862 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8", 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.863 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-20T03:12:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-20T03:12:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "45", 2024-08-20T03:12:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.864 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ACCESS_KEY": { 2024-08-20T03:12:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-20T03:12:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.865 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-20T03:12:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-20T03:12:29.866 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-08-20T03:12:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.867 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SECRET_KEY": { 2024-08-20T03:12:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-20T03:12:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.868 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SSL_VERIFY": { 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.869 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "UNSAFE_TLS_v1_2": { 2024-08-20T03:12:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-20T03:12:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.870 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-20T03:12:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-20T03:12:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.871 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-20T03:12:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-20T03:12:29.872 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.873 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-20T03:12:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-20T03:12:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.874 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "cross_origin_url": { 2024-08-20T03:12:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cross_origin_url", 2024-08-20T03:12:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.875 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "crt_file": { 2024-08-20T03:12:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crt_file", 2024-08-20T03:12:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.876 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "debug": { 2024-08-20T03:12:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "debug", 2024-08-20T03:12:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.877 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable/disable debug options", 2024-08-20T03:12:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.878 INFO:teuthology.orchestra.run.smithi044.stdout: "jwt_token_ttl": { 2024-08-20T03:12:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "jwt_token_ttl", 2024-08-20T03:12:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "28800", 2024-08-20T03:12:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.879 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-08-20T03:12:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-08-20T03:12:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.880 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.881 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.882 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.883 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.884 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.885 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.886 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "motd": { 2024-08-20T03:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "motd", 2024-08-20T03:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.887 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "The message of the day", 2024-08-20T03:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect_resolve_ip_addr": { 2024-08-20T03:12:29.888 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "redirect_resolve_ip_addr", 2024-08-20T03:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.889 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-08-20T03:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-08-20T03:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-08-20T03:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.890 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-08-20T03:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-08-20T03:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8080", 2024-08-20T03:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.891 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.892 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-08-20T03:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-08-20T03:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.893 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.894 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl_server_port": { 2024-08-20T03:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl_server_port", 2024-08-20T03:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8443", 2024-08-20T03:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.895 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-08-20T03:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-08-20T03:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "redirect", 2024-08-20T03:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.896 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect" 2024-08-20T03:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-08-20T03:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-08-20T03:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.897 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-08-20T03:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-08-20T03:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-08-20T03:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "url_prefix": { 2024-08-20T03:12:29.898 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url_prefix", 2024-08-20T03:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.899 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2024-08-20T03:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_monitoring": { 2024-08-20T03:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_monitoring", 2024-08-20T03:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.900 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "monitor device health metrics", 2024-08-20T03:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.901 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.902 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.903 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.904 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.905 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.906 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.907 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "mark_out_threshold": { 2024-08-20T03:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mark_out_threshold", 2024-08-20T03:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2419200", 2024-08-20T03:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-20T03:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.908 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": { 2024-08-20T03:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_name", 2024-08-20T03:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "device_health_metrics", 2024-08-20T03:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.909 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-20T03:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "retention_period": { 2024-08-20T03:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retention_period", 2024-08-20T03:12:29.910 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15552000", 2024-08-20T03:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain device health metrics", 2024-08-20T03:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.911 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_frequency": { 2024-08-20T03:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_frequency", 2024-08-20T03:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-08-20T03:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-20T03:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.912 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "self_heal": { 2024-08-20T03:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "self_heal", 2024-08-20T03:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-20T03:12:29.913 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.914 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and check device health", 2024-08-20T03:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.915 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_threshold": { 2024-08-20T03:12:29.916 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_threshold", 2024-08-20T03:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7257600", 2024-08-20T03:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-20T03:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.917 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "diskprediction_local", 2024-08-20T03:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.918 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.919 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.920 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.921 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.922 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.923 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "predict_interval": { 2024-08-20T03:12:29.924 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predict_interval", 2024-08-20T03:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-08-20T03:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.925 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "predictor_model": { 2024-08-20T03:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predictor_model", 2024-08-20T03:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "prophetstor", 2024-08-20T03:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.926 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.927 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.928 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "influx", 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": false, 2024-08-20T03:12:29.929 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "influxdb python module not found", 2024-08-20T03:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "batch_size": { 2024-08-20T03:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "batch_size", 2024-08-20T03:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5000", 2024-08-20T03:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-20T03:12:29.930 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "database": { 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "database", 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "ceph", 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.931 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": { 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hostname", 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.932 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server hostname", 2024-08-20T03:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "30", 2024-08-20T03:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "5", 2024-08-20T03:12:29.933 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.934 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.935 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.936 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.937 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.938 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.939 INFO:teuthology.orchestra.run.smithi044.stdout: "password": { 2024-08-20T03:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "password", 2024-08-20T03:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "password of InfluxDB server user", 2024-08-20T03:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.940 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "port": { 2024-08-20T03:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "port", 2024-08-20T03:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8086", 2024-08-20T03:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.941 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server port", 2024-08-20T03:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.942 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-08-20T03:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-08-20T03:12:29.943 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "false", 2024-08-20T03:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T03:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "threads": { 2024-08-20T03:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threads", 2024-08-20T03:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-08-20T03:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "32", 2024-08-20T03:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-20T03:12:29.945 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "username": { 2024-08-20T03:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "username", 2024-08-20T03:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.946 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "username of InfluxDB server user", 2024-08-20T03:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "verify_ssl": { 2024-08-20T03:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "verify_ssl", 2024-08-20T03:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.947 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "true", 2024-08-20T03:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T03:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "insights", 2024-08-20T03:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.949 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.950 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.951 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.952 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.953 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.954 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.956 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "iostat", 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.957 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.958 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.959 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.960 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.961 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.962 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.963 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.964 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "k8sevents", 2024-08-20T03:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_event_retention_days": { 2024-08-20T03:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ceph_event_retention_days", 2024-08-20T03:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7", 2024-08-20T03:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.965 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "config_check_secs": { 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_check_secs", 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "10", 2024-08-20T03:12:29.966 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-20T03:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.967 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.968 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.969 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.970 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.971 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.972 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.973 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "localpool", 2024-08-20T03:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "failure_domain": { 2024-08-20T03:12:29.974 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "failure_domain", 2024-08-20T03:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "host", 2024-08-20T03:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "failure domain for any created local pool", 2024-08-20T03:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-20T03:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.975 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.976 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.977 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.978 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.979 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.980 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.981 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.982 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": { 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_size", 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.983 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default min_size for any created local pool", 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "num_rep": { 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "num_rep", 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3", 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.984 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default replica count for any created local pool", 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": { 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_num", 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "128", 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.985 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default pg_num for any created local pool", 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "prefix": { 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prefix", 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.986 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name prefix for any created local pool", 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.987 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "subtree": { 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subtree", 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "rack", 2024-08-20T03:12:29.988 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mds_autoscaler", 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.989 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.990 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.991 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.991 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.991 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.991 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.991 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.991 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.991 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.991 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.991 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.991 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.991 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.991 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.992 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.992 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.992 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.992 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.992 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.992 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.992 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.992 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.992 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.992 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.993 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:29.993 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:29.993 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.993 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.993 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.993 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:29.993 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.993 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.993 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.993 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.993 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:29.994 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:29.995 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:29.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirroring", 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.997 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:29.998 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:29.999 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.000 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.001 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.002 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "nfs", 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.003 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.004 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.005 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.006 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.008 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.009 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: "fail_fs": { 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "fail_fs", 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.010 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.011 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.012 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.013 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.014 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.015 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator": { 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.016 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: "rook", 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: "test_orchestrator" 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Orchestrator backend", 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.017 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.018 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_perf_query", 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.019 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.020 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.020 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.020 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.020 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.020 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.020 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.020 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.020 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.020 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.021 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.022 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.023 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.023 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.023 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.023 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.023 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.023 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.023 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.023 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.023 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.023 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.024 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.024 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.024 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.024 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.024 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.024 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.024 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.024 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.024 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.024 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.024 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.025 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.025 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.025 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.025 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.025 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.025 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.025 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.025 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.025 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.025 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.025 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.026 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.026 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.026 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.026 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.026 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.026 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.026 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.026 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.026 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_support", 2024-08-20T03:12:30.027 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.027 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.027 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.027 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.027 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.027 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.027 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.027 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.027 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.027 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.027 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.027 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.028 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.028 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.028 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.028 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.028 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.028 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.028 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.028 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.028 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.028 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.028 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.028 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.029 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.029 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.029 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.029 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.029 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.029 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.029 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.029 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.029 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.029 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.029 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.030 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.030 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.030 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.030 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.030 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.030 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.030 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.030 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.030 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.030 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.030 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.031 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.031 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.031 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.031 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.031 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.031 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.031 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.031 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.031 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.031 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.031 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.032 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.033 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.033 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.033 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.033 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.033 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.033 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.033 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.033 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.033 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.033 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.034 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.034 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.034 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.034 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.034 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.034 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.034 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.034 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.034 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_autoscaler", 2024-08-20T03:12:30.034 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.035 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.035 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.035 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.035 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.035 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.035 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.035 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.035 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.035 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.035 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.035 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.035 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.036 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.037 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.038 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.039 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.040 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.041 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.042 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "threshold": { 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threshold", 2024-08-20T03:12:30.043 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-08-20T03:12:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1.0", 2024-08-20T03:12:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "scaling threshold", 2024-08-20T03:12:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-20T03:12:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.044 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "progress", 2024-08-20T03:12:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_pg_recovery_event": { 2024-08-20T03:12:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_pg_recovery_event", 2024-08-20T03:12:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.045 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-20T03:12:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-08-20T03:12:30.046 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-08-20T03:12:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.047 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.048 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.049 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.050 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.051 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.051 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.051 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.051 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.051 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.051 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.051 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.051 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.051 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.051 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.051 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.051 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.052 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.053 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.053 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.053 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.053 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.053 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.053 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.053 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.053 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.053 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.053 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.053 INFO:teuthology.orchestra.run.smithi044.stdout: "max_completed_events": { 2024-08-20T03:12:30.054 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_completed_events", 2024-08-20T03:12:30.054 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.054 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.054 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.054 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "50", 2024-08-20T03:12:30.054 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.054 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.054 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.054 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "number of past completed events to remember", 2024-08-20T03:12:30.054 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.054 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.055 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.055 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.055 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:30.055 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:30.055 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.055 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.055 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.055 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:30.055 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.055 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.055 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.055 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long the module is going to sleep", 2024-08-20T03:12:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.056 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.057 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.057 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.057 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.057 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.057 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.057 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.057 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.057 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.057 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.057 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus", 2024-08-20T03:12:30.057 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.057 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.058 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.058 INFO:teuthology.orchestra.run.smithi044.stdout: "cache": { 2024-08-20T03:12:30.058 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cache", 2024-08-20T03:12:30.058 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.058 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.058 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.058 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.058 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.058 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.058 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.058 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.059 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.059 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.059 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.059 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.059 INFO:teuthology.orchestra.run.smithi044.stdout: "exclude_perf_counters": { 2024-08-20T03:12:30.059 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "exclude_perf_counters", 2024-08-20T03:12:30.059 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.059 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.059 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.059 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.059 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.059 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-20T03:12:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-20T03:12:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.060 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.061 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.062 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.063 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.064 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.065 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools": { 2024-08-20T03:12:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools", 2024-08-20T03:12:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.066 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300", 2024-08-20T03:12:30.067 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_interval": { 2024-08-20T03:12:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_interval", 2024-08-20T03:12:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.068 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15.0", 2024-08-20T03:12:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-08-20T03:12:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-08-20T03:12:30.069 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-08-20T03:12:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-20T03:12:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.070 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-08-20T03:12:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-08-20T03:12:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "9283", 2024-08-20T03:12:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-20T03:12:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.071 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.072 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "stale_cache_strategy": { 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stale_cache_strategy", 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "log", 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.073 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-08-20T03:12:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-08-20T03:12:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.074 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "default", 2024-08-20T03:12:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "default", 2024-08-20T03:12:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "error" 2024-08-20T03:12:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.075 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-08-20T03:12:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-08-20T03:12:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-08-20T03:12:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-08-20T03:12:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-08-20T03:12:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.076 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2024-08-20T03:12:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.077 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.078 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.079 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.080 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.081 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.082 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.083 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_snap_create": { 2024-08-20T03:12:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_snap_create", 2024-08-20T03:12:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.084 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "mirror_snapshot_schedule": { 2024-08-20T03:12:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirror_snapshot_schedule", 2024-08-20T03:12:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.085 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.086 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.087 INFO:teuthology.orchestra.run.smithi044.stdout: "trash_purge_schedule": { 2024-08-20T03:12:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "trash_purge_schedule", 2024-08-20T03:12:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.088 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "restful", 2024-08-20T03:12:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_auth": { 2024-08-20T03:12:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_auth", 2024-08-20T03:12:30.089 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.090 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-08-20T03:12:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-08-20T03:12:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.091 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.092 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.093 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.094 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.095 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.096 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.097 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-08-20T03:12:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-08-20T03:12:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.098 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-08-20T03:12:30.099 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-08-20T03:12:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.100 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.101 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rgw", 2024-08-20T03:12:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.102 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.103 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.104 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.105 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.106 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.107 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "secondary_zone_period_retry_limit": { 2024-08-20T03:12:30.108 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-20T03:12:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-20T03:12:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.109 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.110 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rook", 2024-08-20T03:12:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.111 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.112 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.113 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.114 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.115 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.116 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.117 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.118 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "storage_class": { 2024-08-20T03:12:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "storage_class", 2024-08-20T03:12:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.119 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "local", 2024-08-20T03:12:30.120 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.120 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.120 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.120 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-20T03:12:30.120 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "selftest", 2024-08-20T03:12:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.121 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.122 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.123 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.124 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.125 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.126 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "roption1": { 2024-08-20T03:12:30.127 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption1", 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "roption2": { 2024-08-20T03:12:30.128 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption2", 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "xyz", 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.129 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption1": { 2024-08-20T03:12:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption1", 2024-08-20T03:12:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.130 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption2": { 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption2", 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.131 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption3": { 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption3", 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.132 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption4": { 2024-08-20T03:12:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption4", 2024-08-20T03:12:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.133 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption5": { 2024-08-20T03:12:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption5", 2024-08-20T03:12:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.134 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption6": { 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption6", 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.135 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption7": { 2024-08-20T03:12:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption7", 2024-08-20T03:12:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.136 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-08-20T03:12:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "42", 2024-08-20T03:12:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.137 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.138 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "testkey": { 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testkey", 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.139 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "testlkey": { 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testlkey", 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.140 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "testnewline": { 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testnewline", 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.141 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smb", 2024-08-20T03:12:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "internal_store_backend": { 2024-08-20T03:12:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "internal_store_backend", 2024-08-20T03:12:30.142 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.143 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.144 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.145 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.146 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.147 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.148 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.149 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "update_orchestration": { 2024-08-20T03:12:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "update_orchestration", 2024-08-20T03:12:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.150 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.151 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_m_granularity": { 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_m_granularity", 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow minute scheduled snapshots", 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.152 INFO:teuthology.orchestra.run.smithi044.stdout: "dump_on_update": { 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dump_on_update", 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "dump database to debug log on update", 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.153 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.154 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.155 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.156 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.157 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.158 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.159 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.160 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stats", 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.161 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.162 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.163 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.164 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.165 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.166 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "status", 2024-08-20T03:12:30.167 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.169 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.170 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.171 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.172 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.173 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.174 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telegraf", 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "address": { 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "address", 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.175 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15", 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.176 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.177 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.178 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.179 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.180 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.181 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.182 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.183 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.183 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.183 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.183 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.183 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.183 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telemetry", 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_basic": { 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_basic", 2024-08-20T03:12:30.184 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_crash": { 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_crash", 2024-08-20T03:12:30.185 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.186 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.186 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.186 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.186 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.186 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.186 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.186 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-20T03:12:30.186 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.186 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.186 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_device": { 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_device", 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.187 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.188 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.188 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_ident": { 2024-08-20T03:12:30.189 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_ident", 2024-08-20T03:12:30.189 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.189 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.189 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.189 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_perf": { 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_perf", 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.190 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "contact": { 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "contact", 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.191 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.192 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.192 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.192 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.192 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.192 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.192 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.192 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.193 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.193 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.193 INFO:teuthology.orchestra.run.smithi044.stdout: "description": { 2024-08-20T03:12:30.193 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "description", 2024-08-20T03:12:30.193 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.193 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.193 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.193 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.193 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.193 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.193 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: "device_url": { 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_url", 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.194 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.195 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.196 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.196 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.196 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.196 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.196 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.196 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.196 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.196 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:30.196 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:30.196 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.196 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.196 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "24", 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "8", 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "last_opt_revision": { 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "last_opt_revision", 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.197 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.198 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1", 2024-08-20T03:12:30.198 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.198 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.198 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.198 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.198 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.198 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.198 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.198 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.198 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard": { 2024-08-20T03:12:30.198 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard", 2024-08-20T03:12:30.198 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard_description": { 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard_description", 2024-08-20T03:12:30.199 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.200 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.201 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.202 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.203 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.204 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.205 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.206 INFO:teuthology.orchestra.run.smithi044.stdout: "organization": { 2024-08-20T03:12:30.206 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "organization", 2024-08-20T03:12:30.206 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.206 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.206 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.206 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.206 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.206 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.207 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.207 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.207 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.207 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.207 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.207 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.207 INFO:teuthology.orchestra.run.smithi044.stdout: "proxy": { 2024-08-20T03:12:30.208 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "proxy", 2024-08-20T03:12:30.208 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.208 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.208 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.208 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.208 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.209 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.209 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.209 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.209 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.209 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.209 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.210 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.210 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.210 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.210 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.210 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.210 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.211 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.211 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.211 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.211 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.211 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.211 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.212 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.212 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.212 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.212 INFO:teuthology.orchestra.run.smithi044.stdout: "url": { 2024-08-20T03:12:30.212 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url", 2024-08-20T03:12:30.212 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.213 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.213 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.213 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-20T03:12:30.213 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.213 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.214 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.214 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.214 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.214 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.214 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.214 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.215 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.215 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.215 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.215 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "test_orchestrator", 2024-08-20T03:12:30.215 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.216 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.216 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.216 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.216 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.216 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.216 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.217 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.217 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.217 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.217 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.217 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.217 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.218 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.218 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.218 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.218 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.218 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.218 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.219 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.219 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.219 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.219 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.219 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.219 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.220 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.220 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.220 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.220 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.220 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.221 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.221 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.221 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.221 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.221 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.222 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.222 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.222 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.222 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.222 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.222 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.223 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.223 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.223 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.223 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.223 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.223 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.224 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.224 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.224 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.224 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.224 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.225 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.225 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.225 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.225 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.225 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.225 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.226 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.226 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.226 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.226 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.226 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.226 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.227 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.227 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.227 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.227 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.227 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.227 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.228 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.228 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.228 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.228 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.228 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.229 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.229 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.229 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.229 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.229 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.229 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.230 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.230 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.230 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.230 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.230 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.230 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.231 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.231 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.231 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.231 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2024-08-20T03:12:30.231 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.231 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.232 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.232 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.232 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.232 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.232 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.232 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.233 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.233 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.233 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.233 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.233 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.234 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.234 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.234 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.234 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.234 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.234 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.235 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.235 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.235 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.235 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.235 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.236 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.237 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.237 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.237 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.237 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.237 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.237 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.237 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.237 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.237 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.237 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.238 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.238 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.238 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.238 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.238 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.238 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.238 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.238 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.239 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.239 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.239 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.239 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.239 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.239 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.239 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.239 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.239 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.239 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.240 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.240 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.240 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.240 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.240 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.240 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.240 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.240 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.240 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.241 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.241 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.241 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.241 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_clones": { 2024-08-20T03:12:30.241 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_clones", 2024-08-20T03:12:30.241 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.241 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.241 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.241 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4", 2024-08-20T03:12:30.241 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.242 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.242 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.242 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-20T03:12:30.242 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.242 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.242 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.242 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.242 INFO:teuthology.orchestra.run.smithi044.stdout: "periodic_async_work": { 2024-08-20T03:12:30.242 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "periodic_async_work", 2024-08-20T03:12:30.243 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.243 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.243 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.243 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.243 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.243 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.243 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.243 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Periodically check for async work", 2024-08-20T03:12:30.243 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.243 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.244 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.244 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.244 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_delay": { 2024-08-20T03:12:30.244 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_delay", 2024-08-20T03:12:30.244 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.244 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.244 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.244 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.244 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.245 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.245 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.245 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-20T03:12:30.245 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.245 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.245 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.245 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.245 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_no_wait": { 2024-08-20T03:12:30.245 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_no_wait", 2024-08-20T03:12:30.246 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.246 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.246 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.246 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.246 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.246 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.246 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.246 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-20T03:12:30.246 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.247 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.247 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.247 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.247 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.247 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.247 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.247 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.248 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.248 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.248 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.248 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.248 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.248 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.249 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.249 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.249 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.249 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.249 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.249 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.249 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.250 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix", 2024-08-20T03:12:30.250 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.250 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.250 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.250 INFO:teuthology.orchestra.run.smithi044.stdout: "discovery_interval": { 2024-08-20T03:12:30.250 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "discovery_interval", 2024-08-20T03:12:30.250 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:30.251 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.251 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.251 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "100", 2024-08-20T03:12:30.251 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.251 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.251 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.251 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.252 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.252 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.252 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.252 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.252 INFO:teuthology.orchestra.run.smithi044.stdout: "identifier": { 2024-08-20T03:12:30.252 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "identifier", 2024-08-20T03:12:30.253 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.253 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.253 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.253 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.253 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.253 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.254 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.254 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.254 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.254 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.254 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.254 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.255 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:30.255 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:30.255 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.255 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.255 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.255 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:30.255 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.256 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.256 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.256 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.256 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.256 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.256 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.257 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.257 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.257 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.257 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.257 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.258 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.258 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.258 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.258 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.258 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.259 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.259 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.259 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.259 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.259 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.259 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.260 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.260 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.260 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.260 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.260 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.261 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.261 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.261 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.261 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.261 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.261 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.262 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.262 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.262 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.262 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.262 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.263 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.263 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.263 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.263 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.263 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.263 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.264 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.264 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.264 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.264 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.264 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.265 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.265 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.265 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.265 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.265 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.265 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.266 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.266 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.266 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.266 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.266 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.267 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.267 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.267 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.267 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.267 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.267 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.268 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.268 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.268 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.268 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.268 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.268 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.269 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.269 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.269 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.269 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.269 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.269 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.269 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.269 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.269 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_host": { 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_host", 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.270 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_port": { 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_port", 2024-08-20T03:12:30.271 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10051", 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_sender": { 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_sender", 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.272 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-08-20T03:12:30.273 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: "modules": [ 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: "dashboard", 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: "iostat", 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: "nfs", 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: "restful", 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: "snap_schedule" 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: "available_modules": [ 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "alerts", 2024-08-20T03:12:30.274 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How frequently to reexamine health status", 2024-08-20T03:12:30.275 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.276 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.277 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.278 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.279 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.280 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_destination": { 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_destination", 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.281 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email address to send alerts to", 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_from_name": { 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_from_name", 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "Ceph", 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.282 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Email From: name", 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_host": { 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_host", 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.283 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP server", 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_password": { 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_password", 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.284 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Password to authenticate with", 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_port": { 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_port", 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "465", 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.285 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP port", 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_sender": { 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_sender", 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.286 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SMTP envelope sender", 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_ssl": { 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_ssl", 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.287 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "smtp_user": { 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smtp_user", 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.288 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "User to authenticate as", 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.289 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "balancer", 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.290 INFO:teuthology.orchestra.run.smithi044.stdout: "active": { 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "active", 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically balance PGs across cluster", 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.291 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_time": { 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_time", 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0000", 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "beginning time of day to automatically balance", 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.292 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.293 INFO:teuthology.orchestra.run.smithi044.stdout: "begin_weekday": { 2024-08-20T03:12:30.293 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "begin_weekday", 2024-08-20T03:12:30.293 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:30.293 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.293 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.293 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.293 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-08-20T03:12:30.293 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-08-20T03:12:30.293 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_max_iterations": { 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_max_iterations", 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "25", 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-08-20T03:12:30.294 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "250", 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_metrics": { 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_metrics", 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "pgs,objects,bytes", 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.295 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: "crush_compat_step": { 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crush_compat_step", 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.5", 2024-08-20T03:12:30.296 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0.001", 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "0.999", 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "aggressiveness of optimization", 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "end_time": { 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_time", 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2359", 2024-08-20T03:12:30.297 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ending time of day to automatically balance", 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: "end_weekday": { 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "end_weekday", 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.298 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "0", 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "6", 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.299 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.300 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.301 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.302 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.303 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "min_score": { 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_score", 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.304 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.305 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "upmap", 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "crush-compat", 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "none", 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "read", 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap", 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap-read" 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Balancer mode", 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.306 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_ids": { 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_ids", 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-20T03:12:30.307 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.308 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.309 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_deviation": { 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_deviation", 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-08-20T03:12:30.310 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "upmap_max_optimizations": { 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "upmap_max_optimizations", 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.311 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm", 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.312 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_down_multiplier": { 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_down_multiplier", 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.313 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_refresh_rate": { 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_refresh_rate", 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "20", 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.314 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "agent_starting_port": { 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "agent_starting_port", 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4721", 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.315 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_ptrace": { 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_ptrace", 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.316 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_interval": { 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_interval", 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.317 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "autotune_memory_target_ratio": { 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "autotune_memory_target_ratio", 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0.7", 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.318 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm_log_destination": { 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cephadm_log_destination", 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: "file", 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: "file,syslog", 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: "syslog" 2024-08-20T03:12:30.319 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "cgroups_split": { 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cgroups_split", 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.320 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "config_checks_enabled": { 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_checks_enabled", 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.321 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: "config_dashboard": { 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_dashboard", 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.322 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_alertmanager": { 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_alertmanager", 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-20T03:12:30.323 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_base": { 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_base", 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-20T03:12:30.324 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Container image name, without the tag", 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_elasticsearch": { 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_elasticsearch", 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-20T03:12:30.325 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.326 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.326 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.326 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "elasticsearch container image", 2024-08-20T03:12:30.326 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.326 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.326 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.326 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.326 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_grafana": { 2024-08-20T03:12:30.326 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_grafana", 2024-08-20T03:12:30.326 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.326 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.326 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_haproxy": { 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_haproxy", 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.327 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "HAproxy container image", 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_agent": { 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_agent", 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.328 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger agent container image", 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_collector": { 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_collector", 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.329 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger collector container image", 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_jaeger_query": { 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_jaeger_query", 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.330 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Jaeger query container image", 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_keepalived": { 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_keepalived", 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.331 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Keepalived container image", 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_loki": { 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_loki", 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.332 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-20T03:12:30.333 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.333 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.333 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.333 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Loki container image", 2024-08-20T03:12:30.333 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.333 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.333 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.333 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.333 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nginx": { 2024-08-20T03:12:30.333 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nginx", 2024-08-20T03:12:30.333 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.333 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nginx container image", 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_node_exporter": { 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_node_exporter", 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.334 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.335 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.335 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-20T03:12:30.335 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.335 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.335 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.335 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-08-20T03:12:30.335 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.335 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.335 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.335 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.335 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_nvmeof": { 2024-08-20T03:12:30.335 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_nvmeof", 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Nvme-of container image", 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_prometheus": { 2024-08-20T03:12:30.336 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_prometheus", 2024-08-20T03:12:30.337 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.337 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.337 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.337 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-20T03:12:30.337 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.337 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.337 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.337 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Prometheus container image", 2024-08-20T03:12:30.337 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.337 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.337 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.337 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_promtail": { 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_promtail", 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Promtail container image", 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.338 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_samba": { 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_samba", 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Samba/SMB container image", 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.339 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.340 INFO:teuthology.orchestra.run.smithi044.stdout: "container_image_snmp_gateway": { 2024-08-20T03:12:30.340 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_image_snmp_gateway", 2024-08-20T03:12:30.340 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.340 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.340 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.340 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-20T03:12:30.340 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.340 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.340 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.340 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "SNMP Gateway container image", 2024-08-20T03:12:30.340 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.340 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "container_init": { 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "container_init", 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Run podman/docker with `--init`", 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.341 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: "daemon_cache_timeout": { 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "daemon_cache_timeout", 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-20T03:12:30.342 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "default_cephadm_command_timeout": { 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_cephadm_command_timeout", 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "900", 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.343 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "default_registry": { 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "default_registry", 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "docker.io", 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.344 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: "device_cache_timeout": { 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_cache_timeout", 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1800", 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache device inventory", 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.345 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: "device_enhanced_scan": { 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_enhanced_scan", 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-20T03:12:30.346 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "facts_cache_timeout": { 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "facts_cache_timeout", 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "seconds to cache host facts data", 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.347 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "grafana_dashboards_path": { 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "grafana_dashboards_path", 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.348 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "host_check_interval": { 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "host_check_interval", 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to perform a host check", 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.349 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: "hw_monitoring": { 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hw_monitoring", 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.350 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "inventory_list_all": { 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "inventory_list_all", 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.351 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.352 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.352 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.352 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.352 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.352 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.352 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.352 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.352 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.352 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.352 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.352 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.353 INFO:teuthology.orchestra.run.smithi044.stdout: "log_refresh_metadata": { 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_refresh_metadata", 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.354 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.355 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.356 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.357 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.358 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.359 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "*", 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "max_count_per_host": { 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_count_per_host", 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.360 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of daemons per service per host", 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "max_osd_draining_count": { 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_osd_draining_count", 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.361 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: "migration_current": { 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "migration_current", 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.362 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "internal - do not modify", 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "mode": { 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mode", 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.363 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "root", 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm-package", 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: "root" 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "mode for remote execution of cephadm", 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.364 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "oob_default_addr": { 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "oob_default_addr", 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "169.254.1.1", 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.365 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "prometheus_alerts_path": { 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus_alerts_path", 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.366 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_insecure": { 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_insecure", 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.367 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_password": { 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_password", 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.368 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_url": { 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_url", 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.369 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: "registry_username": { 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "registry_username", 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-20T03:12:30.370 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "secure_monitoring_stack": { 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secure_monitoring_stack", 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-20T03:12:30.371 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: "service_discovery_port": { 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "service_discovery_port", 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8765", 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.372 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "cephadm service discovery port", 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.373 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: "ssh_config_file": { 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssh_config_file", 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.374 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: "use_agent": { 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_agent", 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.375 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "use_repo_digest": { 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "use_repo_digest", 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.376 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_failed_host_check": { 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_failed_host_check", 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.377 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if the host check fails", 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_daemons": { 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_daemons", 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.378 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_on_stray_hosts": { 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_on_stray_hosts", 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.379 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crash", 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.380 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.381 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.382 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.383 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.384 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.385 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "retain_interval": { 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retain_interval", 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "31536000", 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.386 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.387 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_recent_interval": { 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_recent_interval", 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1209600", 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.388 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dashboard", 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.389 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_HOST": { 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.390 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.393 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.393 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.394 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.394 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.394 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.394 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.394 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.394 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.394 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.394 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.394 INFO:teuthology.orchestra.run.smithi044.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-20T03:12:30.394 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-20T03:12:30.394 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.394 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_ENABLED": { 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_ENABLED", 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.395 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-20T03:12:30.396 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "ENABLE_BROWSABLE_API": { 2024-08-20T03:12:30.397 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.398 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.399 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.400 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.401 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.402 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.403 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.404 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.405 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-20T03:12:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-20T03:12:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.406 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_PASSWORD": { 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.407 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.408 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_URL": { 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_URL", 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.409 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_API_USERNAME": { 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.410 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.411 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.412 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.413 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.414 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "[]", 2024-08-20T03:12:30.415 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "MULTICLUSTER_CONFIG": { 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "{}", 2024-08-20T03:12:30.416 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_HOST": { 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.417 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.418 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.419 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.420 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.421 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.422 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.423 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.424 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.425 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_ENABLED": { 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.426 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.427 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.428 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.429 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8", 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.430 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "45", 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ACCESS_KEY": { 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.431 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.432 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "admin", 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SECRET_KEY": { 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.433 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "RGW_API_SSL_VERIFY": { 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.434 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "UNSAFE_TLS_v1_2": { 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.435 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.436 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.437 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.438 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "cross_origin_url": { 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cross_origin_url", 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.439 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "crt_file": { 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "crt_file", 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.440 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "debug": { 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "debug", 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.441 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Enable/disable debug options", 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "jwt_token_ttl": { 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "jwt_token_ttl", 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.442 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "28800", 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.443 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.444 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.445 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.446 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.447 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.448 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "motd": { 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "motd", 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.449 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "The message of the day", 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect_resolve_ip_addr": { 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "redirect_resolve_ip_addr", 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.450 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.451 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.452 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8080", 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.453 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-08-20T03:12:30.454 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl_server_port": { 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl_server_port", 2024-08-20T03:12:30.455 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8443", 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-08-20T03:12:30.456 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "redirect", 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "redirect" 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.457 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.458 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "url_prefix": { 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url_prefix", 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.459 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_monitoring": { 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_monitoring", 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.460 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "monitor device health metrics", 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.461 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.462 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.463 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.464 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.465 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "mark_out_threshold": { 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mark_out_threshold", 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.466 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "2419200", 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "pool_name": { 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pool_name", 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.467 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "device_health_metrics", 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "retention_period": { 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "retention_period", 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.468 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15552000", 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long to retain device health metrics", 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_frequency": { 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_frequency", 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-08-20T03:12:30.469 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "self_heal": { 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "self_heal", 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.470 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.471 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how frequently to wake up and check device health", 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.472 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "warn_threshold": { 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "warn_threshold", 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.473 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7257600", 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "diskprediction_local", 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.474 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.475 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.476 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.477 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.478 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.479 INFO:teuthology.orchestra.run.smithi044.stdout: "predict_interval": { 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predict_interval", 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "86400", 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.480 INFO:teuthology.orchestra.run.smithi044.stdout: "predictor_model": { 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "predictor_model", 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "prophetstor", 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.481 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "600", 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.482 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.483 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "influx", 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": false, 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "influxdb python module not found", 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "batch_size": { 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "batch_size", 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5000", 2024-08-20T03:12:30.484 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "database": { 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "database", 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "ceph", 2024-08-20T03:12:30.485 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": { 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "hostname", 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.486 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server hostname", 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.487 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "30", 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "5", 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.488 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.489 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.490 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.491 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.492 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "password": { 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "password", 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.493 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "password of InfluxDB server user", 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "port": { 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "port", 2024-08-20T03:12:30.494 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "8086", 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "InfluxDB server port", 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.495 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "ssl": { 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ssl", 2024-08-20T03:12:30.496 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "false", 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "threads": { 2024-08-20T03:12:30.497 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threads", 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "32", 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "username": { 2024-08-20T03:12:30.498 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "username", 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "username of InfluxDB server user", 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "verify_ssl": { 2024-08-20T03:12:30.499 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "verify_ssl", 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "true", 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.500 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "insights", 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.501 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.502 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.503 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.504 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.505 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.506 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "iostat", 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.507 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.508 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.509 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.510 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.511 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.512 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "k8sevents", 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.513 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_event_retention_days": { 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "ceph_event_retention_days", 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "7", 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.514 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: "config_check_secs": { 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "config_check_secs", 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "10", 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.515 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.516 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.517 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.518 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.519 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.520 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.521 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "localpool", 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: "failure_domain": { 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "failure_domain", 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.522 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "host", 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "failure domain for any created local pool", 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.523 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.524 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.525 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.526 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.527 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "min_size": { 2024-08-20T03:12:30.528 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "min_size", 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default min_size for any created local pool", 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "num_rep": { 2024-08-20T03:12:30.529 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "num_rep", 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3", 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default replica count for any created local pool", 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_num": { 2024-08-20T03:12:30.530 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_num", 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "128", 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "default pg_num for any created local pool", 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "prefix": { 2024-08-20T03:12:30.531 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prefix", 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "name prefix for any created local pool", 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.532 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "subtree": { 2024-08-20T03:12:30.533 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "subtree", 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "rack", 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.534 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mds_autoscaler", 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.535 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.536 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.537 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.538 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.539 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.540 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirroring", 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.541 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.542 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.543 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.544 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.545 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.546 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.547 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "nfs", 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.548 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.549 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.550 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.551 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.552 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.553 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.554 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "fail_fs": { 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "fail_fs", 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.555 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.556 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.557 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.558 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.559 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.560 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator": { 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "orchestrator", 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: "cephadm", 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: "rook", 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: "test_orchestrator" 2024-08-20T03:12:30.561 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Orchestrator backend", 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.562 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_perf_query", 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.563 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.564 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.565 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.566 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.567 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.568 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.569 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "osd_support", 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.570 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.571 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.572 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.572 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.572 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.572 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.572 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.572 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.572 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.572 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.572 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.572 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.572 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.572 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.573 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.574 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.575 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "pg_autoscaler", 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.576 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.577 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.578 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.579 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.580 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:30.581 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.582 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "threshold": { 2024-08-20T03:12:30.583 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "threshold", 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "3.0", 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1.0", 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "scaling threshold", 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.584 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "progress", 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_pg_recovery_event": { 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_pg_recovery_event", 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.585 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.586 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.587 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.588 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.589 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.590 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.590 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.590 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.590 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.590 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.590 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.590 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.590 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.590 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.590 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.590 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.590 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.591 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "max_completed_events": { 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_completed_events", 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.592 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "50", 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "number of past completed events to remember", 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "sleep_interval": { 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sleep_interval", 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.593 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "how long the module is going to sleep", 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.594 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.595 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.595 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.595 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.595 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.595 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.595 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.597 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "prometheus", 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: "cache": { 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "cache", 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.598 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "exclude_perf_counters": { 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "exclude_perf_counters", 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.599 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.600 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.601 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.602 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.603 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.604 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools": { 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools", 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.605 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "300", 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.606 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "scrape_interval": { 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "scrape_interval", 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15.0", 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.607 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "::", 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.608 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "9283", 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.609 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.610 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "stale_cache_strategy": { 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stale_cache_strategy", 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "log", 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.611 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_behaviour": { 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_behaviour", 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "default", 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.612 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "default", 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "error" 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "standby_error_status_code": { 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "standby_error_status_code", 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "500", 2024-08-20T03:12:30.613 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "400", 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "599", 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.614 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.615 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.616 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.617 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.618 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.619 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_snap_create": { 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_snap_create", 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10", 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.620 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "mirror_snapshot_schedule": { 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "mirror_snapshot_schedule", 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.621 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.622 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: "trash_purge_schedule": { 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "trash_purge_schedule", 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.623 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "restful", 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: "enable_auth": { 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enable_auth", 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.624 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "key_file": { 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "key_file", 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.625 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.626 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.627 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.628 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.629 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.630 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.631 INFO:teuthology.orchestra.run.smithi044.stdout: "server_addr": { 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_addr", 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.632 INFO:teuthology.orchestra.run.smithi044.stdout: "server_port": { 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "server_port", 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.633 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.634 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rgw", 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.635 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.636 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.637 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.638 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.639 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "secondary_zone_period_retry_limit": { 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "5", 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.640 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.641 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rook", 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.642 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.643 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.644 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.645 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.646 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.647 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "storage_class": { 2024-08-20T03:12:30.648 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "storage_class", 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "local", 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.649 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "selftest", 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.650 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.651 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.652 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.653 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.654 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: "roption1": { 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption1", 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.655 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "roption2": { 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "roption2", 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "xyz", 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.656 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption1": { 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption1", 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.657 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption2": { 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption2", 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.658 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption3": { 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption3", 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "float", 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.659 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption4": { 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption4", 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.660 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption5": { 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption5", 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.661 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption6": { 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption6", 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.662 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: "rwoption7": { 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rwoption7", 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.663 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "1", 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "42", 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.664 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: "testkey": { 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testkey", 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.665 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "testlkey": { 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testlkey", 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.666 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "testnewline": { 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "testnewline", 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.667 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "smb", 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.668 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "internal_store_backend": { 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "internal_store_backend", 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.669 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.670 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.671 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.672 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.673 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.674 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.675 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "update_orchestration": { 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "update_orchestration", 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.676 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: "allow_m_granularity": { 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "allow_m_granularity", 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.677 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "allow minute scheduled snapshots", 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "dump_on_update": { 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "dump_on_update", 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.678 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "dump database to debug log on update", 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.679 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.680 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.681 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.682 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.683 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.684 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "stats", 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.685 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.686 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.686 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.686 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.686 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.686 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.686 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.686 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.686 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.686 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.686 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.686 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.686 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.687 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.688 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.689 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.690 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.691 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "status", 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.692 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.693 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.694 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.695 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.696 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.697 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.698 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telegraf", 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "address": { 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "address", 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.699 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "15", 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.700 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.701 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.702 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.703 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.704 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.705 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.706 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "telemetry", 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_basic": { 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_basic", 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.707 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_crash": { 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_crash", 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.708 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_device": { 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_device", 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-20T03:12:30.709 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_ident": { 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_ident", 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-20T03:12:30.710 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "channel_perf": { 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "channel_perf", 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-20T03:12:30.711 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "contact": { 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "contact", 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.712 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "description": { 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "description", 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.713 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: "device_url": { 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "device_url", 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.714 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "enabled": { 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "enabled", 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.715 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "24", 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "8", 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.716 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "last_opt_revision": { 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "last_opt_revision", 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "1", 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.717 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard": { 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard", 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.718 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "leaderboard_description": { 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "leaderboard_description", 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.719 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.720 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.721 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.722 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.723 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.724 INFO:teuthology.orchestra.run.smithi044.stdout: "organization": { 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "organization", 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.725 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "proxy": { 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "proxy", 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.726 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.727 INFO:teuthology.orchestra.run.smithi044.stdout: "url": { 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "url", 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.728 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "test_orchestrator", 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.729 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.730 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.731 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.732 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.733 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.734 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.735 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.736 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.737 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.738 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.739 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "max_concurrent_clones": { 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "max_concurrent_clones", 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.740 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "4", 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: "periodic_async_work": { 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "periodic_async_work", 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.741 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Periodically check for async work", 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_delay": { 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_delay", 2024-08-20T03:12:30.742 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "snapshot_clone_no_wait": { 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snapshot_clone_no_wait", 2024-08-20T03:12:30.743 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "True", 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.744 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.745 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix", 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "can_run": true, 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "error_string": "", 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "module_options": { 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "discovery_interval": { 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "discovery_interval", 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "uint", 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "100", 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.746 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "identifier": { 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "identifier", 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.747 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "interval": { 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "interval", 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "secs", 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "60", 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.748 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "log_level": { 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_level", 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.749 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster": { 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster", 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.750 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_cluster_level": { 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_cluster_level", 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "info", 2024-08-20T03:12:30.751 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [ 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "", 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "critical", 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "debug", 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "error", 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "info", 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "warning" 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.752 INFO:teuthology.orchestra.run.smithi044.stdout: "log_to_file": { 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "log_to_file", 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "bool", 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "False", 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "sqlite3_killpoint": { 2024-08-20T03:12:30.753 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "sqlite3_killpoint", 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "dev", 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 1, 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "0", 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_host": { 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_host", 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.754 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "", 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_port": { 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_port", 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "int", 2024-08-20T03:12:30.755 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "10051", 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "zabbix_sender": { 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "zabbix_sender", 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "str", 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "level": "advanced", 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "flags": 0, 2024-08-20T03:12:30.756 INFO:teuthology.orchestra.run.smithi044.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: "min": "", 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: "max": "", 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: "enum_allowed": [], 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: "desc": "", 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: "long_desc": "", 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: "tags": [], 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: "see_also": [] 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: "services": { 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: "dashboard": "https://172.21.15.44:8443/" 2024-08-20T03:12:30.757 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "always_on_modules": { 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "octopus": [ 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.758 INFO:teuthology.orchestra.run.smithi044.stdout: "pacific": [ 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "quincy": [ 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-08-20T03:12:30.759 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: "reef": [ 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-08-20T03:12:30.760 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: ], 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "squid": [ 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "balancer", 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "crash", 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "devicehealth", 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "orchestrator", 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "pg_autoscaler", 2024-08-20T03:12:30.761 INFO:teuthology.orchestra.run.smithi044.stdout: "progress", 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: "rbd_support", 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: "status", 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: "telemetry", 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: "volumes" 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: "last_failure_osd_epoch": 5, 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: "active_clients": [ 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "devicehealth", 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.762 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2273808426 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "libcephsqlite", 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 4115618083 2024-08-20T03:12:30.763 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "rbd_support", 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2487088235 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.764 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2322376513 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "snap_schedule", 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-08-20T03:12:30.765 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 4212190056 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "volumes", 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: "addrvec": [ 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v2", 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "172.21.15.44:0", 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 1832265932 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.766 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-08-20T03:12:30.767 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:30.767 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-08-20T03:12:30.767 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:12:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.135+0000 7f8b6eb31640 1 -- 172.21.15.44:0/979261349 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8b500b3f50 msgr2=0x7f8b500b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.135+0000 7f8b6eb31640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8b500b3f50 0x7f8b500b6410 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8b58000c00 tx=0x7f8b580105a0 comp rx=0 tx=0).stop 2024-08-20T03:12:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.135+0000 7f8b6eb31640 1 -- 172.21.15.44:0/979261349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b68101cb0 msgr2=0x7f8b6806cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.135+0000 7f8b6eb31640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b68101cb0 0x7f8b6806cba0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400d770 tx=0x7f8b6400dc30 comp rx=0 tx=0).stop 2024-08-20T03:12:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.136+0000 7f8b6eb31640 1 -- 172.21.15.44:0/979261349 shutdown_connections 2024-08-20T03:12:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.136+0000 7f8b6eb31640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f8b500b3f50 0x7f8b500b6410 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.136+0000 7f8b6eb31640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b68104a70 0x7f8b68072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.136+0000 7f8b6eb31640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b681040d0 0x7f8b6806d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:30.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.136+0000 7f8b6eb31640 1 --2- 172.21.15.44:0/979261349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b68101cb0 0x7f8b6806cba0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:30.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.136+0000 7f8b6eb31640 1 -- 172.21.15.44:0/979261349 >> 172.21.15.44:0/979261349 conn(0x7f8b680773c0 msgr2=0x7f8b681052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:30.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.136+0000 7f8b6eb31640 1 -- 172.21.15.44:0/979261349 shutdown_connections 2024-08-20T03:12:30.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:29.137+0000 7f8b6eb31640 1 -- 172.21.15.44:0/979261349 wait complete. 2024-08-20T03:12:30.775 DEBUG:tasks.check_counter:Getting stats from a 2024-08-20T03:12:30.775 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph daemon mds.a perf dump 2024-08-20T03:12:30.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:29 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/509344002' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T03:12:30.775 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:30 smithi044 ceph-mon[18790]: pgmap v627: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 793 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 0 B/s wr, 11 op/s 2024-08-20T03:12:30.776 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:30 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/979261349' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-20T03:12:30.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:29 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/509344002' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T03:12:30.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:30 smithi099 ceph-mon[21984]: pgmap v627: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 793 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 0 B/s wr, 11 op/s 2024-08-20T03:12:30.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:30 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/979261349' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-20T03:12:30.777 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:29 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/509344002' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T03:12:30.777 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:30 smithi165 ceph-mon[22357]: pgmap v627: 97 pgs: 15 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 48 active+clean; 793 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 0 B/s wr, 11 op/s 2024-08-20T03:12:30.777 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:30 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/979261349' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-20T03:12:31.201 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:31.891 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:12:31.892 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T03:12:31.892 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1061, 2024-08-20T03:12:31.892 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 1036, 2024-08-20T03:12:31.892 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1673238, 2024-08-20T03:12:31.892 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 386918, 2024-08-20T03:12:31.892 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 6, 2024-08-20T03:12:31.892 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 2, 2024-08-20T03:12:31.892 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.140725895, 2024-08-20T03:12:31.892 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.060212796, 2024-08-20T03:12:31.892 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.055949904, 2024-08-20T03:12:31.893 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.028927213, 2024-08-20T03:12:31.893 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:31.893 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1035, 2024-08-20T03:12:31.893 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.058516357, 2024-08-20T03:12:31.893 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000056537 2024-08-20T03:12:31.893 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.893 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:31.893 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.893 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.893 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1406832, 2024-08-20T03:12:31.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 125840 2024-08-20T03:12:31.894 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.894 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T03:12:31.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 696, 2024-08-20T03:12:31.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 846, 2024-08-20T03:12:31.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 6660085, 2024-08-20T03:12:31.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 570798, 2024-08-20T03:12:31.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-08-20T03:12:31.894 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 4, 2024-08-20T03:12:31.895 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.089160117, 2024-08-20T03:12:31.895 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.039691296, 2024-08-20T03:12:31.895 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.023235911, 2024-08-20T03:12:31.895 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.020017010, 2024-08-20T03:12:31.895 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:31.895 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 845, 2024-08-20T03:12:31.895 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.050446114, 2024-08-20T03:12:31.895 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000059699 2024-08-20T03:12:31.895 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.895 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:31.895 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.896 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.896 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.896 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-08-20T03:12:31.896 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 350992 2024-08-20T03:12:31.896 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.896 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T03:12:31.896 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 697, 2024-08-20T03:12:31.896 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 699, 2024-08-20T03:12:31.896 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 8567871, 2024-08-20T03:12:31.897 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 221647, 2024-08-20T03:12:31.897 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-08-20T03:12:31.897 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 4, 2024-08-20T03:12:31.897 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.075360271, 2024-08-20T03:12:31.897 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.030504100, 2024-08-20T03:12:31.897 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.021902641, 2024-08-20T03:12:31.897 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.018780211, 2024-08-20T03:12:31.897 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:31.897 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 697, 2024-08-20T03:12:31.897 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.040108589, 2024-08-20T03:12:31.897 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000057544 2024-08-20T03:12:31.897 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.898 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:31.898 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.898 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.898 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.898 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T03:12:31.898 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T03:12:31.898 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.899 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-08-20T03:12:31.899 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-08-20T03:12:31.899 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-08-20T03:12:31.899 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.899 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-08-20T03:12:31.899 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-08-20T03:12:31.899 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-08-20T03:12:31.899 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3724, 2024-08-20T03:12:31.900 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.100123784, 2024-08-20T03:12:31.900 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000026886 2024-08-20T03:12:31.900 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:31.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.900 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-08-20T03:12:31.900 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-08-20T03:12:31.900 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-08-20T03:12:31.900 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.900 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.900 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.900 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:31.900 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.901 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-08-20T03:12:31.901 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 0, 2024-08-20T03:12:31.901 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 0, 2024-08-20T03:12:31.901 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-08-20T03:12:31.901 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.901 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.901 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.901 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.901 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-08-20T03:12:31.901 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 0, 2024-08-20T03:12:31.901 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-08-20T03:12:31.902 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-08-20T03:12:31.902 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 0, 2024-08-20T03:12:31.902 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-08-20T03:12:31.902 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-08-20T03:12:31.902 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 1659, 2024-08-20T03:12:31.902 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 0, 2024-08-20T03:12:31.902 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 1659, 2024-08-20T03:12:31.902 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-08-20T03:12:31.902 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 1659, 2024-08-20T03:12:31.902 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 0, 2024-08-20T03:12:31.902 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 0, 2024-08-20T03:12:31.902 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 0, 2024-08-20T03:12:31.903 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 1, 2024-08-20T03:12:31.903 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 0, 2024-08-20T03:12:31.903 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 0, 2024-08-20T03:12:31.903 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2024-08-20T03:12:31.903 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2024-08-20T03:12:31.903 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-08-20T03:12:31.903 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-08-20T03:12:31.903 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-08-20T03:12:31.903 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-08-20T03:12:31.903 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-08-20T03:12:31.904 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-08-20T03:12:31.904 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-08-20T03:12:31.904 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2024-08-20T03:12:31.904 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-08-20T03:12:31.904 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-08-20T03:12:31.904 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2024-08-20T03:12:31.904 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-08-20T03:12:31.904 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 0, 2024-08-20T03:12:31.904 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 0, 2024-08-20T03:12:31.904 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 0, 2024-08-20T03:12:31.904 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T03:12:31.905 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-08-20T03:12:31.905 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T03:12:31.905 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-08-20T03:12:31.905 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-08-20T03:12:31.905 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T03:12:31.905 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T03:12:31.905 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2024-08-20T03:12:31.905 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-08-20T03:12:31.905 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2024-08-20T03:12:31.905 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2024-08-20T03:12:31.905 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T03:12:31.906 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T03:12:31.906 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T03:12:31.906 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 0, 2024-08-20T03:12:31.906 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2024-08-20T03:12:31.906 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2024-08-20T03:12:31.906 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 0 2024-08-20T03:12:31.906 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.906 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-08-20T03:12:31.906 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-08-20T03:12:31.906 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2024-08-20T03:12:31.906 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2024-08-20T03:12:31.906 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2024-08-20T03:12:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2024-08-20T03:12:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-08-20T03:12:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-08-20T03:12:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-08-20T03:12:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2024-08-20T03:12:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2024-08-20T03:12:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-08-20T03:12:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-08-20T03:12:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-08-20T03:12:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-08-20T03:12:31.907 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-08-20T03:12:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-08-20T03:12:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-08-20T03:12:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 0, 2024-08-20T03:12:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 0, 2024-08-20T03:12:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T03:12:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-08-20T03:12:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-08-20T03:12:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-08-20T03:12:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-08-20T03:12:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-08-20T03:12:31.908 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-08-20T03:12:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-08-20T03:12:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 0, 2024-08-20T03:12:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-08-20T03:12:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-08-20T03:12:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 8613, 2024-08-20T03:12:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-08-20T03:12:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-08-20T03:12:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 0, 2024-08-20T03:12:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-08-20T03:12:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-08-20T03:12:31.909 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 11, 2024-08-20T03:12:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-08-20T03:12:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-08-20T03:12:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-08-20T03:12:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-08-20T03:12:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 19062816, 2024-08-20T03:12:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 19058836, 2024-08-20T03:12:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-08-20T03:12:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.910 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 8613 2024-08-20T03:12:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-08-20T03:12:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 12, 2024-08-20T03:12:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 3765, 2024-08-20T03:12:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 3753, 2024-08-20T03:12:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 10, 2024-08-20T03:12:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 96, 2024-08-20T03:12:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 86, 2024-08-20T03:12:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 1659, 2024-08-20T03:12:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 6335, 2024-08-20T03:12:31.911 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 4676, 2024-08-20T03:12:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 0, 2024-08-20T03:12:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 0, 2024-08-20T03:12:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 0, 2024-08-20T03:12:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 59236, 2024-08-20T03:12:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 262025216 2024-08-20T03:12:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-08-20T03:12:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 0, 2024-08-20T03:12:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2024-08-20T03:12:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 0, 2024-08-20T03:12:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 0, 2024-08-20T03:12:31.912 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 0, 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.913 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-08-20T03:12:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-08-20T03:12:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.914 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-08-20T03:12:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-08-20T03:12:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.915 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-08-20T03:12:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-08-20T03:12:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-08-20T03:12:31.916 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-08-20T03:12:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.917 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-08-20T03:12:31.918 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.918 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.918 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.918 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-08-20T03:12:31.918 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-08-20T03:12:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-08-20T03:12:31.919 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-08-20T03:12:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-08-20T03:12:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.920 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-08-20T03:12:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-08-20T03:12:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-08-20T03:12:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-08-20T03:12:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.922 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-08-20T03:12:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-08-20T03:12:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-08-20T03:12:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.923 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-08-20T03:12:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-08-20T03:12:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.924 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-08-20T03:12:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-08-20T03:12:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.925 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-08-20T03:12:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-08-20T03:12:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-08-20T03:12:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-08-20T03:12:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.926 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-08-20T03:12:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 2, 2024-08-20T03:12:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 3, 2024-08-20T03:12:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 1, 2024-08-20T03:12:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 3, 2024-08-20T03:12:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-08-20T03:12:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 0, 2024-08-20T03:12:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 0, 2024-08-20T03:12:31.927 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 2205, 2024-08-20T03:12:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-08-20T03:12:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-08-20T03:12:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-08-20T03:12:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-08-20T03:12:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T03:12:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-08-20T03:12:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T03:12:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-08-20T03:12:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T03:12:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T03:12:31.928 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T03:12:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T03:12:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T03:12:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-08-20T03:12:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T03:12:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T03:12:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-08-20T03:12:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-08-20T03:12:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-08-20T03:12:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-08-20T03:12:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T03:12:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T03:12:31.929 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T03:12:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T03:12:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T03:12:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-08-20T03:12:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-08-20T03:12:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-08-20T03:12:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T03:12:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T03:12:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-08-20T03:12:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-08-20T03:12:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-08-20T03:12:31.930 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-08-20T03:12:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T03:12:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-08-20T03:12:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T03:12:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-08-20T03:12:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 207111, 2024-08-20T03:12:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 62, 2024-08-20T03:12:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-08-20T03:12:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-08-20T03:12:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-08-20T03:12:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-08-20T03:12:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-08-20T03:12:31.931 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-08-20T03:12:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-08-20T03:12:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-08-20T03:12:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 8096, 2024-08-20T03:12:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 180, 2024-08-20T03:12:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T03:12:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-08-20T03:12:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-08-20T03:12:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-08-20T03:12:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 1183304, 2024-08-20T03:12:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 6782, 2024-08-20T03:12:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-08-20T03:12:31.932 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-08-20T03:12:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-08-20T03:12:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-08-20T03:12:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-08-20T03:12:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-08-20T03:12:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-08-20T03:12:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 2209, 2024-08-20T03:12:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 21, 2024-08-20T03:12:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 3, 2024-08-20T03:12:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 2209, 2024-08-20T03:12:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-08-20T03:12:31.933 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2209, 2024-08-20T03:12:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 5.324917566, 2024-08-20T03:12:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.002410555 2024-08-20T03:12:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-08-20T03:12:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-08-20T03:12:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2209, 2024-08-20T03:12:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 2210 2024-08-20T03:12:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 2209, 2024-08-20T03:12:31.934 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 2209, 2024-08-20T03:12:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 0, 2024-08-20T03:12:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-08-20T03:12:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-08-20T03:12:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 1330, 2024-08-20T03:12:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 0, 2024-08-20T03:12:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 877, 2024-08-20T03:12:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 0, 2024-08-20T03:12:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 0, 2024-08-20T03:12:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-08-20T03:12:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-08-20T03:12:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-08-20T03:12:31.935 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-08-20T03:12:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 0, 2024-08-20T03:12:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-08-20T03:12:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-08-20T03:12:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-08-20T03:12:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 1, 2024-08-20T03:12:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 0, 2024-08-20T03:12:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-08-20T03:12:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-08-20T03:12:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-08-20T03:12:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-08-20T03:12:31.936 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-08-20T03:12:31.937 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-08-20T03:12:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-08-20T03:12:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-08-20T03:12:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-08-20T03:12:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-08-20T03:12:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-08-20T03:12:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-08-20T03:12:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-08-20T03:12:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-08-20T03:12:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 84, 2024-08-20T03:12:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-08-20T03:12:31.938 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 9, 2024-08-20T03:12:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 7, 2024-08-20T03:12:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 7, 2024-08-20T03:12:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 0, 2024-08-20T03:12:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-08-20T03:12:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 0, 2024-08-20T03:12:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 2, 2024-08-20T03:12:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 0 2024-08-20T03:12:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-08-20T03:12:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 0, 2024-08-20T03:12:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-08-20T03:12:31.939 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-08-20T03:12:31.940 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-08-20T03:12:31.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.940 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-08-20T03:12:31.940 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-08-20T03:12:31.940 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2024-08-20T03:12:31.940 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-08-20T03:12:31.940 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2024-08-20T03:12:31.940 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 0, 2024-08-20T03:12:31.940 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2024-08-20T03:12:31.940 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-08-20T03:12:31.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.941 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T03:12:31.941 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-08-20T03:12:31.941 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-08-20T03:12:31.941 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-08-20T03:12:31.941 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 2450, 2024-08-20T03:12:31.941 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 16665431, 2024-08-20T03:12:31.941 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:31.941 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 2450, 2024-08-20T03:12:31.941 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-08-20T03:12:31.941 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-08-20T03:12:31.941 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2450, 2024-08-20T03:12:31.942 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 16665431, 2024-08-20T03:12:31.942 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-08-20T03:12:31.942 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.942 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.942 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.942 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:31.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.942 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-08-20T03:12:31.942 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-08-20T03:12:31.942 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-08-20T03:12:31.942 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-08-20T03:12:31.943 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-08-20T03:12:31.943 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-08-20T03:12:31.943 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:31.943 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:31.943 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 2209, 2024-08-20T03:12:31.943 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 14870732, 2024-08-20T03:12:31.943 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2209, 2024-08-20T03:12:31.943 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 14870732, 2024-08-20T03:12:31.943 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-08-20T03:12:31.943 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.943 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.943 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.944 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:31.944 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.944 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-08-20T03:12:31.944 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-08-20T03:12:31.944 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-08-20T03:12:31.944 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-08-20T03:12:31.944 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-08-20T03:12:31.944 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-08-20T03:12:31.944 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:31.944 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:31.945 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 2209, 2024-08-20T03:12:31.945 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 2209, 2024-08-20T03:12:31.945 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2209, 2024-08-20T03:12:31.945 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 2209, 2024-08-20T03:12:31.945 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-08-20T03:12:31.945 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.945 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.945 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.945 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:31.945 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.945 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-08-20T03:12:31.946 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-08-20T03:12:31.946 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-08-20T03:12:31.946 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-08-20T03:12:31.946 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-08-20T03:12:31.946 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-08-20T03:12:31.946 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:31.946 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:31.946 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-08-20T03:12:31.946 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-08-20T03:12:31.946 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-08-20T03:12:31.946 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-08-20T03:12:31.946 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-08-20T03:12:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x55d27cb12960": { 2024-08-20T03:12:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-08-20T03:12:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-08-20T03:12:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-08-20T03:12:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-08-20T03:12:31.947 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:31.948 INFO:teuthology.orchestra.run.smithi044.stdout: "trackedop": { 2024-08-20T03:12:31.949 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-08-20T03:12:31.949 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:31.949 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:12:32.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:32 smithi165 ceph-mon[22357]: pgmap v628: 97 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 53 active+clean; 788 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 11 op/s 2024-08-20T03:12:32.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:32 smithi099 ceph-mon[21984]: pgmap v628: 97 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 53 active+clean; 788 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 11 op/s 2024-08-20T03:12:32.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:32 smithi044 ceph-mon[18790]: pgmap v628: 97 pgs: 13 active+clean+snaptrim, 31 active+clean+snaptrim_wait, 53 active+clean; 788 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 0 B/s wr, 11 op/s 2024-08-20T03:12:32.741 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-08-20T03:12:32.741 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-08-20T03:12:32.741 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-08-20T03:12:32.741 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-08-20T03:12:32.741 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-08-20T03:12:32.741 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-08-20T03:12:32.741 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T03:12:32.741 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-08-20T03:12:32.742 DEBUG:tasks.check_counter:Getting stats from d 2024-08-20T03:12:32.742 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph daemon mds.d perf dump 2024-08-20T03:12:33.080 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:33.634 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:12:33.635 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T03:12:33.635 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 4, 2024-08-20T03:12:33.635 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 4, 2024-08-20T03:12:33.635 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 3040, 2024-08-20T03:12:33.635 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 448, 2024-08-20T03:12:33.635 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-08-20T03:12:33.635 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2024-08-20T03:12:33.635 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.003321767, 2024-08-20T03:12:33.635 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000053539, 2024-08-20T03:12:33.635 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.002395602, 2024-08-20T03:12:33.635 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3, 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.004273151, 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001424383 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 448 2024-08-20T03:12:33.636 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1, 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 157, 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 160, 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 107552, 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 3, 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 2, 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.013973762, 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.011261024, 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.001239943, 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:33.637 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 155, 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.013662683, 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000088146 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 107552 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T03:12:33.638 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 214, 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 206, 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1271744, 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 126624, 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.039655959, 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.017490322, 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.032501535, 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 205, 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.011113518, 2024-08-20T03:12:33.639 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000054212 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1271744, 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 126624 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:33.640 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-08-20T03:12:33.641 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-08-20T03:12:33.642 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 148984, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 30, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-08-20T03:12:33.643 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-08-20T03:12:33.644 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 215, 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1241522, 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 215, 2024-08-20T03:12:33.646 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-08-20T03:12:33.646 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-08-20T03:12:33.646 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 215, 2024-08-20T03:12:33.646 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1241522, 2024-08-20T03:12:33.646 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-08-20T03:12:33.646 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:33.646 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:33.646 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:33.646 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:33.646 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:33.646 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:12:34.285 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:34 smithi044 ceph-mon[18790]: pgmap v629: 97 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 60 active+clean; 788 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 0 B/s wr, 12 op/s 2024-08-20T03:12:34.345 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-08-20T03:12:34.345 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-08-20T03:12:34.345 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-08-20T03:12:34.346 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-08-20T03:12:34.346 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-08-20T03:12:34.346 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-08-20T03:12:34.346 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-08-20T03:12:34.346 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-08-20T03:12:34.346 DEBUG:tasks.check_counter:Getting stats from g 2024-08-20T03:12:34.346 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph daemon mds.g perf dump 2024-08-20T03:12:34.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:34 smithi099 ceph-mon[21984]: pgmap v629: 97 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 60 active+clean; 788 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 0 B/s wr, 12 op/s 2024-08-20T03:12:34.600 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:34.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:34 smithi165 ceph-mon[22357]: pgmap v629: 97 pgs: 10 active+clean+snaptrim, 27 active+clean+snaptrim_wait, 60 active+clean; 788 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 0 B/s wr, 12 op/s 2024-08-20T03:12:35.128 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:12:35.128 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T03:12:35.128 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 959, 2024-08-20T03:12:35.128 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 934, 2024-08-20T03:12:35.128 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1648288, 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 355126, 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.136182228, 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.064409419, 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.053899913, 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.022229540, 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 933, 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.063106317, 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000067638 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:35.129 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1407376, 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 127120 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1430, 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 1583, 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 3739691, 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 805199, 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-08-20T03:12:35.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 4, 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.133096758, 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.057426372, 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.029181319, 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.039445562, 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1581, 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.109960637, 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000069551 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.131 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 350720 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 8, 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 8, 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 4669, 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 1732, 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 4, 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.003421675, 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000162596, 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.003140583, 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000087374, 2024-08-20T03:12:35.132 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 7, 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.006638482, 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000948354 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 448 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.133 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3585, 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.097692739, 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000027250 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-08-20T03:12:35.134 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 0, 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 0, 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.135 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 0, 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 0, 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 346, 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 0, 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 346, 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-08-20T03:12:35.136 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 346, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 0, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 0, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 0, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 1, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 0, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 0, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-08-20T03:12:35.137 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-08-20T03:12:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-08-20T03:12:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2024-08-20T03:12:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-08-20T03:12:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-08-20T03:12:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2024-08-20T03:12:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-08-20T03:12:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 0, 2024-08-20T03:12:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 0, 2024-08-20T03:12:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 0, 2024-08-20T03:12:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T03:12:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-08-20T03:12:35.138 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2024-08-20T03:12:35.139 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 0 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2024-08-20T03:12:35.140 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_path": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_quiesce_inode": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-08-20T03:12:35.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-08-20T03:12:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-08-20T03:12:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-08-20T03:12:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-08-20T03:12:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 0, 2024-08-20T03:12:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-08-20T03:12:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-08-20T03:12:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 1965, 2024-08-20T03:12:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-08-20T03:12:35.142 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-08-20T03:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 0, 2024-08-20T03:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-08-20T03:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-08-20T03:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 3, 2024-08-20T03:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-08-20T03:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-08-20T03:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-08-20T03:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-08-20T03:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 7570765, 2024-08-20T03:12:35.143 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 7566312, 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 1965 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 5, 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 840, 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 835, 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 3, 2024-08-20T03:12:35.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 55, 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 52, 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 346, 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 1510, 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 1164, 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 0, 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 0, 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 0, 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 54248, 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 254664704 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 0, 2024-08-20T03:12:35.145 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 0, 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 0, 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 0, 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-08-20T03:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-08-20T03:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-08-20T03:12:35.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.148 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-08-20T03:12:35.150 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.151 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-08-20T03:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-08-20T03:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-08-20T03:12:35.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.155 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.156 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 2, 2024-08-20T03:12:35.157 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 4, 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 2, 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 3, 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 0, 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 0, 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 2206, 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-08-20T03:12:35.158 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-08-20T03:12:35.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-08-20T03:12:35.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 197520, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 56, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-08-20T03:12:35.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 8096, 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 180, 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 261544, 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 1441, 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.162 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 2150, 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 21, 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 3, 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 2150, 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2150, 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 10.622835359, 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.004940853 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-08-20T03:12:35.163 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2150, 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 2151 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 2150, 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 2150, 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 0, 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 1345, 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 0, 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 803, 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 0, 2024-08-20T03:12:35.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 1, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-08-20T03:12:35.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-08-20T03:12:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-08-20T03:12:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-08-20T03:12:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T03:12:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-08-20T03:12:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-08-20T03:12:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-08-20T03:12:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-08-20T03:12:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-08-20T03:12:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-08-20T03:12:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-08-20T03:12:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-08-20T03:12:35.166 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 84, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 9, 2024-08-20T03:12:35.167 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 6, 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 6, 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 0, 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 0, 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 2, 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 0 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 0, 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.168 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 0, 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 2393, 2024-08-20T03:12:35.169 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 5161911, 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 2393, 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2393, 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 5161911, 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.170 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 2150, 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 3376466, 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2150, 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 3376466, 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.171 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 2150, 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 2150, 2024-08-20T03:12:35.172 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 2150, 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 2150, 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-08-20T03:12:35.173 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x55e46a9d8960": { 2024-08-20T03:12:35.174 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:35.175 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:35.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:35.176 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:35.176 INFO:teuthology.orchestra.run.smithi044.stdout: "trackedop": { 2024-08-20T03:12:35.176 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-08-20T03:12:35.176 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:35.176 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:12:35.846 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-08-20T03:12:35.846 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-08-20T03:12:35.846 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-08-20T03:12:35.847 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-08-20T03:12:35.847 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-08-20T03:12:35.847 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-08-20T03:12:35.847 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T03:12:35.847 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-08-20T03:12:35.847 DEBUG:tasks.check_counter:Getting stats from j 2024-08-20T03:12:35.847 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph daemon mds.j perf dump 2024-08-20T03:12:36.097 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:36.399 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:36 smithi044 ceph-mon[18790]: pgmap v630: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 785 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-08-20T03:12:36.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:36 smithi099 ceph-mon[21984]: pgmap v630: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 785 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-08-20T03:12:36.607 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-08-20T03:12:36.608 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T03:12:36.608 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 0, 2024-08-20T03:12:36.608 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2, 2024-08-20T03:12:36.608 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 0, 2024-08-20T03:12:36.608 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 0, 2024-08-20T03:12:36.608 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-08-20T03:12:36.608 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2024-08-20T03:12:36.608 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.001123138, 2024-08-20T03:12:36.608 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000000000, 2024-08-20T03:12:36.608 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.000204907, 2024-08-20T03:12:36.608 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T03:12:36.609 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:36.609 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:36.609 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:36.609 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:36.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:36.609 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:36.609 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:36.609 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:36.609 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:36.609 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:36.609 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T03:12:36.609 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T03:12:36.610 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:36.610 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T03:12:36.610 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 220, 2024-08-20T03:12:36.610 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 210, 2024-08-20T03:12:36.610 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1275328, 2024-08-20T03:12:36.610 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 128352, 2024-08-20T03:12:36.610 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2024-08-20T03:12:36.610 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-08-20T03:12:36.610 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.045732520, 2024-08-20T03:12:36.610 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.017868573, 2024-08-20T03:12:36.610 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.038094678, 2024-08-20T03:12:36.610 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T03:12:36.611 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:36.611 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 210, 2024-08-20T03:12:36.611 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.014545157, 2024-08-20T03:12:36.611 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000069262 2024-08-20T03:12:36.611 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:36.612 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:36.612 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:36.612 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:36.612 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:36.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:36.612 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1275328, 2024-08-20T03:12:36.612 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 128352 2024-08-20T03:12:36.612 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:36.612 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T03:12:36.613 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 1, 2024-08-20T03:12:36.613 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 158, 2024-08-20T03:12:36.613 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 160, 2024-08-20T03:12:36.613 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 107728, 2024-08-20T03:12:36.613 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 3, 2024-08-20T03:12:36.613 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 2, 2024-08-20T03:12:36.613 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.015284210, 2024-08-20T03:12:36.614 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.011981593, 2024-08-20T03:12:36.614 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.001234379, 2024-08-20T03:12:36.614 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T03:12:36.614 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:36.615 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 156, 2024-08-20T03:12:36.615 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.007010125, 2024-08-20T03:12:36.615 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000044936 2024-08-20T03:12:36.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:36.615 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:36.615 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:36.615 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:36.615 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:36.615 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:36.615 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T03:12:36.616 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 107728 2024-08-20T03:12:36.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:36.616 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-08-20T03:12:36.616 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2024-08-20T03:12:36.616 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-08-20T03:12:36.616 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:36.616 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-08-20T03:12:36.616 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-08-20T03:12:36.617 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-08-20T03:12:36.618 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 148967, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 29, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-08-20T03:12:36.619 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-08-20T03:12:36.620 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-08-20T03:12:36.621 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-08-20T03:12:36.621 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-08-20T03:12:36.621 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T03:12:36.621 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-08-20T03:12:36.621 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-08-20T03:12:36.621 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-08-20T03:12:36.621 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 217, 2024-08-20T03:12:36.621 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1241782, 2024-08-20T03:12:36.621 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:36.621 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 217, 2024-08-20T03:12:36.621 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-08-20T03:12:36.621 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-08-20T03:12:36.622 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 217, 2024-08-20T03:12:36.622 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1241782, 2024-08-20T03:12:36.622 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-08-20T03:12:36.622 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-08-20T03:12:36.622 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-08-20T03:12:36.622 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-08-20T03:12:36.622 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:36.622 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-08-20T03:12:36.622 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-08-20T03:12:36.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:36 smithi165 ceph-mon[22357]: pgmap v630: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 785 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 0 B/s wr, 12 op/s 2024-08-20T03:12:37.392 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-08-20T03:12:37.392 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-08-20T03:12:37.392 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-08-20T03:12:37.393 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-08-20T03:12:37.393 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-08-20T03:12:37.393 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-08-20T03:12:37.393 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-08-20T03:12:37.393 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-08-20T03:12:37.393 DEBUG:tasks.check_counter:Getting stats from b 2024-08-20T03:12:37.393 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph daemon mds.b perf dump 2024-08-20T03:12:37.653 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T03:12:38.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:38 smithi099 ceph-mon[21984]: pgmap v631: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-08-20T03:12:38.415 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:38 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:12:38.498 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:38 smithi044 ceph-mon[18790]: pgmap v631: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-08-20T03:12:38.498 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:38 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:12:38.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:38 smithi165 ceph-mon[22357]: pgmap v631: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-08-20T03:12:38.564 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:38 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T03:12:38.613 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-08-20T03:12:38.613 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T03:12:38.613 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 57055, 2024-08-20T03:12:38.613 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 61357, 2024-08-20T03:12:38.613 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 21793373, 2024-08-20T03:12:38.613 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 44712186, 2024-08-20T03:12:38.613 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 30, 2024-08-20T03:12:38.614 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2024-08-20T03:12:38.614 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 3.021861486, 2024-08-20T03:12:38.614 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 1.623970353, 2024-08-20T03:12:38.614 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 2.071595787, 2024-08-20T03:12:38.614 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.071201648, 2024-08-20T03:12:38.614 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:38.614 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 61355, 2024-08-20T03:12:38.614 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2.403561590, 2024-08-20T03:12:38.614 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000039174 2024-08-20T03:12:38.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.614 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:38.614 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 70660, 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.014621943, 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000206 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 4070, 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 4385, 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1228798, 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 2776591, 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 15, 2024-08-20T03:12:38.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 7, 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.300371654, 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.140603988, 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.112322091, 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.049104619, 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4383, 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.120654352, 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000027527 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3447, 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000516994, 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000149 2024-08-20T03:12:38.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.617 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-08-20T03:12:38.617 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 360464 2024-08-20T03:12:38.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.617 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T03:12:38.617 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 2676, 2024-08-20T03:12:38.617 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2884, 2024-08-20T03:12:38.617 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 2468596, 2024-08-20T03:12:38.617 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 5866655, 2024-08-20T03:12:38.617 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 10, 2024-08-20T03:12:38.617 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 7, 2024-08-20T03:12:38.617 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.238419168, 2024-08-20T03:12:38.618 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.110758157, 2024-08-20T03:12:38.618 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.092825497, 2024-08-20T03:12:38.618 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.041461025, 2024-08-20T03:12:38.618 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:38.618 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2884, 2024-08-20T03:12:38.618 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.128217310, 2024-08-20T03:12:38.618 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000044458 2024-08-20T03:12:38.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.618 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:38.618 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1293, 2024-08-20T03:12:38.618 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000266321, 2024-08-20T03:12:38.618 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000205 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1441936, 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 147424 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3883, 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 5.127744004, 2024-08-20T03:12:38.619 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001320562 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4952, 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.310413567, 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000062684 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 16504, 2024-08-20T03:12:38.620 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 14802, 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 14802, 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 95.015406569, 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.006419092 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 1702, 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 14, 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 54, 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 5, 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-08-20T03:12:38.621 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 19, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 4, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 5, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 10, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 13, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 5486, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 3, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 30323, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 26033, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 1701, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 389, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-08-20T03:12:38.622 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 335, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 48, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 4303, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 13, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 1024, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 3517, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 4, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 234, 2024-08-20T03:12:38.623 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 6, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 3517, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 3517, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 103, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 2622, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 5367, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T03:12:38.624 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 5186, 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 2437, 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 159, 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 9775 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 1036, 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 6019, 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 3535, 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 3881, 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 966, 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-08-20T03:12:38.625 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 2485, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 2485, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T03:12:38.626 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 48, 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 5, 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 12619, 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 12619, 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-08-20T03:12:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 17, 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 17, 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 3, 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 26689930, 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 445, 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 11.436590625, 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.025700203 2024-08-20T03:12:38.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 23, 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 5430, 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 5407, 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 22, 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 2711, 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 2689, 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 19, 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 8113, 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 8094, 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-08-20T03:12:38.629 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 3325, 2024-08-20T03:12:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 3325, 2024-08-20T03:12:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 75016, 2024-08-20T03:12:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 286117888 2024-08-20T03:12:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-08-20T03:12:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 17499, 2024-08-20T03:12:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 789, 2024-08-20T03:12:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 16504, 2024-08-20T03:12:38.630 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 25, 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 1737, 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2028, 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 4.118816504, 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002030974 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.002464956, 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001232478 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-08-20T03:12:38.631 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 7539, 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 81.527064186, 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.010814042 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.632 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.635 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1972, 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2.633493784, 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001335443 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 48, 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.052668499, 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001097260 2024-08-20T03:12:38.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-08-20T03:12:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2031, 2024-08-20T03:12:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 5.199342753, 2024-08-20T03:12:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002559991 2024-08-20T03:12:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-08-20T03:12:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-08-20T03:12:38.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.639 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.640 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.640 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-08-20T03:12:38.640 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.640 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.640 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.640 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-08-20T03:12:38.640 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.640 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.640 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1182, 2024-08-20T03:12:38.641 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.481555887, 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001253431 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2024-08-20T03:12:38.642 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 4, 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 2, 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 2, 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 1580, 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 790, 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 565, 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T03:12:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-08-20T03:12:38.644 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-08-20T03:12:38.645 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 277979, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 131, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 880, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 10, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-08-20T03:12:38.646 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 8096, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 180, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 136013, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 543, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-08-20T03:12:38.647 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 7080, 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 23841180, 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 28, 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 7080, 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 7080, 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 60.406586901, 2024-08-20T03:12:38.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.008532003 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 7080, 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 7403 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 7080, 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 3658, 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 3422, 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2024-08-20T03:12:38.649 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 10, 2024-08-20T03:12:38.650 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 111, 2024-08-20T03:12:38.650 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 557, 2024-08-20T03:12:38.650 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 91, 2024-08-20T03:12:38.650 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-08-20T03:12:38.650 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-08-20T03:12:38.650 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-08-20T03:12:38.650 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 5, 2024-08-20T03:12:38.650 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 2502, 2024-08-20T03:12:38.650 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 3517, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 20, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-08-20T03:12:38.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-08-20T03:12:38.652 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 84, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 9, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 12, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 12, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 462, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 44, 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 84 2024-08-20T03:12:38.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 6, 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 6, 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 2485, 2024-08-20T03:12:38.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 2485, 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 63797, 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 20262269, 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 63797, 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-08-20T03:12:38.655 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 63797, 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 20262269, 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-08-20T03:12:38.656 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 7080, 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 24129347, 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 7080, 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 24129347, 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-08-20T03:12:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 7080, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 7080, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 7080, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 7080, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 2485, 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 288945, 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 2485, 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 112, 2024-08-20T03:12:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 288945, 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x5647372c26e0": { 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 12619, 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 22495626, 2024-08-20T03:12:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 12619, 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 445, 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 22495626, 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "trackedop": { 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-08-20T03:12:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:38.662 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-08-20T03:12:39.489 INFO:tasks.check_counter:Daemon mds.b mds.exported=48 2024-08-20T03:12:39.489 INFO:tasks.check_counter:Daemon mds.b mds.imported=13 2024-08-20T03:12:39.489 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1036 2024-08-20T03:12:39.489 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=6019 2024-08-20T03:12:39.489 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-08-20T03:12:39.489 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-08-20T03:12:39.489 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T03:12:39.489 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=5 2024-08-20T03:12:39.489 DEBUG:tasks.check_counter:Getting stats from e 2024-08-20T03:12:39.490 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph daemon mds.e perf dump 2024-08-20T03:12:40.388 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T03:12:40.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:40 smithi165 ceph-mon[22357]: pgmap v632: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 0 B/s wr, 11 op/s 2024-08-20T03:12:40.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:40.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:40.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:40.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:40 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:40.662 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:40 smithi099 ceph-mon[21984]: pgmap v632: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 0 B/s wr, 11 op/s 2024-08-20T03:12:40.662 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:40.663 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:40.663 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:40.663 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:40 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:40.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:40 smithi044 ceph-mon[18790]: pgmap v632: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 0 B/s wr, 11 op/s 2024-08-20T03:12:40.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:40.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:40.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:40.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:40 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:40.881 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-08-20T03:12:40.881 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T03:12:40.881 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 2201, 2024-08-20T03:12:40.881 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2202, 2024-08-20T03:12:40.881 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 428590992, 2024-08-20T03:12:40.881 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 699610, 2024-08-20T03:12:40.881 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 8, 2024-08-20T03:12:40.882 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-08-20T03:12:40.882 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.393743488, 2024-08-20T03:12:40.882 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.085943505, 2024-08-20T03:12:40.882 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.245101382, 2024-08-20T03:12:40.882 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.047518376, 2024-08-20T03:12:40.882 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:40.882 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2200, 2024-08-20T03:12:40.882 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.079933861, 2024-08-20T03:12:40.882 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000036333 2024-08-20T03:12:40.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.882 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 961, 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1116, 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 386170900, 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 657562, 2024-08-20T03:12:40.883 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 7, 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 6, 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.263186166, 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.036953019, 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.199401449, 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.020072601, 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1115, 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.041673896, 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000037375 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 353488 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1047, 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1022, 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 253301425, 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 384241, 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 5, 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.230068086, 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.042351950, 2024-08-20T03:12:40.885 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.178117566, 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.017807824, 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1021, 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.032138119, 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000031477 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1408624, 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 129840 2024-08-20T03:12:40.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5701, 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.371720701, 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000240610 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-08-20T03:12:40.887 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.888 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.889 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-08-20T03:12:40.889 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-08-20T03:12:40.889 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-08-20T03:12:40.889 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-08-20T03:12:40.889 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 0, 2024-08-20T03:12:40.889 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-08-20T03:12:40.889 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-08-20T03:12:40.889 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 184872, 2024-08-20T03:12:40.889 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 124366, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 60506, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 7206, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 0, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 2, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 0, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 0, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-08-20T03:12:40.890 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 4, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 234, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 6, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T03:12:40.891 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T03:12:40.892 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-08-20T03:12:40.893 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-08-20T03:12:40.894 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 0, 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 326985, 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 0, 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-08-20T03:12:40.895 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 11, 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 43, 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 1052257721, 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 1069644886, 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 1069644774, 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 326985 2024-08-20T03:12:40.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 54801, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 115701, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 60900, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 1593, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 1983, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 390, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 184872, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 189034, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 4162, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 347064, 2024-08-20T03:12:40.897 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 558034944 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 0, 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.898 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-08-20T03:12:40.899 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.899 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.899 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.899 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-08-20T03:12:40.899 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.899 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.899 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.899 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-08-20T03:12:40.899 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.900 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-08-20T03:12:40.901 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.901 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.901 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.901 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-08-20T03:12:40.901 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.901 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.901 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.901 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-08-20T03:12:40.901 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.902 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.902 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.902 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-08-20T03:12:40.902 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.902 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.902 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.902 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-08-20T03:12:40.902 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.902 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.903 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.903 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-08-20T03:12:40.903 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.903 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.903 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.903 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-08-20T03:12:40.903 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.904 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.906 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.907 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-08-20T03:12:40.908 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.909 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.909 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.909 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-08-20T03:12:40.909 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.909 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.909 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.909 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-08-20T03:12:40.909 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.909 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 4, 2024-08-20T03:12:40.910 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 2, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 3, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 2200, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-08-20T03:12:40.911 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T03:12:40.912 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T03:12:40.913 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T03:12:40.913 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-08-20T03:12:40.913 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-08-20T03:12:40.913 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-08-20T03:12:40.913 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T03:12:40.913 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T03:12:40.913 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-08-20T03:12:40.913 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-08-20T03:12:40.913 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-08-20T03:12:40.913 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-08-20T03:12:40.913 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 2297902, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 55732, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 8096, 2024-08-20T03:12:40.914 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 180, 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 255703180, 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2660810, 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-08-20T03:12:40.915 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 3956, 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 26, 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 4, 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 3956, 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3956, 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 23.529449359, 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.005947788 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3956, 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 3957 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.916 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 3956, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 3956, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 0, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 2583, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 0, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 1370, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 0, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 0, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 0, 2024-08-20T03:12:40.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 2, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-08-20T03:12:40.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-08-20T03:12:40.919 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 84, 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 9, 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 11, 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 11, 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 0, 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 2, 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 0 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-08-20T03:12:40.920 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 0, 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.921 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T03:12:40.922 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:40.922 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-08-20T03:12:40.922 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:40.922 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 4205, 2024-08-20T03:12:40.922 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1067669246, 2024-08-20T03:12:40.922 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:40.922 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 4205, 2024-08-20T03:12:40.922 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-08-20T03:12:40.922 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-08-20T03:12:40.922 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 4205, 2024-08-20T03:12:40.922 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1067669246, 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:40.923 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 3956, 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 1065579829, 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3956, 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1065579829, 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-08-20T03:12:40.924 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:40.925 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-08-20T03:12:40.925 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-08-20T03:12:40.925 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:40.925 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:40.925 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 3956, 2024-08-20T03:12:40.925 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 3956, 2024-08-20T03:12:40.925 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3956, 2024-08-20T03:12:40.925 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 3956, 2024-08-20T03:12:40.925 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:40.925 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.925 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-08-20T03:12:40.926 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x56041b464960": { 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-08-20T03:12:40.927 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:40.928 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:40.928 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-08-20T03:12:40.928 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-08-20T03:12:40.928 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-08-20T03:12:40.928 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-08-20T03:12:40.928 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:40.928 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:40.928 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:40.928 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:40.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:40.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:40.929 INFO:teuthology.orchestra.run.smithi099.stdout: "trackedop": { 2024-08-20T03:12:40.929 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-08-20T03:12:40.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:40.929 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-08-20T03:12:42.204 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-08-20T03:12:42.204 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-08-20T03:12:42.204 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-08-20T03:12:42.204 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-08-20T03:12:42.204 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-08-20T03:12:42.204 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-08-20T03:12:42.204 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T03:12:42.204 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-08-20T03:12:42.204 DEBUG:tasks.check_counter:Getting stats from h 2024-08-20T03:12:42.205 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph daemon mds.h perf dump 2024-08-20T03:12:42.431 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:42 smithi099 ceph-mon[21984]: pgmap v633: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:12:42.431 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:42 smithi099 ceph-mon[21984]: osdmap e85: 12 total, 12 up, 12 in 2024-08-20T03:12:42.431 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:42 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:42.431 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:42 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:42.559 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T03:12:42.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:42 smithi165 ceph-mon[22357]: pgmap v633: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:12:42.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:42 smithi165 ceph-mon[22357]: osdmap e85: 12 total, 12 up, 12 in 2024-08-20T03:12:42.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:42 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:42.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:42 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:42.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:42 smithi044 ceph-mon[18790]: pgmap v633: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-08-20T03:12:42.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:42 smithi044 ceph-mon[18790]: osdmap e85: 12 total, 12 up, 12 in 2024-08-20T03:12:42.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:42 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:42.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:42 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:43.169 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-08-20T03:12:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T03:12:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 18171, 2024-08-20T03:12:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 10546, 2024-08-20T03:12:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 10337481, 2024-08-20T03:12:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 6593909, 2024-08-20T03:12:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 23, 2024-08-20T03:12:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2024-08-20T03:12:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 1.011660550, 2024-08-20T03:12:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.436557511, 2024-08-20T03:12:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.754464350, 2024-08-20T03:12:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.023612975, 2024-08-20T03:12:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 10546, 2024-08-20T03:12:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.276884468, 2024-08-20T03:12:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000026254 2024-08-20T03:12:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 26290, 2024-08-20T03:12:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.004036944, 2024-08-20T03:12:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000153 2024-08-20T03:12:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1426352, 2024-08-20T03:12:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 140768 2024-08-20T03:12:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T03:12:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1229, 2024-08-20T03:12:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1188, 2024-08-20T03:12:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 453165, 2024-08-20T03:12:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 856353, 2024-08-20T03:12:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 19, 2024-08-20T03:12:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 7, 2024-08-20T03:12:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.101112480, 2024-08-20T03:12:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.049262152, 2024-08-20T03:12:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.043723037, 2024-08-20T03:12:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.009654603, 2024-08-20T03:12:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1186, 2024-08-20T03:12:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.063668702, 2024-08-20T03:12:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000053683 2024-08-20T03:12:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1336, 2024-08-20T03:12:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000808361, 2024-08-20T03:12:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000605 2024-08-20T03:12:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 5808, 2024-08-20T03:12:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 369632 2024-08-20T03:12:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T03:12:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 13734, 2024-08-20T03:12:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 14889, 2024-08-20T03:12:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 4154584, 2024-08-20T03:12:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 9784956, 2024-08-20T03:12:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 13, 2024-08-20T03:12:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 7, 2024-08-20T03:12:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.594330369, 2024-08-20T03:12:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.341075321, 2024-08-20T03:12:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.346288592, 2024-08-20T03:12:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.011498483, 2024-08-20T03:12:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 14887, 2024-08-20T03:12:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.454166388, 2024-08-20T03:12:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000030507 2024-08-20T03:12:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 12837, 2024-08-20T03:12:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.003692132, 2024-08-20T03:12:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000287 2024-08-20T03:12:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T03:12:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T03:12:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-08-20T03:12:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-08-20T03:12:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-08-20T03:12:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-08-20T03:12:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-08-20T03:12:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-08-20T03:12:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1096, 2024-08-20T03:12:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.780990941, 2024-08-20T03:12:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000712582 2024-08-20T03:12:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-08-20T03:12:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-08-20T03:12:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-08-20T03:12:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 779, 2024-08-20T03:12:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.077396541, 2024-08-20T03:12:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000099353 2024-08-20T03:12:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-08-20T03:12:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 8945, 2024-08-20T03:12:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 8915, 2024-08-20T03:12:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-08-20T03:12:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 8915, 2024-08-20T03:12:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 117.316529164, 2024-08-20T03:12:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.013159453 2024-08-20T03:12:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-08-20T03:12:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 30, 2024-08-20T03:12:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 3, 2024-08-20T03:12:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-08-20T03:12:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 37, 2024-08-20T03:12:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 3, 2024-08-20T03:12:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-08-20T03:12:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 24, 2024-08-20T03:12:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 7, 2024-08-20T03:12:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 7, 2024-08-20T03:12:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 10, 2024-08-20T03:12:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 13, 2024-08-20T03:12:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 1567, 2024-08-20T03:12:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 2, 2024-08-20T03:12:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 2, 2024-08-20T03:12:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 3, 2024-08-20T03:12:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 20969, 2024-08-20T03:12:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 16321, 2024-08-20T03:12:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 27, 2024-08-20T03:12:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 316, 2024-08-20T03:12:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-08-20T03:12:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-08-20T03:12:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 67, 2024-08-20T03:12:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-08-20T03:12:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-08-20T03:12:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 24, 2024-08-20T03:12:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 822, 2024-08-20T03:12:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 3, 2024-08-20T03:12:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 423, 2024-08-20T03:12:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-08-20T03:12:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 945, 2024-08-20T03:12:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-08-20T03:12:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 4, 2024-08-20T03:12:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 234, 2024-08-20T03:12:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 6, 2024-08-20T03:12:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 945, 2024-08-20T03:12:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-08-20T03:12:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T03:12:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-08-20T03:12:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-08-20T03:12:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T03:12:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T03:12:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 945, 2024-08-20T03:12:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 8, 2024-08-20T03:12:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 273, 2024-08-20T03:12:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 518, 2024-08-20T03:12:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T03:12:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T03:12:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T03:12:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 646, 2024-08-20T03:12:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 297, 2024-08-20T03:12:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 26, 2024-08-20T03:12:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 1147 2024-08-20T03:12:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-08-20T03:12:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 162, 2024-08-20T03:12:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 6090, 2024-08-20T03:12:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 3598, 2024-08-20T03:12:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 4211, 2024-08-20T03:12:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 129, 2024-08-20T03:12:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-08-20T03:12:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-08-20T03:12:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-08-20T03:12:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 344, 2024-08-20T03:12:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 344, 2024-08-20T03:12:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-08-20T03:12:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-08-20T03:12:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-08-20T03:12:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-08-20T03:12:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-08-20T03:12:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-08-20T03:12:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-08-20T03:12:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-08-20T03:12:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-08-20T03:12:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T03:12:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 24, 2024-08-20T03:12:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-08-20T03:12:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 3, 2024-08-20T03:12:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-08-20T03:12:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-08-20T03:12:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-08-20T03:12:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-08-20T03:12:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 1965, 2024-08-20T03:12:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-08-20T03:12:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-08-20T03:12:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 1965, 2024-08-20T03:12:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-08-20T03:12:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-08-20T03:12:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 3, 2024-08-20T03:12:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-08-20T03:12:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-08-20T03:12:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 3, 2024-08-20T03:12:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-08-20T03:12:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-08-20T03:12:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-08-20T03:12:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-08-20T03:12:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 7570765, 2024-08-20T03:12:43.187 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2024-08-20T03:12:43.187 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-08-20T03:12:43.187 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 132, 2024-08-20T03:12:43.187 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 3.649513615, 2024-08-20T03:12:43.187 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.027647830 2024-08-20T03:12:43.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.187 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2024-08-20T03:12:43.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.187 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-08-20T03:12:43.187 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 28, 2024-08-20T03:12:43.187 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 1164, 2024-08-20T03:12:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 1136, 2024-08-20T03:12:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 27, 2024-08-20T03:12:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 2651, 2024-08-20T03:12:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 2624, 2024-08-20T03:12:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 24, 2024-08-20T03:12:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 2095, 2024-08-20T03:12:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 2071, 2024-08-20T03:12:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 2, 2024-08-20T03:12:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 669, 2024-08-20T03:12:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 667, 2024-08-20T03:12:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 64628, 2024-08-20T03:12:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 274587648 2024-08-20T03:12:43.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.189 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-08-20T03:12:43.189 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 9093, 2024-08-20T03:12:43.189 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 30, 2024-08-20T03:12:43.189 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 8945, 2024-08-20T03:12:43.189 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 25, 2024-08-20T03:12:43.189 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 107, 2024-08-20T03:12:43.189 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-08-20T03:12:43.189 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 153, 2024-08-20T03:12:43.189 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.305450303, 2024-08-20T03:12:43.189 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.008532354 2024-08-20T03:12:43.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.190 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-08-20T03:12:43.190 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4, 2024-08-20T03:12:43.190 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000802409, 2024-08-20T03:12:43.190 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000200602 2024-08-20T03:12:43.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.190 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-08-20T03:12:43.190 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.190 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.190 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 8107, 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 115.278403229, 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.014219613 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.191 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-08-20T03:12:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1, 2024-08-20T03:12:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000136394, 2024-08-20T03:12:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000136394 2024-08-20T03:12:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-08-20T03:12:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.193 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-08-20T03:12:43.193 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.193 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.193 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.193 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-08-20T03:12:43.193 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.193 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.193 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.193 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-08-20T03:12:43.193 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.194 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.194 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.194 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-08-20T03:12:43.194 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.195 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.195 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.195 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-08-20T03:12:43.195 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.195 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.195 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.195 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-08-20T03:12:43.195 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.195 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.195 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.196 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-08-20T03:12:43.196 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 154, 2024-08-20T03:12:43.196 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.111153197, 2024-08-20T03:12:43.196 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000721774 2024-08-20T03:12:43.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.196 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-08-20T03:12:43.196 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 24, 2024-08-20T03:12:43.196 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.014598611, 2024-08-20T03:12:43.196 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000608275 2024-08-20T03:12:43.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.197 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-08-20T03:12:43.197 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 154, 2024-08-20T03:12:43.197 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.204448379, 2024-08-20T03:12:43.197 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001327586 2024-08-20T03:12:43.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.197 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-08-20T03:12:43.197 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.197 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.197 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.197 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-08-20T03:12:43.197 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.198 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.198 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.198 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-08-20T03:12:43.198 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.198 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.198 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.198 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-08-20T03:12:43.198 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.199 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.199 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.199 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-08-20T03:12:43.199 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.199 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.199 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.199 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-08-20T03:12:43.199 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.199 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.199 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.200 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-08-20T03:12:43.200 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.200 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.200 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.200 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-08-20T03:12:43.200 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.200 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.200 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.200 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-08-20T03:12:43.201 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.201 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.201 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.201 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-08-20T03:12:43.201 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.201 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.201 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.201 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-08-20T03:12:43.201 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.201 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.202 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.202 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-08-20T03:12:43.202 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 318, 2024-08-20T03:12:43.202 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.401536642, 2024-08-20T03:12:43.202 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001262693 2024-08-20T03:12:43.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.202 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-08-20T03:12:43.202 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-08-20T03:12:43.202 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-08-20T03:12:43.202 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-08-20T03:12:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2024-08-20T03:12:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 4, 2024-08-20T03:12:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 2, 2024-08-20T03:12:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 2, 2024-08-20T03:12:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-08-20T03:12:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 811, 2024-08-20T03:12:43.203 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 405, 2024-08-20T03:12:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 600, 2024-08-20T03:12:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-08-20T03:12:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-08-20T03:12:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-08-20T03:12:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-08-20T03:12:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T03:12:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-08-20T03:12:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T03:12:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-08-20T03:12:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T03:12:43.204 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T03:12:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T03:12:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T03:12:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T03:12:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-08-20T03:12:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T03:12:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T03:12:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-08-20T03:12:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-08-20T03:12:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-08-20T03:12:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-08-20T03:12:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T03:12:43.205 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T03:12:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T03:12:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T03:12:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T03:12:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-08-20T03:12:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-08-20T03:12:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-08-20T03:12:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T03:12:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T03:12:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-08-20T03:12:43.206 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-08-20T03:12:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-08-20T03:12:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-08-20T03:12:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T03:12:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-08-20T03:12:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T03:12:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-08-20T03:12:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 306600, 2024-08-20T03:12:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 124, 2024-08-20T03:12:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 440, 2024-08-20T03:12:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 5, 2024-08-20T03:12:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-08-20T03:12:43.207 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-08-20T03:12:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-08-20T03:12:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-08-20T03:12:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-08-20T03:12:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-08-20T03:12:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 8096, 2024-08-20T03:12:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 180, 2024-08-20T03:12:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T03:12:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-08-20T03:12:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-08-20T03:12:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-08-20T03:12:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 126621, 2024-08-20T03:12:43.208 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 470, 2024-08-20T03:12:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-08-20T03:12:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-08-20T03:12:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-08-20T03:12:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-08-20T03:12:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-08-20T03:12:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-08-20T03:12:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-08-20T03:12:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 1673, 2024-08-20T03:12:43.209 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 3852627, 2024-08-20T03:12:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 32, 2024-08-20T03:12:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 1673, 2024-08-20T03:12:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-08-20T03:12:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1673, 2024-08-20T03:12:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 8.780794892, 2024-08-20T03:12:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.005248532 2024-08-20T03:12:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-08-20T03:12:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-08-20T03:12:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1673, 2024-08-20T03:12:43.210 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1808 2024-08-20T03:12:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 1673, 2024-08-20T03:12:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 983, 2024-08-20T03:12:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 690, 2024-08-20T03:12:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-08-20T03:12:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-08-20T03:12:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2024-08-20T03:12:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 10, 2024-08-20T03:12:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 32, 2024-08-20T03:12:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 164, 2024-08-20T03:12:43.211 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 49, 2024-08-20T03:12:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-08-20T03:12:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-08-20T03:12:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-08-20T03:12:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 3, 2024-08-20T03:12:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 356, 2024-08-20T03:12:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-08-20T03:12:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-08-20T03:12:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-08-20T03:12:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 945, 2024-08-20T03:12:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 20, 2024-08-20T03:12:43.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-08-20T03:12:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-08-20T03:12:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-08-20T03:12:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-08-20T03:12:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-08-20T03:12:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-08-20T03:12:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T03:12:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-08-20T03:12:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-08-20T03:12:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-08-20T03:12:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-08-20T03:12:43.213 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-08-20T03:12:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-08-20T03:12:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-08-20T03:12:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-08-20T03:12:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-08-20T03:12:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-08-20T03:12:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-08-20T03:12:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-08-20T03:12:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-08-20T03:12:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-08-20T03:12:43.214 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-08-20T03:12:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-08-20T03:12:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-08-20T03:12:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-08-20T03:12:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-08-20T03:12:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 85, 2024-08-20T03:12:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-08-20T03:12:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 10, 2024-08-20T03:12:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 12, 2024-08-20T03:12:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 12, 2024-08-20T03:12:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-08-20T03:12:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-08-20T03:12:43.215 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 184, 2024-08-20T03:12:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 9, 2024-08-20T03:12:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 36 2024-08-20T03:12:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-08-20T03:12:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2024-08-20T03:12:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-08-20T03:12:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-08-20T03:12:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-08-20T03:12:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-08-20T03:12:43.216 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 6, 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 6, 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 344, 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 344, 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 33130, 2024-08-20T03:12:43.217 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 12149014, 2024-08-20T03:12:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 33130, 2024-08-20T03:12:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-08-20T03:12:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-08-20T03:12:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 33130, 2024-08-20T03:12:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 12149014, 2024-08-20T03:12:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:43.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-08-20T03:12:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-08-20T03:12:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-08-20T03:12:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-08-20T03:12:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 1673, 2024-08-20T03:12:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 3896076, 2024-08-20T03:12:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 1673, 2024-08-20T03:12:43.219 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 3896076, 2024-08-20T03:12:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-08-20T03:12:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-08-20T03:12:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-08-20T03:12:43.220 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-08-20T03:12:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 1673, 2024-08-20T03:12:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 1673, 2024-08-20T03:12:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 1673, 2024-08-20T03:12:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1673, 2024-08-20T03:12:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-08-20T03:12:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-08-20T03:12:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 344, 2024-08-20T03:12:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 43592, 2024-08-20T03:12:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 344, 2024-08-20T03:12:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-08-20T03:12:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-08-20T03:12:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 32, 2024-08-20T03:12:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 43592, 2024-08-20T03:12:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x56522c5d46e0": { 2024-08-20T03:12:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-08-20T03:12:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 1965, 2024-08-20T03:12:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 3376461, 2024-08-20T03:12:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 1965, 2024-08-20T03:12:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-08-20T03:12:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-08-20T03:12:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 132, 2024-08-20T03:12:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 3376461, 2024-08-20T03:12:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "trackedop": { 2024-08-20T03:12:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-08-20T03:12:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:43.225 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-08-20T03:12:43.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:12:43.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:12:43.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:43 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:43.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:12:43.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:12:43.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:43 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:43.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T03:12:43.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T03:12:43.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:43 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' 2024-08-20T03:12:44.039 INFO:tasks.check_counter:Daemon mds.h mds.exported=24 2024-08-20T03:12:44.040 INFO:tasks.check_counter:Daemon mds.h mds.imported=3 2024-08-20T03:12:44.040 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=162 2024-08-20T03:12:44.040 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=6090 2024-08-20T03:12:44.040 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-08-20T03:12:44.040 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-08-20T03:12:44.040 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T03:12:44.040 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=3 2024-08-20T03:12:44.040 DEBUG:tasks.check_counter:Getting stats from k 2024-08-20T03:12:44.040 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph daemon mds.k perf dump 2024-08-20T03:12:44.289 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T03:12:44.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:44 smithi165 ceph-mon[22357]: pgmap v635: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:44.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:44 smithi099 ceph-mon[21984]: pgmap v635: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:44.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:44 smithi044 ceph-mon[18790]: pgmap v635: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:44.795 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-08-20T03:12:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 488, 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 490, 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 4321439, 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 155248, 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 2, 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.047688450, 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.022493270, 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.011676895, 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.009393855, 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 488, 2024-08-20T03:12:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.018948357, 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000038828 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 649, 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 808, 2024-08-20T03:12:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 4356022, 2024-08-20T03:12:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 559461, 2024-08-20T03:12:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 5, 2024-08-20T03:12:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-08-20T03:12:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.073902913, 2024-08-20T03:12:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.033659433, 2024-08-20T03:12:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.016379247, 2024-08-20T03:12:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.017486112, 2024-08-20T03:12:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 806, 2024-08-20T03:12:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.036239368, 2024-08-20T03:12:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000044961 2024-08-20T03:12:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T03:12:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 353392 2024-08-20T03:12:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1286, 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1259, 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 3852296, 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 460564, 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.158150329, 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.068099608, 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.062682193, 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.029861986, 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1259, 2024-08-20T03:12:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.064084976, 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000050901 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1413392, 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 133024 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-08-20T03:12:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3688, 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.092139936, 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000024983 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-08-20T03:12:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-08-20T03:12:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 0, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 1081, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 0, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 1081, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 1081, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 0, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-08-20T03:12:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 1, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-08-20T03:12:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T03:12:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-08-20T03:12:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-08-20T03:12:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 0, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 6048, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 0, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 9, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-08-20T03:12:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 14701272, 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 14696819, 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 6048 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 10, 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 2745, 2024-08-20T03:12:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 2735, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 8, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 70, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 62, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 1081, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 4220, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 3139, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 58324, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 260186112 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-08-20T03:12:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 0, 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-08-20T03:12:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-08-20T03:12:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-08-20T03:12:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-08-20T03:12:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-08-20T03:12:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 3, 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 1, 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 3, 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2024-08-20T03:12:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 2206, 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T03:12:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T03:12:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 198287, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 57, 2024-08-20T03:12:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 7976, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 177, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-08-20T03:12:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 783064, 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 4446, 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 2166, 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 21, 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 3, 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 2166, 2024-08-20T03:12:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2166, 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 25.181410631, 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.011625766 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2166, 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2167 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 2166, 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 2166, 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 0, 2024-08-20T03:12:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 1317, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 0, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 847, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 0, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 0, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 0, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-08-20T03:12:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 1, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-08-20T03:12:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-08-20T03:12:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 85, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 10, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 6, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 6, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 0, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 2, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 0 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 0, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-08-20T03:12:44.834 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:44.835 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 2419, 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 12296170, 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 2419, 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2419, 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 12296170, 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.836 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 2166, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 10506973, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2166, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 10506973, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.837 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 2166, 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 2166, 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 2166, 2024-08-20T03:12:44.838 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 2166, 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-08-20T03:12:44.839 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-08-20T03:12:44.840 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-08-20T03:12:44.840 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-08-20T03:12:44.840 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:44.840 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.840 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.840 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:44.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.840 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x56521452e960": { 2024-08-20T03:12:44.840 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-08-20T03:12:44.840 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:44.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-08-20T03:12:44.842 INFO:teuthology.orchestra.run.smithi099.stdout: "trackedop": { 2024-08-20T03:12:44.842 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-08-20T03:12:44.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-08-20T03:12:44.842 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-08-20T03:12:45.618 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-08-20T03:12:45.618 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-08-20T03:12:45.619 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-08-20T03:12:45.619 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-08-20T03:12:45.619 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-08-20T03:12:45.619 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-08-20T03:12:45.619 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T03:12:45.619 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-08-20T03:12:45.619 DEBUG:tasks.check_counter:Getting stats from c 2024-08-20T03:12:45.619 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph daemon mds.c perf dump 2024-08-20T03:12:45.939 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T03:12:46.487 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-08-20T03:12:46.487 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T03:12:46.487 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2485, 2024-08-20T03:12:46.487 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 2267, 2024-08-20T03:12:46.487 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2139937, 2024-08-20T03:12:46.487 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 5358831, 2024-08-20T03:12:46.487 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 33, 2024-08-20T03:12:46.487 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-08-20T03:12:46.487 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.229148761, 2024-08-20T03:12:46.487 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.104973697, 2024-08-20T03:12:46.487 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.122651418, 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.016932115, 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2267, 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.124293163, 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000054827 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2834, 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000510459, 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000180 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1439488, 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 144400 2024-08-20T03:12:46.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1715, 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1992, 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 477691, 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 4536659, 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 9, 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 7, 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.171337994, 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.079535762, 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.040858316, 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.038599709, 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1990, 2024-08-20T03:12:46.489 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.107108675, 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000053823 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 726, 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000148919, 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000205 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 363296 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 50991, 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 48968, 2024-08-20T03:12:46.490 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 19162833, 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 26280307, 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 13, 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 7, 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 2.831624259, 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 1.563134767, 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 1.724854297, 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.058917078, 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 48966, 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2.368620843, 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000048372 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.491 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 61388, 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.014219334, 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000231 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-08-20T03:12:46.492 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2340, 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2.801114877, 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001197057 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2199, 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.212074340, 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000096441 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.493 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 17337, 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 15436, 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 15436, 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 294.480773726, 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.019077531 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 1901, 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 46, 2024-08-20T03:12:46.494 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 4, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 24, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 6, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 8, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 10, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 13, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 3512, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 3, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 34391, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 28288, 2024-08-20T03:12:46.495 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 1901, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 386, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 178, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 32, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 2702, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 4, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 709, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 2181, 2024-08-20T03:12:46.496 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 6, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 2181, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 2181, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 100, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 1631, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 2560, 2024-08-20T03:12:46.497 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 2788, 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 1124, 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 67, 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 4048 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 560, 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 5956, 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 3444, 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 4016, 2024-08-20T03:12:46.498 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 377, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 1074, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 1074, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-08-20T03:12:46.499 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 32, 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 4, 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 6048, 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 6048, 2024-08-20T03:12:46.500 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 9, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 9, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 14701272, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 257, 2024-08-20T03:12:46.501 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 5.223820513, 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.020326149 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 28, 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 3108, 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 3080, 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 27, 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 2785, 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 2758, 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 24, 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 4889, 2024-08-20T03:12:46.502 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 4865, 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 1815, 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 1815, 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 73060, 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 280879104 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 17842, 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 730, 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 17337, 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 21, 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 1542, 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-08-20T03:12:46.503 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 735, 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 3.814930401, 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.005190381 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.504 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 12436, 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 286.568977611, 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.023043500 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.505 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-08-20T03:12:46.506 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.506 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.506 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.506 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-08-20T03:12:46.506 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.506 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.506 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.506 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-08-20T03:12:46.506 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.506 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-08-20T03:12:46.507 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 760, 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.150460306, 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001513763 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 32, 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.052411426, 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001637857 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 737, 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.778359090, 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002412970 2024-08-20T03:12:46.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.510 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-08-20T03:12:46.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.512 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.512 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.512 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.512 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-08-20T03:12:46.513 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.514 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 736, 2024-08-20T03:12:46.515 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.115634892, 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001515808 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 3, 2024-08-20T03:12:46.516 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 1, 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 1981, 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 990, 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 571, 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T03:12:46.517 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T03:12:46.518 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-08-20T03:12:46.519 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 307799, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 127, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 528, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 6, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7976, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 177, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T03:12:46.520 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 154221, 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 548, 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-08-20T03:12:46.521 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 3969, 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 11352524, 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 32, 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 3969, 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3969, 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 26.865062183, 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.006768723 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3969, 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 4213 2024-08-20T03:12:46.522 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 3969, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 2256, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 1713, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 10, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 75, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 332, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 77, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 4, 2024-08-20T03:12:46.523 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 1088, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 2181, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 20, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-08-20T03:12:46.524 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-08-20T03:12:46.525 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 85, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 10, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 12, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 12, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 354, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 72 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-08-20T03:12:46.526 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 6, 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 6, 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 1074, 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 1074, 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T03:12:46.527 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 55187, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 17255388, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 55187, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 55187, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 17255388, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:46.528 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 3969, 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 11485686, 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 3969, 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 11485686, 2024-08-20T03:12:46.529 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 3969, 2024-08-20T03:12:46.530 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 3969, 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 3969, 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 3969, 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:46.531 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 1074, 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 133162, 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 1074, 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 75, 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 133162, 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x556f797366e0": { 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:46.532 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 6048, 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 10506968, 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 6048, 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 257, 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 10506968, 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:46.533 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:46.534 INFO:teuthology.orchestra.run.smithi165.stdout: "trackedop": { 2024-08-20T03:12:46.534 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-08-20T03:12:46.534 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:46.534 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-08-20T03:12:46.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:46 smithi165 ceph-mon[22357]: pgmap v636: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:46.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:46 smithi099 ceph-mon[21984]: pgmap v636: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:46.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:46 smithi044 ceph-mon[18790]: pgmap v636: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:47.362 INFO:tasks.check_counter:Daemon mds.c mds.exported=32 2024-08-20T03:12:47.363 INFO:tasks.check_counter:Daemon mds.c mds.imported=4 2024-08-20T03:12:47.363 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=560 2024-08-20T03:12:47.363 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=5956 2024-08-20T03:12:47.363 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-08-20T03:12:47.363 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-08-20T03:12:47.363 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T03:12:47.363 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=4 2024-08-20T03:12:47.363 DEBUG:tasks.check_counter:Getting stats from f 2024-08-20T03:12:47.363 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph daemon mds.f perf dump 2024-08-20T03:12:47.624 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T03:12:48.240 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-08-20T03:12:48.240 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T03:12:48.240 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 47785, 2024-08-20T03:12:48.240 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 54773, 2024-08-20T03:12:48.240 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 15049313, 2024-08-20T03:12:48.240 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 37845685, 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 33, 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 2.585709555, 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 1.421814538, 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 1.448091767, 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.284128206, 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 54771, 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 3.710699323, 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000067749 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:48.241 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 45651, 2024-08-20T03:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.013819116, 2024-08-20T03:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000302 2024-08-20T03:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T03:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T03:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T03:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 3287, 2024-08-20T03:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 3398, 2024-08-20T03:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 2376845, 2024-08-20T03:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 6000716, 2024-08-20T03:12:48.242 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 11, 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 7, 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.509046800, 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.120098663, 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.117360157, 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.281778387, 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3397, 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.144708369, 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000042598 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2739, 2024-08-20T03:12:48.243 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000458933, 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000167 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1431840, 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 146416 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 22464, 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 15903, 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 10571107, 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 9191468, 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 11, 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 7, 2024-08-20T03:12:48.244 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 2.053447537, 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.715165816, 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.862407387, 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.669385753, 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 15902, 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.578991751, 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000036409 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 32079, 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.005299477, 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000165 2024-08-20T03:12:48.245 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 4768, 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 363872 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3782, 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 4.377819342, 2024-08-20T03:12:48.246 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001157540 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3364, 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.285947807, 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000085002 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 28260, 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 26533, 2024-08-20T03:12:48.247 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 26533, 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 665.194158873, 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.025070446 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 1727, 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 22, 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 64, 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 6, 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 23, 2024-08-20T03:12:48.248 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 7, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 6, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 10, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 13, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 5199, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 3, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 55698, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 43398, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 1727, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 390, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-08-20T03:12:48.249 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 335, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 48, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 3987, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 6, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 1401, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 3770, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 6, 2024-08-20T03:12:48.250 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 3770, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 3770, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 108, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 1757, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 3351, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T03:12:48.251 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 3767, 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 1721, 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 127, 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 6038 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 842, 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 6049, 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 3449, 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 3940, 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 590, 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-08-20T03:12:48.252 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-08-20T03:12:48.253 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 1650, 2024-08-20T03:12:48.253 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 1650, 2024-08-20T03:12:48.253 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-08-20T03:12:48.253 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-08-20T03:12:48.253 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-08-20T03:12:48.253 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-08-20T03:12:48.253 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-08-20T03:12:48.253 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-08-20T03:12:48.253 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-08-20T03:12:48.254 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-08-20T03:12:48.254 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-08-20T03:12:48.254 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T03:12:48.254 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 48, 2024-08-20T03:12:48.254 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-08-20T03:12:48.254 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 6, 2024-08-20T03:12:48.254 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-08-20T03:12:48.254 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-08-20T03:12:48.254 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 8613, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 8613, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 11, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 11, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 2, 2024-08-20T03:12:48.255 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 19062816, 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 378, 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 7.255356255, 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.019194064 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 27, 2024-08-20T03:12:48.256 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 4216, 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 4189, 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 26, 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 2898, 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 2872, 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 23, 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 7027, 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 7004, 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 2427, 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 2427, 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 75536, 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 286126080 2024-08-20T03:12:48.257 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 29151, 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 606, 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 28260, 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 21, 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 1318, 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1031, 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 6.874062291, 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.006667373 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-08-20T03:12:48.258 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.259 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 22147, 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 650.904124109, 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.029390171 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.260 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-08-20T03:12:48.261 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.262 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.262 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.262 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-08-20T03:12:48.262 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.262 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.262 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.262 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.262 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-08-20T03:12:48.262 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-08-20T03:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-08-20T03:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1016, 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.424443211, 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001402011 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 48, 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.179825171, 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.003746357 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.264 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-08-20T03:12:48.265 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1032, 2024-08-20T03:12:48.265 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 4.093424785, 2024-08-20T03:12:48.265 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.003966496 2024-08-20T03:12:48.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.265 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-08-20T03:12:48.265 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.265 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.265 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.265 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.265 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-08-20T03:12:48.265 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.265 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.266 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-08-20T03:12:48.267 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.267 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.267 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.267 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-08-20T03:12:48.267 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.267 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.267 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.267 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.268 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-08-20T03:12:48.269 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1259, 2024-08-20T03:12:48.270 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.718279306, 2024-08-20T03:12:48.270 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001364796 2024-08-20T03:12:48.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.270 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-08-20T03:12:48.270 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-08-20T03:12:48.270 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-08-20T03:12:48.270 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.270 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.270 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.270 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:48.270 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.270 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 3, 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 1, 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 2587, 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 1293, 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 609, 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T03:12:48.271 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-08-20T03:12:48.272 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-08-20T03:12:48.273 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 301771, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 136, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 880, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 10, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-08-20T03:12:48.274 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7976, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 177, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 144621, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 550, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-08-20T03:12:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 6419, 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 16239075, 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 33, 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 6419, 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6419, 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 56.764130759, 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.008843142 2024-08-20T03:12:48.276 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6419, 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 6739 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 6419, 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 3923, 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 2496, 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 10, 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 109, 2024-08-20T03:12:48.277 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 489, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 87, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 6, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 1665, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 3770, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 20, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-08-20T03:12:48.278 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-08-20T03:12:48.279 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 85, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-08-20T03:12:48.280 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 10, 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 12, 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 12, 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 436, 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 66, 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 81 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-08-20T03:12:48.281 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 6, 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 6, 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 1650, 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 1650, 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:48.282 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-08-20T03:12:48.283 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:48.283 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 73532, 2024-08-20T03:12:48.283 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 22080202, 2024-08-20T03:12:48.283 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:48.283 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 73532, 2024-08-20T03:12:48.283 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-08-20T03:12:48.283 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-08-20T03:12:48.283 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 73532, 2024-08-20T03:12:48.283 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 22080202, 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-08-20T03:12:48.284 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 6419, 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 16445275, 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 6419, 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 16445275, 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-08-20T03:12:48.285 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 6419, 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 6419, 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 6419, 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 6419, 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.286 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 1650, 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 207322, 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 1650, 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-08-20T03:12:48.287 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 111, 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 207322, 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55d498d006e0": { 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:48.288 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 8613, 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 14868512, 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 8613, 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 378, 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 14868512, 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:48.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:48.290 INFO:teuthology.orchestra.run.smithi165.stdout: "trackedop": { 2024-08-20T03:12:48.290 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-08-20T03:12:48.290 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:48.290 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-08-20T03:12:48.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:48 smithi165 ceph-mon[22357]: pgmap v637: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:48.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:48 smithi099 ceph-mon[21984]: pgmap v637: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:48.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:48 smithi044 ceph-mon[18790]: pgmap v637: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:49.109 INFO:tasks.check_counter:Daemon mds.f mds.exported=48 2024-08-20T03:12:49.109 INFO:tasks.check_counter:Daemon mds.f mds.imported=6 2024-08-20T03:12:49.109 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=842 2024-08-20T03:12:49.109 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=6049 2024-08-20T03:12:49.109 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-08-20T03:12:49.109 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-08-20T03:12:49.110 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T03:12:49.110 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=6 2024-08-20T03:12:49.110 DEBUG:tasks.check_counter:Getting stats from i 2024-08-20T03:12:49.110 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph daemon mds.i perf dump 2024-08-20T03:12:49.365 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T03:12:49.968 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-08-20T03:12:49.969 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T03:12:49.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 468489, 2024-08-20T03:12:49.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 726836, 2024-08-20T03:12:49.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 162404203, 2024-08-20T03:12:49.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 816885140, 2024-08-20T03:12:49.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 26, 2024-08-20T03:12:49.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 8, 2024-08-20T03:12:49.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 33.223123646, 2024-08-20T03:12:49.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 20.291061711, 2024-08-20T03:12:49.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 286.098494049, 2024-08-20T03:12:49.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 2.936888803, 2024-08-20T03:12:49.969 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 726828, 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 112.202518784, 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000154372 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 572706, 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.140004600, 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000244 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 364944, 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 115456 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.970 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 130984, 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 138004, 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 63316173, 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 517572781, 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 38, 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 13.182740306, 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 6.713024163, 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 2.337803590, 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 3.775641438, 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 138007, 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 135.046272093, 2024-08-20T03:12:49.971 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000978546 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 32589, 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.007719045, 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000236 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1451808, 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 155072 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 74588, 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 87915, 2024-08-20T03:12:49.972 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 32712143, 2024-08-20T03:12:49.973 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 307436431, 2024-08-20T03:12:49.973 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 121, 2024-08-20T03:12:49.973 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 5, 2024-08-20T03:12:49.973 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 7.630360854, 2024-08-20T03:12:49.973 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 3.928001768, 2024-08-20T03:12:49.973 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 1.890729409, 2024-08-20T03:12:49.973 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 1.760620840, 2024-08-20T03:12:49.973 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:49.973 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 87916, 2024-08-20T03:12:49.973 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 44.109098988, 2024-08-20T03:12:49.973 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000501718 2024-08-20T03:12:49.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 66659, 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.012504644, 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000187 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 36224, 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 369456 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.974 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 47645, 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 351.473088919, 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.007376914 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 108733, 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 40.626800631, 2024-08-20T03:12:49.975 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000373638 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 224869, 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 219365, 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 219365, 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 5594.320034212, 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.025502336 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 5504, 2024-08-20T03:12:49.976 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 4, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 1, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 14549, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 197, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 20, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 94292, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 61690, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 32587, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 15, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 5334, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 5729, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 2, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 3, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 62, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 339480, 2024-08-20T03:12:49.977 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 317434, 2024-08-20T03:12:49.978 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 5475, 2024-08-20T03:12:49.978 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 39, 2024-08-20T03:12:49.978 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-08-20T03:12:49.978 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-08-20T03:12:49.978 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 1389, 2024-08-20T03:12:49.978 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-08-20T03:12:49.978 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-08-20T03:12:49.978 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 26, 2024-08-20T03:12:49.978 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 3557, 2024-08-20T03:12:49.978 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 152, 2024-08-20T03:12:49.978 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 11814, 2024-08-20T03:12:49.978 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 100402, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 4, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 234, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 6, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 100402, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 795, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 11, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 784, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 99607, 2024-08-20T03:12:49.979 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 38, 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 42852, 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 130691, 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 8, 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 137747, 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 65262, 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 7072, 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 125725 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 7488, 2024-08-20T03:12:49.980 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-08-20T03:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-08-20T03:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 39, 2024-08-20T03:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 14025, 2024-08-20T03:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 217, 2024-08-20T03:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-08-20T03:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-08-20T03:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 59410, 2024-08-20T03:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 59193, 2024-08-20T03:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-08-20T03:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-08-20T03:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-08-20T03:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 11, 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 26, 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 217, 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.982 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 326985, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 320864, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 320864, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 6121, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 10803, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 490, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 479, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 479, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 11, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 1, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 18, 2024-08-20T03:12:49.983 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 1052257721, 2024-08-20T03:12:49.984 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 1069644886, 2024-08-20T03:12:49.984 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-08-20T03:12:49.984 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-08-20T03:12:49.984 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4320, 2024-08-20T03:12:49.984 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1170.869776272, 2024-08-20T03:12:49.984 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.271034670 2024-08-20T03:12:49.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.984 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-08-20T03:12:49.984 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.984 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-08-20T03:12:49.984 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 54827, 2024-08-20T03:12:49.984 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 115777, 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 60950, 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 1619, 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 2010, 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 391, 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 94292, 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 193187, 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 98895, 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 3, 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 62347, 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 62344, 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 359564, 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 655220736 2024-08-20T03:12:49.985 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.986 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-08-20T03:12:49.986 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 240688, 2024-08-20T03:12:49.986 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 433, 2024-08-20T03:12:49.986 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 224869, 2024-08-20T03:12:49.986 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 53, 2024-08-20T03:12:49.986 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 3037, 2024-08-20T03:12:49.986 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-08-20T03:12:49.986 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 60792, 2024-08-20T03:12:49.986 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1019.647159458, 2024-08-20T03:12:49.986 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.016772719 2024-08-20T03:12:49.986 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.986 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 11939, 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1632.448359812, 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.136732419 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.987 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 41183, 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1506.792981592, 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.036587742 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000174796, 2024-08-20T03:12:49.988 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000174796 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.989 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 10, 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.016166700, 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001616670 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 231, 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 6.104131079, 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.026424809 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.990 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-08-20T03:12:49.991 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.991 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.994 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.994 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.994 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-08-20T03:12:49.994 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 10, 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 8.859015480, 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.885901548 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 9628, 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 150.577151877, 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.015639504 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1510, 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 4.545936444, 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.003010553 2024-08-20T03:12:49.995 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 60407, 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1177.347783321, 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.019490254 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.996 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 225, 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 31.455487649, 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.139802167 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4, 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.173601196, 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.043400299 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.997 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 9, 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.901132040, 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.100125782 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.998 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8, 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.019797939, 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002474742 2024-08-20T03:12:49.999 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3, 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.001944942, 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000648314 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 33405, 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 55.429209887, 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001659308 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-08-20T03:12:50.000 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 4, 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 2, 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 2, 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 19176, 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 9588, 2024-08-20T03:12:50.001 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 613, 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T03:12:50.002 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-08-20T03:12:50.003 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 4016015, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 53167, 2024-08-20T03:12:50.004 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 176, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 2, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7976, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 177, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 195366118, 2024-08-20T03:12:50.005 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 1919080, 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 245785, 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 1234999365, 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 37, 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 245785, 2024-08-20T03:12:50.006 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 245785, 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 38595.890371158, 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.157031105 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 245785, 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 406739 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 245785, 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 101453, 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 144332, 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-08-20T03:12:50.007 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 12727, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 62778, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 1035, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 5345, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 559, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 197, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 60380, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 100418, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 125556, 2024-08-20T03:12:50.008 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 8, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-08-20T03:12:50.009 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 85, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 10, 2024-08-20T03:12:50.010 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 12, 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 12, 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 29098, 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 10, 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 8628 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-08-20T03:12:50.011 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 9, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 6, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 59217, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 59193, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 674057, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 203733148, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:50.012 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 674057, 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 674057, 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 203733148, 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-08-20T03:12:50.013 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 245785, 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 1241991598, 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 245785, 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1241991598, 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:50.014 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 245785, 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 245785, 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 245785, 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 245785, 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:50.015 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 59193, 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 6992301, 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 59193, 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-08-20T03:12:50.016 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 1025, 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 6992301, 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55e4534506e0": { 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 326985, 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 1065450582, 2024-08-20T03:12:50.017 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 326985, 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 4320, 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 1065450582, 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: "trackedop": { 2024-08-20T03:12:50.018 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 7 2024-08-20T03:12:50.019 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:50.019 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-08-20T03:12:50.446 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:50 smithi099 ceph-mon[21984]: pgmap v638: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:50.565 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:50 smithi165 ceph-mon[22357]: pgmap v638: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:50.592 INFO:tasks.check_counter:Daemon mds.i mds.exported=26 2024-08-20T03:12:50.592 INFO:tasks.check_counter:Daemon mds.i mds.imported=152 2024-08-20T03:12:50.592 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=7488 2024-08-20T03:12:50.592 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-08-20T03:12:50.592 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-08-20T03:12:50.592 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=10 2024-08-20T03:12:50.592 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=4 2024-08-20T03:12:50.593 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=197 2024-08-20T03:12:50.593 DEBUG:tasks.check_counter:Getting stats from l 2024-08-20T03:12:50.593 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph daemon mds.l perf dump 2024-08-20T03:12:50.785 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:50 smithi044 ceph-mon[18790]: pgmap v638: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:50.880 INFO:teuthology.orchestra.run.smithi165.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T03:12:51.315 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-08-20T03:12:51.315 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T03:12:51.315 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1747, 2024-08-20T03:12:51.315 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1749, 2024-08-20T03:12:51.315 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 17284591, 2024-08-20T03:12:51.315 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 555553, 2024-08-20T03:12:51.315 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-08-20T03:12:51.315 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 2, 2024-08-20T03:12:51.316 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.164885183, 2024-08-20T03:12:51.316 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.068203916, 2024-08-20T03:12:51.316 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.043149930, 2024-08-20T03:12:51.316 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.042816159, 2024-08-20T03:12:51.316 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:51.316 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1747, 2024-08-20T03:12:51.316 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.056097235, 2024-08-20T03:12:51.316 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000032110 2024-08-20T03:12:51.316 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.316 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:51.316 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 545, 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 520, 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 5677970, 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 226456, 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 4, 2024-08-20T03:12:51.317 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-08-20T03:12:51.318 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.070955706, 2024-08-20T03:12:51.318 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.027106540, 2024-08-20T03:12:51.318 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.047292883, 2024-08-20T03:12:51.318 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.008230718, 2024-08-20T03:12:51.318 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:51.318 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 519, 2024-08-20T03:12:51.318 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.022886407, 2024-08-20T03:12:51.318 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000044097 2024-08-20T03:12:51.318 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.318 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:51.318 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.318 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.319 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.319 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1410896, 2024-08-20T03:12:51.319 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 133856 2024-08-20T03:12:51.319 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.319 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T03:12:51.319 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 240, 2024-08-20T03:12:51.319 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 399, 2024-08-20T03:12:51.319 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1588050, 2024-08-20T03:12:51.319 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 432154, 2024-08-20T03:12:51.319 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 5, 2024-08-20T03:12:51.319 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-08-20T03:12:51.320 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.036358727, 2024-08-20T03:12:51.320 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.019397108, 2024-08-20T03:12:51.320 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.008382700, 2024-08-20T03:12:51.320 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.005630015, 2024-08-20T03:12:51.320 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T03:12:51.320 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 398, 2024-08-20T03:12:51.320 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.019807880, 2024-08-20T03:12:51.320 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000049768 2024-08-20T03:12:51.320 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.320 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-08-20T03:12:51.320 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.320 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 357424 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-08-20T03:12:51.321 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3852, 2024-08-20T03:12:51.322 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.091340372, 2024-08-20T03:12:51.322 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000023712 2024-08-20T03:12:51.322 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:51.322 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.322 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-08-20T03:12:51.322 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-08-20T03:12:51.322 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-08-20T03:12:51.322 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.322 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.322 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.322 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 0, 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 0, 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-08-20T03:12:51.323 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 0, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 2495, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 2495, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 2495, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 0, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 0, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 1, 2024-08-20T03:12:51.324 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-08-20T03:12:51.325 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-08-20T03:12:51.326 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-08-20T03:12:51.326 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-08-20T03:12:51.326 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-08-20T03:12:51.326 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-08-20T03:12:51.326 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-08-20T03:12:51.326 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T03:12:51.326 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-08-20T03:12:51.326 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T03:12:51.326 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-08-20T03:12:51.326 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-08-20T03:12:51.326 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 0, 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 0, 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 0, 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 0, 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 0, 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 0 2024-08-20T03:12:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 0, 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 0, 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 0, 2024-08-20T03:12:51.328 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_path": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_quiesce_inode": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-08-20T03:12:51.329 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 0, 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 12619, 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 0, 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-08-20T03:12:51.330 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 17, 2024-08-20T03:12:51.331 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 3, 2024-08-20T03:12:51.331 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-08-20T03:12:51.331 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-08-20T03:12:51.331 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-08-20T03:12:51.331 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 26689930, 2024-08-20T03:12:51.331 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 26685004, 2024-08-20T03:12:51.331 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-08-20T03:12:51.331 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.331 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.331 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.331 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.331 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 12619 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 13, 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 5010, 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 4997, 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 11, 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 94, 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 83, 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 2495, 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 7374, 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 4879, 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 0, 2024-08-20T03:12:51.332 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 0, 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 0, 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 60876, 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 264245248 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 0, 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 0, 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 0, 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.333 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.334 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.334 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-08-20T03:12:51.334 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.334 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.334 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.334 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-08-20T03:12:51.334 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.334 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.334 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.335 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.336 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.337 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.338 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-08-20T03:12:51.338 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.338 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.338 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.339 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-08-20T03:12:51.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.339 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.339 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-08-20T03:12:51.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.339 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.339 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-08-20T03:12:51.340 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.340 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.340 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.340 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-08-20T03:12:51.340 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.340 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.340 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.340 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-08-20T03:12:51.340 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.340 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.341 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-08-20T03:12:51.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.341 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.341 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-08-20T03:12:51.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.341 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.342 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.343 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.345 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-08-20T03:12:51.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.345 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.345 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-08-20T03:12:51.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.345 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.346 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-08-20T03:12:51.346 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-08-20T03:12:51.346 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-08-20T03:12:51.346 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.346 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.346 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.346 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:51.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.346 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-08-20T03:12:51.346 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 2, 2024-08-20T03:12:51.346 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 4, 2024-08-20T03:12:51.347 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 2, 2024-08-20T03:12:51.347 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 3, 2024-08-20T03:12:51.347 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-08-20T03:12:51.347 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 0, 2024-08-20T03:12:51.347 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 0, 2024-08-20T03:12:51.347 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 2190, 2024-08-20T03:12:51.347 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-08-20T03:12:51.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.347 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-08-20T03:12:51.347 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-08-20T03:12:51.347 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-08-20T03:12:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T03:12:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-08-20T03:12:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T03:12:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-08-20T03:12:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T03:12:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T03:12:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T03:12:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T03:12:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T03:12:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-08-20T03:12:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T03:12:51.348 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T03:12:51.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-08-20T03:12:51.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-08-20T03:12:51.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-08-20T03:12:51.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-08-20T03:12:51.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T03:12:51.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T03:12:51.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T03:12:51.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T03:12:51.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T03:12:51.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-08-20T03:12:51.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-08-20T03:12:51.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-08-20T03:12:51.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T03:12:51.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T03:12:51.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-08-20T03:12:51.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-08-20T03:12:51.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-08-20T03:12:51.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-08-20T03:12:51.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T03:12:51.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-08-20T03:12:51.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T03:12:51.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-08-20T03:12:51.350 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 199071, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 59, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 0, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 0, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 7976, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 177, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T03:12:51.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-08-20T03:12:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-08-20T03:12:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-08-20T03:12:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 1762824, 2024-08-20T03:12:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 10148, 2024-08-20T03:12:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-08-20T03:12:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-08-20T03:12:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-08-20T03:12:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-08-20T03:12:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-08-20T03:12:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-08-20T03:12:51.352 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-08-20T03:12:51.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 2273, 2024-08-20T03:12:51.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 21, 2024-08-20T03:12:51.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 3, 2024-08-20T03:12:51.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 2273, 2024-08-20T03:12:51.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-08-20T03:12:51.353 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2273, 2024-08-20T03:12:51.353 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 15.701035570, 2024-08-20T03:12:51.353 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.006907626 2024-08-20T03:12:51.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-08-20T03:12:51.353 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-08-20T03:12:51.353 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2273, 2024-08-20T03:12:51.354 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2274 2024-08-20T03:12:51.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.354 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 2273, 2024-08-20T03:12:51.354 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 2273, 2024-08-20T03:12:51.354 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 0, 2024-08-20T03:12:51.354 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-08-20T03:12:51.354 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-08-20T03:12:51.354 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 1367, 2024-08-20T03:12:51.354 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 0, 2024-08-20T03:12:51.354 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 904, 2024-08-20T03:12:51.354 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 0, 2024-08-20T03:12:51.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 0, 2024-08-20T03:12:51.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-08-20T03:12:51.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-08-20T03:12:51.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 0, 2024-08-20T03:12:51.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-08-20T03:12:51.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 0, 2024-08-20T03:12:51.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-08-20T03:12:51.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-08-20T03:12:51.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-08-20T03:12:51.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 1, 2024-08-20T03:12:51.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 0, 2024-08-20T03:12:51.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-08-20T03:12:51.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-08-20T03:12:51.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-08-20T03:12:51.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-08-20T03:12:51.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-08-20T03:12:51.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-08-20T03:12:51.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T03:12:51.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-08-20T03:12:51.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-08-20T03:12:51.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-08-20T03:12:51.356 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-08-20T03:12:51.356 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-08-20T03:12:51.357 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-08-20T03:12:51.357 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-08-20T03:12:51.357 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-08-20T03:12:51.357 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-08-20T03:12:51.357 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-08-20T03:12:51.357 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-08-20T03:12:51.357 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-08-20T03:12:51.357 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-08-20T03:12:51.357 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-08-20T03:12:51.357 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-08-20T03:12:51.357 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-08-20T03:12:51.357 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-08-20T03:12:51.358 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-08-20T03:12:51.358 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-08-20T03:12:51.358 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 85, 2024-08-20T03:12:51.358 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-08-20T03:12:51.358 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 10, 2024-08-20T03:12:51.358 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 6, 2024-08-20T03:12:51.358 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 6, 2024-08-20T03:12:51.358 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-08-20T03:12:51.358 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-08-20T03:12:51.358 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 0, 2024-08-20T03:12:51.358 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 2, 2024-08-20T03:12:51.359 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 0 2024-08-20T03:12:51.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.359 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-08-20T03:12:51.359 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 0, 2024-08-20T03:12:51.359 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-08-20T03:12:51.359 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-08-20T03:12:51.359 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-08-20T03:12:51.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.359 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-08-20T03:12:51.359 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-08-20T03:12:51.359 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 0, 2024-08-20T03:12:51.359 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 0, 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 0, 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 0, 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 2528, 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 24307110, 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:51.360 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 2528, 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2528, 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 24307110, 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-08-20T03:12:51.361 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:51.362 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-08-20T03:12:51.362 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-08-20T03:12:51.362 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:51.362 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:51.362 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2273, 2024-08-20T03:12:51.362 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 22498109, 2024-08-20T03:12:51.362 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2273, 2024-08-20T03:12:51.362 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 22498109, 2024-08-20T03:12:51.362 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:51.362 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.362 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.362 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 2273, 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 2273, 2024-08-20T03:12:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 2273, 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 2273, 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-08-20T03:12:51.364 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.365 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55ce9d5ee960": { 2024-08-20T03:12:51.366 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-08-20T03:12:51.366 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-08-20T03:12:51.366 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-08-20T03:12:51.366 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-08-20T03:12:51.366 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-08-20T03:12:51.366 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-08-20T03:12:51.366 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-08-20T03:12:51.366 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-08-20T03:12:51.366 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-08-20T03:12:51.366 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 0, 2024-08-20T03:12:51.366 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 0, 2024-08-20T03:12:51.366 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-08-20T03:12:51.367 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-08-20T03:12:51.367 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-08-20T03:12:51.367 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-08-20T03:12:51.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:51.367 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-08-20T03:12:51.367 INFO:teuthology.orchestra.run.smithi165.stdout: "trackedop": { 2024-08-20T03:12:51.367 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-08-20T03:12:51.367 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-08-20T03:12:51.367 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-08-20T03:12:52.105 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-08-20T03:12:52.105 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-08-20T03:12:52.105 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-08-20T03:12:52.105 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-08-20T03:12:52.105 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-08-20T03:12:52.105 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-08-20T03:12:52.105 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T03:12:52.105 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-08-20T03:12:52.106 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-08-20T03:12:52.116 INFO:tasks.kclient:Unmounting kernel clients... 2024-08-20T03:12:52.116 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T03:12:52.117 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-20T03:12:52.137 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-08-20T03:12:52.138 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-20T03:12:52.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-20T03:12:52.507 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:52.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:52 smithi044 ceph-mon[18790]: pgmap v639: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:52.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:52 smithi099 ceph-mon[21984]: pgmap v639: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:52.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:52 smithi165 ceph-mon[22357]: pgmap v639: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-08-20T03:12:53.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.985+0000 7f1f810f7640 1 -- 172.21.15.44:0/2924959732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1073e0 msgr2=0x7f1f7c107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:53.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.985+0000 7f1f810f7640 1 --2- 172.21.15.44:0/2924959732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1073e0 0x7f1f7c107820 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009900 tx=0x7f1f7002a410 comp rx=0 tx=0).stop 2024-08-20T03:12:53.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.985+0000 7f1f810f7640 1 -- 172.21.15.44:0/2924959732 shutdown_connections 2024-08-20T03:12:53.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.985+0000 7f1f810f7640 1 --2- 172.21.15.44:0/2924959732 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f7c107d60 0x7f1f7c10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:53.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.985+0000 7f1f810f7640 1 --2- 172.21.15.44:0/2924959732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1073e0 0x7f1f7c107820 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:53.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.985+0000 7f1f810f7640 1 --2- 172.21.15.44:0/2924959732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c1019d0 0x7f1f7c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:53.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.985+0000 7f1f810f7640 1 -- 172.21.15.44:0/2924959732 >> 172.21.15.44:0/2924959732 conn(0x7f1f7c0fd440 msgr2=0x7f1f7c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.985+0000 7f1f810f7640 1 -- 172.21.15.44:0/2924959732 shutdown_connections 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.985+0000 7f1f810f7640 1 -- 172.21.15.44:0/2924959732 wait complete. 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.985+0000 7f1f810f7640 1 Processor -- start 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f810f7640 1 -- start start 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f810f7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c1019d0 0x7f1f7c10f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f810f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1073e0 0x7f1f7c10f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f810f7640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f7c107d60 0x7f1f7c10faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f810f7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c112b40 con 0x7f1f7c1073e0 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f810f7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c1129c0 con 0x7f1f7c1019d0 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f810f7640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c112cc0 con 0x7f1f7c107d60 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1073e0 0x7f1f7c10f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1073e0 0x7f1f7c10f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50392/0 (socket says 172.21.15.44:50392) 2024-08-20T03:12:53.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7a59c640 1 -- 172.21.15.44:0/3041096233 learned_addr learned my addr 172.21.15.44:0/3041096233 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7b59e640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f7c107d60 0x7f1f7c10faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7ad9d640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c1019d0 0x7f1f7c10f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7a59c640 1 -- 172.21.15.44:0/3041096233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f7c107d60 msgr2=0x7f1f7c10faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7a59c640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f7c107d60 0x7f1f7c10faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7a59c640 1 -- 172.21.15.44:0/3041096233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c1019d0 msgr2=0x7f1f7c10f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7a59c640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c1019d0 0x7f1f7c10f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7a59c640 1 -- 172.21.15.44:0/3041096233 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7c1a7410 con 0x7f1f7c1073e0 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7b59e640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f7c107d60 0x7f1f7c10faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7ad9d640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c1019d0 0x7f1f7c10f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.986+0000 7f1f7a59c640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1073e0 0x7f1f7c10f5b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1f70005940 tx=0x7f1f700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.987+0000 7f1f63fff640 1 -- 172.21.15.44:0/3041096233 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7003f030 con 0x7f1f7c1073e0 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.987+0000 7f1f63fff640 1 -- 172.21.15.44:0/3041096233 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f70003cb0 con 0x7f1f7c1073e0 2024-08-20T03:12:53.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.987+0000 7f1f810f7640 1 -- 172.21.15.44:0/3041096233 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7c1a76a0 con 0x7f1f7c1073e0 2024-08-20T03:12:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.987+0000 7f1f63fff640 1 -- 172.21.15.44:0/3041096233 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70035070 con 0x7f1f7c1073e0 2024-08-20T03:12:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.987+0000 7f1f810f7640 1 -- 172.21.15.44:0/3041096233 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f7c1a7c00 con 0x7f1f7c1073e0 2024-08-20T03:12:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.988+0000 7f1f810f7640 1 -- 172.21.15.44:0/3041096233 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f7c101db0 con 0x7f1f7c1073e0 2024-08-20T03:12:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.989+0000 7f1f63fff640 1 -- 172.21.15.44:0/3041096233 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f1f70042030 con 0x7f1f7c1073e0 2024-08-20T03:12:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.990+0000 7f1f63fff640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f640b3d40 0x7f1f640b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.990+0000 7f1f7ad9d640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f640b3d40 0x7f1f640b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.990+0000 7f1f63fff640 1 -- 172.21.15.44:0/3041096233 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f1f70104ba0 con 0x7f1f7c1073e0 2024-08-20T03:12:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.990+0000 7f1f7ad9d640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f640b3d40 0x7f1f640b6200 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009930 tx=0x7f1f6c008040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:53.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:52.993+0000 7f1f63fff640 1 -- 172.21.15.44:0/3041096233 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f1f700cbef0 con 0x7f1f7c1073e0 2024-08-20T03:12:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.192+0000 7f1f810f7640 1 -- 172.21.15.44:0/3041096233 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1f7c02e340 con 0x7f1f7c1073e0 2024-08-20T03:12:53.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.193+0000 7f1f63fff640 1 -- 172.21.15.44:0/3041096233 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v85) ==== 81+0+888 (secure 0 0 0) 0x7f1f700d41c0 con 0x7f1f7c1073e0 2024-08-20T03:12:53.195 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2597827045 2024-08-21T02:53:58.312659+0000 2024-08-20T03:12:53.195 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/3582612619 2024-08-21T02:53:58.312659+0000 2024-08-20T03:12:53.195 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/1916809746 2024-08-21T02:52:33.983380+0000 2024-08-20T03:12:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2919033419 2024-08-21T02:52:33.983380+0000 2024-08-20T03:12:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/1293546305 2024-08-21T02:53:58.312659+0000 2024-08-20T03:12:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/1961932124 2024-08-21T02:52:33.983380+0000 2024-08-20T03:12:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2331034359 2024-08-21T02:52:33.983380+0000 2024-08-20T03:12:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/1961932124 2024-08-21T02:52:33.983380+0000 2024-08-20T03:12:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2572185146 2024-08-21T02:52:58.451601+0000 2024-08-20T03:12:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/1293546305 2024-08-21T02:53:58.312659+0000 2024-08-20T03:12:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/4176802819 2024-08-21T02:52:58.451601+0000 2024-08-20T03:12:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6800/3842608703 2024-08-21T02:52:58.451601+0000 2024-08-20T03:12:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/3654439062 2024-08-21T02:52:58.451601+0000 2024-08-20T03:12:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:6801/3842608703 2024-08-21T02:52:58.451601+0000 2024-08-20T03:12:53.196 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44:0/2376733275 2024-08-21T02:53:58.312659+0000 2024-08-20T03:12:53.197 INFO:teuthology.orchestra.run.smithi044.stderr:listed 15 entries 2024-08-20T03:12:53.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.195+0000 7f1f810f7640 1 -- 172.21.15.44:0/3041096233 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f640b3d40 msgr2=0x7f1f640b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:53.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.195+0000 7f1f810f7640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f640b3d40 0x7f1f640b6200 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009930 tx=0x7f1f6c008040 comp rx=0 tx=0).stop 2024-08-20T03:12:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.195+0000 7f1f810f7640 1 -- 172.21.15.44:0/3041096233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1073e0 msgr2=0x7f1f7c10f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.195+0000 7f1f810f7640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1073e0 0x7f1f7c10f5b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f1f70005940 tx=0x7f1f700042f0 comp rx=0 tx=0).stop 2024-08-20T03:12:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.195+0000 7f1f810f7640 1 -- 172.21.15.44:0/3041096233 shutdown_connections 2024-08-20T03:12:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.195+0000 7f1f810f7640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f1f640b3d40 0x7f1f640b6200 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.195+0000 7f1f810f7640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1f7c107d60 0x7f1f7c10faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.195+0000 7f1f810f7640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f7c1073e0 0x7f1f7c10f5b0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.195+0000 7f1f810f7640 1 --2- 172.21.15.44:0/3041096233 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c1019d0 0x7f1f7c10f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.195+0000 7f1f810f7640 1 -- 172.21.15.44:0/3041096233 >> 172.21.15.44:0/3041096233 conn(0x7f1f7c0fd440 msgr2=0x7f1f7c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.195+0000 7f1f810f7640 1 -- 172.21.15.44:0/3041096233 shutdown_connections 2024-08-20T03:12:53.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:53.195+0000 7f1f810f7640 1 -- 172.21.15.44:0/3041096233 wait complete. 2024-08-20T03:12:53.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:53 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3041096233' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T03:12:53.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:53 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3041096233' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T03:12:53.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:53 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3041096233' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T03:12:54.347 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-08-20T03:12:54.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:12:54.347 DEBUG:teuthology.orchestra.run.smithi044:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-08-20T03:12:54.535 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-08-20T03:12:54.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:12:54.535 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-20T03:12:54.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:12:54.594 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-08-20T03:12:54.646 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-20T03:12:54.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:12:54.646 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-20T03:12:54.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:54 smithi099 ceph-mon[21984]: pgmap v640: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-08-20T03:12:54.693 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:54 smithi044 ceph-mon[18790]: pgmap v640: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-08-20T03:12:54.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T03:12:54.703 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-08-20T03:12:54.787 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:54 smithi165 ceph-mon[22357]: pgmap v640: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-08-20T03:12:54.816 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-08-20T03:12:54.828 INFO:tasks.cephadm:Teardown begin 2024-08-20T03:12:54.828 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-20T03:12:54.828 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:12:55.447 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:55.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:55 smithi044 ceph-mon[18790]: pgmap v641: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-08-20T03:12:55.801 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:55 smithi165 ceph-mon[22357]: pgmap v641: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-08-20T03:12:55.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:55 smithi099 ceph-mon[21984]: pgmap v641: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-08-20T03:12:56.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.023+0000 7f77b6464640 1 -- 172.21.15.44:0/1341841586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b0104300 msgr2=0x7f77b010c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:56.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.023+0000 7f77b6464640 1 --2- 172.21.15.44:0/1341841586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b0104300 0x7f77b010c7f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009900 tx=0x7f77a402a410 comp rx=0 tx=0).stop 2024-08-20T03:12:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.023+0000 7f77b6464640 1 -- 172.21.15.44:0/1341841586 shutdown_connections 2024-08-20T03:12:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.023+0000 7f77b6464640 1 --2- 172.21.15.44:0/1341841586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77b010ce40 0x7f77b010f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.023+0000 7f77b6464640 1 --2- 172.21.15.44:0/1341841586 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b0104300 0x7f77b010c7f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.023+0000 7f77b6464640 1 --2- 172.21.15.44:0/1341841586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77b01039e0 0x7f77b0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.023+0000 7f77b6464640 1 -- 172.21.15.44:0/1341841586 >> 172.21.15.44:0/1341841586 conn(0x7f77b00fd440 msgr2=0x7f77b00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.023+0000 7f77b6464640 1 -- 172.21.15.44:0/1341841586 shutdown_connections 2024-08-20T03:12:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.023+0000 7f77b6464640 1 -- 172.21.15.44:0/1341841586 wait complete. 2024-08-20T03:12:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.024+0000 7f77b6464640 1 Processor -- start 2024-08-20T03:12:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.024+0000 7f77b6464640 1 -- start start 2024-08-20T03:12:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.024+0000 7f77b6464640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77b01039e0 0x7f77b0198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.024+0000 7f77b6464640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b0104300 0x7f77b0199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:56.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.024+0000 7f77b6464640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77b010ce40 0x7f77b019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.024+0000 7f77b6464640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0112ab0 con 0x7f77b0104300 2024-08-20T03:12:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.024+0000 7f77b6464640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0112930 con 0x7f77b01039e0 2024-08-20T03:12:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.024+0000 7f77b6464640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0112c30 con 0x7f77b010ce40 2024-08-20T03:12:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b0104300 0x7f77b0199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b0104300 0x7f77b0199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:50416/0 (socket says 172.21.15.44:50416) 2024-08-20T03:12:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77af7fe640 1 -- 172.21.15.44:0/3200419166 learned_addr learned my addr 172.21.15.44:0/3200419166 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77b4a01640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77b010ce40 0x7f77b019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77affff640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77b01039e0 0x7f77b0198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77af7fe640 1 -- 172.21.15.44:0/3200419166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77b010ce40 msgr2=0x7f77b019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77af7fe640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77b010ce40 0x7f77b019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77af7fe640 1 -- 172.21.15.44:0/3200419166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77b01039e0 msgr2=0x7f77b0198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:56.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77af7fe640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77b01039e0 0x7f77b0198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77af7fe640 1 -- 172.21.15.44:0/3200419166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b019f090 con 0x7f77b0104300 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77af7fe640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b0104300 0x7f77b0199520 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f77a4032fd0 tx=0x7f77a4031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77ad7fa640 1 -- 172.21.15.44:0/3200419166 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a403f030 con 0x7f77b0104300 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77b6464640 1 -- 172.21.15.44:0/3200419166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b019f320 con 0x7f77b0104300 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77ad7fa640 1 -- 172.21.15.44:0/3200419166 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77a4004490 con 0x7f77b0104300 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77ad7fa640 1 -- 172.21.15.44:0/3200419166 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4035070 con 0x7f77b0104300 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.025+0000 7f77b6464640 1 -- 172.21.15.44:0/3200419166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b01a5a40 con 0x7f77b0104300 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.027+0000 7f77b6464640 1 -- 172.21.15.44:0/3200419166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b0103dc0 con 0x7f77b0104300 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.028+0000 7f77ad7fa640 1 -- 172.21.15.44:0/3200419166 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f77a4042030 con 0x7f77b0104300 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.029+0000 7f77ad7fa640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77980b3d40 0x7f77980b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.029+0000 7f77affff640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77980b3d40 0x7f77980b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.029+0000 7f77ad7fa640 1 -- 172.21.15.44:0/3200419166 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f77a41047b0 con 0x7f77b0104300 2024-08-20T03:12:56.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.029+0000 7f77affff640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77980b3d40 0x7f77980b6200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009bd0 tx=0x7f77a0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:56.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.032+0000 7f77ad7fa640 1 -- 172.21.15.44:0/3200419166 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f77a40cb3b0 con 0x7f77b0104300 2024-08-20T03:12:56.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.252+0000 7f77b6464640 1 -- 172.21.15.44:0/3200419166 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77b0107e20 con 0x7f77b0104300 2024-08-20T03:12:56.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.253+0000 7f77ad7fa640 1 -- 172.21.15.44:0/3200419166 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11973 (secure 0 0 0) 0x7f77a40cf900 con 0x7f77b0104300 2024-08-20T03:12:56.256 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:12:56.256 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":44,"btime":"2024-08-20T03:11:43:976127+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:11:43.142495+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24569,24583,24596,24589,24604]},"id":1}]} 2024-08-20T03:12:56.257 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 44 2024-08-20T03:12:56.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.255+0000 7f77b6464640 1 -- 172.21.15.44:0/3200419166 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77980b3d40 msgr2=0x7f77980b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:56.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.255+0000 7f77b6464640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77980b3d40 0x7f77980b6200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009bd0 tx=0x7f77a0005f90 comp rx=0 tx=0).stop 2024-08-20T03:12:56.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.255+0000 7f77b6464640 1 -- 172.21.15.44:0/3200419166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b0104300 msgr2=0x7f77b0199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:56.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.255+0000 7f77b6464640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b0104300 0x7f77b0199520 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f77a4032fd0 tx=0x7f77a4031a20 comp rx=0 tx=0).stop 2024-08-20T03:12:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.255+0000 7f77b6464640 1 -- 172.21.15.44:0/3200419166 shutdown_connections 2024-08-20T03:12:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.255+0000 7f77b6464640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f77980b3d40 0x7f77980b6200 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.255+0000 7f77b6464640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77b010ce40 0x7f77b019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.255+0000 7f77b6464640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77b0104300 0x7f77b0199520 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.255+0000 7f77b6464640 1 --2- 172.21.15.44:0/3200419166 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77b01039e0 0x7f77b0198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.255+0000 7f77b6464640 1 -- 172.21.15.44:0/3200419166 >> 172.21.15.44:0/3200419166 conn(0x7f77b00fd440 msgr2=0x7f77b00fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.255+0000 7f77b6464640 1 -- 172.21.15.44:0/3200419166 shutdown_connections 2024-08-20T03:12:56.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:56.255+0000 7f77b6464640 1 -- 172.21.15.44:0/3200419166 wait complete. 2024-08-20T03:12:56.786 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:56 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3200419166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:12:56.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:56 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3200419166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:12:56.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:56 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3200419166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:12:56.963 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T03:12:57.217 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:57.593 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:57 smithi044 ceph-mon[18790]: pgmap v642: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-08-20T03:12:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.672+0000 7f187f2ba640 1 -- 172.21.15.44:0/2095579817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1878101520 msgr2=0x7f1878101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.672+0000 7f187f2ba640 1 --2- 172.21.15.44:0/2095579817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1878101520 0x7f1878101900 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1874009900 tx=0x7f187402a410 comp rx=0 tx=0).stop 2024-08-20T03:12:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.672+0000 7f187f2ba640 1 -- 172.21.15.44:0/2095579817 shutdown_connections 2024-08-20T03:12:57.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.672+0000 7f187f2ba640 1 --2- 172.21.15.44:0/2095579817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18781074d0 0x7f187810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.672+0000 7f187f2ba640 1 --2- 172.21.15.44:0/2095579817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1878106b50 0x7f1878106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.672+0000 7f187f2ba640 1 --2- 172.21.15.44:0/2095579817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1878101520 0x7f1878101900 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.672+0000 7f187f2ba640 1 -- 172.21.15.44:0/2095579817 >> 172.21.15.44:0/2095579817 conn(0x7f1878076c80 msgr2=0x7f1878077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.672+0000 7f187f2ba640 1 -- 172.21.15.44:0/2095579817 shutdown_connections 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.672+0000 7f187f2ba640 1 -- 172.21.15.44:0/2095579817 wait complete. 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.673+0000 7f187f2ba640 1 Processor -- start 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.673+0000 7f187f2ba640 1 -- start start 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.673+0000 7f187f2ba640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1878101520 0x7f1878196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.673+0000 7f187f2ba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1878106b50 0x7f1878197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.673+0000 7f187f2ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18781074d0 0x7f187819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.673+0000 7f187f2ba640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1878110910 con 0x7f1878106b50 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.673+0000 7f187f2ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1878110790 con 0x7f18781074d0 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.673+0000 7f187f2ba640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f1878110a90 con 0x7f1878101520 2024-08-20T03:12:57.711 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.673+0000 7f187c855640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1878106b50 0x7f1878197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.673+0000 7f187c855640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1878106b50 0x7f1878197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45444/0 (socket says 172.21.15.44:45444) 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.673+0000 7f187c855640 1 -- 172.21.15.44:0/41552777 learned_addr learned my addr 172.21.15.44:0/41552777 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.673+0000 7f187d857640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18781074d0 0x7f187819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.674+0000 7f187c855640 1 -- 172.21.15.44:0/41552777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1878101520 msgr2=0x7f1878196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.674+0000 7f187c855640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1878101520 0x7f1878196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.674+0000 7f187c855640 1 -- 172.21.15.44:0/41552777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18781074d0 msgr2=0x7f187819c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.674+0000 7f187c855640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18781074d0 0x7f187819c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.674+0000 7f187c855640 1 -- 172.21.15.44:0/41552777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187819ceb0 con 0x7f1878106b50 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.674+0000 7f187d857640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18781074d0 0x7f187819c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.674+0000 7f187c855640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1878106b50 0x7f1878197340 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f186800fe70 tx=0x7f186800d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.674+0000 7f186e7fc640 1 -- 172.21.15.44:0/41552777 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186801c070 con 0x7f1878106b50 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.674+0000 7f186e7fc640 1 -- 172.21.15.44:0/41552777 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f186800ca90 con 0x7f1878106b50 2024-08-20T03:12:57.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.674+0000 7f187f2ba640 1 -- 172.21.15.44:0/41552777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187819d1a0 con 0x7f1878106b50 2024-08-20T03:12:57.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.674+0000 7f186e7fc640 1 -- 172.21.15.44:0/41552777 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868017ab0 con 0x7f1878106b50 2024-08-20T03:12:57.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.674+0000 7f187f2ba640 1 -- 172.21.15.44:0/41552777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18781a3860 con 0x7f1878106b50 2024-08-20T03:12:57.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.675+0000 7f187f2ba640 1 -- 172.21.15.44:0/41552777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1878101900 con 0x7f1878106b50 2024-08-20T03:12:57.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.676+0000 7f186e7fc640 1 -- 172.21.15.44:0/41552777 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f186801f070 con 0x7f1878106b50 2024-08-20T03:12:57.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.677+0000 7f186e7fc640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f18640b3d40 0x7f18640b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:57.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.677+0000 7f187d056640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f18640b3d40 0x7f18640b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:57.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.677+0000 7f186e7fc640 1 -- 172.21.15.44:0/41552777 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f18680e55d0 con 0x7f1878106b50 2024-08-20T03:12:57.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.678+0000 7f187d056640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f18640b3d40 0x7f18640b6200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f18740096c0 tx=0x7f1874031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:57.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.680+0000 7f186e7fc640 1 -- 172.21.15.44:0/41552777 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f18680ac920 con 0x7f1878106b50 2024-08-20T03:12:57.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:57 smithi165 ceph-mon[22357]: pgmap v642: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-08-20T03:12:57.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.905+0000 7f187f2ba640 1 -- 172.21.15.44:0/41552777 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f187802e340 con 0x7f1878106b50 2024-08-20T03:12:57.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.905+0000 7f186e7fc640 1 -- 172.21.15.44:0/41552777 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11969 (secure 0 0 0) 0x7f18680ac220 con 0x7f1878106b50 2024-08-20T03:12:57.909 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:12:57.909 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":45,"btime":"2024-08-20T03:12:57:463709+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14742,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1875462475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1875462475},{"type":"v1","addr":"172.21.15.44:6835","nonce":1875462475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14754,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/3289736917","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":3289736917},{"type":"v1","addr":"172.21.15.44:6837","nonce":3289736917}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T03:00:18.935328+0000","modified":"2024-08-20T03:12:56.703652+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24604,"mds_1":24596,"mds_2":24589,"mds_3":24583,"mds_4":24569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14796":{"gid":14796,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6839/335802127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":335802127},{"type":"v1","addr":"172.21.15.44:6839","nonce":335802127}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14808":{"gid":14808,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6841/370123956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6840","nonce":370123956},{"type":"v1","addr":"172.21.15.44:6841","nonce":370123956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6833/2092228440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2092228440},{"type":"v1","addr":"172.21.15.99:6833","nonce":2092228440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24550":{"gid":24550,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6833/1910345968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":1910345968},{"type":"v1","addr":"172.21.15.165:6833","nonce":1910345968}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24557":{"gid":24557,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/797787628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":797787628},{"type":"v1","addr":"172.21.15.99:6835","nonce":797787628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.99:6837/728486889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":728486889},{"type":"v1","addr":"172.21.15.99:6837","nonce":728486889}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.165:6835/3019941967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3019941967},{"type":"v1","addr":"172.21.15.165:6835","nonce":3019941967}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24589":{"gid":24589,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.165:6837/3478633280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3478633280},{"type":"v1","addr":"172.21.15.165:6837","nonce":3478633280}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.99:6839/1272675547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":1272675547},{"type":"v1","addr":"172.21.15.99:6839","nonce":1272675547}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24604":{"gid":24604,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.165:6839/3029973590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6838","nonce":3029973590},{"type":"v1","addr":"172.21.15.165:6839","nonce":3029973590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24604,"qdb_cluster":[24604,24589,24596,24583,24569]},"id":1}]} 2024-08-20T03:12:57.909 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 45 2024-08-20T03:12:57.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.908+0000 7f187f2ba640 1 -- 172.21.15.44:0/41552777 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f18640b3d40 msgr2=0x7f18640b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:57.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.908+0000 7f187f2ba640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f18640b3d40 0x7f18640b6200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f18740096c0 tx=0x7f1874031400 comp rx=0 tx=0).stop 2024-08-20T03:12:57.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.908+0000 7f187f2ba640 1 -- 172.21.15.44:0/41552777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1878106b50 msgr2=0x7f1878197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:57.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.908+0000 7f187f2ba640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1878106b50 0x7f1878197340 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f186800fe70 tx=0x7f186800d7b0 comp rx=0 tx=0).stop 2024-08-20T03:12:57.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.908+0000 7f187f2ba640 1 -- 172.21.15.44:0/41552777 shutdown_connections 2024-08-20T03:12:57.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.908+0000 7f187f2ba640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f18640b3d40 0x7f18640b6200 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:57.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.908+0000 7f187f2ba640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18781074d0 0x7f187819c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:57.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.908+0000 7f187f2ba640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1878106b50 0x7f1878197340 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.908+0000 7f187f2ba640 1 --2- 172.21.15.44:0/41552777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1878101520 0x7f1878196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.908+0000 7f187f2ba640 1 -- 172.21.15.44:0/41552777 >> 172.21.15.44:0/41552777 conn(0x7f1878076c80 msgr2=0x7f1878100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.908+0000 7f187f2ba640 1 -- 172.21.15.44:0/41552777 shutdown_connections 2024-08-20T03:12:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:57.908+0000 7f187f2ba640 1 -- 172.21.15.44:0/41552777 wait complete. 2024-08-20T03:12:57.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:57 smithi099 ceph-mon[21984]: pgmap v642: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 426 B/s wr, 10 op/s 2024-08-20T03:12:58.586 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T03:12:58.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:58 smithi044 ceph-mon[18790]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:12:58.609 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:58 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/41552777' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:12:58.841 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:12:58.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:58 smithi165 ceph-mon[22357]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:12:58.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:58 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/41552777' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:12:58.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:58 smithi099 ceph-mon[21984]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-08-20T03:12:58.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:58 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/41552777' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T03:12:59.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.365+0000 7f830c4f6640 1 -- 172.21.15.44:0/4198378588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83041039e0 msgr2=0x7f8304103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:59.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.365+0000 7f830c4f6640 1 --2- 172.21.15.44:0/4198378588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83041039e0 0x7f8304103dc0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8300009900 tx=0x7f830002a410 comp rx=0 tx=0).stop 2024-08-20T03:12:59.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 -- 172.21.15.44:0/4198378588 shutdown_connections 2024-08-20T03:12:59.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 --2- 172.21.15.44:0/4198378588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f830410ce40 0x7f830410f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:59.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 --2- 172.21.15.44:0/4198378588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8304104300 0x7f830410c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:59.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 --2- 172.21.15.44:0/4198378588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83041039e0 0x7f8304103dc0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 -- 172.21.15.44:0/4198378588 >> 172.21.15.44:0/4198378588 conn(0x7f83040fd440 msgr2=0x7f83040ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 -- 172.21.15.44:0/4198378588 shutdown_connections 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 -- 172.21.15.44:0/4198378588 wait complete. 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 Processor -- start 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 -- start start 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83041039e0 0x7f8304101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8304104300 0x7f8304101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830410ce40 0x7f8304102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8304112ab0 con 0x7f83041039e0 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8304112930 con 0x7f830410ce40 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.366+0000 7f830c4f6640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f8304112c30 con 0x7f8304104300 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f830a292640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83041039e0 0x7f8304101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:59.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f830a292640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83041039e0 0x7f8304101690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45452/0 (socket says 172.21.15.44:45452) 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f830a292640 1 -- 172.21.15.44:0/2943792316 learned_addr learned my addr 172.21.15.44:0/2943792316 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f8309a91640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8304104300 0x7f8304101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f830aa93640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830410ce40 0x7f8304102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f830a292640 1 -- 172.21.15.44:0/2943792316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8304104300 msgr2=0x7f8304101bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f830a292640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8304104300 0x7f8304101bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f830a292640 1 -- 172.21.15.44:0/2943792316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830410ce40 msgr2=0x7f8304102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f830a292640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830410ce40 0x7f8304102110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f830a292640 1 -- 172.21.15.44:0/2943792316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83041a73c0 con 0x7f83041039e0 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f8309a91640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8304104300 0x7f8304101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f830a292640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83041039e0 0x7f8304101690 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8300005940 tx=0x7f83000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f82fb7fe640 1 -- 172.21.15.44:0/2943792316 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830003f030 con 0x7f83041039e0 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f82fb7fe640 1 -- 172.21.15.44:0/2943792316 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8300003cb0 con 0x7f83041039e0 2024-08-20T03:12:59.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f830c4f6640 1 -- 172.21.15.44:0/2943792316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83041a7650 con 0x7f83041039e0 2024-08-20T03:12:59.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f82fb7fe640 1 -- 172.21.15.44:0/2943792316 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300035070 con 0x7f83041039e0 2024-08-20T03:12:59.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.367+0000 7f830c4f6640 1 -- 172.21.15.44:0/2943792316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83041a7b30 con 0x7f83041039e0 2024-08-20T03:12:59.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.368+0000 7f830c4f6640 1 -- 172.21.15.44:0/2943792316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8304103dc0 con 0x7f83041039e0 2024-08-20T03:12:59.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.370+0000 7f82fb7fe640 1 -- 172.21.15.44:0/2943792316 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f8300042030 con 0x7f83041039e0 2024-08-20T03:12:59.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.370+0000 7f82fb7fe640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f82f00b3d40 0x7f82f00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:12:59.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.371+0000 7f8309a91640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f82f00b3d40 0x7f82f00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:12:59.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.371+0000 7f82fb7fe640 1 -- 172.21.15.44:0/2943792316 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7f8300104c60 con 0x7f83041039e0 2024-08-20T03:12:59.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.371+0000 7f8309a91640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f82f00b3d40 0x7f82f00b6200 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005fd0 tx=0x7f82f4005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:12:59.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.374+0000 7f82fb7fe640 1 -- 172.21.15.44:0/2943792316 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f83000cbf50 con 0x7f83041039e0 2024-08-20T03:12:59.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.562+0000 7f830c4f6640 1 -- 172.21.15.44:0/2943792316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83041a7ea0 con 0x7f83041039e0 2024-08-20T03:12:59.566 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:12:59 smithi044 ceph-mon[18790]: pgmap v643: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-08-20T03:12:59.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.564+0000 7f82fb7fe640 1 -- 172.21.15.44:0/2943792316 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23044 (secure 0 0 0) 0x7f83000d4220 con 0x7f83041039e0 2024-08-20T03:12:59.567 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-08-20T03:12:59.568 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":85,"fsid":"0284e16c-5e9f-11ef-bccf-c7b262605968","created":"2024-08-20T02:52:10.979264+0000","modified":"2024-08-20T03:12:41.815232+0000","last_up_change":"2024-08-20T02:59:26.404917+0000","last_in_change":"2024-08-20T02:59:03.707380+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T02:55:46.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T03:00:14.354568+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T03:00:16.869335+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7708f8c-a799-4b8f-ab1e-78482d73ca1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6803","nonce":858898316}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6805","nonce":858898316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6809","nonce":858898316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":858898316},{"type":"v1","addr":"172.21.15.44:6807","nonce":858898316}]},"public_addr":"172.21.15.44:6803/858898316","cluster_addr":"172.21.15.44:6805/858898316","heartbeat_back_addr":"172.21.15.44:6809/858898316","heartbeat_front_addr":"172.21.15.44:6807/858898316","state":["exists","up"]},{"osd":1,"uuid":"12f07986-60bf-498f-a012-cd7ea607b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6801","nonce":4252911663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6803","nonce":4252911663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6807","nonce":4252911663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4252911663},{"type":"v1","addr":"172.21.15.99:6805","nonce":4252911663}]},"public_addr":"172.21.15.99:6801/4252911663","cluster_addr":"172.21.15.99:6803/4252911663","heartbeat_back_addr":"172.21.15.99:6807/4252911663","heartbeat_front_addr":"172.21.15.99:6805/4252911663","state":["exists","up"]},{"osd":2,"uuid":"0f189000-ba7f-4412-b0d3-75e8265f6685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6801","nonce":1492740465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6803","nonce":1492740465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6807","nonce":1492740465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":1492740465},{"type":"v1","addr":"172.21.15.165:6805","nonce":1492740465}]},"public_addr":"172.21.15.165:6801/1492740465","cluster_addr":"172.21.15.165:6803/1492740465","heartbeat_back_addr":"172.21.15.165:6807/1492740465","heartbeat_front_addr":"172.21.15.165:6805/1492740465","state":["exists","up"]},{"osd":3,"uuid":"8f9b042c-d6b5-4eee-aa29-ffc41a48c20a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6811","nonce":3423983702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6813","nonce":3423983702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6817","nonce":3423983702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":3423983702},{"type":"v1","addr":"172.21.15.44:6815","nonce":3423983702}]},"public_addr":"172.21.15.44:6811/3423983702","cluster_addr":"172.21.15.44:6813/3423983702","heartbeat_back_addr":"172.21.15.44:6817/3423983702","heartbeat_front_addr":"172.21.15.44:6815/3423983702","state":["exists","up"]},{"osd":4,"uuid":"4edebfdb-af10-49ee-b724-2a63b9219221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6809","nonce":3099617325}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6811","nonce":3099617325}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6815","nonce":3099617325}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":3099617325},{"type":"v1","addr":"172.21.15.99:6813","nonce":3099617325}]},"public_addr":"172.21.15.99:6809/3099617325","cluster_addr":"172.21.15.99:6811/3099617325","heartbeat_back_addr":"172.21.15.99:6815/3099617325","heartbeat_front_addr":"172.21.15.99:6813/3099617325","state":["exists","up"]},{"osd":5,"uuid":"6d22c21f-cc43-4ef5-a75c-d4fc1a361c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6809","nonce":523245255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6811","nonce":523245255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6815","nonce":523245255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":523245255},{"type":"v1","addr":"172.21.15.165:6813","nonce":523245255}]},"public_addr":"172.21.15.165:6809/523245255","cluster_addr":"172.21.15.165:6811/523245255","heartbeat_back_addr":"172.21.15.165:6815/523245255","heartbeat_front_addr":"172.21.15.165:6813/523245255","state":["exists","up"]},{"osd":6,"uuid":"03df0c2f-1b7c-4242-888c-e047322fd8f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6819","nonce":1039098986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6821","nonce":1039098986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6825","nonce":1039098986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":1039098986},{"type":"v1","addr":"172.21.15.44:6823","nonce":1039098986}]},"public_addr":"172.21.15.44:6819/1039098986","cluster_addr":"172.21.15.44:6821/1039098986","heartbeat_back_addr":"172.21.15.44:6825/1039098986","heartbeat_front_addr":"172.21.15.44:6823/1039098986","state":["exists","up"]},{"osd":7,"uuid":"84bbc683-9b13-448e-897c-114458dae33b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6817","nonce":1438613564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6819","nonce":1438613564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6823","nonce":1438613564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1438613564},{"type":"v1","addr":"172.21.15.99:6821","nonce":1438613564}]},"public_addr":"172.21.15.99:6817/1438613564","cluster_addr":"172.21.15.99:6819/1438613564","heartbeat_back_addr":"172.21.15.99:6823/1438613564","heartbeat_front_addr":"172.21.15.99:6821/1438613564","state":["exists","up"]},{"osd":8,"uuid":"14d7ffd3-db05-4228-a12e-4c69ef65ef8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6817","nonce":632957688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6819","nonce":632957688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6823","nonce":632957688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":632957688},{"type":"v1","addr":"172.21.15.165:6821","nonce":632957688}]},"public_addr":"172.21.15.165:6817/632957688","cluster_addr":"172.21.15.165:6819/632957688","heartbeat_back_addr":"172.21.15.165:6823/632957688","heartbeat_front_addr":"172.21.15.165:6821/632957688","state":["exists","up"]},{"osd":9,"uuid":"1ae468a5-3d72-4eb0-a1a6-cd87eae740b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6827","nonce":2426940093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6829","nonce":2426940093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6833","nonce":2426940093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2426940093},{"type":"v1","addr":"172.21.15.44:6831","nonce":2426940093}]},"public_addr":"172.21.15.44:6827/2426940093","cluster_addr":"172.21.15.44:6829/2426940093","heartbeat_back_addr":"172.21.15.44:6833/2426940093","heartbeat_front_addr":"172.21.15.44:6831/2426940093","state":["exists","up"]},{"osd":10,"uuid":"fbf8bb7c-96ad-4372-ad9e-d0eb9c25b0b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6825","nonce":3275793038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6827","nonce":3275793038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6831","nonce":3275793038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":3275793038},{"type":"v1","addr":"172.21.15.99:6829","nonce":3275793038}]},"public_addr":"172.21.15.99:6825/3275793038","cluster_addr":"172.21.15.99:6827/3275793038","heartbeat_back_addr":"172.21.15.99:6831/3275793038","heartbeat_front_addr":"172.21.15.99:6829/3275793038","state":["exists","up"]},{"osd":11,"uuid":"50715410-1fc2-4818-b600-604a4b7cce9a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6825","nonce":107688415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6827","nonce":107688415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6831","nonce":107688415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":107688415},{"type":"v1","addr":"172.21.15.165:6829","nonce":107688415}]},"public_addr":"172.21.15.165:6825/107688415","cluster_addr":"172.21.15.165:6827/107688415","heartbeat_back_addr":"172.21.15.165:6831/107688415","heartbeat_front_addr":"172.21.15.165:6829/107688415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:54:56.697075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:20.486846+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:55:43.353376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:08.333879+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:32.186492+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:56:55.891366+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:19.681421+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:57:45.475926+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:10.109795+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:34.444169+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:58:59.147646+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T02:59:25.403071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.44:0/2597827045":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/3582612619":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/1916809746":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2919033419":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:6800/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2331034359":"2024-08-21T02:52:33.983380+0000","172.21.15.44:6801/1961932124":"2024-08-21T02:52:33.983380+0000","172.21.15.44:0/2572185146":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/1293546305":"2024-08-21T02:53:58.312659+0000","172.21.15.44:0/4176802819":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6800/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/3654439062":"2024-08-21T02:52:58.451601+0000","172.21.15.44:6801/3842608703":"2024-08-21T02:52:58.451601+0000","172.21.15.44:0/2376733275":"2024-08-21T02:53:58.312659+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T03:12:59.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.567+0000 7f830c4f6640 1 -- 172.21.15.44:0/2943792316 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f82f00b3d40 msgr2=0x7f82f00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:59.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.567+0000 7f830c4f6640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f82f00b3d40 0x7f82f00b6200 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005fd0 tx=0x7f82f4005850 comp rx=0 tx=0).stop 2024-08-20T03:12:59.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.567+0000 7f830c4f6640 1 -- 172.21.15.44:0/2943792316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83041039e0 msgr2=0x7f8304101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:12:59.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.567+0000 7f830c4f6640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83041039e0 0x7f8304101690 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8300005940 tx=0x7f83000042f0 comp rx=0 tx=0).stop 2024-08-20T03:12:59.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.567+0000 7f830c4f6640 1 -- 172.21.15.44:0/2943792316 shutdown_connections 2024-08-20T03:12:59.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.567+0000 7f830c4f6640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f82f00b3d40 0x7f82f00b6200 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:59.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.567+0000 7f830c4f6640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f830410ce40 0x7f8304102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:59.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.567+0000 7f830c4f6640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8304104300 0x7f8304101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:59.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.567+0000 7f830c4f6640 1 --2- 172.21.15.44:0/2943792316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83041039e0 0x7f8304101690 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:12:59.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.567+0000 7f830c4f6640 1 -- 172.21.15.44:0/2943792316 >> 172.21.15.44:0/2943792316 conn(0x7f83040fd440 msgr2=0x7f83040fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:12:59.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.567+0000 7f830c4f6640 1 -- 172.21.15.44:0/2943792316 shutdown_connections 2024-08-20T03:12:59.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:12:59.567+0000 7f830c4f6640 1 -- 172.21.15.44:0/2943792316 wait complete. 2024-08-20T03:12:59.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:12:59 smithi165 ceph-mon[22357]: pgmap v643: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-08-20T03:12:59.914 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:12:59 smithi099 ceph-mon[21984]: pgmap v643: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-08-20T03:13:00.300 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs fail cephfs 2024-08-20T03:13:00.535 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:00 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2943792316' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:13:00.596 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:13:00.881 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:00 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2943792316' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:13:00.885 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:00 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2943792316' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T03:13:01.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.108+0000 7fd4b285d640 1 -- 172.21.15.44:0/3012701373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ac108e40 msgr2=0x7fd4ac10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:01.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.108+0000 7fd4b285d640 1 --2- 172.21.15.44:0/3012701373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ac108e40 0x7fd4ac10f640 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009ca0 tx=0x7fd4a402a8c0 comp rx=0 tx=0).stop 2024-08-20T03:13:01.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 -- 172.21.15.44:0/3012701373 shutdown_connections 2024-08-20T03:13:01.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 --2- 172.21.15.44:0/3012701373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ac108e40 0x7fd4ac10f640 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:01.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 --2- 172.21.15.44:0/3012701373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ac1084c0 0x7fd4ac108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:01.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 --2- 172.21.15.44:0/3012701373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4ac102ab0 0x7fd4ac102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:01.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 -- 172.21.15.44:0/3012701373 >> 172.21.15.44:0/3012701373 conn(0x7fd4ac0fe500 msgr2=0x7fd4ac100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 -- 172.21.15.44:0/3012701373 shutdown_connections 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 -- 172.21.15.44:0/3012701373 wait complete. 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 Processor -- start 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 -- start start 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4ac102ab0 0x7fd4ac198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ac1084c0 0x7fd4ac1994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ac108e40 0x7fd4ac19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac112b00 con 0x7fd4ac108e40 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac112980 con 0x7fd4ac102ab0 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b285d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac112c80 con 0x7fd4ac1084c0 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.109+0000 7fd4b0dfa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ac108e40 0x7fd4ac19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4b0dfa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ac108e40 0x7fd4ac19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45464/0 (socket says 172.21.15.44:45464) 2024-08-20T03:13:01.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4b0dfa640 1 -- 172.21.15.44:0/3389247601 learned_addr learned my addr 172.21.15.44:0/3389247601 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4abfff640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4ac102ab0 0x7fd4ac198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4a3fff640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ac1084c0 0x7fd4ac1994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4b0dfa640 1 -- 172.21.15.44:0/3389247601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ac1084c0 msgr2=0x7fd4ac1994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4b0dfa640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ac1084c0 0x7fd4ac1994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4b0dfa640 1 -- 172.21.15.44:0/3389247601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4ac102ab0 msgr2=0x7fd4ac198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4b0dfa640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4ac102ab0 0x7fd4ac198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4b0dfa640 1 -- 172.21.15.44:0/3389247601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ac19f060 con 0x7fd4ac108e40 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4abfff640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4ac102ab0 0x7fd4ac198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4a3fff640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ac1084c0 0x7fd4ac1994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4b0dfa640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ac108e40 0x7fd4ac19e890 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4005cc0 tx=0x7fd4a402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4a9ffb640 1 -- 172.21.15.44:0/3389247601 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4003bf0 con 0x7fd4ac108e40 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4b285d640 1 -- 172.21.15.44:0/3389247601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ac19f2f0 con 0x7fd4ac108e40 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4a9ffb640 1 -- 172.21.15.44:0/3389247601 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a4003d90 con 0x7fd4ac108e40 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4a9ffb640 1 -- 172.21.15.44:0/3389247601 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a402fbc0 con 0x7fd4ac108e40 2024-08-20T03:13:01.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.110+0000 7fd4b285d640 1 -- 172.21.15.44:0/3389247601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ac1a5a90 con 0x7fd4ac108e40 2024-08-20T03:13:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.111+0000 7fd4b285d640 1 -- 172.21.15.44:0/3389247601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ac102e90 con 0x7fd4ac108e40 2024-08-20T03:13:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.113+0000 7fd4a9ffb640 1 -- 172.21.15.44:0/3389247601 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7fd4a402fd60 con 0x7fd4ac108e40 2024-08-20T03:13:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.114+0000 7fd4a9ffb640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4980b3d40 0x7fd4980b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.114+0000 7fd4abfff640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4980b3d40 0x7fd4980b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.114+0000 7fd4a9ffb640 1 -- 172.21.15.44:0/3389247601 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8449+0+0 (secure 0 0 0) 0x7fd4a41056e0 con 0x7fd4ac108e40 2024-08-20T03:13:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.114+0000 7fd4abfff640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4980b3d40 0x7fd4980b6200 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0058c0 tx=0x7fd49c005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:13:01.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.118+0000 7fd4a9ffb640 1 -- 172.21.15.44:0/3389247601 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd4a40ccb40 con 0x7fd4ac108e40 2024-08-20T03:13:01.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.342+0000 7fd4b285d640 1 -- 172.21.15.44:0/3389247601 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd4ac10ac80 con 0x7fd4ac108e40 2024-08-20T03:13:01.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.344+0000 7fd4a9ffb640 1 -- 172.21.15.44:0/3389247601 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 161+0+0 (secure 0 0 0) 0x7fd4a40098b0 con 0x7fd4ac108e40 2024-08-20T03:13:01.348 INFO:teuthology.orchestra.run.smithi044.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-20T03:13:01.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.348+0000 7fd4b285d640 1 -- 172.21.15.44:0/3389247601 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4980b3d40 msgr2=0x7fd4980b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.348+0000 7fd4b285d640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4980b3d40 0x7fd4980b6200 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0058c0 tx=0x7fd49c005850 comp rx=0 tx=0).stop 2024-08-20T03:13:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.348+0000 7fd4b285d640 1 -- 172.21.15.44:0/3389247601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ac108e40 msgr2=0x7fd4ac19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.348+0000 7fd4b285d640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ac108e40 0x7fd4ac19e890 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4005cc0 tx=0x7fd4a402f380 comp rx=0 tx=0).stop 2024-08-20T03:13:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.349+0000 7fd4b285d640 1 -- 172.21.15.44:0/3389247601 shutdown_connections 2024-08-20T03:13:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.349+0000 7fd4b285d640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7fd4980b3d40 0x7fd4980b6200 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.349+0000 7fd4b285d640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4ac108e40 0x7fd4ac19e890 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.349+0000 7fd4b285d640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ac1084c0 0x7fd4ac1994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.349+0000 7fd4b285d640 1 --2- 172.21.15.44:0/3389247601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4ac102ab0 0x7fd4ac198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.349+0000 7fd4b285d640 1 -- 172.21.15.44:0/3389247601 >> 172.21.15.44:0/3389247601 conn(0x7fd4ac0fe500 msgr2=0x7fd4ac1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:13:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.349+0000 7fd4b285d640 1 -- 172.21.15.44:0/3389247601 shutdown_connections 2024-08-20T03:13:01.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:01.349+0000 7fd4b285d640 1 -- 172.21.15.44:0/3389247601 wait complete. 2024-08-20T03:13:01.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-mon[21984]: pgmap v644: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-08-20T03:13:01.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3389247601' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-20T03:13:01.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-mon[21984]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-20T03:13:01.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-mon[21984]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-20T03:13:01.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/3389247601' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-20T03:13:01.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-mon[21984]: osdmap e86: 12 total, 12 up, 12 in 2024-08-20T03:13:01.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-mon[21984]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-08-20T03:13:01.665 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-b[53751]: ignoring --setuser ceph since I am not root 2024-08-20T03:13:01.665 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-b[53751]: ignoring --setgroup ceph since I am not root 2024-08-20T03:13:01.665 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-mds[53755]: -- 172.21.15.99:0/2639541431 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d284df5860 con 0x55d284dd2800 2024-08-20T03:13:01.665 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-b[53751]: starting mds.b at 2024-08-20T03:13:01.666 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h[53448]: ignoring --setuser ceph since I am not root 2024-08-20T03:13:01.666 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h[53448]: ignoring --setgroup ceph since I am not root 2024-08-20T03:13:01.666 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-mds[53452]: -- 172.21.15.99:0/2053119074 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ffcce07860 con 0x55ffccde5400 2024-08-20T03:13:01.666 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h[53448]: starting mds.h at 2024-08-20T03:13:01.666 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-k[52833]: ignoring --setuser ceph since I am not root 2024-08-20T03:13:01.666 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-k[52833]: ignoring --setgroup ceph since I am not root 2024-08-20T03:13:01.666 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-mds[52837]: -- 172.21.15.99:0/2495503085 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa034f9860 con 0x55aa034d7400 2024-08-20T03:13:01.666 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-k[52833]: starting mds.k at 2024-08-20T03:13:01.667 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-e[53147]: ignoring --setuser ceph since I am not root 2024-08-20T03:13:01.667 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-e[53147]: ignoring --setgroup ceph since I am not root 2024-08-20T03:13:01.667 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-mds[53151]: -- 172.21.15.99:0/1077301191 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55eabc5d7520 con 0x55eabc5ab000 2024-08-20T03:13:01.667 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-mds[53151]: -- 172.21.15.99:0/1077301191 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eabc5cd860 con 0x55eabc5ab000 2024-08-20T03:13:01.667 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 03:13:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-e[53147]: starting mds.e at 2024-08-20T03:13:01.685 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-a[63876]: ignoring --setuser ceph since I am not root 2024-08-20T03:13:01.685 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-a[63876]: ignoring --setgroup ceph since I am not root 2024-08-20T03:13:01.685 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-mds[63880]: -- 172.21.15.44:0/402611282 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565b5c2d860 con 0x5565b5c0b000 2024-08-20T03:13:01.686 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-a[63876]: starting mds.a at 2024-08-20T03:13:01.686 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-g[63491]: ignoring --setuser ceph since I am not root 2024-08-20T03:13:01.686 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-g[63491]: ignoring --setgroup ceph since I am not root 2024-08-20T03:13:01.686 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-mds[63508]: -- 172.21.15.44:0/2973017394 <== mon.0 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f6b519860 con 0x561f6b4f7000 2024-08-20T03:13:01.686 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-g[63491]: starting mds.g at 2024-08-20T03:13:01.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a[18786]: 2024-08-20T03:13:01.343+0000 7f1e9073b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-20T03:13:01.686 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-mon[18790]: pgmap v644: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-08-20T03:13:01.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3389247601' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-20T03:13:01.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-mon[18790]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-20T03:13:01.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-mon[18790]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-20T03:13:01.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/3389247601' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-20T03:13:01.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-mon[18790]: osdmap e86: 12 total, 12 up, 12 in 2024-08-20T03:13:01.687 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:01 smithi044 ceph-mon[18790]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-08-20T03:13:01.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-mon[22357]: pgmap v644: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-08-20T03:13:01.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3389247601' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-20T03:13:01.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-mon[22357]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-20T03:13:01.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-mon[22357]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-20T03:13:01.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/3389247601' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-20T03:13:01.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-mon[22357]: osdmap e86: 12 total, 12 up, 12 in 2024-08-20T03:13:01.887 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-mon[22357]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-08-20T03:13:01.887 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f[53859]: ignoring --setuser ceph since I am not root 2024-08-20T03:13:01.887 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f[53859]: ignoring --setgroup ceph since I am not root 2024-08-20T03:13:01.887 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-mds[53863]: -- 172.21.15.165:0/2951122780 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559278725860 con 0x559278703000 2024-08-20T03:13:01.887 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f[53859]: starting mds.f at 2024-08-20T03:13:01.887 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: ignoring --setuser ceph since I am not root 2024-08-20T03:13:01.887 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: ignoring --setgroup ceph since I am not root 2024-08-20T03:13:01.888 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-mds[54463]: -- 172.21.15.165:0/1097853301 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55892c631c20 con 0x55892c60f400 2024-08-20T03:13:01.888 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: starting mds.i at 2024-08-20T03:13:01.888 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l[53546]: ignoring --setuser ceph since I am not root 2024-08-20T03:13:01.888 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l[53546]: ignoring --setgroup ceph since I am not root 2024-08-20T03:13:01.888 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-mds[53550]: -- 172.21.15.165:0/126921397 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f3b1253c20 con 0x55f3b1231400 2024-08-20T03:13:01.888 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l[53546]: starting mds.l at 2024-08-20T03:13:01.888 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c[54158]: ignoring --setuser ceph since I am not root 2024-08-20T03:13:01.888 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c[54158]: ignoring --setgroup ceph since I am not root 2024-08-20T03:13:01.889 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-mds[54162]: -- 172.21.15.165:0/176977856 <== mon.1 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55df30039520 con 0x55df3000d400 2024-08-20T03:13:01.889 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-mds[54162]: -- 172.21.15.165:0/176977856 <== mon.1 v2:172.21.15.165:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df3002f860 con 0x55df3000d400 2024-08-20T03:13:01.889 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 03:13:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c[54158]: starting mds.c at 2024-08-20T03:13:02.145 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-08-20T03:13:02.408 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:13:02.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.914+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/3269803555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff4101990 msgr2=0x7f5ff4101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:02.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.914+0000 7f5ffc3e8640 1 --2- 172.21.15.44:0/3269803555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff4101990 0x7f5ff4101d70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009900 tx=0x7f5ff002a3f0 comp rx=0 tx=0).stop 2024-08-20T03:13:02.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.914+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/3269803555 shutdown_connections 2024-08-20T03:13:02.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.914+0000 7f5ffc3e8640 1 --2- 172.21.15.44:0/3269803555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ff4107d20 0x7f5ff410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:02.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.914+0000 7f5ffc3e8640 1 --2- 172.21.15.44:0/3269803555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ff41073a0 0x7f5ff41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.914+0000 7f5ffc3e8640 1 --2- 172.21.15.44:0/3269803555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff4101990 0x7f5ff4101d70 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.914+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/3269803555 >> 172.21.15.44:0/3269803555 conn(0x7f5ff40fd400 msgr2=0x7f5ff40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:13:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.914+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/3269803555 shutdown_connections 2024-08-20T03:13:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.914+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/3269803555 wait complete. 2024-08-20T03:13:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ffc3e8640 1 Processor -- start 2024-08-20T03:13:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ffc3e8640 1 -- start start 2024-08-20T03:13:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ffc3e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ff4101990 0x7f5ff40fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ffc3e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff41073a0 0x7f5ff40fde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ffc3e8640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ff4107d20 0x7f5ff40fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ffc3e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41119e0 con 0x7f5ff41073a0 2024-08-20T03:13:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ffc3e8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4111860 con 0x7f5ff4101990 2024-08-20T03:13:02.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ffc3e8640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4111b60 con 0x7f5ff4107d20 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ff9983640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff41073a0 0x7f5ff40fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ff9983640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff41073a0 0x7f5ff40fde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45506/0 (socket says 172.21.15.44:45506) 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ff9983640 1 -- 172.21.15.44:0/2960481241 learned_addr learned my addr 172.21.15.44:0/2960481241 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ffa184640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ff4101990 0x7f5ff40fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ffa985640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ff4107d20 0x7f5ff40fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ff9983640 1 -- 172.21.15.44:0/2960481241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ff4107d20 msgr2=0x7f5ff40fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ff9983640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ff4107d20 0x7f5ff40fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ff9983640 1 -- 172.21.15.44:0/2960481241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ff4101990 msgr2=0x7f5ff40fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ff9983640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ff4101990 0x7f5ff40fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ff9983640 1 -- 172.21.15.44:0/2960481241 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff41a7450 con 0x7f5ff41073a0 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.915+0000 7f5ffa985640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ff4107d20 0x7f5ff40fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.916+0000 7f5ff9983640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff41073a0 0x7f5ff40fde40 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400d770 tx=0x7f5fe400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.916+0000 7f5feb7fe640 1 -- 172.21.15.44:0/2960481241 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4017070 con 0x7f5ff41073a0 2024-08-20T03:13:02.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.916+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/2960481241 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff41a7740 con 0x7f5ff41073a0 2024-08-20T03:13:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.916+0000 7f5feb7fe640 1 -- 172.21.15.44:0/2960481241 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe40054d0 con 0x7f5ff41073a0 2024-08-20T03:13:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.916+0000 7f5feb7fe640 1 -- 172.21.15.44:0/2960481241 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4004c30 con 0x7f5ff41073a0 2024-08-20T03:13:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.916+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/2960481241 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff41a7c80 con 0x7f5ff41073a0 2024-08-20T03:13:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.917+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/2960481241 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff4101d70 con 0x7f5ff41073a0 2024-08-20T03:13:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.919+0000 7f5feb7fe640 1 -- 172.21.15.44:0/2960481241 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f5fe4004dd0 con 0x7f5ff41073a0 2024-08-20T03:13:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.919+0000 7f5feb7fe640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5fe00b3dc0 0x7f5fe00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.920+0000 7f5ffa184640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5fe00b3dc0 0x7f5fe00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.920+0000 7f5feb7fe640 1 -- 172.21.15.44:0/2960481241 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8835+0+0 (secure 0 0 0) 0x7f5fe40e3fe0 con 0x7f5ff41073a0 2024-08-20T03:13:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.920+0000 7f5ffa184640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5fe00b3dc0 0x7f5fe00b6280 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0031440 tx=0x7f5ff0005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:13:02.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:02.926+0000 7f5feb7fe640 1 -- 172.21.15.44:0/2960481241 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5fe40ab210 con 0x7f5ff41073a0 2024-08-20T03:13:03.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.158+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/2960481241 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5ff410ab70 con 0x7f5ff41073a0 2024-08-20T03:13:03.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.351+0000 7f5feb7fe640 1 -- 172.21.15.44:0/2960481241 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) ==== 104+0+0 (secure 0 0 0) 0x7f5fe40aab10 con 0x7f5ff41073a0 2024-08-20T03:13:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.353+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/2960481241 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5fe00b3dc0 msgr2=0x7f5fe00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.353+0000 7f5ffc3e8640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5fe00b3dc0 0x7f5fe00b6280 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0031440 tx=0x7f5ff0005b50 comp rx=0 tx=0).stop 2024-08-20T03:13:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.353+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/2960481241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff41073a0 msgr2=0x7f5ff40fde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.353+0000 7f5ffc3e8640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff41073a0 0x7f5ff40fde40 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400d770 tx=0x7f5fe400dc30 comp rx=0 tx=0).stop 2024-08-20T03:13:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.354+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/2960481241 shutdown_connections 2024-08-20T03:13:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.354+0000 7f5ffc3e8640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f5fe00b3dc0 0x7f5fe00b6280 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.354+0000 7f5ffc3e8640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ff4107d20 0x7f5ff40fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.354+0000 7f5ffc3e8640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ff41073a0 0x7f5ff40fde40 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.354+0000 7f5ffc3e8640 1 --2- 172.21.15.44:0/2960481241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ff4101990 0x7f5ff40fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.354+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/2960481241 >> 172.21.15.44:0/2960481241 conn(0x7f5ff40fd400 msgr2=0x7f5ff406ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:13:03.357 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.354+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/2960481241 shutdown_connections 2024-08-20T03:13:03.358 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:03.354+0000 7f5ffc3e8640 1 -- 172.21.15.44:0/2960481241 wait complete. 2024-08-20T03:13:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.44:6838/2154721726,v1:172.21.15.44:6839/2154721726] up:boot 2024-08-20T03:13:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.165:6832/770332932,v1:172.21.15.165:6835/770332932] up:boot 2024-08-20T03:13:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.165:6834/1588169155,v1:172.21.15.165:6837/1588169155] up:boot 2024-08-20T03:13:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.165:6833/1519008852,v1:172.21.15.165:6836/1519008852] up:boot 2024-08-20T03:13:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.44:6840/1707513409,v1:172.21.15.44:6841/1707513409] up:boot 2024-08-20T03:13:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.99:6832/867240580,v1:172.21.15.99:6833/867240580] up:boot 2024-08-20T03:13:03.536 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.99:6834/2247325440,v1:172.21.15.99:6835/2247325440] up:boot 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.99:6836/179989704,v1:172.21.15.99:6837/179989704] up:boot 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.165:6838/3281848973,v1:172.21.15.165:6839/3281848973] up:boot 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: mds.? [v2:172.21.15.99:6838/1607652262,v1:172.21.15.99:6839/1607652262] up:boot 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T03:13:03.537 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T03:13:03.538 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:03 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2960481241' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T03:13:03.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.44:6838/2154721726,v1:172.21.15.44:6839/2154721726] up:boot 2024-08-20T03:13:03.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.165:6832/770332932,v1:172.21.15.165:6835/770332932] up:boot 2024-08-20T03:13:03.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.165:6834/1588169155,v1:172.21.15.165:6837/1588169155] up:boot 2024-08-20T03:13:03.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.165:6833/1519008852,v1:172.21.15.165:6836/1519008852] up:boot 2024-08-20T03:13:03.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.44:6840/1707513409,v1:172.21.15.44:6841/1707513409] up:boot 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.99:6832/867240580,v1:172.21.15.99:6833/867240580] up:boot 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.99:6834/2247325440,v1:172.21.15.99:6835/2247325440] up:boot 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.99:6836/179989704,v1:172.21.15.99:6837/179989704] up:boot 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.165:6838/3281848973,v1:172.21.15.165:6839/3281848973] up:boot 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: mds.? [v2:172.21.15.99:6838/1607652262,v1:172.21.15.99:6839/1607652262] up:boot 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-20T03:13:03.637 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T03:13:03.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T03:13:03.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T03:13:03.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T03:13:03.638 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:03 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2960481241' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T03:13:03.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.44:6838/2154721726,v1:172.21.15.44:6839/2154721726] up:boot 2024-08-20T03:13:03.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.165:6832/770332932,v1:172.21.15.165:6835/770332932] up:boot 2024-08-20T03:13:03.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.165:6834/1588169155,v1:172.21.15.165:6837/1588169155] up:boot 2024-08-20T03:13:03.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.165:6833/1519008852,v1:172.21.15.165:6836/1519008852] up:boot 2024-08-20T03:13:03.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.44:6840/1707513409,v1:172.21.15.44:6841/1707513409] up:boot 2024-08-20T03:13:03.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.99:6832/867240580,v1:172.21.15.99:6833/867240580] up:boot 2024-08-20T03:13:03.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.99:6834/2247325440,v1:172.21.15.99:6835/2247325440] up:boot 2024-08-20T03:13:03.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.99:6836/179989704,v1:172.21.15.99:6837/179989704] up:boot 2024-08-20T03:13:03.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.165:6838/3281848973,v1:172.21.15.165:6839/3281848973] up:boot 2024-08-20T03:13:03.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: mds.? [v2:172.21.15.99:6838/1607652262,v1:172.21.15.99:6839/1607652262] up:boot 2024-08-20T03:13:03.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-08-20T03:13:03.665 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-20T03:13:03.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T03:13:03.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-20T03:13:03.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T03:13:03.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T03:13:03.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-20T03:13:03.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T03:13:03.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T03:13:03.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T03:13:03.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: from='mgr.14220 172.21.15.44:0/4035587332' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T03:13:03.666 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:03 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2960481241' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T03:13:04.072 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-20T03:13:04.327 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:13:04.619 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:04 smithi044 ceph-mon[18790]: pgmap v646: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-08-20T03:13:04.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:04 smithi044 ceph-mon[18790]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-20T03:13:04.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:04 smithi044 ceph-mon[18790]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-20T03:13:04.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:04 smithi044 ceph-mon[18790]: Cluster is now healthy 2024-08-20T03:13:04.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:04 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2960481241' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-20T03:13:04.620 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:04 smithi044 ceph-mon[18790]: fsmap 12 up:standby 2024-08-20T03:13:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:04 smithi165 ceph-mon[22357]: pgmap v646: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-08-20T03:13:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:04 smithi165 ceph-mon[22357]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-20T03:13:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:04 smithi165 ceph-mon[22357]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-20T03:13:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:04 smithi165 ceph-mon[22357]: Cluster is now healthy 2024-08-20T03:13:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:04 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2960481241' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-20T03:13:04.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:04 smithi165 ceph-mon[22357]: fsmap 12 up:standby 2024-08-20T03:13:04.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:04 smithi099 ceph-mon[21984]: pgmap v646: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-08-20T03:13:04.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:04 smithi099 ceph-mon[21984]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-20T03:13:04.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:04 smithi099 ceph-mon[21984]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-20T03:13:04.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:04 smithi099 ceph-mon[21984]: Cluster is now healthy 2024-08-20T03:13:04.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:04 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2960481241' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-20T03:13:04.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:04 smithi099 ceph-mon[21984]: fsmap 12 up:standby 2024-08-20T03:13:04.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.800+0000 7f2d1891d640 1 -- 172.21.15.44:0/2752762072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d14102ad0 msgr2=0x7f2d14102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:04.818 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.800+0000 7f2d1891d640 1 --2- 172.21.15.44:0/2752762072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d14102ad0 0x7f2d14102eb0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009900 tx=0x7f2d0402a410 comp rx=0 tx=0).stop 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.800+0000 7f2d1891d640 1 -- 172.21.15.44:0/2752762072 shutdown_connections 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.800+0000 7f2d1891d640 1 --2- 172.21.15.44:0/2752762072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d14108e60 0x7f2d1410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.800+0000 7f2d1891d640 1 --2- 172.21.15.44:0/2752762072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d141084e0 0x7f2d14108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.800+0000 7f2d1891d640 1 --2- 172.21.15.44:0/2752762072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d14102ad0 0x7f2d14102eb0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.800+0000 7f2d1891d640 1 -- 172.21.15.44:0/2752762072 >> 172.21.15.44:0/2752762072 conn(0x7f2d140fe540 msgr2=0x7f2d14100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.800+0000 7f2d1891d640 1 -- 172.21.15.44:0/2752762072 shutdown_connections 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.800+0000 7f2d1891d640 1 -- 172.21.15.44:0/2752762072 wait complete. 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.801+0000 7f2d1891d640 1 Processor -- start 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.801+0000 7f2d1891d640 1 -- start start 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.801+0000 7f2d1891d640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d14102ad0 0x7f2d140751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.801+0000 7f2d1891d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d141084e0 0x7f2d14075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.801+0000 7f2d1891d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d14108e60 0x7f2d14075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:04.819 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.801+0000 7f2d1891d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14112b20 con 0x7f2d141084e0 2024-08-20T03:13:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.801+0000 7f2d1891d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141129a0 con 0x7f2d14108e60 2024-08-20T03:13:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.801+0000 7f2d1891d640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14112ca0 con 0x7f2d14102ad0 2024-08-20T03:13:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.801+0000 7f2d0bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d141084e0 0x7f2d14075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.801+0000 7f2d0bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d141084e0 0x7f2d14075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45522/0 (socket says 172.21.15.44:45522) 2024-08-20T03:13:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.801+0000 7f2d0bfff640 1 -- 172.21.15.44:0/2732809077 learned_addr learned my addr 172.21.15.44:0/2732809077 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:13:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.801+0000 7f2d1259c640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d14102ad0 0x7f2d140751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d12d9d640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d14108e60 0x7f2d14075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d0bfff640 1 -- 172.21.15.44:0/2732809077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d14102ad0 msgr2=0x7f2d140751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:04.820 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d0bfff640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d14102ad0 0x7f2d140751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d0bfff640 1 -- 172.21.15.44:0/2732809077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d14108e60 msgr2=0x7f2d14075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d0bfff640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d14108e60 0x7f2d14075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d0bfff640 1 -- 172.21.15.44:0/2732809077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d141a7390 con 0x7f2d141084e0 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d12d9d640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d14108e60 0x7f2d14075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d0bfff640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d141084e0 0x7f2d14075710 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00c970 tx=0x7f2cfc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d0b7fe640 1 -- 172.21.15.44:0/2732809077 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc007bb0 con 0x7f2d141084e0 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d0b7fe640 1 -- 172.21.15.44:0/2732809077 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc007d50 con 0x7f2d141084e0 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d1891d640 1 -- 172.21.15.44:0/2732809077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d141a7680 con 0x7f2d141084e0 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d0b7fe640 1 -- 172.21.15.44:0/2732809077 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc00f450 con 0x7f2d141084e0 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.802+0000 7f2d1891d640 1 -- 172.21.15.44:0/2732809077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d141a7b70 con 0x7f2d141084e0 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.803+0000 7f2d1891d640 1 -- 172.21.15.44:0/2732809077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d14102eb0 con 0x7f2d141084e0 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.805+0000 7f2d0b7fe640 1 -- 172.21.15.44:0/2732809077 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f2cfc004f00 con 0x7f2d141084e0 2024-08-20T03:13:04.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.805+0000 7f2d0b7fe640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2d000b3d40 0x7f2d000b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:04.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.805+0000 7f2d1259c640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2d000b3d40 0x7f2d000b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:04.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.805+0000 7f2d0b7fe640 1 -- 172.21.15.44:0/2732809077 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8835+0+0 (secure 0 0 0) 0x7f2cfc0e4360 con 0x7f2d141084e0 2024-08-20T03:13:04.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.806+0000 7f2d1259c640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2d000b3d40 0x7f2d000b6200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2d04031440 tx=0x7f2d040095f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:13:04.822 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.808+0000 7f2d0b7fe640 1 -- 172.21.15.44:0/2732809077 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2cfc014030 con 0x7f2d141084e0 2024-08-20T03:13:04.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:04.995+0000 7f2d1891d640 1 -- 172.21.15.44:0/2732809077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2d1402e340 con 0x7f2d141084e0 2024-08-20T03:13:05.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.358+0000 7f2d0b7fe640 1 -- 172.21.15.44:0/2732809077 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) ==== 181+0+0 (secure 0 0 0) 0x7f2cfc0ab530 con 0x7f2d141084e0 2024-08-20T03:13:05.361 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' removed 2024-08-20T03:13:05.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.361+0000 7f2d1891d640 1 -- 172.21.15.44:0/2732809077 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2d000b3d40 msgr2=0x7f2d000b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:05.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.361+0000 7f2d1891d640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2d000b3d40 0x7f2d000b6200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2d04031440 tx=0x7f2d040095f0 comp rx=0 tx=0).stop 2024-08-20T03:13:05.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.361+0000 7f2d1891d640 1 -- 172.21.15.44:0/2732809077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d141084e0 msgr2=0x7f2d14075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:05.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.361+0000 7f2d1891d640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d141084e0 0x7f2d14075710 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00c970 tx=0x7f2cfc00ce30 comp rx=0 tx=0).stop 2024-08-20T03:13:05.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.362+0000 7f2d1891d640 1 -- 172.21.15.44:0/2732809077 shutdown_connections 2024-08-20T03:13:05.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.362+0000 7f2d1891d640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f2d000b3d40 0x7f2d000b6200 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:05.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.362+0000 7f2d1891d640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d14108e60 0x7f2d14075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:05.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.362+0000 7f2d1891d640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d141084e0 0x7f2d14075710 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:05.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.362+0000 7f2d1891d640 1 --2- 172.21.15.44:0/2732809077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d14102ad0 0x7f2d140751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:05.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.362+0000 7f2d1891d640 1 -- 172.21.15.44:0/2732809077 >> 172.21.15.44:0/2732809077 conn(0x7f2d140fe540 msgr2=0x7f2d14100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:13:05.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.362+0000 7f2d1891d640 1 -- 172.21.15.44:0/2732809077 shutdown_connections 2024-08-20T03:13:05.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:05.362+0000 7f2d1891d640 1 -- 172.21.15.44:0/2732809077 wait complete. 2024-08-20T03:13:05.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:05 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2732809077' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T03:13:05.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:05 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2732809077' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T03:13:05.741 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:05 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2732809077' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T03:13:06.109 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-20T03:13:06.362 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:06 smithi044 ceph-mon[18790]: pgmap v647: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 31 op/s 2024-08-20T03:13:06.365 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T03:13:06.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:06 smithi165 ceph-mon[22357]: pgmap v647: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 31 op/s 2024-08-20T03:13:06.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:06 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/2732809077' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T03:13:06.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:06 smithi165 ceph-mon[22357]: osdmap e87: 12 total, 12 up, 12 in 2024-08-20T03:13:06.653 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:06 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/2732809077' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T03:13:06.653 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:06 smithi044 ceph-mon[18790]: osdmap e87: 12 total, 12 up, 12 in 2024-08-20T03:13:06.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:06 smithi099 ceph-mon[21984]: pgmap v647: 97 pgs: 97 active+clean; 784 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 31 op/s 2024-08-20T03:13:06.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:06 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/2732809077' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T03:13:06.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:06 smithi099 ceph-mon[21984]: osdmap e87: 12 total, 12 up, 12 in 2024-08-20T03:13:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.887+0000 7f9429d1f640 1 -- 172.21.15.44:0/1976204570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d60 msgr2=0x7f942410e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.887+0000 7f9429d1f640 1 --2- 172.21.15.44:0/1976204570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d60 0x7f942410e560 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f941c009900 tx=0x7f941c02a410 comp rx=0 tx=0).stop 2024-08-20T03:13:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.887+0000 7f9429d1f640 1 -- 172.21.15.44:0/1976204570 shutdown_connections 2024-08-20T03:13:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.887+0000 7f9429d1f640 1 --2- 172.21.15.44:0/1976204570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d60 0x7f942410e560 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.887+0000 7f9429d1f640 1 --2- 172.21.15.44:0/1976204570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94241073e0 0x7f9424107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.887+0000 7f9429d1f640 1 --2- 172.21.15.44:0/1976204570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94241019d0 0x7f9424101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.887+0000 7f9429d1f640 1 -- 172.21.15.44:0/1976204570 >> 172.21.15.44:0/1976204570 conn(0x7f94240fd440 msgr2=0x7f94240ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:13:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.887+0000 7f9429d1f640 1 -- 172.21.15.44:0/1976204570 shutdown_connections 2024-08-20T03:13:06.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.887+0000 7f9429d1f640 1 -- 172.21.15.44:0/1976204570 wait complete. 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.887+0000 7f9429d1f640 1 Processor -- start 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9429d1f640 1 -- start start 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9429d1f640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94241019d0 0x7f94240fdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9429d1f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94241073e0 0x7f94240fe090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9429d1f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d60 0x7f94240fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9429d1f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f94241119e0 con 0x7f9424107d60 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9429d1f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9424111860 con 0x7f94241073e0 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9429d1f640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 -- 0x7f9424111b60 con 0x7f94241019d0 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9423fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d60 0x7f94240fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9423fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d60 0x7f94240fe5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:45544/0 (socket says 172.21.15.44:45544) 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9423fff640 1 -- 172.21.15.44:0/557642630 learned_addr learned my addr 172.21.15.44:0/557642630 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f94237fe640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94241019d0 0x7f94240fdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9422ffd640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94241073e0 0x7f94240fe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:06.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9423fff640 1 -- 172.21.15.44:0/557642630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94241019d0 msgr2=0x7f94240fdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9423fff640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94241019d0 0x7f94240fdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9423fff640 1 -- 172.21.15.44:0/557642630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94241073e0 msgr2=0x7f94240fe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9423fff640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94241073e0 0x7f94240fe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f9423fff640 1 -- 172.21.15.44:0/557642630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94241a7380 con 0x7f9424107d60 2024-08-20T03:13:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.888+0000 7f94237fe640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94241019d0 0x7f94240fdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T03:13:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.889+0000 7f9423fff640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d60 0x7f94240fe5d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f941c004770 tx=0x7f941c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:13:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.889+0000 7f9420ff9640 1 -- 172.21.15.44:0/557642630 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c03f030 con 0x7f9424107d60 2024-08-20T03:13:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.889+0000 7f9420ff9640 1 -- 172.21.15.44:0/557642630 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f941c003aa0 con 0x7f9424107d60 2024-08-20T03:13:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.889+0000 7f9429d1f640 1 -- 172.21.15.44:0/557642630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94241a7610 con 0x7f9424107d60 2024-08-20T03:13:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.889+0000 7f9420ff9640 1 -- 172.21.15.44:0/557642630 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c035070 con 0x7f9424107d60 2024-08-20T03:13:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.889+0000 7f9429d1f640 1 -- 172.21.15.44:0/557642630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94241a7b40 con 0x7f9424107d60 2024-08-20T03:13:06.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.890+0000 7f9429d1f640 1 -- 172.21.15.44:0/557642630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9424101db0 con 0x7f9424107d60 2024-08-20T03:13:06.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.891+0000 7f9420ff9640 1 -- 172.21.15.44:0/557642630 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 27) ==== 151190+0+0 (secure 0 0 0) 0x7f941c042030 con 0x7f9424107d60 2024-08-20T03:13:06.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.893+0000 7f9420ff9640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f940c0b3dc0 0x7f940c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T03:13:06.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.893+0000 7f94237fe640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f940c0b3dc0 0x7f940c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T03:13:06.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.893+0000 7f9420ff9640 1 -- 172.21.15.44:0/557642630 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8302+0+0 (secure 0 0 0) 0x7f941c104770 con 0x7f9424107d60 2024-08-20T03:13:06.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.893+0000 7f94237fe640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f940c0b3dc0 0x7f940c0b6280 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9414009fd0 tx=0x7f9414009680 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T03:13:06.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:06.896+0000 7f9420ff9640 1 -- 172.21.15.44:0/557642630 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f941c0cb4b0 con 0x7f9424107d60 2024-08-20T03:13:07.093 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.090+0000 7f9429d1f640 1 -- 172.21.15.44:0/557642630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f94240fee50 con 0x7f9424107d60 2024-08-20T03:13:07.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.366+0000 7f9420ff9640 1 -- 172.21.15.44:0/557642630 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) ==== 169+0+0 (secure 0 0 0) 0x7f941c0cfa00 con 0x7f9424107d60 2024-08-20T03:13:07.370 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' removed 2024-08-20T03:13:07.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.369+0000 7f9429d1f640 1 -- 172.21.15.44:0/557642630 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f940c0b3dc0 msgr2=0x7f940c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:07.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.369+0000 7f9429d1f640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f940c0b3dc0 0x7f940c0b6280 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9414009fd0 tx=0x7f9414009680 comp rx=0 tx=0).stop 2024-08-20T03:13:07.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.370+0000 7f9429d1f640 1 -- 172.21.15.44:0/557642630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d60 msgr2=0x7f94240fe5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T03:13:07.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.370+0000 7f9429d1f640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d60 0x7f94240fe5d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f941c004770 tx=0x7f941c004270 comp rx=0 tx=0).stop 2024-08-20T03:13:07.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.370+0000 7f9429d1f640 1 -- 172.21.15.44:0/557642630 shutdown_connections 2024-08-20T03:13:07.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.370+0000 7f9429d1f640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.44:6800/587429394,v1:172.21.15.44:6801/587429394] conn(0x7f940c0b3dc0 0x7f940c0b6280 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:07.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.370+0000 7f9429d1f640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9424107d60 0x7f94240fe5d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:07.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.370+0000 7f9429d1f640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94241073e0 0x7f94240fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:07.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.370+0000 7f9429d1f640 1 --2- 172.21.15.44:0/557642630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94241019d0 0x7f94240fdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T03:13:07.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.370+0000 7f9429d1f640 1 -- 172.21.15.44:0/557642630 >> 172.21.15.44:0/557642630 conn(0x7f94240fd440 msgr2=0x7f942406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T03:13:07.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.371+0000 7f9429d1f640 1 -- 172.21.15.44:0/557642630 shutdown_connections 2024-08-20T03:13:07.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-08-20T03:13:07.371+0000 7f9429d1f640 1 -- 172.21.15.44:0/557642630 wait complete. 2024-08-20T03:13:07.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:07 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/557642630' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T03:13:07.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:07 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/557642630' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T03:13:07.720 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:07 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/557642630' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T03:13:08.186 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T03:13:08.225 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T03:13:08.265 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T03:13:08.310 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-08-20T03:13:08.310 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-20T03:13:08.332 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-20T03:13:08.353 DEBUG:teuthology.orchestra.run.smithi165:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-20T03:13:08.374 INFO:tasks.cephadm:Stopping all daemons... 2024-08-20T03:13:08.374 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-08-20T03:13:08.374 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.a 2024-08-20T03:13:08.403 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:08 smithi044 ceph-mon[18790]: pgmap v649: 65 pgs: 65 active+clean; 711 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:13:08.404 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:08 smithi044 ceph-mon[18790]: from='client.? 172.21.15.44:0/557642630' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T03:13:08.404 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:08 smithi044 ceph-mon[18790]: osdmap e88: 12 total, 12 up, 12 in 2024-08-20T03:13:08.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:08 smithi165 ceph-mon[22357]: pgmap v649: 65 pgs: 65 active+clean; 711 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:13:08.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:08 smithi165 ceph-mon[22357]: from='client.? 172.21.15.44:0/557642630' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T03:13:08.636 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:08 smithi165 ceph-mon[22357]: osdmap e88: 12 total, 12 up, 12 in 2024-08-20T03:13:08.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:08 smithi099 ceph-mon[21984]: pgmap v649: 65 pgs: 65 active+clean; 711 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T03:13:08.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:08 smithi099 ceph-mon[21984]: from='client.? 172.21.15.44:0/557642630' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T03:13:08.664 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:08 smithi099 ceph-mon[21984]: osdmap e88: 12 total, 12 up, 12 in 2024-08-20T03:13:08.777 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:08 smithi044 systemd[1]: Stopping Ceph mon.a for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:13:09.035 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:08 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a[18786]: 2024-08-20T03:13:08.772+0000 7f1e95f46640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:13:09.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:08 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a[18786]: 2024-08-20T03:13:08.772+0000 7f1e95f46640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-08-20T03:13:09.036 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:08 smithi044 podman[97908]: 2024-08-20 03:13:08.805073016 +0000 UTC m=+0.161642082 container died b0653d7023932256d30639a66904b211b2a5c4af49c1738c9d97586ca38d97ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-08-20T03:13:09.485 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:09 smithi044 podman[97908]: 2024-08-20 03:13:09.19820121 +0000 UTC m=+0.554770268 container cleanup b0653d7023932256d30639a66904b211b2a5c4af49c1738c9d97586ca38d97ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-08-20T03:13:09.485 INFO:journalctl@ceph.mon.a.smithi044.stdout:Aug 20 03:13:09 smithi044 bash[97908]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-a 2024-08-20T03:13:09.631 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.a.service' 2024-08-20T03:13:09.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:13:09.738 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-08-20T03:13:09.738 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-08-20T03:13:09.738 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.b 2024-08-20T03:13:10.110 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:09 smithi099 systemd[1]: Stopping Ceph mon.b for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:13:10.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:10 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-b[21980]: 2024-08-20T03:13:10.107+0000 7f25aa314640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:13:10.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:10 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-b[21980]: 2024-08-20T03:13:10.107+0000 7f25aa314640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-08-20T03:13:10.414 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:10 smithi099 podman[58512]: 2024-08-20 03:13:10.271650125 +0000 UTC m=+0.340473115 container died 79fb5c25db4d0fa4a71f58e58bd192064e4c982b8b32ba1cae33c5283aaf5bca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:13:10.887 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:10 smithi099 podman[58512]: 2024-08-20 03:13:10.675990054 +0000 UTC m=+0.744813036 container cleanup 79fb5c25db4d0fa4a71f58e58bd192064e4c982b8b32ba1cae33c5283aaf5bca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-08-20T03:13:10.887 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:10 smithi099 bash[58512]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-b 2024-08-20T03:13:11.137 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:10 smithi099 podman[58525]: 2024-08-20 03:13:10.885980836 +0000 UTC m=+0.611124191 container remove 79fb5c25db4d0fa4a71f58e58bd192064e4c982b8b32ba1cae33c5283aaf5bca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:13:11.137 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:11 smithi099 systemd[1]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.b.service: Deactivated successfully. 2024-08-20T03:13:11.137 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:11 smithi099 systemd[1]: Stopped Ceph mon.b for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T03:13:11.137 INFO:journalctl@ceph.mon.b.smithi099.stdout:Aug 20 03:13:11 smithi099 systemd[1]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.b.service: Consumed 11.437s CPU time. 2024-08-20T03:13:11.140 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.b.service' 2024-08-20T03:13:11.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:13:11.193 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-08-20T03:13:11.193 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-08-20T03:13:11.193 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.c 2024-08-20T03:13:11.522 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:11 smithi165 systemd[1]: Stopping Ceph mon.c for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:13:11.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:11 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-c[22353]: 2024-08-20T03:13:11.518+0000 7f02135b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:13:11.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:11 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-c[22353]: 2024-08-20T03:13:11.518+0000 7f02135b5640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-08-20T03:13:11.886 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:11 smithi165 podman[58823]: 2024-08-20 03:13:11.683628651 +0000 UTC m=+0.289718787 container died b9c55899951908ba9ae4fcb88b9b5bd7023b2b3aaadfc278e664f7062d9da6f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-c, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-20T03:13:12.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:12 smithi165 podman[58823]: 2024-08-20 03:13:12.087529469 +0000 UTC m=+0.693619604 container cleanup b9c55899951908ba9ae4fcb88b9b5bd7023b2b3aaadfc278e664f7062d9da6f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-c, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:13:12.386 INFO:journalctl@ceph.mon.c.smithi165.stdout:Aug 20 03:13:12 smithi165 bash[58823]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mon-c 2024-08-20T03:13:12.599 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mon.c.service' 2024-08-20T03:13:12.647 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:13:12.647 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-08-20T03:13:12.647 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-08-20T03:13:12.647 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.x 2024-08-20T03:13:12.972 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:13:12 smithi044 systemd[1]: Stopping Ceph mgr.x for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:13:13.285 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:13:12 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x[19015]: 2024-08-20T03:13:12.968+0000 7efd083a8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:13:13.286 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:13:13 smithi044 podman[98028]: 2024-08-20 03:13:13.000676301 +0000 UTC m=+0.154905974 container died 4db061448d4872ea084afbc766844bd40f6d1d190416bfa0a42b3181efadd755 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:13:13.785 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:13:13 smithi044 podman[98028]: 2024-08-20 03:13:13.395986984 +0000 UTC m=+0.550216640 container cleanup 4db061448d4872ea084afbc766844bd40f6d1d190416bfa0a42b3181efadd755 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-20T03:13:13.785 INFO:journalctl@ceph.mgr.x.smithi044.stdout:Aug 20 03:13:13 smithi044 bash[98028]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-x 2024-08-20T03:13:14.088 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.x.service' 2024-08-20T03:13:14.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:13:14.124 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-08-20T03:13:14.124 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-08-20T03:13:14.124 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.y 2024-08-20T03:13:14.414 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Aug 20 03:13:14 smithi099 systemd[1]: Stopping Ceph mgr.y for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:13:14.775 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Aug 20 03:13:14 smithi099 podman[58633]: 2024-08-20 03:13:14.453064624 +0000 UTC m=+0.140153154 container died 78cf05a7f6f0421bd69d7b3cc2eb43cdef3bf12a00f621a8ca42a97ff438586c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-y, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-08-20T03:13:15.164 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Aug 20 03:13:14 smithi099 podman[58633]: 2024-08-20 03:13:14.881176198 +0000 UTC m=+0.568264733 container cleanup 78cf05a7f6f0421bd69d7b3cc2eb43cdef3bf12a00f621a8ca42a97ff438586c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-08-20T03:13:15.165 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Aug 20 03:13:14 smithi099 bash[58633]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-y 2024-08-20T03:13:15.442 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Aug 20 03:13:15 smithi099 podman[58646]: 2024-08-20 03:13:15.188064244 +0000 UTC m=+0.735884622 container remove 78cf05a7f6f0421bd69d7b3cc2eb43cdef3bf12a00f621a8ca42a97ff438586c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-20T03:13:15.442 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Aug 20 03:13:15 smithi099 systemd[1]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-08-20T03:13:15.456 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.y.service' 2024-08-20T03:13:15.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:13:15.491 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-08-20T03:13:15.491 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-08-20T03:13:15.491 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.z 2024-08-20T03:13:15.840 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Aug 20 03:13:15 smithi165 systemd[1]: Stopping Ceph mgr.z for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:13:16.136 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Aug 20 03:13:15 smithi165 podman[58944]: 2024-08-20 03:13:15.838711405 +0000 UTC m=+0.145706733 container died 1a4c2ed740a443d477e7e04c8d3086601abf45f6ef1ef5a3e1afebd25bc9db39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T03:13:16.505 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Aug 20 03:13:16 smithi165 podman[58944]: 2024-08-20 03:13:16.252169151 +0000 UTC m=+0.559164482 container cleanup 1a4c2ed740a443d477e7e04c8d3086601abf45f6ef1ef5a3e1afebd25bc9db39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-z, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0) 2024-08-20T03:13:16.505 INFO:journalctl@ceph.mgr.z.smithi165.stdout:Aug 20 03:13:16 smithi165 bash[58944]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mgr-z 2024-08-20T03:13:16.633 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mgr.z.service' 2024-08-20T03:13:16.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:13:16.668 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-08-20T03:13:16.668 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-08-20T03:13:16.668 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.0 2024-08-20T03:13:16.995 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:16 smithi044 systemd[1]: Stopping Ceph osd.0 for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:13:17.285 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:16 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0[37020]: 2024-08-20T03:13:16.991+0000 7f1b02aac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:13:17.285 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:16 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0[37020]: 2024-08-20T03:13:16.991+0000 7f1b02aac640 -1 osd.0 88 *** Got signal Terminated *** 2024-08-20T03:13:17.285 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:16 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0[37020]: 2024-08-20T03:13:16.991+0000 7f1b02aac640 -1 osd.0 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T03:13:22.692 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:22 smithi044 podman[98145]: 2024-08-20 03:13:22.410176824 +0000 UTC m=+5.538142855 container died ded8da4ebf474fbdfa85c66fba38d0d20de0f7657eb14519d554452c6feeef83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS) 2024-08-20T03:13:23.282 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:23 smithi044 podman[98145]: 2024-08-20 03:13:23.011255513 +0000 UTC m=+6.139221537 container cleanup ded8da4ebf474fbdfa85c66fba38d0d20de0f7657eb14519d554452c6feeef83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:13:23.282 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:23 smithi044 bash[98145]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0 2024-08-20T03:13:23.535 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:23 smithi044 podman[98162]: 2024-08-20 03:13:23.27974188 +0000 UTC m=+0.868535893 container remove ded8da4ebf474fbdfa85c66fba38d0d20de0f7657eb14519d554452c6feeef83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731) 2024-08-20T03:13:23.943 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:23 smithi044 podman[98259]: 2024-08-20 03:13:23.536945743 +0000 UTC m=+0.016592001 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:13:23.943 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:23 smithi044 podman[98259]: 2024-08-20 03:13:23.664081508 +0000 UTC m=+0.143727762 container create 511080be3a19d2b7d57f6f7beea3a091d21d65c1ccb5b7f189a4e9a2292d668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731) 2024-08-20T03:13:23.944 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:23 smithi044 podman[98259]: 2024-08-20 03:13:23.94199846 +0000 UTC m=+0.421644719 container init 511080be3a19d2b7d57f6f7beea3a091d21d65c1ccb5b7f189a4e9a2292d668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-20T03:13:24.286 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:23 smithi044 podman[98259]: 2024-08-20 03:13:23.945668262 +0000 UTC m=+0.425314520 container start 511080be3a19d2b7d57f6f7beea3a091d21d65c1ccb5b7f189a4e9a2292d668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0-deactivate, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:13:24.286 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:24 smithi044 podman[98259]: 2024-08-20 03:13:24.050603066 +0000 UTC m=+0.530249321 container attach 511080be3a19d2b7d57f6f7beea3a091d21d65c1ccb5b7f189a4e9a2292d668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-20T03:13:24.286 INFO:journalctl@ceph.osd.0.smithi044.stdout:Aug 20 03:13:24 smithi044 podman[98259]: 2024-08-20 03:13:24.140409767 +0000 UTC m=+0.620056023 container died 511080be3a19d2b7d57f6f7beea3a091d21d65c1ccb5b7f189a4e9a2292d668e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-08-20T03:13:25.118 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.0.service' 2024-08-20T03:13:25.164 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:13:25.164 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-08-20T03:13:25.165 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-08-20T03:13:25.165 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.1 2024-08-20T03:13:25.664 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:25 smithi099 systemd[1]: Stopping Ceph osd.1 for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:13:26.164 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:25 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1[27731]: 2024-08-20T03:13:25.760+0000 7f4a0a2bf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:13:26.164 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:25 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1[27731]: 2024-08-20T03:13:25.760+0000 7f4a0a2bf640 -1 osd.1 88 *** Got signal Terminated *** 2024-08-20T03:13:26.164 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:25 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1[27731]: 2024-08-20T03:13:25.760+0000 7f4a0a2bf640 -1 osd.1 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T03:13:31.664 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:31 smithi099 podman[58770]: 2024-08-20 03:13:31.358085206 +0000 UTC m=+6.013739942 container died 2bc4321bd17b8a32c013ded16723e0d2eff123f80d758c675f5a852b64a5a6bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-20T03:13:32.130 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:31 smithi099 podman[58770]: 2024-08-20 03:13:31.89412661 +0000 UTC m=+6.549781344 container cleanup 2bc4321bd17b8a32c013ded16723e0d2eff123f80d758c675f5a852b64a5a6bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-20T03:13:32.131 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:31 smithi099 bash[58770]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1 2024-08-20T03:13:32.414 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:32 smithi099 podman[58786]: 2024-08-20 03:13:32.129307768 +0000 UTC m=+0.767986944 container remove 2bc4321bd17b8a32c013ded16723e0d2eff123f80d758c675f5a852b64a5a6bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-20T03:13:32.835 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:32 smithi099 podman[58874]: 2024-08-20 03:13:32.451841508 +0000 UTC m=+0.016659942 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:13:32.835 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:32 smithi099 podman[58874]: 2024-08-20 03:13:32.564034179 +0000 UTC m=+0.128852612 container create 582941482ee7903ecc74e6ae9f4222e04dc082b3a61032cdd31ca9db5ab07c50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True) 2024-08-20T03:13:33.164 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:32 smithi099 podman[58874]: 2024-08-20 03:13:32.833521999 +0000 UTC m=+0.398340432 container init 582941482ee7903ecc74e6ae9f4222e04dc082b3a61032cdd31ca9db5ab07c50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T03:13:33.165 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:32 smithi099 podman[58874]: 2024-08-20 03:13:32.837204086 +0000 UTC m=+0.402022522 container start 582941482ee7903ecc74e6ae9f4222e04dc082b3a61032cdd31ca9db5ab07c50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-08-20T03:13:33.165 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:32 smithi099 podman[58874]: 2024-08-20 03:13:32.942062171 +0000 UTC m=+0.506880619 container attach 582941482ee7903ecc74e6ae9f4222e04dc082b3a61032cdd31ca9db5ab07c50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-08-20T03:13:33.165 INFO:journalctl@ceph.osd.1.smithi099.stdout:Aug 20 03:13:33 smithi099 podman[58874]: 2024-08-20 03:13:33.024706429 +0000 UTC m=+0.589524869 container died 582941482ee7903ecc74e6ae9f4222e04dc082b3a61032cdd31ca9db5ab07c50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-08-20T03:13:34.092 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.1.service' 2024-08-20T03:13:34.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:13:34.138 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-08-20T03:13:34.138 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-08-20T03:13:34.139 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.2 2024-08-20T03:13:34.447 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:34 smithi165 systemd[1]: Stopping Ceph osd.2 for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:13:34.886 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:34 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2[28587]: 2024-08-20T03:13:34.456+0000 7f842f37f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:13:34.886 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:34 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2[28587]: 2024-08-20T03:13:34.456+0000 7f842f37f640 -1 osd.2 88 *** Got signal Terminated *** 2024-08-20T03:13:34.886 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:34 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2[28587]: 2024-08-20T03:13:34.456+0000 7f842f37f640 -1 osd.2 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T03:13:40.136 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:39 smithi165 podman[59065]: 2024-08-20 03:13:39.870697206 +0000 UTC m=+5.534774475 container died 4f787bbc0eb66c82501c5c953460897353d09a2db5db63218f7f5671d0a72773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-20T03:13:40.636 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:40 smithi165 podman[59065]: 2024-08-20 03:13:40.240661212 +0000 UTC m=+5.904738483 container cleanup 4f787bbc0eb66c82501c5c953460897353d09a2db5db63218f7f5671d0a72773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-20T03:13:40.636 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:40 smithi165 bash[59065]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2 2024-08-20T03:13:41.046 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:40 smithi165 podman[59080]: 2024-08-20 03:13:40.733913264 +0000 UTC m=+0.851392124 container remove 4f787bbc0eb66c82501c5c953460897353d09a2db5db63218f7f5671d0a72773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-20T03:13:41.364 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:41 smithi165 podman[59169]: 2024-08-20 03:13:40.948089913 +0000 UTC m=+0.016136190 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:13:41.364 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:41 smithi165 podman[59169]: 2024-08-20 03:13:41.093368379 +0000 UTC m=+0.161414683 container create 608d026199b4ffc9079256940bf1d27ea84b8738db7ee068692f5d93f5f01e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2) 2024-08-20T03:13:41.364 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:41 smithi165 podman[59169]: 2024-08-20 03:13:41.363112732 +0000 UTC m=+0.431159007 container init 608d026199b4ffc9079256940bf1d27ea84b8738db7ee068692f5d93f5f01e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, ceph=True) 2024-08-20T03:13:41.624 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:41 smithi165 podman[59169]: 2024-08-20 03:13:41.366980028 +0000 UTC m=+0.435026296 container start 608d026199b4ffc9079256940bf1d27ea84b8738db7ee068692f5d93f5f01e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T03:13:41.624 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:41 smithi165 podman[59169]: 2024-08-20 03:13:41.463329709 +0000 UTC m=+0.531375992 container attach 608d026199b4ffc9079256940bf1d27ea84b8738db7ee068692f5d93f5f01e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2-deactivate, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-08-20T03:13:41.886 INFO:journalctl@ceph.osd.2.smithi165.stdout:Aug 20 03:13:41 smithi165 podman[59169]: 2024-08-20 03:13:41.622657323 +0000 UTC m=+0.690703585 container died 608d026199b4ffc9079256940bf1d27ea84b8738db7ee068692f5d93f5f01e2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-20T03:13:42.198 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.2.service' 2024-08-20T03:13:42.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:13:42.236 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-08-20T03:13:42.236 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-08-20T03:13:42.237 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.3 2024-08-20T03:13:42.535 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:42 smithi044 systemd[1]: Stopping Ceph osd.3 for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:13:43.035 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:42 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3[43469]: 2024-08-20T03:13:42.550+0000 7f93fda5f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:13:43.036 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:42 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3[43469]: 2024-08-20T03:13:42.550+0000 7f93fda5f640 -1 osd.3 88 *** Got signal Terminated *** 2024-08-20T03:13:43.036 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:42 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3[43469]: 2024-08-20T03:13:42.550+0000 7f93fda5f640 -1 osd.3 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T03:13:43.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:43 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:43.351+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:43.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:43 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:43.403+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:44.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:44 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:44.302+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:44.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:44 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:44.437+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:45.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:44 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:44.749+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:45.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:45 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:45.321+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:45.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:45 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:45.402+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:46.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:45 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:45.766+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:46.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:45 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:45.748+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:46.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:46 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:46.329+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:46.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:46 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:46.391+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:47.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:46 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:46.759+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:47.285 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:46 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:46.809+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:47.286 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:47 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:47.005+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:47.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:47 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:47.305+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:47.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:47 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:47.440+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:48.087 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:47 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:47.971+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:48.087 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:47 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:47.787+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:48.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:47 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:47.743+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:48.369 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:48 smithi044 podman[98363]: 2024-08-20 03:13:48.085507086 +0000 UTC m=+5.648987981 container died a56414a543fbbbed00f3eff682b8257aaa1ce816af759748f77f54cede865450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T03:13:48.638 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:48 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:48.332+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:48.785 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:48 smithi044 podman[98363]: 2024-08-20 03:13:48.567569747 +0000 UTC m=+6.131050683 container cleanup a56414a543fbbbed00f3eff682b8257aaa1ce816af759748f77f54cede865450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-20T03:13:48.786 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:48 smithi044 bash[98363]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3 2024-08-20T03:13:48.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:48 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:48.462+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:49.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:48 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:48.758+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:49.190 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:49 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:49.013+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:49.190 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:48 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:48.791+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:49.190 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:48 smithi044 podman[98380]: 2024-08-20 03:13:48.85278372 +0000 UTC m=+0.766676268 container remove a56414a543fbbbed00f3eff682b8257aaa1ce816af759748f77f54cede865450 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-20T03:13:49.483 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:49 smithi044 podman[98466]: 2024-08-20 03:13:49.091200216 +0000 UTC m=+0.015883312 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:13:49.483 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:49 smithi044 podman[98466]: 2024-08-20 03:13:49.220325476 +0000 UTC m=+0.145008610 container create a87a55de42d0579d8665d4a55af99a7cdbd3d422d585be4aede7bac2ac685fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3-deactivate, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T03:13:49.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:49 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:49.322+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:49.785 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:49 smithi044 podman[98466]: 2024-08-20 03:13:49.481183378 +0000 UTC m=+0.405866471 container init a87a55de42d0579d8665d4a55af99a7cdbd3d422d585be4aede7bac2ac685fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS) 2024-08-20T03:13:49.785 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:49 smithi044 podman[98466]: 2024-08-20 03:13:49.484880423 +0000 UTC m=+0.409563516 container start a87a55de42d0579d8665d4a55af99a7cdbd3d422d585be4aede7bac2ac685fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-20T03:13:49.786 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:49 smithi044 podman[98466]: 2024-08-20 03:13:49.589633887 +0000 UTC m=+0.514316987 container attach a87a55de42d0579d8665d4a55af99a7cdbd3d422d585be4aede7bac2ac685fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T03:13:49.786 INFO:journalctl@ceph.osd.3.smithi044.stdout:Aug 20 03:13:49 smithi044 podman[98466]: 2024-08-20 03:13:49.68544896 +0000 UTC m=+0.610132053 container died a87a55de42d0579d8665d4a55af99a7cdbd3d422d585be4aede7bac2ac685fdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240731) 2024-08-20T03:13:49.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:49 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:49.474+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:50.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:49 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:49.720+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:50.285 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:49 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:49.971+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:50.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:49 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:49.800+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:50.336 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.3.service' 2024-08-20T03:13:50.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:13:50.377 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-08-20T03:13:50.377 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-08-20T03:13:50.377 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.4 2024-08-20T03:13:50.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:50 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:50.319+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:50.664 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:50 smithi099 systemd[1]: Stopping Ceph osd.4 for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:13:50.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:50 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:50.491+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:51.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:50 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:50.970+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:51.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:50 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:50.760+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:51.135 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:50 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:50.747+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:51.164 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:50 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4[34618]: 2024-08-20T03:13:50.706+0000 7fa49e582640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:13:51.164 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:50 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4[34618]: 2024-08-20T03:13:50.706+0000 7fa49e582640 -1 osd.4 88 *** Got signal Terminated *** 2024-08-20T03:13:51.164 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:50 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4[34618]: 2024-08-20T03:13:50.706+0000 7fa49e582640 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T03:13:51.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:51 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:51.369+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:51.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:51 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:51.484+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:52.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:51 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:51.739+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:52.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:51 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:51.932+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:52.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:51 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:51.932+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:13:52.135 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:51 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:51.718+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:52.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:52 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:52.340+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:52.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:52 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:52.522+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:53.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:52 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:52.760+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:53.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:52 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:52.973+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:53.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:52 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:52.973+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:13:53.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:52 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:52.727+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:53.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:53 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:53.292+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:53.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:53 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:53.551+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:53.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:13:53 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:13:53.307+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:13:54.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:53 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:53.934+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:54.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:53 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:53.934+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:13:54.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:53 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:53.743+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:54.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:53 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:53.735+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:54.355 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:54 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4[34618]: 2024-08-20T03:13:54.121+0000 7fa49ab97640 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.313460+0000 front 2024-08-20T03:13:30.313434+0000 (oldest deadline 2024-08-20T03:13:53.213557+0000) 2024-08-20T03:13:54.569 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:54 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:54.296+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:54.569 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:54 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:54.296+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:13:54.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:13:54 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:13:54.353+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:13:54.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:54 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:54.513+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:54.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:13:54 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:13:54.568+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:13:54.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:54 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:54.760+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:54.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:54 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:54.760+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:29.447532+0000 front 2024-08-20T03:13:29.447430+0000 (oldest deadline 2024-08-20T03:13:54.147273+0000) 2024-08-20T03:13:55.188 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:54 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:54.932+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:55.188 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:54 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:54.932+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:13:55.188 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:54 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:54.790+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:55.188 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:54 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:54.790+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:13:55.415 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:55 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4[34618]: 2024-08-20T03:13:55.134+0000 7fa49ab97640 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.313460+0000 front 2024-08-20T03:13:30.313434+0000 (oldest deadline 2024-08-20T03:13:53.213557+0000) 2024-08-20T03:13:55.415 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:13:55 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:13:55.360+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:13:55.571 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:55 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:55.299+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:55.572 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:55 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:55.299+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:13:55.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:13:55 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:13:55.570+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:13:55.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:55 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:55.727+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:55.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:55 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:55.727+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:29.447532+0000 front 2024-08-20T03:13:29.447430+0000 (oldest deadline 2024-08-20T03:13:54.147273+0000) 2024-08-20T03:13:55.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:55 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:55.501+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:55.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:55 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:55.501+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:13:56.285 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:55 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:55.914+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:56.286 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:55 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:55.914+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:13:56.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:55 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:55.802+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:56.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:55 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:55.802+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:13:56.608 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:13:56 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:13:56.405+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:13:56.609 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:56 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:56.499+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:56.609 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:56 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:56.499+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:13:56.609 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:56 smithi099 podman[58976]: 2024-08-20 03:13:56.295311432 +0000 UTC m=+5.726833208 container died d75d62af8a8bacb01f9acb63bbbfeee2c1bc71b78fbf78fcc13a5b3d8ed66eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-08-20T03:13:56.616 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:56 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:56.323+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:56.616 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:56 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:56.323+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:13:56.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:13:56 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:13:56.615+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:13:56.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:56 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:56.706+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:56.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:56 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:56.706+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:29.447532+0000 front 2024-08-20T03:13:29.447430+0000 (oldest deadline 2024-08-20T03:13:54.147273+0000) 2024-08-20T03:13:56.914 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:56 smithi099 podman[58976]: 2024-08-20 03:13:56.698636473 +0000 UTC m=+6.130158257 container cleanup d75d62af8a8bacb01f9acb63bbbfeee2c1bc71b78fbf78fcc13a5b3d8ed66eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-08-20T03:13:56.914 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:56 smithi099 bash[58976]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4 2024-08-20T03:13:57.260 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:56 smithi099 podman[59005]: 2024-08-20 03:13:56.925344753 +0000 UTC m=+0.627592908 container remove d75d62af8a8bacb01f9acb63bbbfeee2c1bc71b78fbf78fcc13a5b3d8ed66eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS) 2024-08-20T03:13:57.285 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:56 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:56.811+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:57.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:56 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:56.811+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:13:57.286 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:56 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:56.892+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:57.286 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:56 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:56.892+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:13:57.530 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:13:57 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:13:57.395+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:13:57.531 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:57 smithi099 podman[59096]: 2024-08-20 03:13:57.1613249 +0000 UTC m=+0.016522786 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:13:57.531 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:57 smithi099 podman[59096]: 2024-08-20 03:13:57.271165242 +0000 UTC m=+0.126363116 container create bc3ce4c46da961dc65279c99bec3948d0719239c5243fd474ac66a7388bb280c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4-deactivate, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-20T03:13:57.586 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:57 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:57.302+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:57.586 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:57 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:57.302+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:13:57.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:13:57 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:13:57.584+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:13:57.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:57 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:57.662+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:57.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:57 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:57.662+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:29.447532+0000 front 2024-08-20T03:13:29.447430+0000 (oldest deadline 2024-08-20T03:13:54.147273+0000) 2024-08-20T03:13:57.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:57 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:57.543+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:57.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:57 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:57.543+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:13:57.915 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:57 smithi099 podman[59096]: 2024-08-20 03:13:57.529400516 +0000 UTC m=+0.384598396 container init bc3ce4c46da961dc65279c99bec3948d0719239c5243fd474ac66a7388bb280c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-08-20T03:13:57.915 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:57 smithi099 podman[59096]: 2024-08-20 03:13:57.532849475 +0000 UTC m=+0.388047350 container start bc3ce4c46da961dc65279c99bec3948d0719239c5243fd474ac66a7388bb280c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-20T03:13:57.915 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:57 smithi099 podman[59096]: 2024-08-20 03:13:57.737725588 +0000 UTC m=+0.592923473 container attach bc3ce4c46da961dc65279c99bec3948d0719239c5243fd474ac66a7388bb280c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T03:13:57.915 INFO:journalctl@ceph.osd.4.smithi099.stdout:Aug 20 03:13:57 smithi099 podman[59096]: 2024-08-20 03:13:57.738004491 +0000 UTC m=+0.593202375 container died bc3ce4c46da961dc65279c99bec3948d0719239c5243fd474ac66a7388bb280c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-08-20T03:13:58.285 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:57 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:57.850+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:58.286 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:57 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:57.850+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:13:58.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:57 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:57.833+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:58.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:57 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:57.833+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:13:58.388 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.4.service' 2024-08-20T03:13:58.429 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:13:58.429 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-08-20T03:13:58.429 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-08-20T03:13:58.429 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.5 2024-08-20T03:13:58.627 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:58 smithi165 systemd[1]: Stopping Ceph osd.5 for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:13:58.627 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:58 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:58.300+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:58.627 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:58 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:58.300+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:13:58.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:13:58 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:13:58.405+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:13:58.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:58 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:58.581+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:58.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:58 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:58.581+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:13:58.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:58 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:58.685+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:13:58.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:58 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:58.685+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:29.447532+0000 front 2024-08-20T03:13:29.447430+0000 (oldest deadline 2024-08-20T03:13:54.147273+0000) 2024-08-20T03:13:58.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:58 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:58.745+0000 7f94f5b32640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:13:58.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:58 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:58.745+0000 7f94f5b32640 -1 osd.5 88 *** Got signal Terminated *** 2024-08-20T03:13:58.886 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:58 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:58.745+0000 7f94f5b32640 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T03:13:58.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:13:58 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:13:58.625+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:13:59.285 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:58 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:58.851+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:13:59.285 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:58 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:58.851+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:13:59.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:58 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:58.809+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:13:59.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:58 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:58.809+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:13:59.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:59 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:59.299+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:13:59.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:13:59 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:13:59.299+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:13:59.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:13:59 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:13:59.370+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:13:59.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:59 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:59.594+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:13:59.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:13:59 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:13:59.594+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:00.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:59 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:59.899+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:14:00.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:13:59 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:13:59.899+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:14:00.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:59 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:59.781+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:00.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:13:59 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:13:59.781+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:00.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:13:59 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:13:59.672+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:00.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:59 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:59.639+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:14:00.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:13:59 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:13:59.639+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:29.447532+0000 front 2024-08-20T03:13:29.447430+0000 (oldest deadline 2024-08-20T03:13:54.147273+0000) 2024-08-20T03:14:00.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:00 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:00.301+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:00.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:00 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:00.301+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:00.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:00 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:00.301+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:00.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:00 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:00.392+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:00.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:00 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:00.570+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:00.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:00 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:00.570+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:01.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:00 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:00.658+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:01.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:00 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:14:00.667+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:14:01.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:00 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:14:00.667+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:29.447532+0000 front 2024-08-20T03:13:29.447430+0000 (oldest deadline 2024-08-20T03:13:54.147273+0000) 2024-08-20T03:14:01.285 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:00.905+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:14:01.286 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:00.905+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:14:01.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:00.810+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:01.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:00.810+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:01.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:00 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:00.810+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:01.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:01.288+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:01.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:01.288+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:01.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:01.288+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:01.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:01.402+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:01.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:01.549+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:01.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:01.549+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:01.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:01 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:01.549+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:02.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:01.858+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:14:02.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:01.858+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:14:02.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:01.765+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:02.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:01.765+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:02.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:01 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:01.765+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:02.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:01.679+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:02.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:14:01.667+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:14:02.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:01 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:14:01.667+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:29.447532+0000 front 2024-08-20T03:13:29.447430+0000 (oldest deadline 2024-08-20T03:13:54.147273+0000) 2024-08-20T03:14:02.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:02 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:02.336+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:02.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:02 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:02.336+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:02.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:02 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:02.336+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:02.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:02 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:02.371+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:02.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:02 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:02.371+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:02.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:02 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:02.561+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:02.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:02 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:02.561+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:02.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:02 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:02.561+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:03.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:02 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:02.893+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:14:03.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:02 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:02.893+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:14:03.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:02 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:02.893+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.266275+0000 front 2024-08-20T03:13:38.266293+0000 (oldest deadline 2024-08-20T03:14:02.366431+0000) 2024-08-20T03:14:03.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:02 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:02.743+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:03.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:02 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:02.743+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:03.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:02 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:02.743+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:03.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:02 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:02.702+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:03.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:02 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:14:02.670+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:14:03.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:02 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:14:02.670+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:29.447532+0000 front 2024-08-20T03:13:29.447430+0000 (oldest deadline 2024-08-20T03:13:54.147273+0000) 2024-08-20T03:14:03.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:02 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:14:02.670+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.748700+0000 front 2024-08-20T03:13:38.748683+0000 (oldest deadline 2024-08-20T03:14:02.248686+0000) 2024-08-20T03:14:03.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:03 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:03.373+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:03.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:03 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:03.373+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:03.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:03 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:03.373+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:03.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:03 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:03.322+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:03.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:03 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:03.322+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:03.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:03 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:03.542+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:03.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:03 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:03.542+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:03.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:03 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:03.542+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:04.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:03 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:03.889+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:14:04.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:03 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:03.889+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:14:04.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:03 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:03.889+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.266275+0000 front 2024-08-20T03:13:38.266293+0000 (oldest deadline 2024-08-20T03:14:02.366431+0000) 2024-08-20T03:14:04.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:03 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:03.710+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:04.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:03 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:03.710+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:04.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:03 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:03.710+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:04.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:03 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:14:03.671+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.846642+0000 front 2024-08-20T03:13:21.846522+0000 (oldest deadline 2024-08-20T03:13:44.146424+0000) 2024-08-20T03:14:04.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:03 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:14:03.671+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:29.447532+0000 front 2024-08-20T03:13:29.447430+0000 (oldest deadline 2024-08-20T03:13:54.147273+0000) 2024-08-20T03:14:04.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:03 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5[35013]: 2024-08-20T03:14:03.671+0000 7f94ee13f640 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.748700+0000 front 2024-08-20T03:13:38.748683+0000 (oldest deadline 2024-08-20T03:14:02.248686+0000) 2024-08-20T03:14:04.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:03 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:03.749+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:04.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:03 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:03.749+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:04.490 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:04 smithi165 podman[59272]: 2024-08-20 03:14:04.194508234 +0000 UTC m=+5.565919621 container died fe942d45eace7d4f6e72b20475beb8319e4cbe6028a5d240827d25edaec9cd69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:14:04.490 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:04 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:04.356+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:04.490 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:04 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:04.356+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:04.490 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:04 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:04.356+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:04.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:04 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:04.323+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:04.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:04 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:04.323+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:04.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:04 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:04.496+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:04.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:04 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:04.496+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:04.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:04 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:04.496+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:04.758 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:04 smithi165 podman[59272]: 2024-08-20 03:14:04.596193968 +0000 UTC m=+5.967605406 container cleanup fe942d45eace7d4f6e72b20475beb8319e4cbe6028a5d240827d25edaec9cd69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-08-20T03:14:04.758 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:04 smithi165 bash[59272]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5 2024-08-20T03:14:05.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:04 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:04.905+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:14:05.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:04 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:04.905+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:14:05.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:04 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:04.905+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.266275+0000 front 2024-08-20T03:13:38.266293+0000 (oldest deadline 2024-08-20T03:14:02.366431+0000) 2024-08-20T03:14:05.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:04 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:04.662+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:05.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:04 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:04.662+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:05.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:04 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:04.662+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:05.136 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:04 smithi165 podman[59289]: 2024-08-20 03:14:04.847623224 +0000 UTC m=+0.650981781 container remove fe942d45eace7d4f6e72b20475beb8319e4cbe6028a5d240827d25edaec9cd69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-08-20T03:14:05.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:04 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:04.756+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:05.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:04 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:04.756+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:05.444 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:05 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:05.330+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:05.445 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:05 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:05.330+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:05.445 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:05 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:05.330+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:05.445 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:05 smithi165 podman[59377]: 2024-08-20 03:14:05.066374436 +0000 UTC m=+0.016437713 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:14:05.445 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:05 smithi165 podman[59377]: 2024-08-20 03:14:05.182159496 +0000 UTC m=+0.132222803 container create 9cf5b5f7a3328c4bd32b1fd672ec305ded5e7b17334b85e8d12cc94b8e46ac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-08-20T03:14:05.445 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:05 smithi165 podman[59377]: 2024-08-20 03:14:05.44349664 +0000 UTC m=+0.393559914 container init 9cf5b5f7a3328c4bd32b1fd672ec305ded5e7b17334b85e8d12cc94b8e46ac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5-deactivate, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T03:14:05.531 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:05 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:05.274+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:05.531 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:05 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:05.274+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:05.762 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:05 smithi165 podman[59377]: 2024-08-20 03:14:05.446988293 +0000 UTC m=+0.397051561 container start 9cf5b5f7a3328c4bd32b1fd672ec305ded5e7b17334b85e8d12cc94b8e46ac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-20T03:14:05.762 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:05 smithi165 podman[59377]: 2024-08-20 03:14:05.543499314 +0000 UTC m=+0.493562581 container attach 9cf5b5f7a3328c4bd32b1fd672ec305ded5e7b17334b85e8d12cc94b8e46ac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.36.0) 2024-08-20T03:14:05.762 INFO:journalctl@ceph.osd.5.smithi165.stdout:Aug 20 03:14:05 smithi165 podman[59377]: 2024-08-20 03:14:05.638771915 +0000 UTC m=+0.588835183 container died 9cf5b5f7a3328c4bd32b1fd672ec305ded5e7b17334b85e8d12cc94b8e46ac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-5-deactivate, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T03:14:05.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:05 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:05.529+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:05.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:05 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:05.529+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:05.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:05 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:05.529+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:06.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:05 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:05.861+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:14:06.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:05 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:05.861+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:14:06.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:05 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:05.861+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.266275+0000 front 2024-08-20T03:13:38.266293+0000 (oldest deadline 2024-08-20T03:14:02.366431+0000) 2024-08-20T03:14:06.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:05 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:05.696+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:06.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:05 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:05.696+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:06.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:05 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:05.696+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:06.135 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:05 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:05.760+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:06.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:05 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:05.760+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:06.320 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.5.service' 2024-08-20T03:14:06.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:14:06.363 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-08-20T03:14:06.363 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-08-20T03:14:06.363 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.6 2024-08-20T03:14:06.572 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:06 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:06.277+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:06.572 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:06 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:06.277+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:06.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:06 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:06.355+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:06.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:06 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:06.355+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:06.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:06 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:06.355+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:06.665 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:06 smithi044 systemd[1]: Stopping Ceph osd.6 for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:14:06.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:06 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:06.570+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:06.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:06 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:06.570+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:06.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:06 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:06.570+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:07.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:06 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:06.720+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:07.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:06 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:06.720+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:07.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:06 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:06.720+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:07.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:06 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:06.662+0000 7f26e5e5c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:14:07.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:06 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:06.662+0000 7f26e5e5c640 -1 osd.6 88 *** Got signal Terminated *** 2024-08-20T03:14:07.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:06 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:06.662+0000 7f26e5e5c640 -1 osd.6 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T03:14:07.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:06 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:06.868+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:14:07.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:06 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:06.868+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:14:07.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:06 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:06.868+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.266275+0000 front 2024-08-20T03:13:38.266293+0000 (oldest deadline 2024-08-20T03:14:02.366431+0000) 2024-08-20T03:14:07.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:06 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:06.711+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:07.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:06 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:06.711+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:07.544 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:07 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:07.234+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:07.544 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:07 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:07.234+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:07.695 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:07 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:07.401+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:07.695 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:07 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:07.401+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:07.695 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:07 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:07.401+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:07.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:07 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:07.542+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:07.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:07 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:07.542+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:07.915 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:07 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:07.542+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:08.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:07 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:07.820+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:14:08.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:07 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:07.820+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:14:08.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:07 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:07.820+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.266275+0000 front 2024-08-20T03:13:38.266293+0000 (oldest deadline 2024-08-20T03:14:02.366431+0000) 2024-08-20T03:14:08.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:07 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:07.730+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:08.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:07 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:07.730+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:08.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:07 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:07.730+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:08.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:07 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:07.693+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:08.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:07 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:07.693+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:08.516 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:08 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:08.258+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:08.516 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:08 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:08.258+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:08.671 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:08 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:08.390+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:08.671 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:08 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:08.390+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:08.671 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:08 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:08.390+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:08.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:08 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:08.514+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:08.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:08 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:08.514+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:08.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:08 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:08.514+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:09.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:08 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:08.863+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:14:09.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:08 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:08.863+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:14:09.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:08 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:08.863+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.266275+0000 front 2024-08-20T03:13:38.266293+0000 (oldest deadline 2024-08-20T03:14:02.366431+0000) 2024-08-20T03:14:09.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:08 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:08.696+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:09.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:08 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:08.696+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:09.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:08 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:08.696+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:09.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:08 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:08.669+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:09.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:08 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:08.669+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:09.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:08 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:08.669+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:09.502 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:09 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:09.212+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:09.502 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:09 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:09.212+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:09.658 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:09 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:09.401+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:09.658 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:09 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:09.401+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:09.658 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:09 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:09.401+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:09.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:09 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:09.500+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:09.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:09 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:09.500+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:09.915 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:09 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:09.500+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:09.915 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:09 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:09.500+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:10.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:09 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:09.876+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:14:10.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:09 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:09.876+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:14:10.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:09 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:09.876+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.266275+0000 front 2024-08-20T03:13:38.266293+0000 (oldest deadline 2024-08-20T03:14:02.366431+0000) 2024-08-20T03:14:10.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:09 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:09.676+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:10.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:09 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:09.676+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:10.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:09 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:09.676+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:10.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:09 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:09.656+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:10.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:09 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:09.656+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:10.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:09 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:09.656+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:10.472 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:10 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:10.183+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:10.472 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:10 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:10.183+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:10.473 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:10 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:10.183+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:45.900753+0000 front 2024-08-20T03:13:45.900841+0000 (oldest deadline 2024-08-20T03:14:09.400781+0000) 2024-08-20T03:14:10.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:10 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:10.646+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:10.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:10 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:10.646+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:10.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:10 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:10.646+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:10.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:10 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:10.436+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:10.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:10 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:10.436+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:10.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:10 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:10.436+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:10.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:10 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:10.471+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:10.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:10 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:10.471+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:10.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:10 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:10.471+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:10.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:10 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:10.471+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:11.035 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:10 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:10.875+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:20.361983+0000 front 2024-08-20T03:13:20.362090+0000 (oldest deadline 2024-08-20T03:13:46.262198+0000) 2024-08-20T03:14:11.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:10 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:10.875+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.262887+0000 front 2024-08-20T03:13:26.262774+0000 (oldest deadline 2024-08-20T03:13:51.562853+0000) 2024-08-20T03:14:11.036 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:10 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6[50240]: 2024-08-20T03:14:10.875+0000 7f26de469640 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.266275+0000 front 2024-08-20T03:13:38.266293+0000 (oldest deadline 2024-08-20T03:14:02.366431+0000) 2024-08-20T03:14:11.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:10 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:10.701+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:11.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:10 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:10.701+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:11.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:10 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:10.701+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:11.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:10 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:10.701+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:11.501 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:11 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:11.181+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:11.501 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:11 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:11.181+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:11.501 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:11 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:11.181+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:45.900753+0000 front 2024-08-20T03:13:45.900841+0000 (oldest deadline 2024-08-20T03:14:09.400781+0000) 2024-08-20T03:14:11.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:11 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:11.650+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:11.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:11 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:11.650+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:11.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:11 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:11.650+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:11.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:11 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:11.447+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:11.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:11 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:11.447+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:11.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:11 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:11.447+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:11.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:11 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:11.499+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:11.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:11 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:11.499+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:11.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:11 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:11.499+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:11.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:11 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:11.499+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:12.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:11 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:11.741+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:12.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:11 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:11.741+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:12.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:11 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:11.741+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:12.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:11 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:11.741+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:12.475 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:12 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:12.207+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:12.476 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:12 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:12.207+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:12.476 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:12 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:12.207+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:45.900753+0000 front 2024-08-20T03:13:45.900841+0000 (oldest deadline 2024-08-20T03:14:09.400781+0000) 2024-08-20T03:14:12.515 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:12 smithi044 podman[98581]: 2024-08-20 03:14:12.110171199 +0000 UTC m=+5.550945117 container died 79e044a853fcbc6fd908a130224938dae4e807e4a90ad9eb2709e0642bad809a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-08-20T03:14:12.659 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:12 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:12.404+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:12.659 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:12 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:12.404+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:12.660 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:12 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:12.404+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:12.785 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:12 smithi044 podman[98581]: 2024-08-20 03:14:12.51357748 +0000 UTC m=+5.954351387 container cleanup 79e044a853fcbc6fd908a130224938dae4e807e4a90ad9eb2709e0642bad809a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2) 2024-08-20T03:14:12.786 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:12 smithi044 bash[98581]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6 2024-08-20T03:14:12.786 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:12 smithi044 podman[98598]: 2024-08-20 03:14:12.748340567 +0000 UTC m=+0.631456933 container remove 79e044a853fcbc6fd908a130224938dae4e807e4a90ad9eb2709e0642bad809a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-20T03:14:12.786 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:12 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:12.743+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:12.786 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:12 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:12.743+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:12.786 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:12 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:12.743+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:12.786 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:12 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:12.743+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:12.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:12 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:12.474+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:12.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:12 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:12.474+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:12.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:12 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:12.474+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:12.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:12 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:12.474+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:13.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:12 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:12.657+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:13.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:12 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:12.657+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:13.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:12 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:12.657+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:13.370 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:13 smithi044 podman[98687]: 2024-08-20 03:14:12.98230133 +0000 UTC m=+0.016469428 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:14:13.370 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:13 smithi044 podman[98687]: 2024-08-20 03:14:13.090923512 +0000 UTC m=+0.125091598 container create 13e5a6005b68472778de622f1c02c640eb1732952dd2ba6f9d7375e3287c7d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:14:13.614 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:13 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:13.356+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:13.614 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:13 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:13.356+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:13.614 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:13 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:13.356+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:13.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:13 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:13.227+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:13.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:13 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:13.227+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:13.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:13 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:13.227+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:45.900753+0000 front 2024-08-20T03:13:45.900841+0000 (oldest deadline 2024-08-20T03:14:09.400781+0000) 2024-08-20T03:14:13.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:13 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:13.426+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:13.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:13 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:13.426+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:13.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:13 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:13.426+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:13.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:13 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:13.426+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:13.767 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:13 smithi044 podman[98687]: 2024-08-20 03:14:13.368428786 +0000 UTC m=+0.402596871 container init 13e5a6005b68472778de622f1c02c640eb1732952dd2ba6f9d7375e3287c7d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6-deactivate, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-20T03:14:13.767 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:13 smithi044 podman[98687]: 2024-08-20 03:14:13.372032804 +0000 UTC m=+0.406200887 container start 13e5a6005b68472778de622f1c02c640eb1732952dd2ba6f9d7375e3287c7d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T03:14:13.767 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:13 smithi044 podman[98687]: 2024-08-20 03:14:13.485465468 +0000 UTC m=+0.519633567 container attach 13e5a6005b68472778de622f1c02c640eb1732952dd2ba6f9d7375e3287c7d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-20T03:14:13.767 INFO:journalctl@ceph.osd.6.smithi044.stdout:Aug 20 03:14:13 smithi044 podman[98687]: 2024-08-20 03:14:13.566133661 +0000 UTC m=+0.600301759 container died 13e5a6005b68472778de622f1c02c640eb1732952dd2ba6f9d7375e3287c7d82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-6-deactivate, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-20T03:14:13.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:13 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:13.611+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:13.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:13 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:13.611+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:13.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:13 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:13.611+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:14.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:13 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:13.764+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:14.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:13 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:13.764+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:14.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:13 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:13.764+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:14.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:13 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:13.764+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:14.263 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.6.service' 2024-08-20T03:14:14.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:14:14.308 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-08-20T03:14:14.308 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-08-20T03:14:14.308 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.7 2024-08-20T03:14:14.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:14 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:14.254+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:14.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:14 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:14.254+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:14.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:14 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:14.254+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:45.900753+0000 front 2024-08-20T03:13:45.900841+0000 (oldest deadline 2024-08-20T03:14:09.400781+0000) 2024-08-20T03:14:14.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:14 smithi099 systemd[1]: Stopping Ceph osd.7 for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:14:14.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:14 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:14.456+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:14.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:14 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:14.456+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:14.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:14 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:14.456+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:14.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:14 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:14.456+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:14.785 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:14 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:14.765+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:14.786 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:14 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:14.765+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:14.786 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:14 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:14.765+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:14.786 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:14 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:14.765+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:14.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:14 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:14.634+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:14.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:14 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:14.634+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:14.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:14 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:14.634+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:14.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:14 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:14.401+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:14.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:14 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:14.401+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:14.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:14 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:14.401+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:15.164 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:14 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:14.892+0000 7fe863dd5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:14:15.164 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:14 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:14.892+0000 7fe863dd5640 -1 osd.7 88 *** Got signal Terminated *** 2024-08-20T03:14:15.164 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:14 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:14.892+0000 7fe863dd5640 -1 osd.7 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T03:14:15.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:15 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:15.265+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:15.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:15 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:15.265+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:15.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:15 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:15.265+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:45.900753+0000 front 2024-08-20T03:13:45.900841+0000 (oldest deadline 2024-08-20T03:14:09.400781+0000) 2024-08-20T03:14:15.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:15 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:15.471+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:15.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:15 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:15.471+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:15.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:15 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:15.471+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:15.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:15 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:15.471+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:15.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:15 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:15.676+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:15.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:15 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:15.676+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:15.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:15 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:15.676+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:15.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:15 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:15.437+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:15.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:15 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:15.437+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:15.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:15 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:15.437+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:16.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:15 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:15.734+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:16.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:15 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:15.734+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:16.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:15 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:15.734+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:16.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:15 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:15.734+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:16.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:16 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:16.288+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:16.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:16 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:16.288+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:16.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:16 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:16.288+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:45.900753+0000 front 2024-08-20T03:13:45.900841+0000 (oldest deadline 2024-08-20T03:14:09.400781+0000) 2024-08-20T03:14:16.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:16 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:16.424+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:16.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:16 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:16.424+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:16.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:16 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:16.424+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:16.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:16 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:16.424+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:16.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:16 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:16.626+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:16.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:16 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:16.627+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:16.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:16 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:16.627+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:16.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:16 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:16.452+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:16.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:16 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:16.452+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:16.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:16 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:16.452+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:17.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:16 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:16.692+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:17.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:16 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:16.692+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:17.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:16 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:16.692+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:17.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:16 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:16.692+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:17.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:17 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:17.283+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:17.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:17 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:17.283+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:17.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:17 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:17.283+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:45.900753+0000 front 2024-08-20T03:13:45.900841+0000 (oldest deadline 2024-08-20T03:14:09.400781+0000) 2024-08-20T03:14:17.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:17 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:17.283+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.101432+0000 front 2024-08-20T03:13:54.101449+0000 (oldest deadline 2024-08-20T03:14:16.401593+0000) 2024-08-20T03:14:17.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:17 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:17.432+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:17.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:17 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:17.432+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:17.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:17 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:17.432+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:17.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:17 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:17.432+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:17.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:17 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:17.432+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:17.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:17 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:17.614+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:17.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:17 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:17.614+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:17.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:17 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:17.614+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:17.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:17 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:17.614+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.236110+0000 front 2024-08-20T03:13:53.236092+0000 (oldest deadline 2024-08-20T03:14:17.335971+0000) 2024-08-20T03:14:17.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:17 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:17.446+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:17.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:17 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:17.446+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:17.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:17 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:17.446+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:17.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:17 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:17.446+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:18.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:17 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:17.738+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:18.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:17 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:17.738+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:18.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:17 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:17.738+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:18.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:17 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:17.738+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:18.664 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:18 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:18.297+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:18.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:18 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:18.297+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:18.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:18 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:18.297+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:45.900753+0000 front 2024-08-20T03:13:45.900841+0000 (oldest deadline 2024-08-20T03:14:09.400781+0000) 2024-08-20T03:14:18.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:18 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:18.297+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.101432+0000 front 2024-08-20T03:13:54.101449+0000 (oldest deadline 2024-08-20T03:14:16.401593+0000) 2024-08-20T03:14:18.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:18 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:18.387+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:18.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:18 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:18.387+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:18.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:18 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:18.387+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:18.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:18 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:18.387+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:18.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:18 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:18.387+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:18.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:18 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:18.622+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:18.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:18 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:18.622+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:18.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:18 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:18.622+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:18.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:18 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:18.622+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.236110+0000 front 2024-08-20T03:13:53.236092+0000 (oldest deadline 2024-08-20T03:14:17.335971+0000) 2024-08-20T03:14:18.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:18 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:18.441+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:18.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:18 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:18.441+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:18.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:18 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:18.441+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:18.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:18 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:18.441+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:19.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:18 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:18.777+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:19.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:18 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:18.777+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:19.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:18 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:18.777+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:19.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:18 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:18.777+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:19.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:19 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:19.393+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:19.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:19 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:19.393+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:19.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:19 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:19.393+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:19.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:19 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:19.393+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:19.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:19 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:19.393+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:19.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:19 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:19.341+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:26.697542+0000 front 2024-08-20T03:13:26.697676+0000 (oldest deadline 2024-08-20T03:13:52.597732+0000) 2024-08-20T03:14:19.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:19 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:19.341+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:38.399062+0000 front 2024-08-20T03:13:38.399216+0000 (oldest deadline 2024-08-20T03:14:01.899120+0000) 2024-08-20T03:14:19.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:19 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:19.341+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:45.900753+0000 front 2024-08-20T03:13:45.900841+0000 (oldest deadline 2024-08-20T03:14:09.400781+0000) 2024-08-20T03:14:19.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:19 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7[41403]: 2024-08-20T03:14:19.341+0000 7fe85c3e2640 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.101432+0000 front 2024-08-20T03:13:54.101449+0000 (oldest deadline 2024-08-20T03:14:16.401593+0000) 2024-08-20T03:14:19.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:19 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:19.629+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:19.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:19 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:19.629+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:19.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:19 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:19.629+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:19.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:19 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:19.629+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.236110+0000 front 2024-08-20T03:13:53.236092+0000 (oldest deadline 2024-08-20T03:14:17.335971+0000) 2024-08-20T03:14:19.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:19 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:19.469+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:19.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:19 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:19.469+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:19.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:19 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:19.469+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:19.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:19 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:19.469+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:20.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:19 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:19.748+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:20.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:19 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:19.748+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:20.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:19 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:19.748+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:20.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:19 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:19.748+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:20.665 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:20 smithi099 podman[59199]: 2024-08-20 03:14:20.417323614 +0000 UTC m=+5.916765155 container died 40e93aabc69f52640778831d183657c23bc327a1adfda6c5242110fe452c62a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-08-20T03:14:20.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:20 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:20.396+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:20.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:20 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:20.396+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:20.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:20 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:20.396+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:20.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:20 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:20.396+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:20.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:20 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:20.396+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:20.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:20 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:20.672+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:20.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:20 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:20.672+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:20.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:20 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:20.672+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:20.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:20 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:20.672+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.236110+0000 front 2024-08-20T03:13:53.236092+0000 (oldest deadline 2024-08-20T03:14:17.335971+0000) 2024-08-20T03:14:20.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:20 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:20.453+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:20.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:20 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:20.453+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:20.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:20 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:20.453+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:20.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:20 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:20.453+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:21.029 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:20 smithi099 podman[59199]: 2024-08-20 03:14:20.842794854 +0000 UTC m=+6.342236397 container cleanup 40e93aabc69f52640778831d183657c23bc327a1adfda6c5242110fe452c62a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-08-20T03:14:21.029 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:20 smithi099 bash[59199]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7 2024-08-20T03:14:21.285 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:20 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:20.797+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:21.285 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:20 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:20.797+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:21.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:20 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:20.797+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:21.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:20 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:20.797+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:21.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:20 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:20.797+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:21.414 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:21 smithi099 podman[59214]: 2024-08-20 03:14:21.027554314 +0000 UTC m=+0.607618366 container remove 40e93aabc69f52640778831d183657c23bc327a1adfda6c5242110fe452c62a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-20T03:14:21.726 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:21 smithi099 podman[59306]: 2024-08-20 03:14:21.357593982 +0000 UTC m=+0.015718548 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:14:21.726 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:21 smithi099 podman[59306]: 2024-08-20 03:14:21.455510376 +0000 UTC m=+0.113634943 container create b8462b6aef3fba2a344e12764598e906eeb82cf9d018342c87ea6b4f60b02e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-08-20T03:14:21.726 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:21 smithi099 podman[59306]: 2024-08-20 03:14:21.724964337 +0000 UTC m=+0.383088903 container init b8462b6aef3fba2a344e12764598e906eeb82cf9d018342c87ea6b4f60b02e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:14:21.727 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:21 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:21.444+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:21.727 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:21 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:21.444+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:21.727 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:21 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:21.444+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:21.727 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:21 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:21.444+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:21.727 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:21 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:21.444+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:21.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:21 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:21.645+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:21.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:21 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:21.645+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:21.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:21 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:21.645+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:21.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:21 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:21.645+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.236110+0000 front 2024-08-20T03:13:53.236092+0000 (oldest deadline 2024-08-20T03:14:17.335971+0000) 2024-08-20T03:14:21.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:21 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:21.452+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:21.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:21 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:21.452+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:21.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:21 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:21.452+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:21.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:21 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:21.452+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:22.164 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:21 smithi099 podman[59306]: 2024-08-20 03:14:21.728921592 +0000 UTC m=+0.387046159 container start b8462b6aef3fba2a344e12764598e906eeb82cf9d018342c87ea6b4f60b02e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-08-20T03:14:22.165 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:21 smithi099 podman[59306]: 2024-08-20 03:14:21.841662799 +0000 UTC m=+0.499787366 container attach b8462b6aef3fba2a344e12764598e906eeb82cf9d018342c87ea6b4f60b02e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-08-20T03:14:22.165 INFO:journalctl@ceph.osd.7.smithi099.stdout:Aug 20 03:14:21 smithi099 podman[59306]: 2024-08-20 03:14:21.917591833 +0000 UTC m=+0.575716412 container died b8462b6aef3fba2a344e12764598e906eeb82cf9d018342c87ea6b4f60b02e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-20T03:14:22.187 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:21 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:21.806+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:22.187 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:21 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:21.806+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:22.187 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:21 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:21.806+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:22.188 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:21 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:21.806+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:22.188 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:21 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:21.806+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:22.505 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:22 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:22.414+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:22.505 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:22 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:22.414+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:22.505 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:22 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:22.414+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:22.505 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:22 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:22.414+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:22.505 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:22 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:22.414+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:22.529 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.7.service' 2024-08-20T03:14:22.568 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:14:22.568 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-08-20T03:14:22.568 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-08-20T03:14:22.568 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.8 2024-08-20T03:14:22.885 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:22 smithi165 systemd[1]: Stopping Ceph osd.8 for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:14:22.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:22 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:22.658+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:22.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:22 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:22.658+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:22.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:22 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:22.658+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:22.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:22 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:22.658+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.236110+0000 front 2024-08-20T03:13:53.236092+0000 (oldest deadline 2024-08-20T03:14:17.335971+0000) 2024-08-20T03:14:22.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:22 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:22.883+0000 7f0201cf7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:14:22.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:22 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:22.883+0000 7f0201cf7640 -1 osd.8 88 *** Got signal Terminated *** 2024-08-20T03:14:22.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:22 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:22.467+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:22.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:22 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:22.467+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:22.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:22 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:22.467+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:22.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:22 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:22.467+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:23.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:22 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:22.795+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:23.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:22 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:22.795+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:23.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:22 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:22.795+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:23.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:22 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:22.795+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:23.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:22 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:22.795+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:23.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:23 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:23.379+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:23.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:23 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:23.379+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:23.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:23 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:23.379+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:23.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:23 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:23.379+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:23.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:23 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:23.379+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:23.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:22 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:22.883+0000 7f0201cf7640 -1 osd.8 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T03:14:23.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:23 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:23.660+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:23.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:23 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:23.660+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:23.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:23 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:23.660+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:23.887 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:23 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:23.660+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.236110+0000 front 2024-08-20T03:13:53.236092+0000 (oldest deadline 2024-08-20T03:14:17.335971+0000) 2024-08-20T03:14:23.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:23 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:23.417+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:23.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:23 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:23.417+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:23.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:23 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:23.417+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:23.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:23 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:23.417+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:24.285 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:23 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:23.784+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:24.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:23 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:23.784+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:24.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:23 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:23.784+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:24.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:23 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:23.784+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:24.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:23 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:23.784+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:24.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:24 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:24.359+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:24.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:24 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:24.359+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:24.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:24 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:24.359+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:24.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:24 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:24.359+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:24.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:24 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:24.359+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:24.708 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:24 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:24.415+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:24.709 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:24 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:24.415+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:24.709 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:24 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:24.415+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:24.709 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:24 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:24.415+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:25.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:24 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:24.706+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:25.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:24 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:24.706+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:25.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:24 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:24.706+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:25.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:24 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:24.706+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.236110+0000 front 2024-08-20T03:13:53.236092+0000 (oldest deadline 2024-08-20T03:14:17.335971+0000) 2024-08-20T03:14:25.285 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:24 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:24.785+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:25.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:24 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:24.785+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:25.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:24 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:24.785+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:25.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:24 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:24.785+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:25.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:24 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:24.785+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:25.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:25 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:25.377+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:25.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:25 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:25.377+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:25.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:25 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:25.377+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:25.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:25 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:25.377+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:25.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:25 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:25.377+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:25.672 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:25 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:25.413+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:25.672 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:25 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:25.413+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:25.672 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:25 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:25.413+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:25.672 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:25 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:25.413+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:25.672 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:25 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:25.413+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:26.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:25 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:25.757+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:26.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:25 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:25.757+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:26.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:25 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:25.757+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:26.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:25 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:25.757+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:26.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:25 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:25.757+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:26.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:25 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:25.670+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:26.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:25 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:25.670+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:26.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:25 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:25.670+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:26.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:25 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:25.670+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.236110+0000 front 2024-08-20T03:13:53.236092+0000 (oldest deadline 2024-08-20T03:14:17.335971+0000) 2024-08-20T03:14:26.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:26 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:26.336+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:26.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:26 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:26.336+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:26.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:26 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:26.336+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:26.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:26 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:26.336+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:26.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:26 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:26.336+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:26.697 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:26 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:26.392+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:26.698 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:26 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:26.392+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:26.698 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:26 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:26.392+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:26.698 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:26 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:26.392+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:26.698 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:26 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:26.392+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:27.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:26 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:26.774+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:27.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:26 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:26.774+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:27.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:26 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:26.774+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:27.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:26 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:26.774+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:27.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:26 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:26.774+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:27.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:26 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:26.696+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:27.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:26 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:26.696+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:27.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:26 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:26.696+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:27.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:26 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:26.696+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.236110+0000 front 2024-08-20T03:13:53.236092+0000 (oldest deadline 2024-08-20T03:14:17.335971+0000) 2024-08-20T03:14:27.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:26 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:26.696+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:03.236920+0000 front 2024-08-20T03:14:03.236944+0000 (oldest deadline 2024-08-20T03:14:26.136984+0000) 2024-08-20T03:14:27.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:27 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:27.376+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:27.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:27 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:27.376+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:27.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:27 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:27.376+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:27.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:27 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:27.376+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:27.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:27 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:27.376+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:27.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:27 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:27.352+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:27.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:27 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:27.352+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:27.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:27 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:27.352+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:27.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:27 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:27.352+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:27.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:27 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:27.352+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:28.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:27 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:27.700+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.731563+0000 front 2024-08-20T03:13:30.731535+0000 (oldest deadline 2024-08-20T03:13:53.631528+0000) 2024-08-20T03:14:28.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:27 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:27.700+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:37.632698+0000 front 2024-08-20T03:13:37.632766+0000 (oldest deadline 2024-08-20T03:14:03.532886+0000) 2024-08-20T03:14:28.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:27 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:27.700+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:46.934234+0000 front 2024-08-20T03:13:46.934252+0000 (oldest deadline 2024-08-20T03:14:08.634293+0000) 2024-08-20T03:14:28.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:27 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:27.700+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.236110+0000 front 2024-08-20T03:13:53.236092+0000 (oldest deadline 2024-08-20T03:14:17.335971+0000) 2024-08-20T03:14:28.136 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:27 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8[41935]: 2024-08-20T03:14:27.700+0000 7f01fa304640 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:03.236920+0000 front 2024-08-20T03:14:03.236944+0000 (oldest deadline 2024-08-20T03:14:26.136984+0000) 2024-08-20T03:14:28.285 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:27 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:27.806+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:28.285 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:27 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:27.806+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:28.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:27 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:27.806+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:28.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:27 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:27.806+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:28.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:27 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:27.806+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:28.636 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:28 smithi165 podman[59481]: 2024-08-20 03:14:28.376298597 +0000 UTC m=+5.614255671 container died bca2bd2f250606fdb3b4a12dea2381c7647d43cc48fbc2866d5bccff58c4cb66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:14:28.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:28 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:28.371+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:28.637 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:28 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:28.371+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:28.637 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:28 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:28.371+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:28.637 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:28 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:28.371+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:28.637 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:28 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:28.371+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:28.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:28 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:28.303+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:28.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:28 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:28.303+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:28.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:28 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:28.303+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:28.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:28 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:28.303+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:28.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:28 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:28.303+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:28.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:28 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:28.303+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:29.011 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:28 smithi165 podman[59481]: 2024-08-20 03:14:28.717026629 +0000 UTC m=+5.954983704 container cleanup bca2bd2f250606fdb3b4a12dea2381c7647d43cc48fbc2866d5bccff58c4cb66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:14:29.012 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:28 smithi165 bash[59481]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8 2024-08-20T03:14:29.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:28 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:28.795+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:29.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:28 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:28.795+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:29.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:28 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:28.795+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:29.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:28 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:28.795+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:29.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:28 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:28.795+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:29.332 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:29 smithi165 podman[59496]: 2024-08-20 03:14:29.010219787 +0000 UTC m=+0.635541202 container remove bca2bd2f250606fdb3b4a12dea2381c7647d43cc48fbc2866d5bccff58c4cb66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-08-20T03:14:29.632 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:29 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:29.406+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:29.632 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:29 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:29.406+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:29.632 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:29 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:29.406+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:29.632 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:29 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:29.406+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:29.632 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:29 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:29.406+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:29.632 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:29 smithi165 podman[59586]: 2024-08-20 03:14:29.23353042 +0000 UTC m=+0.015949525 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:14:29.632 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:29 smithi165 podman[59586]: 2024-08-20 03:14:29.344508788 +0000 UTC m=+0.126927925 container create 47b7701fc4cf954367981fbb61df493f20b4c9f5ca103dc550e8ee23f2e26c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-08-20T03:14:29.632 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:29 smithi165 podman[59586]: 2024-08-20 03:14:29.547292013 +0000 UTC m=+0.329711113 container init 47b7701fc4cf954367981fbb61df493f20b4c9f5ca103dc550e8ee23f2e26c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8-deactivate, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-08-20T03:14:29.633 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:29 smithi165 podman[59586]: 2024-08-20 03:14:29.550550913 +0000 UTC m=+0.332970012 container start 47b7701fc4cf954367981fbb61df493f20b4c9f5ca103dc550e8ee23f2e26c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:14:29.633 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:29 smithi165 podman[59586]: 2024-08-20 03:14:29.630537156 +0000 UTC m=+0.412956264 container attach 47b7701fc4cf954367981fbb61df493f20b4c9f5ca103dc550e8ee23f2e26c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8-deactivate, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-20T03:14:29.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:29 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:29.277+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:29.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:29 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:29.277+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:29.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:29 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:29.277+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:29.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:29 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:29.277+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:29.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:29 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:29.277+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:29.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:29 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:29.277+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:29.886 INFO:journalctl@ceph.osd.8.smithi165.stdout:Aug 20 03:14:29 smithi165 podman[59586]: 2024-08-20 03:14:29.73869367 +0000 UTC m=+0.521112767 container died 47b7701fc4cf954367981fbb61df493f20b4c9f5ca103dc550e8ee23f2e26c32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-08-20T03:14:30.285 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:29 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:29.799+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:30.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:29 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:29.799+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:30.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:29 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:29.799+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:30.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:29 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:29.799+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:30.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:29 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:29.799+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:30.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:29 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:29.799+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:03.499982+0000 front 2024-08-20T03:14:03.499961+0000 (oldest deadline 2024-08-20T03:14:29.399826+0000) 2024-08-20T03:14:30.354 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.8.service' 2024-08-20T03:14:30.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:14:30.394 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-08-20T03:14:30.394 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-08-20T03:14:30.395 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.9 2024-08-20T03:14:30.616 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:30 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:30.400+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:30.616 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:30 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:30.400+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:30.616 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:30 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:30.400+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:30.616 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:30 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:30.400+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:30.616 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:30 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:30.400+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:30.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:30 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:30.323+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:30.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:30 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:30.323+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:30.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:30 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:30.323+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:30.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:30 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:30.323+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:30.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:30 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:30.323+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:30.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:30 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:30.323+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:30.702 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:30 smithi044 systemd[1]: Stopping Ceph osd.9 for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:14:31.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:30 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:30.700+0000 7f835062a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:14:31.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:30 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:30.700+0000 7f835062a640 -1 osd.9 88 *** Got signal Terminated *** 2024-08-20T03:14:31.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:30 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:30.700+0000 7f835062a640 -1 osd.9 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T03:14:31.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:30 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:30.774+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:31.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:30 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:30.774+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:31.037 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:30 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:30.774+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:31.037 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:30 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:30.774+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:31.037 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:30 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:30.774+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:31.037 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:30 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:30.774+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:03.499982+0000 front 2024-08-20T03:14:03.499961+0000 (oldest deadline 2024-08-20T03:14:29.399826+0000) 2024-08-20T03:14:31.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:31 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:31.285+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:31.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:31 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:31.285+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:31.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:31 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:31.285+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:31.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:31 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:31.285+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:31.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:31 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:31.285+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:31.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:31 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:31.285+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:31.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:31 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:31.391+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:31.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:31 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:31.391+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:31.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:31 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:31.391+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:31.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:31 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:31.391+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:31.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:31 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:31.391+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:32.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:31 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:31.776+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:32.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:31 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:31.776+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:32.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:31 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:31.776+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:32.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:31 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:31.776+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:32.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:31 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:31.776+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:32.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:31 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:31.776+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:03.499982+0000 front 2024-08-20T03:14:03.499961+0000 (oldest deadline 2024-08-20T03:14:29.399826+0000) 2024-08-20T03:14:32.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:32 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:32.285+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:32.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:32 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:32.285+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:32.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:32 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:32.285+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:32.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:32 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:32.285+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:32.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:32 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:32.285+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:32.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:32 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:32.285+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:32.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:32 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:32.427+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:32.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:32 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:32.427+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:32.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:32 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:32.427+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:32.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:32 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:32.427+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:32.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:32 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:32.427+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:33.285 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:32 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:32.796+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:33.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:32 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:32.796+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:33.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:32 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:32.796+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:33.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:32 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:32.796+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:33.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:32 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:32.796+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:33.286 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:32 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:32.796+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:03.499982+0000 front 2024-08-20T03:14:03.499961+0000 (oldest deadline 2024-08-20T03:14:29.399826+0000) 2024-08-20T03:14:33.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:33 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:33.332+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:33.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:33 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:33.332+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:33.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:33 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:33.332+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:33.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:33 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:33.332+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:33.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:33 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:33.332+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:33.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:33 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:33.332+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:33.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:33 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:33.457+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:33.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:33 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:33.457+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:33.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:33 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:33.457+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:33.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:33 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:33.457+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:33.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:33 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:33.457+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:33.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:33 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:33.457+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:34.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:33 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:33.760+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:34.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:33 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:33.760+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:34.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:33 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:33.760+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:34.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:33 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:33.760+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:34.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:33 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:33.760+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:34.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:33 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:33.760+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:03.499982+0000 front 2024-08-20T03:14:03.499961+0000 (oldest deadline 2024-08-20T03:14:29.399826+0000) 2024-08-20T03:14:34.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:33 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:33.760+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:09.400199+0000 front 2024-08-20T03:14:09.400231+0000 (oldest deadline 2024-08-20T03:14:33.500271+0000) 2024-08-20T03:14:34.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:34 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:34.377+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:34.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:34 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:34.377+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:34.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:34 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:34.377+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:34.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:34 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:34.377+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:34.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:34 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:34.377+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:34.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:34 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:34.377+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:34.787 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:34 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:34.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:34.787 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:34 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:34.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:34.787 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:34 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:34.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:34.787 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:34 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:34.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:34.788 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:34 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:34.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:34.788 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:34 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:34.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:35.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:34 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:34.723+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:35.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:34 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:34.723+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:35.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:34 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:34.723+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:35.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:34 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:34.723+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:35.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:34 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:34.723+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:35.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:34 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:34.723+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:03.499982+0000 front 2024-08-20T03:14:03.499961+0000 (oldest deadline 2024-08-20T03:14:29.399826+0000) 2024-08-20T03:14:35.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:34 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:34.723+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:09.400199+0000 front 2024-08-20T03:14:09.400231+0000 (oldest deadline 2024-08-20T03:14:33.500271+0000) 2024-08-20T03:14:35.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:35 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:35.354+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:35.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:35 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:35.354+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:35.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:35 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:35.354+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:35.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:35 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:35.354+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:35.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:35 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:35.354+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:35.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:35 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:35.354+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:35.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:35 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:35.441+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:35.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:35 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:35.441+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:35.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:35 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:35.441+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:35.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:35 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:35.441+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:35.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:35 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:35.441+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:35.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:35 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:35.441+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:36.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:35 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:35.698+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:21.493066+0000 front 2024-08-20T03:13:21.493033+0000 (oldest deadline 2024-08-20T03:13:44.993155+0000) 2024-08-20T03:14:36.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:35 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:35.698+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:24.993713+0000 front 2024-08-20T03:13:24.993700+0000 (oldest deadline 2024-08-20T03:13:54.394172+0000) 2024-08-20T03:14:36.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:35 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:35.698+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:39.095527+0000 front 2024-08-20T03:13:39.095442+0000 (oldest deadline 2024-08-20T03:14:00.795587+0000) 2024-08-20T03:14:36.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:35 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:35.698+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:44.296628+0000 front 2024-08-20T03:13:44.296725+0000 (oldest deadline 2024-08-20T03:14:10.196830+0000) 2024-08-20T03:14:36.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:35 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:35.698+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.198094+0000 front 2024-08-20T03:13:54.198063+0000 (oldest deadline 2024-08-20T03:14:20.098043+0000) 2024-08-20T03:14:36.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:35 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:35.698+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:03.499982+0000 front 2024-08-20T03:14:03.499961+0000 (oldest deadline 2024-08-20T03:14:29.399826+0000) 2024-08-20T03:14:36.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:35 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9[57105]: 2024-08-20T03:14:35.698+0000 7f834cc3f640 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:09.400199+0000 front 2024-08-20T03:14:09.400231+0000 (oldest deadline 2024-08-20T03:14:33.500271+0000) 2024-08-20T03:14:36.578 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:36 smithi044 podman[98801]: 2024-08-20 03:14:36.297718013 +0000 UTC m=+5.705543911 container died cf6bf4a563a9104b33e7708820818d7abe387a76ce72a160e891bd7088836cca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-20T03:14:36.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:36 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:36.380+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:36.678 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:36 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:36.380+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:36.678 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:36 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:36.380+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:36.678 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:36 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:36.380+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:36.678 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:36 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:36.380+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:36.678 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:36 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:36.380+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:36.678 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:36 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:36.380+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.309525+0000 front 2024-08-20T03:14:10.309495+0000 (oldest deadline 2024-08-20T03:14:36.209457+0000) 2024-08-20T03:14:36.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:36 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:36.470+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:36.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:36 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:36.470+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:36.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:36 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:36.470+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:36.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:36 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:36.470+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:36.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:36 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:36.470+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:36.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:36 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:36.470+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:36.928 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:36 smithi044 podman[98801]: 2024-08-20 03:14:36.742126811 +0000 UTC m=+6.149952683 container cleanup cf6bf4a563a9104b33e7708820818d7abe387a76ce72a160e891bd7088836cca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-20T03:14:36.928 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:36 smithi044 bash[98801]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9 2024-08-20T03:14:37.285 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:36 smithi044 podman[98815]: 2024-08-20 03:14:36.927105706 +0000 UTC m=+0.623710279 container remove cf6bf4a563a9104b33e7708820818d7abe387a76ce72a160e891bd7088836cca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-20T03:14:37.640 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:37 smithi044 podman[98905]: 2024-08-20 03:14:37.264778987 +0000 UTC m=+0.015999093 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:14:37.640 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:37 smithi044 podman[98905]: 2024-08-20 03:14:37.386470906 +0000 UTC m=+0.137690994 container create d2497351fb2bdff9a6462e9e5060e01ea0911bba3bcab54548cfcf2bd7422043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.36.0) 2024-08-20T03:14:37.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:37 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:37.428+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:37.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:37 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:37.428+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:37.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:37 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:37.428+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:37.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:37 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:37.428+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:37.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:37 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:37.428+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:37.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:37 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:37.428+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:37.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:37 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:37.413+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:37.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:37 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:37.413+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:37.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:37 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:37.413+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:37.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:37 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:37.413+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:37.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:37 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:37.413+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:37.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:37 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:37.413+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:37.915 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:37 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:37.413+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.309525+0000 front 2024-08-20T03:14:10.309495+0000 (oldest deadline 2024-08-20T03:14:36.209457+0000) 2024-08-20T03:14:38.035 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:37 smithi044 podman[98905]: 2024-08-20 03:14:37.638766639 +0000 UTC m=+0.389986728 container init d2497351fb2bdff9a6462e9e5060e01ea0911bba3bcab54548cfcf2bd7422043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9-deactivate, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-20T03:14:38.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:37 smithi044 podman[98905]: 2024-08-20 03:14:37.64274278 +0000 UTC m=+0.393962870 container start d2497351fb2bdff9a6462e9e5060e01ea0911bba3bcab54548cfcf2bd7422043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:14:38.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:37 smithi044 podman[98905]: 2024-08-20 03:14:37.722040993 +0000 UTC m=+0.473261084 container attach d2497351fb2bdff9a6462e9e5060e01ea0911bba3bcab54548cfcf2bd7422043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-20T03:14:38.036 INFO:journalctl@ceph.osd.9.smithi044.stdout:Aug 20 03:14:37 smithi044 podman[98905]: 2024-08-20 03:14:37.837905717 +0000 UTC m=+0.589125813 container died d2497351fb2bdff9a6462e9e5060e01ea0911bba3bcab54548cfcf2bd7422043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-20T03:14:38.452 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.9.service' 2024-08-20T03:14:38.493 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:14:38.494 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-08-20T03:14:38.494 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-08-20T03:14:38.494 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.10 2024-08-20T03:14:38.798 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:38 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:38.446+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:38.798 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:38 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:38.446+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:38.799 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:38 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:38.446+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:38.799 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:38 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:38.446+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:38.799 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:38 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:38.446+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:38.799 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:38 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:38.446+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:38.799 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:38 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:38.446+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.309525+0000 front 2024-08-20T03:14:10.309495+0000 (oldest deadline 2024-08-20T03:14:36.209457+0000) 2024-08-20T03:14:38.799 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:38 smithi099 systemd[1]: Stopping Ceph osd.10 for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:14:38.799 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:38 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:38.796+0000 7f7c5b141640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:14:38.799 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:38 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:38.796+0000 7f7c5b141640 -1 osd.10 88 *** Got signal Terminated *** 2024-08-20T03:14:38.799 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:38 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:38.796+0000 7f7c5b141640 -1 osd.10 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T03:14:38.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:38 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:38.470+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:38.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:38 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:38.470+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:38.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:38 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:38.470+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:38.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:38 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:38.470+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:38.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:38 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:38.470+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:38.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:38 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:38.470+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:39.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:39 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:39.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:39.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:39 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:39.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:39.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:39 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:39.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:39.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:39 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:39.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:39.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:39 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:39.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:39.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:39 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:39.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:39.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:39 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:39.479+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:39.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:39 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:39.479+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:39.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:39 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:39.479+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:39.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:39 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:39.479+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:39.915 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:39 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:39.479+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:39.915 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:39 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:39.479+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:39.915 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:39 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:39.479+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.309525+0000 front 2024-08-20T03:14:10.309495+0000 (oldest deadline 2024-08-20T03:14:36.209457+0000) 2024-08-20T03:14:40.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:40 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:40.505+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:40.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:40 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:40.506+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:40.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:40 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:40.506+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:40.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:40 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:40.506+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:40.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:40 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:40.506+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:40.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:40 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:40.506+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:40.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:40 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:40.443+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:40.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:40 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:40.443+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:40.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:40 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:40.443+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:40.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:40 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:40.443+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:40.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:40 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:40.443+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:40.915 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:40 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:40.443+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:40.915 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:40 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:40.443+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.309525+0000 front 2024-08-20T03:14:10.309495+0000 (oldest deadline 2024-08-20T03:14:36.209457+0000) 2024-08-20T03:14:41.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:41 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:41.401+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:41.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:41 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:41.401+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:41.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:41 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:41.401+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:41.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:41 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:41.401+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:41.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:41 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:41.401+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:41.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:41 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:41.401+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:41.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:41 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:41.401+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.309525+0000 front 2024-08-20T03:14:10.309495+0000 (oldest deadline 2024-08-20T03:14:36.209457+0000) 2024-08-20T03:14:41.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:41 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:41.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:41.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:41 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:41.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:41.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:41 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:41.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:41.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:41 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:41.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:41.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:41 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:41.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:41.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:41 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:41.490+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:42.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:42 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:42.379+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:42.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:42 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:42.379+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:42.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:42 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:42.379+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:42.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:42 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:42.379+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:42.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:42 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:42.379+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:42.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:42 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:42.379+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:42.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:42 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:42.379+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.309525+0000 front 2024-08-20T03:14:10.309495+0000 (oldest deadline 2024-08-20T03:14:36.209457+0000) 2024-08-20T03:14:42.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:42 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:42.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:42.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:42 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:42.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:42.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:42 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:42.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:42.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:42 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:42.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:42.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:42 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:42.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:42.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:42 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:42.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:43.664 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:43 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:43.368+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:17.102515+0000 front 2024-08-20T03:13:17.102450+0000 (oldest deadline 2024-08-20T03:13:43.002604+0000) 2024-08-20T03:14:43.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:43 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:43.368+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:28.804337+0000 front 2024-08-20T03:13:28.804287+0000 (oldest deadline 2024-08-20T03:13:54.704494+0000) 2024-08-20T03:14:43.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:43 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:43.368+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.705055+0000 front 2024-08-20T03:13:34.705134+0000 (oldest deadline 2024-08-20T03:14:00.605160+0000) 2024-08-20T03:14:43.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:43 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:43.368+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6814 osd.3 since back 2024-08-20T03:13:43.406583+0000 front 2024-08-20T03:13:43.406671+0000 (oldest deadline 2024-08-20T03:14:09.306471+0000) 2024-08-20T03:14:43.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:43 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:43.368+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:53.407299+0000 front 2024-08-20T03:13:53.407267+0000 (oldest deadline 2024-08-20T03:14:16.907464+0000) 2024-08-20T03:14:43.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:43 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:43.368+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:01.508891+0000 front 2024-08-20T03:14:01.508979+0000 (oldest deadline 2024-08-20T03:14:27.408706+0000) 2024-08-20T03:14:43.665 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:43 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10[48396]: 2024-08-20T03:14:43.368+0000 7f7c57756640 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.309525+0000 front 2024-08-20T03:14:10.309495+0000 (oldest deadline 2024-08-20T03:14:36.209457+0000) 2024-08-20T03:14:43.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:43 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:43.497+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:43.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:43 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:43.497+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:43.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:43 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:43.497+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:43.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:43 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:43.497+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:43.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:43 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:43.497+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:43.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:43 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:43.497+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:44.632 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:44 smithi099 podman[59423]: 2024-08-20 03:14:44.34555114 +0000 UTC m=+5.661943632 container died b0ea8d8553c9acfeff9dc3ad8af013b51251a3f4a4a8349689eb478b1c7a638d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-08-20T03:14:44.840 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:44 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:44.453+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:44.841 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:44 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:44.453+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:44.841 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:44 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:44.453+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:44.841 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:44 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:44.453+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:44.841 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:44 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:44.453+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:44.841 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:44 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:44.453+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:44.841 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:44 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:44.453+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-08-20T03:14:18.926793+0000 front 2024-08-20T03:14:18.926753+0000 (oldest deadline 2024-08-20T03:14:43.626657+0000) 2024-08-20T03:14:44.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:44 smithi099 podman[59423]: 2024-08-20 03:14:44.712252014 +0000 UTC m=+6.028644491 container cleanup b0ea8d8553c9acfeff9dc3ad8af013b51251a3f4a4a8349689eb478b1c7a638d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-20T03:14:44.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:44 smithi099 bash[59423]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10 2024-08-20T03:14:45.312 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:45 smithi099 podman[59438]: 2024-08-20 03:14:45.005098905 +0000 UTC m=+0.649670214 container remove b0ea8d8553c9acfeff9dc3ad8af013b51251a3f4a4a8349689eb478b1c7a638d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:14:45.584 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:45 smithi099 podman[59527]: 2024-08-20 03:14:45.213456253 +0000 UTC m=+0.016273197 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:14:45.584 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:45 smithi099 podman[59527]: 2024-08-20 03:14:45.32257618 +0000 UTC m=+0.125393153 container create 634c732eb0b8b801b202f60e4a030a665df47e6e63afc5ebd91dfd727f26af6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T03:14:45.585 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:45 smithi099 podman[59527]: 2024-08-20 03:14:45.533329069 +0000 UTC m=+0.336146000 container init 634c732eb0b8b801b202f60e4a030a665df47e6e63afc5ebd91dfd727f26af6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:14:45.585 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:45 smithi099 podman[59527]: 2024-08-20 03:14:45.536749445 +0000 UTC m=+0.339566377 container start 634c732eb0b8b801b202f60e4a030a665df47e6e63afc5ebd91dfd727f26af6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-08-20T03:14:45.585 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:45 smithi099 podman[59527]: 2024-08-20 03:14:45.582982165 +0000 UTC m=+0.385799103 container attach 634c732eb0b8b801b202f60e4a030a665df47e6e63afc5ebd91dfd727f26af6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-20T03:14:45.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:45 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:45.451+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:45.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:45 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:45.451+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:45.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:45 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:45.451+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:45.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:45 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:45.451+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:45.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:45 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:45.451+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:45.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:45 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:45.451+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:45.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:45 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:45.451+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-08-20T03:14:18.926793+0000 front 2024-08-20T03:14:18.926753+0000 (oldest deadline 2024-08-20T03:14:43.626657+0000) 2024-08-20T03:14:45.914 INFO:journalctl@ceph.osd.10.smithi099.stdout:Aug 20 03:14:45 smithi099 podman[59527]: 2024-08-20 03:14:45.724290043 +0000 UTC m=+0.527106988 container died 634c732eb0b8b801b202f60e4a030a665df47e6e63afc5ebd91dfd727f26af6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-20T03:14:46.299 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.10.service' 2024-08-20T03:14:46.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:14:46.338 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-08-20T03:14:46.338 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-08-20T03:14:46.338 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.11 2024-08-20T03:14:46.622 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:46 smithi165 systemd[1]: Stopping Ceph osd.11 for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:14:46.622 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:46 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:46.495+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:46.622 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:46 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:46.495+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:46.622 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:46 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:46.495+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:46.622 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:46 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:46.495+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:46.622 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:46 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:46.495+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:46.623 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:46 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:46.495+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:46.623 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:46 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:46.495+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-08-20T03:14:18.926793+0000 front 2024-08-20T03:14:18.926753+0000 (oldest deadline 2024-08-20T03:14:43.626657+0000) 2024-08-20T03:14:46.623 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:46 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:46.619+0000 7f7014ae8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:14:46.623 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:46 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:46.619+0000 7f7014ae8640 -1 osd.11 88 *** Got signal Terminated *** 2024-08-20T03:14:46.623 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:46 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:46.619+0000 7f7014ae8640 -1 osd.11 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T03:14:47.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:47 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:47.545+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:47.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:47 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:47.545+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:47.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:47 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:47.545+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:47.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:47 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:47.545+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:47.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:47 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:47.545+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:47.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:47 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:47.545+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:47.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:47 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:47.545+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-08-20T03:14:18.926793+0000 front 2024-08-20T03:14:18.926753+0000 (oldest deadline 2024-08-20T03:14:43.626657+0000) 2024-08-20T03:14:48.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:48 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:48.518+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:48.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:48 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:48.518+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:48.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:48 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:48.518+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:48.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:48 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:48.518+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:48.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:48 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:48.518+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:48.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:48 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:48.518+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:48.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:48 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:48.518+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-08-20T03:14:18.926793+0000 front 2024-08-20T03:14:18.926753+0000 (oldest deadline 2024-08-20T03:14:43.626657+0000) 2024-08-20T03:14:49.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:49 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:49.535+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:49.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:49 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:49.535+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:49.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:49 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:49.535+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:49.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:49 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:49.535+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:49.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:49 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:49.535+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:49.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:49 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:49.535+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:49.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:49 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:49.535+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-08-20T03:14:18.926793+0000 front 2024-08-20T03:14:18.926753+0000 (oldest deadline 2024-08-20T03:14:43.626657+0000) 2024-08-20T03:14:49.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:49 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:49.535+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-08-20T03:14:24.727387+0000 front 2024-08-20T03:14:24.727422+0000 (oldest deadline 2024-08-20T03:14:48.827468+0000) 2024-08-20T03:14:50.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:50 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:50.491+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:50.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:50 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:50.491+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:50.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:50 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:50.491+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:50.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:50 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:50.491+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:50.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:50 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:50.491+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:50.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:50 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:50.491+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:50.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:50 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:50.491+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-08-20T03:14:18.926793+0000 front 2024-08-20T03:14:18.926753+0000 (oldest deadline 2024-08-20T03:14:43.626657+0000) 2024-08-20T03:14:50.888 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:50 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:50.491+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-08-20T03:14:24.727387+0000 front 2024-08-20T03:14:24.727422+0000 (oldest deadline 2024-08-20T03:14:48.827468+0000) 2024-08-20T03:14:51.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:51 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:51.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6806 osd.0 since back 2024-08-20T03:13:18.015794+0000 front 2024-08-20T03:13:18.015772+0000 (oldest deadline 2024-08-20T03:13:42.715979+0000) 2024-08-20T03:14:51.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:51 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:51.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-08-20T03:13:30.719021+0000 front 2024-08-20T03:13:30.719090+0000 (oldest deadline 2024-08-20T03:13:54.218755+0000) 2024-08-20T03:14:51.886 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:51 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:51.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6804 osd.2 since back 2024-08-20T03:13:34.219260+0000 front 2024-08-20T03:13:34.219660+0000 (oldest deadline 2024-08-20T03:14:00.119294+0000) 2024-08-20T03:14:51.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:51 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:51.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-08-20T03:13:54.022146+0000 front 2024-08-20T03:13:54.022094+0000 (oldest deadline 2024-08-20T03:14:16.922230+0000) 2024-08-20T03:14:51.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:51 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:51.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6812 osd.5 since back 2024-08-20T03:14:00.923859+0000 front 2024-08-20T03:14:00.923820+0000 (oldest deadline 2024-08-20T03:14:25.023919+0000) 2024-08-20T03:14:51.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:51 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:51.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.44:6822 osd.6 since back 2024-08-20T03:14:10.726157+0000 front 2024-08-20T03:14:10.726099+0000 (oldest deadline 2024-08-20T03:14:33.025939+0000) 2024-08-20T03:14:51.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:51 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:51.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-08-20T03:14:18.926793+0000 front 2024-08-20T03:14:18.926753+0000 (oldest deadline 2024-08-20T03:14:43.626657+0000) 2024-08-20T03:14:51.887 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:51 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11[48866]: 2024-08-20T03:14:51.532+0000 7f700d0f5640 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.165:6820 osd.8 since back 2024-08-20T03:14:24.727387+0000 front 2024-08-20T03:14:24.727422+0000 (oldest deadline 2024-08-20T03:14:48.827468+0000) 2024-08-20T03:14:52.363 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:52 smithi165 podman[59703]: 2024-08-20 03:14:52.105987047 +0000 UTC m=+5.572479141 container died 8e1af7e4602a45930bc6c84001ab9d116073e1ebdc7b861dd74e210551e2580e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T03:14:52.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:52 smithi165 podman[59703]: 2024-08-20 03:14:52.452807879 +0000 UTC m=+5.919299972 container cleanup 8e1af7e4602a45930bc6c84001ab9d116073e1ebdc7b861dd74e210551e2580e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-08-20T03:14:52.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:52 smithi165 bash[59703]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11 2024-08-20T03:14:53.031 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:52 smithi165 podman[59720]: 2024-08-20 03:14:52.73752298 +0000 UTC m=+0.628401546 container remove 8e1af7e4602a45930bc6c84001ab9d116073e1ebdc7b861dd74e210551e2580e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:14:53.325 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:53 smithi165 podman[59806]: 2024-08-20 03:14:53.02980937 +0000 UTC m=+0.095571122 container create 6a933d704e9e9d9e66cc8d46e6523538b5c72f7bc9031b1a51427a8d6b24fc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:14:53.325 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:53 smithi165 podman[59806]: 2024-08-20 03:14:52.950312353 +0000 UTC m=+0.016074105 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T03:14:53.325 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:53 smithi165 podman[59806]: 2024-08-20 03:14:53.25745506 +0000 UTC m=+0.323216810 container init 6a933d704e9e9d9e66cc8d46e6523538b5c72f7bc9031b1a51427a8d6b24fc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:14:53.326 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:53 smithi165 podman[59806]: 2024-08-20 03:14:53.260744103 +0000 UTC m=+0.326505855 container start 6a933d704e9e9d9e66cc8d46e6523538b5c72f7bc9031b1a51427a8d6b24fc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-08-20T03:14:53.326 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:53 smithi165 podman[59806]: 2024-08-20 03:14:53.324105334 +0000 UTC m=+0.389867105 container attach 6a933d704e9e9d9e66cc8d46e6523538b5c72f7bc9031b1a51427a8d6b24fc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-08-20T03:14:53.636 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:53 smithi165 podman[59806]: 2024-08-20 03:14:53.453956867 +0000 UTC m=+0.519718618 container died 6a933d704e9e9d9e66cc8d46e6523538b5c72f7bc9031b1a51427a8d6b24fc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-20T03:14:54.000 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.11.service' 2024-08-20T03:14:54.024 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:53 smithi165 podman[59830]: 2024-08-20 03:14:53.969606105 +0000 UTC m=+0.509849069 container remove 6a933d704e9e9d9e66cc8d46e6523538b5c72f7bc9031b1a51427a8d6b24fc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-osd-11-deactivate, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-08-20T03:14:54.024 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:53 smithi165 systemd[1]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.11.service: Deactivated successfully. 2024-08-20T03:14:54.024 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:53 smithi165 systemd[1]: Stopped Ceph osd.11 for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T03:14:54.024 INFO:journalctl@ceph.osd.11.smithi165.stdout:Aug 20 03:14:53 smithi165 systemd[1]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@osd.11.service: Consumed 4min 3.938s CPU time. 2024-08-20T03:14:54.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:14:54.040 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-08-20T03:14:54.041 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-08-20T03:14:54.041 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.a 2024-08-20T03:14:54.338 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:14:54 smithi044 systemd[1]: Stopping Ceph mds.a for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:14:54.785 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:14:54 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-a[63876]: 2024-08-20T03:14:54.335+0000 7f44a0f1a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:14:54.785 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:14:54 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-a[63876]: 2024-08-20T03:14:54.335+0000 7f44a0f1a640 -1 mds.a *** got signal Terminated *** 2024-08-20T03:14:57.696 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:14:57 smithi044 podman[99007]: 2024-08-20 03:14:57.403392021 +0000 UTC m=+3.152790176 container died 37ec51dad20bb89ce993ad9e18e7cd70eacd541b9d3a7ada6111a367a35f1670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:14:58.035 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:14:57 smithi044 podman[99007]: 2024-08-20 03:14:57.910922485 +0000 UTC m=+3.660320673 container cleanup 37ec51dad20bb89ce993ad9e18e7cd70eacd541b9d3a7ada6111a367a35f1670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:14:58.035 INFO:journalctl@ceph.mds.a.smithi044.stdout:Aug 20 03:14:57 smithi044 bash[99007]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-a 2024-08-20T03:14:58.346 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.a.service' 2024-08-20T03:14:58.381 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:14:58.382 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-08-20T03:14:58.382 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-08-20T03:14:58.382 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.d 2024-08-20T03:14:58.709 INFO:journalctl@ceph.mds.d.smithi044.stdout:Aug 20 03:14:58 smithi044 systemd[1]: Stopping Ceph mds.d for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:14:59.035 INFO:journalctl@ceph.mds.d.smithi044.stdout:Aug 20 03:14:58 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-d[62691]: 2024-08-20T03:14:58.707+0000 7fa31e3a8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:14:59.035 INFO:journalctl@ceph.mds.d.smithi044.stdout:Aug 20 03:14:58 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-d[62691]: 2024-08-20T03:14:58.707+0000 7fa31e3a8640 -1 mds.d *** got signal Terminated *** 2024-08-20T03:15:02.634 INFO:journalctl@ceph.mds.d.smithi044.stdout:Aug 20 03:15:02 smithi044 podman[99137]: 2024-08-20 03:15:02.355260281 +0000 UTC m=+3.756179748 container died 91ad41b7d956fd8dd2135e07d9e5b4985749990f6b3ba9b7201ee34d148e31f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-d, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-08-20T03:15:02.968 INFO:journalctl@ceph.mds.d.smithi044.stdout:Aug 20 03:15:02 smithi044 podman[99137]: 2024-08-20 03:15:02.781621798 +0000 UTC m=+4.182541262 container cleanup 91ad41b7d956fd8dd2135e07d9e5b4985749990f6b3ba9b7201ee34d148e31f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-d, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T03:15:02.968 INFO:journalctl@ceph.mds.d.smithi044.stdout:Aug 20 03:15:02 smithi044 bash[99137]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-d 2024-08-20T03:15:03.195 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.d.service' 2024-08-20T03:15:03.219 INFO:journalctl@ceph.mds.d.smithi044.stdout:Aug 20 03:15:02 smithi044 podman[99149]: 2024-08-20 03:15:02.966719764 +0000 UTC m=+0.612396151 container remove 91ad41b7d956fd8dd2135e07d9e5b4985749990f6b3ba9b7201ee34d148e31f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:15:03.219 INFO:journalctl@ceph.mds.d.smithi044.stdout:Aug 20 03:15:03 smithi044 systemd[1]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.d.service: Deactivated successfully. 2024-08-20T03:15:03.219 INFO:journalctl@ceph.mds.d.smithi044.stdout:Aug 20 03:15:03 smithi044 systemd[1]: Stopped Ceph mds.d for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T03:15:03.219 INFO:journalctl@ceph.mds.d.smithi044.stdout:Aug 20 03:15:03 smithi044 systemd[1]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.d.service: Consumed 1.047s CPU time. 2024-08-20T03:15:03.229 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:15:03.229 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-08-20T03:15:03.229 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-08-20T03:15:03.229 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.g 2024-08-20T03:15:03.535 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:15:03 smithi044 systemd[1]: Stopping Ceph mds.g for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:15:04.035 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:15:03 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-g[63491]: 2024-08-20T03:15:03.543+0000 7f9234934640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:15:04.036 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:15:03 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-g[63491]: 2024-08-20T03:15:03.544+0000 7f9234934640 -1 mds.g *** got signal Terminated *** 2024-08-20T03:15:05.713 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:15:05 smithi044 podman[99257]: 2024-08-20 03:15:05.417577707 +0000 UTC m=+1.971636922 container died 98b40491673a910fce95b8a51f96d1cbc9b5f56d0c8a0fab80a9c4c22a5b161d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-g, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True) 2024-08-20T03:15:06.035 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:15:05 smithi044 podman[99257]: 2024-08-20 03:15:05.903078915 +0000 UTC m=+2.457138175 container cleanup 98b40491673a910fce95b8a51f96d1cbc9b5f56d0c8a0fab80a9c4c22a5b161d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:15:06.036 INFO:journalctl@ceph.mds.g.smithi044.stdout:Aug 20 03:15:05 smithi044 bash[99257]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-g 2024-08-20T03:15:06.282 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.g.service' 2024-08-20T03:15:06.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:15:06.315 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-08-20T03:15:06.315 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-08-20T03:15:06.315 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.j 2024-08-20T03:15:06.535 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 03:15:06 smithi044 systemd[1]: Stopping Ceph mds.j for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:15:07.035 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 03:15:06 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-j[62294]: 2024-08-20T03:15:06.640+0000 7f384f587640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:15:07.035 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 03:15:06 smithi044 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-j[62294]: 2024-08-20T03:15:06.640+0000 7f384f587640 -1 mds.j *** got signal Terminated *** 2024-08-20T03:15:10.900 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 03:15:10 smithi044 podman[99379]: 2024-08-20 03:15:10.64834867 +0000 UTC m=+4.105911997 container died 48ce209b7fab2e767801d5befff2c5f7fbdf0f4a56f46efae923003a6d1f49ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-08-20T03:15:11.276 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 03:15:11 smithi044 podman[99379]: 2024-08-20 03:15:11.040553267 +0000 UTC m=+4.498116593 container cleanup 48ce209b7fab2e767801d5befff2c5f7fbdf0f4a56f46efae923003a6d1f49ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-j, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-20T03:15:11.276 INFO:journalctl@ceph.mds.j.smithi044.stdout:Aug 20 03:15:11 smithi044 bash[99379]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-j 2024-08-20T03:15:11.426 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.j.service' 2024-08-20T03:15:11.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:15:11.458 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-08-20T03:15:11.459 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-08-20T03:15:11.459 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.b 2024-08-20T03:15:11.761 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:15:11 smithi099 systemd[1]: Stopping Ceph mds.b for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:15:12.024 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:15:11 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-b[53751]: 2024-08-20T03:15:11.759+0000 7f55ee616640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:15:12.024 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:15:11 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-b[53751]: 2024-08-20T03:15:11.759+0000 7f55ee616640 -1 mds.b *** got signal Terminated *** 2024-08-20T03:15:13.915 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:15:13 smithi099 podman[59630]: 2024-08-20 03:15:13.430913061 +0000 UTC m=+1.777331951 container died 2a1257101341a66597c6096b9e1f16659ec661031c76f2b1b3654c953ba2abc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-08-20T03:15:14.262 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:15:13 smithi099 podman[59630]: 2024-08-20 03:15:13.992648347 +0000 UTC m=+2.339067236 container cleanup 2a1257101341a66597c6096b9e1f16659ec661031c76f2b1b3654c953ba2abc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-b, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:15:14.262 INFO:journalctl@ceph.mds.b.smithi099.stdout:Aug 20 03:15:13 smithi099 bash[59630]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-b 2024-08-20T03:15:14.414 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.b.service' 2024-08-20T03:15:14.450 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:15:14.450 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-08-20T03:15:14.450 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-08-20T03:15:14.450 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.e 2024-08-20T03:15:14.774 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 03:15:14 smithi099 systemd[1]: Stopping Ceph mds.e for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:15:15.164 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 03:15:14 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-e[53147]: 2024-08-20T03:15:14.771+0000 7f32457a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:15:15.164 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 03:15:14 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-e[53147]: 2024-08-20T03:15:14.771+0000 7f32457a7640 -1 mds.e *** got signal Terminated *** 2024-08-20T03:15:17.707 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 03:15:17 smithi099 podman[59763]: 2024-08-20 03:15:17.436317115 +0000 UTC m=+2.770408391 container died 595d45342c7041e680e1fbc231c6a59d95ead5d834489d2ec8f5389312181e41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-20T03:15:18.065 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 03:15:17 smithi099 podman[59763]: 2024-08-20 03:15:17.804246326 +0000 UTC m=+3.138337605 container cleanup 595d45342c7041e680e1fbc231c6a59d95ead5d834489d2ec8f5389312181e41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-08-20T03:15:18.065 INFO:journalctl@ceph.mds.e.smithi099.stdout:Aug 20 03:15:17 smithi099 bash[59763]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-e 2024-08-20T03:15:18.187 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.e.service' 2024-08-20T03:15:18.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:15:18.221 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-08-20T03:15:18.221 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-08-20T03:15:18.221 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.h 2024-08-20T03:15:18.414 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 03:15:18 smithi099 systemd[1]: Stopping Ceph mds.h for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:15:18.914 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 03:15:18 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h[53448]: 2024-08-20T03:15:18.574+0000 7ff74e8cb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:15:18.914 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 03:15:18 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h[53448]: 2024-08-20T03:15:18.574+0000 7ff74e8cb640 -1 mds.h *** got signal Terminated *** 2024-08-20T03:15:21.693 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 03:15:21 smithi099 podman[59884]: 2024-08-20 03:15:21.422078571 +0000 UTC m=+2.978044711 container died 02ec440a0d71d6169cc66d5761e117f7513a278ce0bc88df38a6acd69fa284d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-20T03:15:22.126 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 03:15:21 smithi099 podman[59884]: 2024-08-20 03:15:21.75770769 +0000 UTC m=+3.313673829 container cleanup 02ec440a0d71d6169cc66d5761e117f7513a278ce0bc88df38a6acd69fa284d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-20T03:15:22.126 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 03:15:21 smithi099 bash[59884]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h 2024-08-20T03:15:22.126 INFO:journalctl@ceph.mds.h.smithi099.stdout:Aug 20 03:15:21 smithi099 podman[59897]: 2024-08-20 03:15:21.925496096 +0000 UTC m=+0.499229675 container remove 02ec440a0d71d6169cc66d5761e117f7513a278ce0bc88df38a6acd69fa284d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-h, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-20T03:15:22.138 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.h.service' 2024-08-20T03:15:22.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:15:22.170 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-08-20T03:15:22.170 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-08-20T03:15:22.171 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.k 2024-08-20T03:15:22.414 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 03:15:22 smithi099 systemd[1]: Stopping Ceph mds.k for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:15:22.914 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 03:15:22 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-k[52833]: 2024-08-20T03:15:22.494+0000 7faaf8ccf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:15:22.915 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 03:15:22 smithi099 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-k[52833]: 2024-08-20T03:15:22.494+0000 7faaf8ccf640 -1 mds.k *** got signal Terminated *** 2024-08-20T03:15:25.679 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 03:15:25 smithi099 podman[60006]: 2024-08-20 03:15:25.424377092 +0000 UTC m=+3.038782084 container died b692a8f7e019459dbba23929498832959f7efdb2dbb88b448c2d8663c4019953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-08-20T03:15:26.063 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 03:15:25 smithi099 podman[60006]: 2024-08-20 03:15:25.777954806 +0000 UTC m=+3.392359791 container cleanup b692a8f7e019459dbba23929498832959f7efdb2dbb88b448c2d8663c4019953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-k, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T03:15:26.063 INFO:journalctl@ceph.mds.k.smithi099.stdout:Aug 20 03:15:25 smithi099 bash[60006]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-k 2024-08-20T03:15:26.194 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.k.service' 2024-08-20T03:15:26.225 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:15:26.225 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-08-20T03:15:26.225 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-08-20T03:15:26.225 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.c 2024-08-20T03:15:26.545 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 03:15:26 smithi165 systemd[1]: Stopping Ceph mds.c for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:15:26.808 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 03:15:26 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c[54158]: 2024-08-20T03:15:26.543+0000 7fdf14a84640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:15:26.808 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 03:15:26 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c[54158]: 2024-08-20T03:15:26.543+0000 7fdf14a84640 -1 mds.c *** got signal Terminated *** 2024-08-20T03:15:29.745 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 03:15:29 smithi165 podman[59909]: 2024-08-20 03:15:29.418303245 +0000 UTC m=+2.990587799 container died 3c27108e7bb325e6f8cae18ef9534218e9a733eaa9da3848a1022c0bed44a0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:15:30.004 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 03:15:29 smithi165 podman[59909]: 2024-08-20 03:15:29.826451189 +0000 UTC m=+3.398735741 container cleanup 3c27108e7bb325e6f8cae18ef9534218e9a733eaa9da3848a1022c0bed44a0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T03:15:30.004 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 03:15:29 smithi165 bash[59909]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c 2024-08-20T03:15:30.267 INFO:journalctl@ceph.mds.c.smithi165.stdout:Aug 20 03:15:30 smithi165 podman[59933]: 2024-08-20 03:15:30.002922168 +0000 UTC m=+0.585644761 container remove 3c27108e7bb325e6f8cae18ef9534218e9a733eaa9da3848a1022c0bed44a0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-08-20T03:15:30.271 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.c.service' 2024-08-20T03:15:30.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:15:30.307 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-08-20T03:15:30.307 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-08-20T03:15:30.307 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.f 2024-08-20T03:15:30.608 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:15:30 smithi165 systemd[1]: Stopping Ceph mds.f for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:15:30.885 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:15:30 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f[53859]: 2024-08-20T03:15:30.605+0000 7f41da7b4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:15:30.886 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:15:30 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f[53859]: 2024-08-20T03:15:30.605+0000 7f41da7b4640 -1 mds.f *** got signal Terminated *** 2024-08-20T03:15:33.675 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:15:33 smithi165 podman[60043]: 2024-08-20 03:15:33.41116165 +0000 UTC m=+2.890730944 container died 0783fe6121a0a108f9921c0bebd470f2d1da52b84b7b845660fa4394a93f7f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T03:15:33.942 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:15:33 smithi165 podman[60043]: 2024-08-20 03:15:33.763940733 +0000 UTC m=+3.243510027 container cleanup 0783fe6121a0a108f9921c0bebd470f2d1da52b84b7b845660fa4394a93f7f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0) 2024-08-20T03:15:33.942 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:15:33 smithi165 bash[60043]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f 2024-08-20T03:15:34.190 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.f.service' 2024-08-20T03:15:34.212 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:15:33 smithi165 podman[60055]: 2024-08-20 03:15:33.940619464 +0000 UTC m=+0.523551074 container remove 0783fe6121a0a108f9921c0bebd470f2d1da52b84b7b845660fa4394a93f7f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-08-20T03:15:34.212 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:15:34 smithi165 systemd[1]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.f.service: Deactivated successfully. 2024-08-20T03:15:34.212 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:15:34 smithi165 systemd[1]: Stopped Ceph mds.f for 0284e16c-5e9f-11ef-bccf-c7b262605968. 2024-08-20T03:15:34.212 INFO:journalctl@ceph.mds.f.smithi165.stdout:Aug 20 03:15:34 smithi165 systemd[1]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.f.service: Consumed 44.086s CPU time. 2024-08-20T03:15:34.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:15:34.228 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-08-20T03:15:34.228 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-08-20T03:15:34.228 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.i 2024-08-20T03:15:34.544 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:15:34 smithi165 systemd[1]: Stopping Ceph mds.i for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:15:34.886 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:15:34 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:15:34.542+0000 7fa348919640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:15:34.886 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:15:34 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i[54459]: 2024-08-20T03:15:34.542+0000 7fa348919640 -1 mds.i *** got signal Terminated *** 2024-08-20T03:15:37.636 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:15:37 smithi165 podman[60168]: 2024-08-20 03:15:37.431770742 +0000 UTC m=+2.982807063 container died ef18f09e186f9d8a43c8c005d2ad93071e4adad50b06f4cff0498dd25104f066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-20T03:15:38.029 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:15:37 smithi165 podman[60168]: 2024-08-20 03:15:37.810019316 +0000 UTC m=+3.361055647 container cleanup ef18f09e186f9d8a43c8c005d2ad93071e4adad50b06f4cff0498dd25104f066 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-08-20T03:15:38.029 INFO:journalctl@ceph.mds.i.smithi165.stdout:Aug 20 03:15:37 smithi165 bash[60168]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-i 2024-08-20T03:15:38.159 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.i.service' 2024-08-20T03:15:38.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:15:38.192 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-08-20T03:15:38.192 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-08-20T03:15:38.192 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.l 2024-08-20T03:15:38.386 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 03:15:38 smithi165 systemd[1]: Stopping Ceph mds.l for 0284e16c-5e9f-11ef-bccf-c7b262605968... 2024-08-20T03:15:38.886 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 03:15:38 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l[53546]: 2024-08-20T03:15:38.488+0000 7fa46c7f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T03:15:38.886 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 03:15:38 smithi165 ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l[53546]: 2024-08-20T03:15:38.488+0000 7fa46c7f9640 -1 mds.l *** got signal Terminated *** 2024-08-20T03:15:41.691 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 03:15:41 smithi165 podman[60288]: 2024-08-20 03:15:41.411664736 +0000 UTC m=+3.009624852 container died 99599d5447be1c8f27eafb400bc7a50057dd3a2537cc9b0110059976645c1a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-08-20T03:15:42.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 03:15:41 smithi165 podman[60288]: 2024-08-20 03:15:41.763745219 +0000 UTC m=+3.361705335 container cleanup 99599d5447be1c8f27eafb400bc7a50057dd3a2537cc9b0110059976645c1a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=) 2024-08-20T03:15:42.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 03:15:41 smithi165 bash[60288]: ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l 2024-08-20T03:15:42.136 INFO:journalctl@ceph.mds.l.smithi165.stdout:Aug 20 03:15:41 smithi165 podman[60300]: 2024-08-20 03:15:41.931414357 +0000 UTC m=+0.509461387 container remove 99599d5447be1c8f27eafb400bc7a50057dd3a2537cc9b0110059976645c1a86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-0284e16c-5e9f-11ef-bccf-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-08-20T03:15:42.180 DEBUG:teuthology.orchestra.run.smithi165:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0284e16c-5e9f-11ef-bccf-c7b262605968@mds.l.service' 2024-08-20T03:15:42.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T03:15:42.212 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-08-20T03:15:42.212 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-20T03:15:42.421 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T03:15:44.868 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-20T03:15:45.070 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T03:15:47.475 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-20T03:15:47.677 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T03:15:50.139 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T03:15:50.172 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T03:15:50.205 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T03:15:50.232 INFO:tasks.cephadm:Archiving crash dumps... 2024-08-20T03:15:50.235 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390/remote/smithi044/crash 2024-08-20T03:15:50.236 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/crash -- . 2024-08-20T03:15:50.262 INFO:teuthology.orchestra.run.smithi044.stderr:tar: /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-20T03:15:50.262 INFO:teuthology.orchestra.run.smithi044.stderr:tar: Error is not recoverable: exiting now 2024-08-20T03:15:50.264 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390/remote/smithi099/crash 2024-08-20T03:15:50.265 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/crash -- . 2024-08-20T03:15:50.304 INFO:teuthology.orchestra.run.smithi099.stderr:tar: /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-20T03:15:50.304 INFO:teuthology.orchestra.run.smithi099.stderr:tar: Error is not recoverable: exiting now 2024-08-20T03:15:50.306 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390/remote/smithi165/crash 2024-08-20T03:15:50.306 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/crash -- . 2024-08-20T03:15:50.339 INFO:teuthology.orchestra.run.smithi165.stderr:tar: /var/lib/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-20T03:15:50.339 INFO:teuthology.orchestra.run.smithi165.stderr:tar: Error is not recoverable: exiting now 2024-08-20T03:15:50.340 INFO:tasks.cephadm:Checking cluster log for badness... 2024-08-20T03:15:50.341 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-20T03:15:50.374 INFO:teuthology.orchestra.run.smithi044.stdout:2024-08-20T03:09:16.952178+0000 mds.i (mds.0) 39 : cluster [WRN] Scrub error on inode 0x1000000b0f2 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-163) see mds.i log and `damage ls` output for details 2024-08-20T03:15:50.374 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-08-20T03:15:50.374 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[SEC\]' /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-20T03:15:50.449 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[ERR\]' /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-20T03:15:50.524 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[WRN\]' /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-20T03:15:50.598 INFO:teuthology.orchestra.run.smithi044.stdout:2024-08-20T03:09:16.952178+0000 mds.i (mds.0) 39 : cluster [WRN] Scrub error on inode 0x1000000b0f2 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-163) see mds.i log and `damage ls` output for details 2024-08-20T03:15:50.598 INFO:tasks.cephadm:Compressing logs... 2024-08-20T03:15:50.598 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-20T03:15:50.641 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-20T03:15:50.644 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-20T03:15:50.663 INFO:teuthology.orchestra.run.smithi044.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-20T03:15:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:: No such file or directory 2024-08-20T03:15:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-20T03:15:50.664 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.9.log 2024-08-20T03:15:50.666 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mgr.x.log 2024-08-20T03:15:50.666 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.audit.log: /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log 2024-08-20T03:15:50.666 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.g.log 2024-08-20T03:15:50.666 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.3.log 2024-08-20T03:15:50.666 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-20T03:15:50.666 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.0.log 2024-08-20T03:15:50.667 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-20T03:15:50.667 INFO:teuthology.orchestra.run.smithi044.stderr: 90.1% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-20T03:15:50.668 INFO:teuthology.orchestra.run.smithi044.stderr: 86.5% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-20T03:15:50.668 INFO:teuthology.orchestra.run.smithi044.stderr: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-20T03:15:50.672 INFO:teuthology.orchestra.run.smithi165.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:: No such file or directory 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.2.log 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.5.log 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.f.log 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.c.log 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.8.log 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.i.log 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.l.log 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mon.c.log 2024-08-20T03:15:50.673 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.11.log 2024-08-20T03:15:50.674 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-20T03:15:50.674 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mgr.z.log 2024-08-20T03:15:50.675 INFO:teuthology.orchestra.run.smithi099.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-20T03:15:50.675 INFO:teuthology.orchestra.run.smithi099.stderr:: No such file or directory 2024-08-20T03:15:50.675 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-20T03:15:50.675 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.7.log 2024-08-20T03:15:50.675 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.e.log 2024-08-20T03:15:50.675 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log 2024-08-20T03:15:50.676 INFO:teuthology.orchestra.run.smithi099.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-20T03:15:50.677 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.4.log 2024-08-20T03:15:50.678 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.e.log: /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mon.b.log 2024-08-20T03:15:50.678 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.1.log 2024-08-20T03:15:50.678 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.k.log 2024-08-20T03:15:50.678 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-20T03:15:50.678 INFO:teuthology.orchestra.run.smithi099.stderr: 90.3%/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.k.log: -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-20T03:15:50.678 INFO:teuthology.orchestra.run.smithi099.stderr: 86.5% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-20T03:15:50.680 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.j.log 2024-08-20T03:15:50.680 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mon.a.log 2024-08-20T03:15:50.680 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.a.log 2024-08-20T03:15:50.680 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.6.log 2024-08-20T03:15:50.686 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.a.log: 95.1% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.j.log.gz 2024-08-20T03:15:50.686 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.d.log 2024-08-20T03:15:50.688 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.10.log 2024-08-20T03:15:50.689 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.h.log 2024-08-20T03:15:50.689 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.10.log: 81.4% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-20T03:15:50.690 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mgr.y.log 2024-08-20T03:15:50.698 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-20T03:15:50.709 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.d.log: 95.1% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.d.log.gz 2024-08-20T03:15:50.718 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.b.log 2024-08-20T03:15:50.731 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mgr.y.log: 92.5%gzip -5 --verbose -- /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-20T03:15:50.732 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-volume.log: /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mgr.z.log: 92.6% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mgr.z.log.gz 2024-08-20T03:15:50.736 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-volume.log: 90.9% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-20T03:15:50.738 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.b.log: -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mgr.y.log.gz 2024-08-20T03:15:50.772 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-volume.log: 90.8% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-20T03:15:50.845 INFO:teuthology.orchestra.run.smithi165.stderr: 86.5% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-20T03:15:50.902 INFO:teuthology.orchestra.run.smithi165.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-20T03:15:50.922 INFO:teuthology.orchestra.run.smithi044.stderr: 94.3% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.g.log.gz 2024-08-20T03:15:50.926 INFO:teuthology.orchestra.run.smithi165.stderr: 90.8% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-20T03:15:50.946 INFO:teuthology.orchestra.run.smithi165.stderr: 90.3% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-20T03:15:51.031 INFO:teuthology.orchestra.run.smithi044.stderr: 90.2% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mgr.x.log.gz 2024-08-20T03:15:51.129 INFO:teuthology.orchestra.run.smithi099.stderr: 91.7% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mon.b.log.gz 2024-08-20T03:15:51.235 INFO:teuthology.orchestra.run.smithi165.stderr: 81.4% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-20T03:15:51.265 INFO:teuthology.orchestra.run.smithi099.stderr: 94.8% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.k.log.gz 2024-08-20T03:15:51.516 INFO:teuthology.orchestra.run.smithi044.stderr: 94.8% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.a.log.gz 2024-08-20T03:15:51.940 INFO:teuthology.orchestra.run.smithi044.stderr: 90.3% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mon.a.log.gz 2024-08-20T03:15:52.984 INFO:teuthology.orchestra.run.smithi165.stderr: 91.6% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mon.c.log.gz 2024-08-20T03:15:56.802 INFO:teuthology.orchestra.run.smithi099.stderr: 94.6% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.h.log.gz 2024-08-20T03:16:06.597 INFO:teuthology.orchestra.run.smithi099.stderr: 94.0% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.b.log.gz 2024-08-20T03:16:09.594 INFO:teuthology.orchestra.run.smithi044.stderr: 91.5% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.9.log.gz 2024-08-20T03:16:17.187 INFO:teuthology.orchestra.run.smithi099.stderr: 94.5% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.e.log.gz 2024-08-20T03:16:17.588 INFO:teuthology.orchestra.run.smithi165.stderr: 95.1% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.l.log.gz 2024-08-20T03:16:17.955 INFO:teuthology.orchestra.run.smithi044.stderr: 91.4% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.6.log.gz 2024-08-20T03:16:19.646 INFO:teuthology.orchestra.run.smithi099.stderr: 91.3% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.10.log.gz 2024-08-20T03:16:22.250 INFO:teuthology.orchestra.run.smithi099.stderr: 91.4% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.4.log.gz 2024-08-20T03:16:22.371 INFO:teuthology.orchestra.run.smithi044.stderr: 91.4% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.0.log.gz 2024-08-20T03:16:22.469 INFO:teuthology.orchestra.run.smithi099.stderr: 91.4% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.1.log.gz 2024-08-20T03:16:25.617 INFO:teuthology.orchestra.run.smithi044.stderr: 91.5% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.3.log.gz 2024-08-20T03:16:25.619 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-08-20T03:16:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:real 0m34.965s 2024-08-20T03:16:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:user 1m51.975s 2024-08-20T03:16:25.619 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m3.427s 2024-08-20T03:16:28.030 INFO:teuthology.orchestra.run.smithi099.stderr: 91.6% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.7.log.gz 2024-08-20T03:16:28.032 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-08-20T03:16:28.032 INFO:teuthology.orchestra.run.smithi099.stderr:real 0m37.373s 2024-08-20T03:16:28.032 INFO:teuthology.orchestra.run.smithi099.stderr:user 2m53.340s 2024-08-20T03:16:28.033 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m5.028s 2024-08-20T03:16:44.524 INFO:teuthology.orchestra.run.smithi165.stderr: 94.5% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.c.log.gz 2024-08-20T03:17:08.976 INFO:teuthology.orchestra.run.smithi165.stderr: 94.4% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.f.log.gz 2024-08-20T03:18:19.837 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.2.log.gz 2024-08-20T03:19:08.363 INFO:teuthology.orchestra.run.smithi165.stderr: 91.5% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.8.log.gz 2024-08-20T03:19:39.916 INFO:teuthology.orchestra.run.smithi165.stderr: 91.4% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.5.log.gz 2024-08-20T03:19:58.154 INFO:teuthology.orchestra.run.smithi165.stderr: 91.4% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-osd.11.log.gz 2024-08-20T03:24:45.079 INFO:teuthology.orchestra.run.smithi165.stderr: 92.9% -- replaced with /var/log/ceph/0284e16c-5e9f-11ef-bccf-c7b262605968/ceph-mds.i.log.gz 2024-08-20T03:24:45.083 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-08-20T03:24:45.083 INFO:teuthology.orchestra.run.smithi165.stderr:real 8m54.421s 2024-08-20T03:24:45.083 INFO:teuthology.orchestra.run.smithi165.stderr:user 6m51.152s 2024-08-20T03:24:45.083 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m15.526s 2024-08-20T03:24:45.084 INFO:tasks.cephadm:Archiving logs... 2024-08-20T03:24:45.085 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390/remote/smithi044/log 2024-08-20T03:24:45.086 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-20T03:25:03.520 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390/remote/smithi099/log 2024-08-20T03:25:03.521 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-20T03:25:26.009 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390/remote/smithi165/log 2024-08-20T03:25:26.010 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-20T03:26:32.279 INFO:tasks.cephadm:Removing cluster... 2024-08-20T03:26:32.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 --force 2024-08-20T03:26:32.490 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T03:26:32.923 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 --force 2024-08-20T03:26:33.125 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T03:26:33.799 DEBUG:teuthology.orchestra.run.smithi165:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0284e16c-5e9f-11ef-bccf-c7b262605968 --force 2024-08-20T03:26:34.525 INFO:teuthology.orchestra.run.smithi165.stdout:Deleting cluster with fsid: 0284e16c-5e9f-11ef-bccf-c7b262605968 2024-08-20T03:26:35.152 INFO:tasks.cephadm:Removing cephadm ... 2024-08-20T03:26:35.152 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-20T03:26:35.171 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-20T03:26:35.190 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-20T03:26:35.207 INFO:tasks.cephadm:Teardown complete 2024-08-20T03:26:35.207 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-20T03:26:35.218 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-20T03:26:35.218 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-20T03:26:35.221 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-20T03:26:35.233 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-20T03:26:35.369 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-20T03:26:35.380 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-20T03:26:35.380 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T03:26:35.383 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T03:26:35.385 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T03:26:35.397 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-08-20T03:26:35.400 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-08-20T03:26:35.400 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-08-20T03:26:35.403 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T03:26:35.403 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-08-20T03:26:35.403 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv01.front.sepia.ceph.com 3 6 377 57 -414us[ -409us] +/- 52ms 2024-08-20T03:26:35.403 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 +315us[ +321us] +/- 77ms 2024-08-20T03:26:35.403 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +317us[ +322us] +/- 71ms 2024-08-20T03:26:35.404 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T03:26:35.404 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T03:26:35.404 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-08-20T03:26:35.404 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 -399us[ -391us] +/- 52ms 2024-08-20T03:26:35.404 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 +335us[ +342us] +/- 77ms 2024-08-20T03:26:35.404 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +322us[ +322us] +/- 71ms 2024-08-20T03:26:35.405 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T03:26:35.417 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T03:26:35.417 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-08-20T03:26:35.417 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv01.front.sepia.ceph.com 3 6 377 55 -431us[ -429us] +/- 52ms 2024-08-20T03:26:35.417 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 +314us[ +316us] +/- 77ms 2024-08-20T03:26:35.417 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 +288us[ +290us] +/- 71ms 2024-08-20T03:26:35.417 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T03:26:35.418 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-20T03:26:35.428 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-20T03:26:35.429 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-20T03:26:35.439 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-20T03:26:35.515 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-20T03:26:35.575 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-20T03:26:35.639 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T03:26:35.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T03:26:35.668 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T03:26:35.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T03:26:35.698 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T03:26:35.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T03:26:35.725 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-20T03:26:35.735 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-20T03:26:35.745 INFO:teuthology.task.internal:Duration was 2395.653672 seconds 2024-08-20T03:26:35.745 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-20T03:26:35.755 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-20T03:26:35.755 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-20T03:26:35.758 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-20T03:26:35.760 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-20T03:26:35.796 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T03:26:35.799 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T03:26:35.837 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T03:26:36.206 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-20T03:26:36.207 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-08-20T03:26:36.207 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-20T03:26:36.228 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-08-20T03:26:36.228 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-20T03:26:36.276 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-08-20T03:26:36.276 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-20T03:26:36.304 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-20T03:26:36.304 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-20T03:26:36.307 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-20T03:26:36.319 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-20T03:26:36.371 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-20T03:26:36.372 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-20T03:26:36.375 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-20T03:26:36.391 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-20T03:26:37.329 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-20T03:26:37.340 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-20T03:26:37.341 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-20T03:26:37.365 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-20T03:26:37.393 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-20T03:26:37.424 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-20T03:26:37.434 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-20T03:26:37.437 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-20T03:26:37.440 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-20T03:26:37.457 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-08-20T03:26:37.463 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-08-20T03:26:37.755 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-08-20T03:26:37.775 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-20T03:26:37.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T03:26:37.790 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-20T03:26:37.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T03:26:37.808 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-20T03:26:37.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T03:26:37.826 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-20T03:26:37.838 INFO:teuthology.task.internal:Transferring archived files... 2024-08-20T03:26:37.839 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390/remote/smithi044 2024-08-20T03:26:37.839 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-20T03:26:37.890 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390/remote/smithi099 2024-08-20T03:26:37.891 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-20T03:26:37.928 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861390/remote/smithi165 2024-08-20T03:26:37.928 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-20T03:26:37.966 INFO:teuthology.task.internal:Removing archive directory... 2024-08-20T03:26:37.967 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-20T03:26:37.969 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-20T03:26:37.972 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-20T03:26:38.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-20T03:26:38.035 INFO:teuthology.task.internal:Not uploading archives. 2024-08-20T03:26:38.035 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-20T03:26:38.046 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-20T03:26:38.046 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-20T03:26:38.049 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-20T03:26:38.051 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-20T03:26:38.061 INFO:teuthology.orchestra.run.smithi044.stdout: 262217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 20 03:26 /home/ubuntu/cephtest 2024-08-20T03:26:38.066 INFO:teuthology.orchestra.run.smithi099.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 20 03:26 /home/ubuntu/cephtest 2024-08-20T03:26:38.077 INFO:teuthology.orchestra.run.smithi165.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 20 03:26 /home/ubuntu/cephtest 2024-08-20T03:26:38.078 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-20T03:26:38.089 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-20T03:26:38.120 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/blogbench}} duration: 2395.6536724567413 failure_reason: '"2024-08-20T03:09:16.952178+0000 mds.i (mds.0) 39 : cluster [WRN] Scrub error on inode 0x1000000b0f2 (/volumes/qa/sv_1/015b80c1-9189-4015-8515-87e37dc378f2/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-163) see mds.i log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_hyelloji@teuthology success: false 2024-08-20T03:26:38.120 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-20T03:26:38.217 INFO:teuthology.run:FAIL